converge ppt
TRANSCRIPT
Check Yo’selfWhat Ice Cube taught me about security metrics
DisclaimerAll opinions and thoughts in this presentation are my own and do not represent my employerAll use of Ice Cube’s image, lyrics, movies, and music are for storytelling, not for profitThe data used in this presentation comes from my employer, but is anonymized to protect the guilty and innocent
OverviewSpeedQualityCoverageCharts & Takeaways
Quality
Coverage
Speed
SpeedIf you're foul, you better run a make on that license plate
You coulda had a V8Instead of a tre-eight slug to the cranium
I got six and I'm aimin em
SpeedHow fast did you find the breach?How fast did you stop the breach after it happened?How fast did you clean it up?
How fast did you go from What? to So What? to Now What?
Speed
Comp Find Alert Give a s&*t?
Taking action
Stop the s&*t
Clean up the s&*t
You better check yourself before you wreck yourselfCause I'm bad for your health, I come real stealthDropping bombs on your moms, f*** car alarmsDoing foul crime, I'm that fool with your Alpine
- Check Yourself – Ice Cube
Intellectual Honesty
Time’s are all in the same time zone – goes without sayingThe time of compromise is when something changed in the system – not when you or your system found itMissing that key fact means you miss
Quality of intelligenceCoverage of intelligence
Time dropper hit the file tableTime A/V reported finding the backdoor
Difference = 7 months, 8 days, 13 hours, 34 minutes, 7 seconds
Trusted sources of truth
HostEvent logsMFTs
NetworkFirewall logsNetflow logsSMTP logs (for phish)Proxy logs (for watering-holes)
Comp-to-Find
Speed of intelligence deployment to your tools
How fast did you get it?How fast did you know it?How fast did you use it?
Frequency of scansAlertness of users
Collection
Processing
Exploitation
Dissemination
Tasking
Comp Find Alert Give a s&*t?
Taking action
Stop the s&*t
Clean up the s&*t
How to find?Host
AV logsEvent logsNagiosTripwire
NetworkIDS/IPS alertsFirewall logsProxy logsEmail gateway logs
Find-to-Alert
Speed of the sensorAre your alerts backing up on a DB somewhere?How often are sensors reporting back to their console?
Knowledge of user (protein-based sensor)
Do they know how to report shadiness?
Comp Find Alert Give a s&*t?
Taking action
Stop the s&*t
Clean up the s&*t
Alert-to-Give a s&*t
How long do alerts linger?How long do emails about incidents bounced around inboxes?SIEM logs
When analyst acknowledges the alert
Comp Find Alert Give a s&*t?
Taking action
Stop the s&*t
Clean up the s&*t
Give a s&*t-to-taking action
Speed of triage & initial analysisKnowledge of internal organization
Do your responders know who to call?
Comprehensiveness of response plans and SOPs
Comp Find Alert Give a s&*t?
Taking action
Stop the s&*t
Clean up the s&*t
I found the APT !!!
Taking Action-to-Stopping the s&*t
HostEvent log (shutdown)DHCP logAV log (deleted malz)Phish deleted
NetworkACL in switchIPS rule change logIP block added to routerFirewall block addedProxy log
Not when the rule was added, but when it was confirmed to be working
Comp Find Alert Give a s&*t?
Taking action
Stop the s&*t
Clean up the s&*t
Stopping-to-cleaning up the s&*t
How long the business was impacted by the breach?Did the containment strategy conflict with or support recovery?How fast did you find other breaches?How effective was your recovery?
The fed’s preferred recovery method
Comp Find Alert Give a s&*t?
Taking action
Stop the s&*t
Clean up the s&*t
QualityI hate motherf**kers claimin that they foldin bank
But steady talkin s&*t in the holding tankFirst you wanna step to me
Now you’re a** screamin for the deputy
QualityIt’s great that you’re fast, but are you any good at it?Easy to confuse quality with forensic soundnessEasy to confuse quality with expensive blinking boxesQuality really measures
Are you focusing on what’s really important (customer)?Are you focusing on what really works (performance)?Do you track failures as much as you do successes (defects)?Do you learn from mistakes and do you repeat them (improvement)?
Comp Find Alert Give a s&*t?
Taking action
Stop the s&*t
Clean up the s&*t
First time rightIn this process, how often were mistakes made
Do you track and categorize mistakes and misfires?
How many times did you
miss the breach?
Did the alerts go to the
right place the first time?
Did the person
viewing the alert make the right
call?
Did the person who gives a s&*t do the right
thing?
Did the actions
actually stop the breach?
Was your cleanup
effective?
Measuring QualityGet granularAvoid “other” or “unknown”
If given an option, analysts will choose “other” two out of every three times.
Set goalsWhat’s acceptable performance?
Forensics & Kill Chain
Reconnaissan
ce
Weaponizatio
n
DeliveryExploitationInstallation
C&C
Actions on
Objectives
Increasing ferocity of Ice Cube movie characters
Increasing cost of response and recovery
NetworkReconnaissanceDeliveryC2AoO
HostExploitInstallationAoO
Forensics & Kill Chain
ReconnaissanceWeaponizationDeliveryExploitationInstallationC2AoO
Forensics & Kill Chain
Know every system/person involved in the incident and how they performed – relative to the Kill Chain
PLAYING WITH MY KILL CHAIN
IS LIKE PLAYING WITH MY EMOTIONS
CoverageTricks wanna step to Cube and then they get played
Cause they b&*ch made pullin out a switchbladeThat's kinda trifle, cause that's a knife-o
[here’s an] AK-47, assault rifle
CoverageAre you looking for the right things in the right places?
Filenames in IDS?IP addresses in AV logs?
What percentage of your install base are you monitoring?First, check yo’self
Use the Kill ChainFind your gaps
Check Yo’selfHow do you get got?
Phishing?Watering holes?Thumbdrives?Websites getting popped?
For one thing, you don’t know how the f**k my company be muthaf**king
owned.
Check Yo’self
Recon Weapon Deliver Exploit Install C2 AoO
#1
#2
#3
#4
#5
Attacks stopped by Kill
Chain
#6
Check Yo’self
Recon Weapon Deliver Exploit Install C2 AoO
$$$
$$$$
$
$$ $
$$
$$$
Cost of the intrusion
Check Yo’self
Recon Weapon Deliver Exploit Install C2 AoO
$$
$$$$$$
$$$
Cost of countermeasur
es
$
Finding GapsLack of process
Misapplying IntelBad deployment of web applications
Lack of TrainingDevelopers building insecure apps
Lack of technologyBuy only when you have a clear blind spot
Not every gap in yo’ security needs to be filled with cash
money
Check yo ‘netDo you have every network ingress/egress point monitored?
3rd Parties/SuppliersVPNMobile/BYOD
Do you have monitoring on every network service?
FTP, SFTP, Web, SMTP, Telnet (yes, telnet)Cloud services (*aaS)
Gary’s manager found an un-instrumented PoP on the
network
Check yo ‘boxesWhat is your host logging policy?Do your logs go to a central location? Do you have a method to search the endpoints and servers for IOCs?How agro are your patching policies?
Will a Java patch f’ your network? http://bit.ly/1pTiodM - for other
derp-ables referring to “the APT”
TakeawaysHere to let you know boy, oh boy
I make dough but don't call me DoughBoyThis ain't no f**kin motion picture
A guy or b^*ch-a, my fool get wit'chaAnd hit ya, takin that yack to the neck
So you better run a check
Telling your story to management
Know the real cost of your breach
Your timeYour team’s timeCost of recoveryClient’s lost productivityData loss
Cost of R&DProfit Margin
Know the real cost of countermeasures
Training costs should include time away and travelProcess improvements requires good data, discipline, and expertiseIf you’re buying a new tool, double the cost of deployment and add 50% to annual O&M
Telling your story to management
6/19/1
0 11:4
5
4/8/10
21:52
3/7/10
4:00
1/3/10
7:55
12/16/
09 22
:32
8/29/0
9 1:28
5/23/0
9 16:3
2
5/20/0
9 15:2
9
4/25/0
9 14:0
3
4/9/09
16:28
3/13/0
9 2:28
3/6/09
14:38
1/29/0
9 20:0
9
1/19/0
9 7:48
12/28
/08 4:0
7
11/4/0
8 8:00
9/10/0
8 10:3
3
6/4/08
9:47
4/19/0
8 7:39
4/4/08
7:25
3/22/0
8 16:0
3
3/11/0
8 2:25
2/6/08
16:11
1/10/0
8 20:5
5$ K
$20 K
$40 K
$60 K
$80 K
$100 K
$120 K
$140 K
$160 K
$135 K
$99 K
$39 K
$119 K
$100 K$97 K
$144 K$152 K
$47 K
$17 K
$110 K
$7 K$6 K$6 K
$45 K$56 K
$42 K
$114 K
$142 K
$122 K$119 K
$64 K
$113 K
$7 K
Per-event cost of our large-scale intrusions (Jan ‘12 – Jul ’14)
(# of days of full-scale response) x (daily rate of employee) x (# of employees involved in the response)
What point in the Kill Chain are attacks being stopped?
Does it cost more to respond to events higher in the KC?
Telling your story to management
Recon Deliver Exploit Install C2 AoO0
100
200
300
400
500
600
700
800
900
1000
0.00
2.00
4.00
6.00
8.00
10.00
12.00
14.00
Num
ber
of In
cide
nts
Day
s
What systems are catching attacks from “the APT”
Telling your story to management
IDS29%
Host-Based Scanner12%
AV12%
Proxy Logs7%
User Report6%
Email Scanner6%
Frequency Analysis5%
Monthly Host Checker4%
IP/Domain Hotlist4%
SIEM Correlations4%
Event Logs3%
Other2%
Netflow2%
3rd Party Notification2%
Cloud-based Proxy1%
IPS1%
Commercial Malware Analysis appliance1%
Registry Scanner1%
Email Logs1%
Don’t buy me another chirping box
Telling your story to management
IDS Crmcl Malware Analysis Device
McAfee User Report Email Scanner
3rd Party (Other)
Event Logs Proxy Logs0
1
2
3
4
5
6
7
8
9
0.00
1.00
2.00
3.00
4.00
5.00
6.00
7.00
Detection Tool
# o
f Fa
lse
Posi
tives
Day
s of
Inve
stig
atio
n
5 10 15 20 25 30 35 400
5
10
15
20
25
f(x) = − 0.0957872009672683 x + 12.2788185508248R² = 0.0181875582165211
# of Analysts on IR Team
# o
f Day
s of
Ful
l-sca
le R
espo
nse
More people, more problems
Practically no correlation between having more people and being able to responding faster
Training vs. Tools
Cost of Training an Analyst for a small network – 10K hosts
SANS Course & Certification = ~$5,500Travel & Meals = ~$1,500Time Away from office = ~$1,750Cost of OS IDS appliance(s) & management servers = $20,000Total = $28,750
Cost of a commercial IDS solution = ~$50,000 - $150,000
Cost of a commercial SIEM product = ~$150,000-$200,000
Annual cost of MSSP services = ~$60,000-$120,000
Questions?@DaveTrollman
(since Jul 10, 2014 – 245PM)