Hacker Hotshots – 11/27/2013

How to Build the

Collective Intelligence Framework

And Start to Protect Your Network

John Bambenek

Chief Forensic Examiner, Bambenek Consulting

jcb@bambenekconsulting.com

Problem Lots of people product lots of data, blacklists,

indicators of badness out there.

They all have their own formats and means of distribution.

How to take multiple datasets, normalize them and take action?

“Does anyone know anything about X?”

Solution: Collective Intelligence Framework Developed by REN-ISAC

http://code.google.com/p/collective-intelligence-framework/

Does not generate data, simply takes sources normalizes it and then outputs by given types

Not really a data sharing tool

Up to user to assess confidence in the data

Limited in the types of data it can handle

Data Types

URLs Domains IPs MD5s

Certainly more to threat intel than this, but it’s a start

CIF Architecture

CIF Architecture By default, everything lives in /opt/cif

Configs in /opt/cif/etc/*.cfg (CIF processes all files ending in .cfg)

cif_smrt – queries the feeds cif_feed – generates feeds by assessment cif – command-line client tool cif_crontool – used for querying all feeds

automatically

Requirements to Install For a “real” instance, you would need some disk (250 GB

– 500 GB) and RAM (16 GB) Disk is driven by how long you want to keep old data

Memory is only needing while parsing data CIF can be placed in a virtual infrastructure easily

Can install it on most everything, Debian/Ubuntu easiest mostly because the instructions are available and clear Ubuntu 12 probably best, 13 has some undocumented changes

that need to be made

Some kernel tweaking is needed

CIF Queries Generally an analyst investigating will use

queries to see what is in the database.

cif –q <IP ADDRESS|DOMAIN NAME|MD5>

Will include search records in the response (unless suppressed)

Exact matching only (can’t search for part of a URL… yet)

CIF Queries

CIF also ships a browser plugin which is a little easier for analysis

Use cif_apikeys –l to get your key, find your amazon IP and configure it now

Can query specific items or feeds

CIF feeds cif –q feed/assessment –p output type [-z 0]

(-z 0 will prevent truncating URLs)

cif –q infrastructure/scanTry with a lower confidence level

cif –q url/phishing –c 45

Not all output plugins work for all feeds

Full list at: http://code.google.com/p/collective-intelligence-framework/wiki/API_FeedTypes_v1

CIF output types

bindzone Bind zone configuration

bro bro (network monitor)

csv comma separated value

html Html-ized table

iptables iptables drop rules

json json

pcapfilter pcap filter (i.e. tcpdump)

snort snort alert rules

table ascii table (default)

CIF Output There are dozens of sources (many don’t have

configs in CIF), but you can integrate them all into CIF and/or a feed.

What to do with this now?Snort RulesFeed to web proxy to block/alertSend to border device to blacklist IPsSet up a sinkhole

You can also put your own data into CIF for later research

Questions?

Thanks for tuning in!

See more courses and hacker hotshot sessions at concise-courses.com