Turning data into actionable intelligenceadvanced features in MISP supporting your analysts and tools

Threat Sharing

Team MISP Project

GSMA Edition

about CIRCL

The Computer Incident Response Center Luxembourg (CIRCL)is a government-driven initiative designed to provide asystematic response facility to computer security threatsand incidents. CIRCL is the CERT for the private sector,communes and non-governmental entities in Luxembourgand is operated by securitymadein.lu g.i.e.

1 37

MISP and CIRCL

CIRCL is mandated by the Ministry of Economy and acting asthe Luxembourg National CERT for private sector.CIRCL leads the development of the Open Source MISPthreat intelligence platform which is used by many militaryor intelligence communities, private companies, �nancialsector, National CERTs and LEAs globally.CIRCL runs multiple large MISP communities performingactive daily threat-intelligence sharing.

2 37

The aim of this presentation

To give some insight into what sort of an evolution of ourvarious communities’ have gone through as observed overthe past 8 yearsShow the importance of strong contextualisation......and how that can be leveraged when trying to make ourdata actionable

3 37

Development based on practical user feedback

There are many di�erent types of users of an informationsharing platform like MISP:I Malware reversers willing to share indicators of analysis withrespective colleagues.

I Security analysts searching, validating and using indicatorsin operational security.

I Intelligence analysts gathering information about speci�cadversary groups.

I Law-enforcement relying on indicators to support orbootstrap their DFIR cases.

I Risk analysis teams willing to know about the new threats,likelyhood and occurences.

I Fraud analysts willing to share �nancial indicators to detect�nancial frauds.

4 37

The initial scope of MISP

Extract information during the analysis processStore and correlate these datapointsShare the data with partnersFocus on technical indicators: IP, domain, hostname, hashes,�lename, pattern in �le/memory/tra�cGenerate protective signatures out of the data: snort,suricata, OpenIOC

5 37

Initial workflow

6 37

Why was it so simplistic?

This was both a re�ection of our maturity as a communityI Capabilities for extracting informationI Capabilities for utilising the informationI Lack of willingness to share contextI Lack of co-operation between teams doing technicalanalysis/monitoring and threat-intel

The more growth we saw in maturity, the more we tried tomatch it with our data-model, often against pushback

7 37

The growing need to contextualise data

There were separate factors that made our data-sets lessand less useful for detection/defense in generalI Growth of our communitiesI Distinguish between information of interest and raw dataI False-positive managementI TTPs and aggregate information may be prevalent comparedto raw data (risk assessment)

I Increased data volumes leads to be able to prioritise

8 37

Our initial solution

Allow users to tag any information created in MISPWe wanted to be lax with what we accept in terms of data,but be strict on what we fed to our tools, with strong �lteroptionsWe had some ideas on how to potentially move forward...

9 37

Our initial failures

Try to capture di�erent aspects of contextualisation intonormalised values (threat level, source reliability, etc)I Didn’t scale with needs other than our ownI Incorporating new types of contextualisation would mean themodi�cation of the software

I Getting communities with established naming conventions touse anything but their go-to vocabularies was a pipe-dream

I Heated arguments over numeric conversions

10 37

Human creativity

We tried an alternate approach instead: Free taggingI Result was spectacularly painful, at least 7 di�erent ways tospell tlp:amber

I No canonisation for common terms lead to tagging ultimatelybecoming a highly �awed tool for �ltering within a sharingcommunity

11 37

How we ended up tackling the issue moresuccessfuly

We ended up with a mixed approach, currently implementedby the MISP-taxonomy systemI Taxonomies are vocabularies of known tagsI Tags would be in a triple tag format

namespace:predicate=”value”I Create your own taxonomies, recipients should be able to usedata you tag with them without knowing it at the �rst place

I Avoid any coding, stick to JSONMassive success, approaching 100 taxonomiesOrganisations can solve their own issues without having torely on us

12 37

We were still missing something...

Taxonomy tags often non self-explanatoryExample: universal understanding of tlp:green vs APT 28For the latter, a single string was ill-suitedSo we needed something new in addition to taxonomies -GalaxiesI Community driven knowledge-base libraries used as tagsI Including descriptions, links, synonyms, meta information,etc.

I Goal was to keep it simple and make it reusableI Internally it works the exact same way as taxonomies (stick toJSON)

13 37

Broadening the scope of what sort of contextwe are interested in

Who can receive our data? What can they do with it?Data accuracy, source reliabilityWhy is this data relevant to us?Who do we think is behind it, what tools were used?What sort of motivations are we dealing with? Who are thetargets?How can we block/detect/remediate the attack?What sort of impact are we dealing with?

14 37

Parallel to the contextualisation efforts: Falsepositive handling

Low quality / false positive prone information being sharedLead to alert-fatigueExclude organisation xy out of the community?False positives are often obvious - can be encodedWarninglist system1 aims to do thatLists of well-known indicators which are oftenfalse-positives like RFC1918 networks, ...

1https://github.com/MISP/misp-warninglists15 37

More complex data-structures for a modern age

Atomic attributes were a great starting point, but lacking inmany aspectsMISP objects2 systemI Simple templating approachI Use templating to build more complex structuresI Decouple it from the core, allow users to de�ne their ownstructures

I MISP should understand the data without knowing thetemplates

I Massive caveat: Building blocks have to be MISP attributetypes

I Allow relationships to be built between objects

2https://github.com/MISP/misp-objects16 37

Supporting specific datamodel

17 37

Continuous feedback loop

Data ingested by MISP was in a sense frozen in timeWe had a creation data, but lacked a way to use the outputof our detectionLead to the introduction of the Sighting systemThe community could sight indicators and convey the timeof sightingPotentially powerful tool for IoC lifecycle management,clumsy query implementation default

18 37

Supporting specific datamodel

19 37

Making use of all this context

Most obvious goal: Improve the way we query dataI Uni�ed all export APIsI Incorporate all contextualisation options into API �ltersI Allow for an on-demand way of excluding potential falsepositives

I Allow users to easily build their own export modules feedtheir various tools

20 37

Example query

/attributes/restSearch

{" returnFormat " : " n e t f i l t e r " ," enforceWarningl is t " : 1 ," tags " : {"NOT " : [" t lp : white " ," type : OSINT "

] ,"OR " : ["misp−galaxy : threat−actor =\" Sofacy \ " " ,"misp−galaxy : sector =\" Chemical \""

] ,}

}

21 37

Synchronisation filters

Make decisions on whom to share data with based oncontextI MISP by default decides based on the information creator’sdecision who data gets shared with

I Community hosts should be able to act as a safety net forsharing

Push �lters - what can I push?Pull �lters - what am I interested in?Local tags allow for information �ow control

22 37

The emergence of ATT&CK and similar galaxies

Standardising on high-level TTPs was a solution to a longlist of issuesAdoption was rapid, tools producing ATT&CK data, familiarinterface for usersA much better take on kill-chain phases in generalFeeds into our �ltering and situational awareness needsextremely wellGave rise to other, ATT&CK-like systems tackling otherconcernsI attck4fraud 3 by Francesco Bigarella from INGI Election guidelines 4 by NIS Cooperation Group

3https://www.misp-project.org/galaxy.html#_attck4fraud4https:

//www.misp-project.org/galaxy.html#_election_guidelines23 37

Example query to generate ATT&CK heatmaps

/events/restSearch

{" returnFormat " : " attack " ," tags " : [

"misp−galaxy : sector =\" Chemical \""] ," timestamp " : "365d"

}

24 37

A sample result for the above query

25 37

Monitor trends outside of MISP (example:dashboard)

26 37

Decaying of indicators

We were still missing a way to use all of these systems incombination to decay indicatorsMove the decision making from complex �lter options tocomplex decay modelsDecay models would take into account various taxonomies,sightings, the type of each indicator Sightings and CreationdateThe �rst iteration of what we have in MISP now took:I 2 years of researchI 3 published research papersI A lot of prototyping

27 37

Scoring Indicators: Our solution

score(Attribute) = base_score(Attribute, Model) • decay(Model, time)

Where,

score ∈ [0, 100]base_score ∈ [0, 100]decay is a function de�ned by model’s parameterscontrolling decay speedAttribute Contains Attribute’s values and metadata(Taxonomies, Galaxies, ...)

Model Contains the Model’s con�guration

28 37

Implementation in MISP: Event/view

Decay score toggle buttonI Shows Score for each Models associated to the Attribute type

29 37

Implementation in MISP: API result

/attributes/restSearch" A t t r ibu te " : [{" category " : "Network a c t i v i t y " ," type " : " ip−src " ," to_ids " : true ," timestamp " : "1565703507" ,[ . . . ]" value " : " 8 . 8 . 8 . 8 " ," decay_score " : [{" score " : 54 .475223849544456 ," decayed " : fa lse ," DecayingModel " : {" id " : "85 " ,"name " : "NIDS Simple Decaying Model "

}}

] ,[ . . . ]

30 37

Implementation in MISP: Index

View, update, add, create, delete, enable, export, import

31 37

Implementation in MISP: Fine tuning tool

Create, modify, visualise, perform mapping

32 37

Implementation in MISP: base_score tool

Adjust Taxonomies relative weights 33 37

Implementation in MISP: simulation tool

Simulate Attributes with di�erent Models

34 37

Implementation in MISP: API query body

/attributes/restSearch

{" includeDecayScore " : 1 ," includeFullModel " : 0 ," excludeDecayed " : 0 ," decayingModel " : [ 8 5 ] ,"modelOverrides " : {

" threshold " : 30}" score " : 30 ,

}

35 37

To sum it all up...

Massive rise in user capabilitiesGrowing need for truly actionable threat intelLessons learned:I Context is king - Enables better decision makingI Intelligence and situational awareness are naturalby-products of context

I Don’t lock users into your work�ows, build tools that enabletheirs

36 37

Get in touch if you have any questions

Contact usI https://twitter.com/mokaddem_samiI https://twitter.com/iglocska

Contact CIRCLI info@circl.luI https://twitter.com/circl_luI https://www.circl.lu/

Contact MISPProjectI https://github.com/MISPI https://gitter.im/MISP/MISPI https://twitter.com/MISPProject

37 / 37