Security events correlation with

Nikolay Klendar bsploit gmail.com

ComplexEventProcessing(correla5on)*-iseventprocessingthatcombinesdatafrommul6plesourcestoinfereventsorpa7ernsthatsuggestmorecomplicatedcircumstances.

INTRO

*Wikipedia

LibraryusedfordevelopmentJava,.NET

ProcesseseventSTREAMSofpredefinedtypes.Esperdoesnotparseevents!

Processingrules(correla6onrules)aredefinedwithEventProcessingLanguage(EPL)similartoSQL

Network scan detection

Typeevent:5mestamp:stringtype:stringsrc_ip:stringdst_ip:stringsrc_port:intdst_port:intbytes_sent:intbytes_recieved:intlogin:string

Allowedmonitoringsystmes

Annota5on

Alldst_ipwithin30sec

@Name('Scan')SELECTsrc_ip,window(dst_ip)FROMevent(type='firewall'

ANDsrc_ipNOTIN('10.0.0.1','10.0.0.2')).win:5me(30sec)/*sliding5mewindow*/GROUPBYsrc_ipHAVINGcount(dis5nctdst_ip)>50outputfirstevery1hour/*1eventperhour*/

Worm spreading detection

INSERTINTOscanningSELECTsrc_ip,window(dst_ip)targetsFROMevent().win:5me(10min).std:unique(dst_ip)GROUPBYsrc_ipHAVINGcount(dis5nctdst_ip)>50;

{src_ip='10.0.0.1',targets=['192.168.0.1','192.168.0.2',…,'192.168.0.254']}{src_ip='192.168.0.2',targets=['192.167.0.1','192.167.0.2',…,'192.167.0.254']}

@Name('warm_spreading')SELECTa.src_ip,b.src_ip,b.targetsFROMpaaern[everya=scanning->b=scanning(

b.src_ip!=a.src_ipANDArrays.asList(a.targets).contains(b.src_ip)

)WHERE5mer:within(1min)];

{a.src_ip='10.0.0.1',b.src_ip='192.168.0.2',b.targets=['192.167.0.2',…,'192.167.0.2','192.167.0.2']}

Moneylaunderingdetec5on

@Name('obnal')SELECTa.transac5on,a.clien5d,a.amountincome,c.sumOf(i=>i.amount)+b.amounttotalFROMPATTERN[EVERYa=event(transac5onlike'card_income')-> b=event(b.clien5d=a.clien5dANDtransac5on='card_outcome')WHERE5mer:within(3hour)->([3:]c=event(c.clien5d=a.clien5dANDtransac5on='card_outcome')un5l5mer:interval(20min))]

Totalmoneytransferredtocard

Totaloutcome

Join & enrichment

SELECTS.src_ip,S.targets,L.login,L.last_seenFROMscanning.std:lastevent()asSLEFTOUTERJOINLoginsIPLonL.ip=S.src_ipGROUPBYS.src_ipoutputfirstevery1hour;

CREATEWINDOWLoginsIP.std:unique(ip)as(ipstring,loginstring,last_seenstring);INSRTINTOLoginsIPSELECTsrc_ipasip,login.toLowerCase()aslogin,5mestampaslast_seenFROMEvent(type='windows'ANDeven5d='4624'ANDsrc_ipISNOTNULLANDloginISNOTNULLANDlogin!='ANONYMOUSLOGON'ANDloginNOTLIKE'%$');

{S.src_ip='10.0.0.1',L.login='ivanov',L.last_seen='17.11.201512:00:00'S.targets=['192.167.0.2',…,'192.167.0.2','192.167.0.2']}

Integration with external sources

SELECTsrc_ipfromevent(type='firewall')asfw,SQL:mysql['selecttornode_ipfromtor_nodes']astorwherefw.src_ip=tor.tornode_ip

Users profiling

Building user profile createwindowloginProfileASN.win:keepall()(loginstring,paramstring,valuestring,v_countlong)

createwindowloginProfileTotal.win:keepall()(loginstring,paramstring,totallong)

ONEVENT()eMERGEloginProfileASNpwherep.login=e.loginandp.value=(e.geoip('number')).toString()whennotmatchedtheninsertselectlogin,'ASN'param,geoip('number')value,1Lv_countwhenmatchedthenupdatesetp.v_count=p.v_count+1

ONEVENT()eMERGEloginProfileTotalpwherep.login=e.loginwhennotmatchedtheninsertselectlogin,'ASN'param,1Ltotalwhenmatchedthenupdatesetp.total=p.total+1

Deviation from profile

SELECTe.login,e.geoip('asn')asn,e.geoip('number'),e.src_ip,v.v_countcount,t.total,cast((100-100*v.v_count/t.total),int)scoreFROMevent().std:lastevent()e,loginProfileASNv,loginProfileTotaltwherev.login=e.loginandv.value=(e.geoip('number')).toString()andt.login=e.loginand(100-100*v.v_count/t.total)>97

CorReactive and integration with ELK Logstashconfigoutput{redis{host=>"127.0.0.1"db=>0data_type=>"list"batch=>truebatch_events=>500key=>"events”codec=>json }}

CorReac5veconfigCollectevents"inputs":[{"type":"redis","config":{"host":"localhost","port":6379,"db":0,"queue":"events","batch_count":500,"reconnect_6meout":60}}]

CorReac5veconfigReturnalerts"outputs":[{"type":"redis","id":1,"config":{"host":"localhost","queue":"alerts","port":6379,"db":0,"reconnect_6meout":60,"batch_count":1}}]

CorReactive configuration steps

1.   conf/types:Extendbaseeventtype“event”,addnewfields

2.   conf/modules:AddnewEPLmodules(correla6onrules)Ifonemoduledependsonanotherusespecialdirec6ve:usesdependent_module;h7p://goo.gl/9pvlIj

3. Configureinputsandoutputs

CorReactive special annotations

Alertgenera6ontooutputchannel@Alert(name='newalert',outID=1)

Savedatafromnamedwindowtodiskevery5minutes.Saveddataisautoma6callyrestoredtonamedwindowduringloadingstage@Persist

Namedwindowdatareloadingevery5minutesfromcsvfilelocatedinvar/winload@Load(file="data.csv",format="csv",delim=";")

Dynamicallyalertenrichmentwithdatafromexternalcommandoutputorondemandquery.Enrichmentofenrichmentissupported.@Enrich(dst="eLogin",type="window",param="selectsrc_ipfromloginipwhere

login='%{login}'")@Enrich(dst="nsresult",type="cmd",param="nslookup%{eLogin}")

Alert example in Kibana

REST API

SendeventinJSONformatPOST/api/eventsViewallregisteredmodulesGET/api/modules/registered

ViewallregisteredEsperstatementsorqueriesGETapi/modules/statementsReloaddatainnamedwindowPOST/api/window/reload/{moduleName}/{winName}

DeployallmodulesPOSTapi/modules/deployModuledele6onDELETE/api/modulesModulesyntaxvalida6onPOSTapi/modules/validateDoondemandqueryPOST/api/query

Links

Esperdocsh7p://www.espertech.com/esper/documenta6on.phpSolu6onpa7ernswithdescrip6onh7p://www.espertech.com/esper/solu6on_pa7erns.phpEPLeditoranddebuggerh7p://esper-epl-tryout.appspot.com/epltryout/mainform.htmlCorReac6veengine(specialforZeroNights2015)h7p://correac6ve.sourceforge.net/

Thank you!

Questions?