the big book of network flows for security4 cisco’s netflow is just one of the many network flow...

24
THE BIG BOOK OF NETWORK FLOWS FOR SECURITY

Upload: others

Post on 26-May-2020

5 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: THE BIG BOOK OF NETWORK FLOWS FOR SECURITY4 Cisco’s NetFlow is just one of the many network flow protocols out there. Others include: • J-Flow • Cflow • IPFIX • sFlow® Flow

THE BIG BOOK OF NETWORK FLOWS FOR SECURITY

Page 2: THE BIG BOOK OF NETWORK FLOWS FOR SECURITY4 Cisco’s NetFlow is just one of the many network flow protocols out there. Others include: • J-Flow • Cflow • IPFIX • sFlow® Flow

2

3 | Introduction to Network Flows

5 | The Different Kinds of Network Flows

8 | Flow Exporters

9 | Flow Collectors

10 | To Sample or Not To Sample?

12 | Network Flows for Visibility

15 | Network Flows for Security

18 | Which Network Flow is Best for DDoS

Detection and Mitigation?

21 | The Scalability of Network Flows

22 |TipsonConfiguringFlowExport

23 | About FlowTraq

© 2016 FlowTraq. All Rights Reserved.

TABLE OF CONTENTS

Page 3: THE BIG BOOK OF NETWORK FLOWS FOR SECURITY4 Cisco’s NetFlow is just one of the many network flow protocols out there. Others include: • J-Flow • Cflow • IPFIX • sFlow® Flow

3

Networkflowsrepresenttheconversationsthatmakeyour

businesswork:emails,Webrequests,VoIPcalls,filetransfers,and

allthelow-levelback-and-forththatmakeanetworkanetwork.

Amongtheseconversationsarealsoattacks:spam,scanning,

malware,dataexfiltrations,andotherpotentialthreats.

Networkflowsprovideup-to-the-minuteinformationaboutthe

communications taking place on the network, including who’s

sending how much data to whom, as well as how and when: IP

addresses,portandprotocol,exportingdevice,timestamps,plus

VLAN,TCPflags,etc.Thisdataiswidelyavailablefromdeviceslike

routers,switches,firewalls,loadbalancers,hypervisors,andeven

assoftwaretoinstallonindividualhosts.Withdatastreamingin

frommultiplesources,acentrallocationcangetanexcellentview

ofthenetwork,includingcross-borderandpurelyinternaltraffic.

Cisco,theinventorsofNetFlow,describeitasaphonebillfor

your network1—alistingofalltheconversationsthattakeplace

onyournetwork,whethertheylasthoursormilliseconds.Unlike

anordinaryphonebillwithhundredsofconversations,there

arethousandsormillionsof“conversations”onthenetworkat

anyonetime.Theseconversationsaretheebbandflowofdata

and control of a modern computer network, embodying the

businessprocessesthatyournetworksupports.Thesizeofthe

conversationdoesn’tmatter—asingle-kilobytecommunication

can be as important to the operation of your network as a multi-

gigabytedownload.

INTRODUCTION TO NETWORK FLOWS

1 “CiscoIOSNetFlowandSecurity,”InternetTechnologiesDivision, February 2005

Page 4: THE BIG BOOK OF NETWORK FLOWS FOR SECURITY4 Cisco’s NetFlow is just one of the many network flow protocols out there. Others include: • J-Flow • Cflow • IPFIX • sFlow® Flow

4

Cisco’sNetFlowisjustoneofthemanynetworkflowprotocolsout

there.Othersinclude:

• J-Flow

• Cflow

• IPFIX

• sFlow®

Flowdataprovidesgranularinsightintoyournetworkthatcanhelp

youdiagnoseapotentialproblem.Withtherighttools,anetwork

administrator can determine the source and destination of network

traffic,prominentpeeringrelationships,andcommonbottlenecks.

Specifically,flowrecordscontaininformationabout:

• Source IP address:whoisoriginatingthetraffic

• Destination IP address:whoisreceivingthetraffic

• Ports:theapplicationutilizingthetraffic

• Class of service:thepriorityofthetraffic

• Device interface:howthetrafficisbeingusedbythe

networkdevice

• Tallied packets and bytes:theamountoftraffic

• And more: including packet timestamps

Page 5: THE BIG BOOK OF NETWORK FLOWS FOR SECURITY4 Cisco’s NetFlow is just one of the many network flow protocols out there. Others include: • J-Flow • Cflow • IPFIX • sFlow® Flow

5

NETFLOW SUITE OF PROTOCOLS

FortheNetFlowsuiteofprotocols—whichincludesIPFIX,Cflow,

andJ-Flow—wemostoftenseeversion5(supportedbythe

majorityofdevices),somecombinedv5/v7(theCatalysts),and

someversion9onthenewerdevices.Don’tbefooledbythe

ASAseriesoffirewalls;theydonotactuallysupportversion9flow

exporting.Instead,theseCiscodevicesuseNetFlow9tofirewall

events,similartologlines:norealtrafficrecordsinthere!NetFlow

v5usesastaticpacketformat(andisinthiswayverysimilartov7),

definingIPv4IPs,protocols,ports,andmillisecondprecisionon

flowstartandendtimes.Version9usesadynamicformat,parsed

basedonatemplatewhichissentaroundfirst.Thesetemplates

areflexibleandallowforexpansionoftheprotocolinthefuture.

Incidentally,IPFIXisalsobasedonNetFlowandisversionedas

NetFlow10.

J-FlowandCflowarethesameasCiscoNetFlowv5.OnlyNetFlow

v9andIPFIXsupportIPv6.

NetFlowdefinesa“flow”asaunidirectionalseriesofpacketsfrom

IPAtoIPB,usingsomeprotocol(TCP/UDP/ICMP/…).Whenthe

packetsuseeitherTCPorUDP,thentheflowisfurtherspecifiedby

apairofports;forinstance,10.20.30.40:53823->50.60.70.80:443

TCP.Often,sincemostcommunicationsrequirebothsidesto

transmitpackets,onewillseeNetFlowreporttwoflowsassociated

witheverycommunication,accountingforthepacketsandbytes

THE DIFFERENT KINDS OF NETWORK FLOWS

Page 6: THE BIG BOOK OF NETWORK FLOWS FOR SECURITY4 Cisco’s NetFlow is just one of the many network flow protocols out there. Others include: • J-Flow • Cflow • IPFIX • sFlow® Flow

6

thatwentineitherdirection.Aproperflowcollectorandanalyzer

will correlate these with each other for you, so you can see a

reportofafullconversation.MostversionsofNetFlowalsosupport

asamplingmodewhereonlyoneineveryNpacketsisused

toupdatetheflowcounters.Thisisnotveryusefulforforensic

analysisofyournetworktraffic,buthelpskeepstheCPUloadon

yourrouterorswitchdown(see“ToSampleorNotToSample?”

below).Iffull-fidelityNetFlowisrequired,considerusingaSPAN,

TAP or mirrored port, and generate NetFlow with a software tool

or dedicated appliance without incurring the additional load on the

routerorswitch.

The NetFlow suite of protocols is a powerful source of security and

networkdebugginginformation.Sinceeachandeverynetwork

communication can be logged with millisecond precision, you

canquicklydeterminewhocommunicatedwithwhomandwhen.

Flow information is also much more tenable than raw packet data,

allowingforamuchquickerfirstlookatthenetwork.Inother

words,youcanuseflowdataasaspringboardtodetermineif

furtherpacketinspectionisnecessary.

SFLOW

ThesFlowprotocolisacompletelydifferentanimal.Easily

configurablethroughSNMP,itsprimaryobjectiveistobea

statisticalnetworkmonitoringtool.Lotsofdifferentperformance

counters can be monitored through the sFlow protocol, and the

biggestbenefitofsFlowcomesfromitsinfinitescalabilityinlarge

networksunderheavyloads;however,thisinnovativestatistical

approachcomesataslightdisadvantageinaccuracy,granularity,

andtimingprecision.Tounderstandthis,wehavetotakeacloser

lookathowsFlowmeasuresnetworktraffic.

Page 7: THE BIG BOOK OF NETWORK FLOWS FOR SECURITY4 Cisco’s NetFlow is just one of the many network flow protocols out there. Others include: • J-Flow • Cflow • IPFIX • sFlow® Flow

7

UnlikeNetFlow,thesFlowprotocolsampleseveryNthpacket

fromthetrafficstream,whereNcanbeone-in-512,one-in-1024,

etc.Thismeansthatsomecommunicationsmayslipbyentirely

undetected, and the sFlow collector software will not know about

them.Largercommunications,suchasbigdownloadsandonline

videocontent,willstandamuchbiggerchanceofbeingreported,

astherearemanypacketsinvolved.Theseconddrawbackis

thelackofaccuratetimestampingofthepacketdata.Sampled

packetsgetforwardedastheyarepickedupfromthedatastream;

however,theyarenottimestamped.Therefore,asmallamountof

uncertaintyabouttheexacttimeofpacketcaptureisintroduced.

Although these tradeoffs render sFlow to be not particularly well

suitedtonetworkforensicinvestigations(thereissomestatistical

uncertainty as to when a communication began, how many

packetsweretransmittedeachway,theirsize,andwhenitended),

theyarenecessarytoallowthevirtuallyunlimitedscalabilitythat

sFlowoffers.Ifthenetworkgetsbusy,itcanfallbacktoaslower

samplingrate,andkeeploadontheexportingdeviceandsFlow

collectordownsignificantly.

WHICH FLOW SOLUTION IS RIGHT FOR YOU?

The answer to this question depends on the intended purpose

oftheimplementation.AttheISPorlargeenterpriselevel,the

hardwarecostassociatedwithtrackingeverycommunication

throughNetFlowissubstantial,andcanonlybejustifiedifthe

NetFlowdataisusedforsecurityandnetworkforensicanalysis.

Ifthegoalistosimplygetaroughoverviewofusage(“Who’s

hoggingmybandwidth?”),thesFlowprotocol,orsampledNetFlow

willsuffice—it’smuchmoremanageableandlesscostly.At

smaller sites, the decision will usually be dictated by the switching

androutinggearinthecurrentnetworkcloset.Usewhatyouhave!

Page 8: THE BIG BOOK OF NETWORK FLOWS FOR SECURITY4 Cisco’s NetFlow is just one of the many network flow protocols out there. Others include: • J-Flow • Cflow • IPFIX • sFlow® Flow

8

Aflowexporterisasoftwareorhardwareenginethatkeepstrack

ofallthecurrentsessionsthatit“sees.”Inhardware,thismight

meanallthepacketsthataswitchswitches;insoftware,it’swhat

canbeseenonaninterfaceorsetofinterfaces.Itdoesthisby

maintainingatableofcurrentsessions,calledaconntrack(as

inconnectiontracking)table.Typically,thistableisscannedfor

updatesatregularintervals,whicharesubsequentlyaddedtoflow

packetsthatareexportedtoaflowcollector.

Onverybusynetworks,sometimesanenormousnumberof

sessionsreceiveupdatesduringthescaninterval,thusresulting

inasurgeofflowpacketstoexport.Thissuddenburstofoutput

canoverwhelmtheUDPinputbuffersoftheoperatingsystemon

thecollector,especiallyifthecollectormachineisheavilyloaded

orunderpowered.Thiscanleadtodroppedflowpacketsand

inaccurate,incompletedata.Tocombatthisproblem,youwantyour

flowexportertospreadalltheexportpacketsovertime,ratherthan

sendtheminbursts.Thissmoothsoutthesurgesconsiderably,

resultinginasteadystreamofflowpackets,evenonverybusy

networks,andminimizesthechancesofoverloadingacollector.

FLOW EXPORTERS

Page 9: THE BIG BOOK OF NETWORK FLOWS FOR SECURITY4 Cisco’s NetFlow is just one of the many network flow protocols out there. Others include: • J-Flow • Cflow • IPFIX • sFlow® Flow

9

Acollectorisaserverwithsoftwarethatcanacceptandinterpret

flowexports.Exporterssendtheirflowsummariestocollectorsfor

storageandanalysis.Mostcollectorssummarizeandaggregate

theflowsbeforestorage,discardingtherecords.Althoughcoarse,

this approach is fastest, but the cost is the loss of forensic

accuracy.Somecollectorsstoreallflowrecords,allowingfullrecall,

andprecisefiltering.Thesefull-fidelityanalyzersaremorepowerful.

Forsimplyanalyzingtopnetworkusersandtheirtopcontent,

youmayonlyneedaflowaggregator.Butbecausenetworkflows

are a summary, they can be stored compactly — a gigabyte of

flowdatacandescribehundredsofthousandsofgigabytesof

actualtraffic.Thisefficiencymakesitsuitableforfull-fidelityrecall,

meaningeverysinglerecordcanbestoredforlateranalysis,down

tothesmallestICMPping.Withafull-fidelitynetworkflowhistory,

it becomes possible to perform intricate analysis of one’s historical

record,downtosmalltransactionsthatoccurredmonthsago.This

enablesananalysttoperformdifficulttasksliketrackinganintruder

through multiple hops of SSH sessions, or separating potential

DNStunnelsfromordinaryrequests.Italsomeansthatanalysts

don’thavetobepsychic—theydon’tneedtoknowtodaywhat

they’llneedtosearchfortomorrow.

Morethanthatlevelofintricateanalysis,afastfull-fidelitysystem

givesanalystsanunparalleledfeelforthenetwork.Simplyclicking

aroundandexploringforahalf-hourcangiveafarbetterideaof

whattoexpectinthetrafficthandaysinalecturehall.

FLOW COLLECTORSFLOW COLLECTORS

Page 10: THE BIG BOOK OF NETWORK FLOWS FOR SECURITY4 Cisco’s NetFlow is just one of the many network flow protocols out there. Others include: • J-Flow • Cflow • IPFIX • sFlow® Flow

10

Onanygivenday,atypicalnetworkedhostwillsendabout30MB

andreceiveabout200MB.About300,000packetsareswitched.

Duringpeaktimes,theaverageworkstationinitiatestwotofour

networkUDPorTCPsessionspersecond,andeachsession

averages34KBinsize,roughly100packets.What’smore,these

sessionsarenegative-exponentiallydistributedwithregardto

packetcount.Whatdoesthatmean?Itmeanstherearealot

moreveryshortsessionsofonlyacoupleofpacketsthanlengthy

sessionswithlotsofpackets.

Whenroutersusesamplingfornetworkflowgeneration,an

interestingthinghappens.Thesamplingisdoneonapacket-

countlevel,soa1:512samplingratewillgrabroughlyevery512th

packettoupdatetheflowstatetables.

ThisisgreatforreducingCPUload.Butitisnotsogreatat

reducingflowupdaterate.Here’swhy:Withanaveragesession

sizeofroughly100packets,eachsampledpacketisverylikely

tobepartofaflowthatisnotyetinthestatetable.Thismeans

anentryiscreated,whichwillleadtoaflowupdatebeingsent.

Comparethisto1:1unsampledflowgeneration,wheremostof

thepacketswillgotowardupdatingexistingentriesintheflow

statetable.Flowstatetablesaretypicallyexportedwhenaflow

is60secondsold,orthetableisfull,andtheoldonesneedto

bepurged.

TO SAMPLE OR NOT TO SAMPLE?

Page 11: THE BIG BOOK OF NETWORK FLOWS FOR SECURITY4 Cisco’s NetFlow is just one of the many network flow protocols out there. Others include: • J-Flow • Cflow • IPFIX • sFlow® Flow

11

Leavingtheexactmathoutforclarity,ifunsampledflowgeneration

resultsinaflowrateofX,thena1:512samplingresultsina

roughly1/5thoftheflowtrafficbeinggenerated.Not1/512th.

Thisistheintuitiveanswer,andthetrueresultsofsamplingdepend

muchontheprecisemixoftrafficpresentonthenetwork.Also,

somerouterswilluseadaptiveflowsamplingratestokeeptheir

flowexportratesconstant.Thismeansthatatbusiertimes,the

granularityofthedatabecomeslessandless.Althoughthisis

niceforCPUtimeconsiderationsontherouter’send,itdoesnot

helpmuchthattheroughestdataiscollectedduringtheheaviest

attack!Samplingissimplynotthecorrectapproachtoreducecost

orgainbestvisibility.Youshoulddesignfor1:1unsampledflow,

becauseitbuildsinasafetymarginduringthebiggestattacks.

Page 12: THE BIG BOOK OF NETWORK FLOWS FOR SECURITY4 Cisco’s NetFlow is just one of the many network flow protocols out there. Others include: • J-Flow • Cflow • IPFIX • sFlow® Flow

12

Whencollectedinacentrallocationandmadeavailabletoview

andsearch,networkflowsprovidevisibilityintowhat’sgoing

overthenetwork,includinglarge-scaleeventslikeDDoSattacks

andsaturatedlinks,andsmall-scaleeventslikeSSHloginsand

databaseconnections.Thisallowsananalysttomonitorfor

attacks,policycompliance,anddatausageforbilling.Whetherit’s

abrute-forceattackoron-the-slyNetflixwatching,ifitcrossesthe

network,it’sinthenetworkflows.

Amoderncomputernetworkencompassesawidevarietyof

Internet-capabledevices,notallofthemphysical,joiningand

leavingyournetworkinalmostnotimeatall.Thenetwork

infrastructure itself can change drastically with just a few

keystrokes,withoutmovingevenasinglewireofyourphysical

infrastructure.Aproperlydeployednetworkflowsolutiongivesyou

exceptionalvisibilityintoyournetwork,providingthebestavailable

balancebetweenscopeanddepth.Networkflowsbythemselves

giveyouanexcellentview,butcombinedwiththeproperanalytical

tools,itgivesyouunparalleledcontroloveryournetwork.

VIRTUALIZED NETWORKS

Theabilitytorapidlydeployvirtualnetworksandvirtualhosts

has been a game-changer for many companies, allowing

unprecedentedflexibility.Networkflowsarefastenoughtokeep

pace—theminuteyourvirtualnetworkgoeslive,sodoesyour

NETWORK FLOWS FOR VISIBILITY

Page 13: THE BIG BOOK OF NETWORK FLOWS FOR SECURITY4 Cisco’s NetFlow is just one of the many network flow protocols out there. Others include: • J-Flow • Cflow • IPFIX • sFlow® Flow

13

flowexportandyoucanseetheflowandbalanceoftraffic

betweenVLANsataglance.

MOBILE DEVICES

Mobiledevicesareparticularlydifficulttomonitorbecausethey

moverapidlyfromnetworktonetwork,arenoteasytoinstrument,

anddonothaveeasilyaccessiblelogfiles.Networkflowsprovide

aconvenientandrobustmeansofmonitoringthesedeviceswhen

theyassociatewithyourwirelessnetworks.

REAL-TIME NETWORK MANAGEMENT

The more you know about your network, the better prepared you’ll

be for decisions you need to make:

• IsyourWebservicesufferingadenialofserviceattack?

• Areallofyourbackupsmadeontime,everytime?

• That foreign IP address that is currently trying a

brute-force attack against a system in your Chicago

network — has it contacted any of your other networks

today? Last week? Last year? Are they trying blindly or

didtheyperformreconnaissancefirst?

• CanyoureduceloadonyourBostonserversbymoving

functionality to San Francisco or do they experience

peaks at the same time?

BANDWIDTH PLANNING

Whether your site is getting more popular, your team is expanding,

yourpartnersstartpreferringvideoconferencestovoicecalls,or

thesizeoftheaverageWebpageisincreasing,theonlycertaintyis

thatyouwillneedmorebandwidth.Butwhere?Long-termnetwork

Page 14: THE BIG BOOK OF NETWORK FLOWS FOR SECURITY4 Cisco’s NetFlow is just one of the many network flow protocols out there. Others include: • J-Flow • Cflow • IPFIX • sFlow® Flow

14

flowtrendscantellyouwhereresourcesaremostlikelytobe

neededmosturgently.Theycantellyouwhichtrafficisdominating

your network and, at a glance, show you the rates at which it’s

increasing.Thismakesiteasytospotpotentialbottlenecksand

strategicallyplanforexpansion.

REGULATORY COMPLIANCE

Differentorganizationshavedifferentrecord-keepingrequirements.

For example, if you handle medical records, you are required to

showHIPAAcompliance;ifyouhandlecreditcarddetails,you

arerequiredtotracktheflowofdatatoandfromthesystemthat

processesthisvitalinformation,showingallaccess,eventhe

smallest.Ifyoumonetizeyournetworkbyhostingservices,you

needtorackbandwidthuseaccuratelytodetermineusagebilling.

Relyingon95th-percentilebillingcanberisky;itcanbegamed

andyoucanmissspikesintrafficthatdegradeperformanceto

othercustomers.

Page 15: THE BIG BOOK OF NETWORK FLOWS FOR SECURITY4 Cisco’s NetFlow is just one of the many network flow protocols out there. Others include: • J-Flow • Cflow • IPFIX • sFlow® Flow

15

Flow is a compact format, meaning it can be processed and

analyzedmuchmorequicklythanafullpacketcaptureformat

(fullcap).Whenanalyzingnetworkpatternsforillicitbehaviors,

botnets,ordataexfiltrations,thespeedofanalysisiskey.Ifittakes

hours to detect a potential data theft, and then a security analyst

takeshoursmoretoinvestigate,howmanyproverbialhorseswill

haveleftthebarnbeforethebarndoorisshut?Flowanalysis

deliversthepowerandspeedtoactquickly.Atthesametime,

point-to-point communications are increasingly being encrypted,

whichdramaticallyandrapidlydecreasesthevalueoffullcap.

Byanalyzingflowdata,networkandsecurityoperationspersonnel

canflagunfamiliarIPaddresses(orIPaddressesknowntohost

malware),analyzedistributeddenialofservice(DDoS)attacks,

identifypotentialwormsandbotnets,trackunusualdatatraffic

patterns,findnon-compliantusers,andgiveyouadetailedaudit

trailofallnetworkactivity.Flowanalysiscanalsohelpyoupinpoint

unwanteddataexfiltrations,identifycausesofslowdowns,and

spotwhereattacksorinformationleaksarecomingfrom.

BOTNET PROTECTION

Onceyoulearnyourorganization’stypicalnetworkpatterns,you

canuseyourflowdatatohelpdetectnetworkanomaliesthatoften

indicatebotnets,rogueservers,unauthorizedclients,orother

networkthreats.

NETWORK FLOWS FOR SECURITY

Page 16: THE BIG BOOK OF NETWORK FLOWS FOR SECURITY4 Cisco’s NetFlow is just one of the many network flow protocols out there. Others include: • J-Flow • Cflow • IPFIX • sFlow® Flow

16

DDOS AND BRUTE FORCE ATTACK DEFENSE

AsignificanttrafficsurgecanindicateapotentialDDoSorbrute

forceattack,twoofthemostcommonnetworkthreatsaround.

DDoS attacks can make your websites, email, or other applications

unavailabletoyourcustomersoremployees,whileasuccessful

bruteforceattackallowsaperpetrator,virus,orwormtopenetrate

yournetworkwithcompromisedcredentials.Byregularlyanalyzing

yourflowtrafficyoucanrecognizethesetypesofmaliciousthreats

in real time so you can immediately react to them and select the

appropriatedefense.

WORMS, SCANS, AND NETWORK RECONNAISSANCE PREVENTION

Worms propagate through your network by rapidly looking

forhostswithcommonvulnerabilitiesandexploitingthose

weaknessestospreadthroughoutyournetwork.Byusingflow

data,thisscanningbehaviorcanbeeasilyidentifiedwhenseveral

internalsystemsshowthesamebadpatterninrapidsuccession.

Thisisaverycommonreconnaissancetechniqueusedby

attackers,andcanserveasanearlywarningforothermore

maliciousattackstocome.Flowdatacanhelpyoucatchand

controltheseincidentsbeforetheybecomearealproblem.

DATA EXFILTRATION RECOGNITION

Youmayalreadyspendtimeandresourcessearchingforviruses

oncomputers,blockingspam,andtrackingdownabuse.

Butit’snotonlywormsandvirusesthatmaybeexfiltrating

yourmostsensitivedata,itcouldbeanyonewithinyourown

walls.Bymonitoringflowdatatodetectanomalies,youcan

Page 17: THE BIG BOOK OF NETWORK FLOWS FOR SECURITY4 Cisco’s NetFlow is just one of the many network flow protocols out there. Others include: • J-Flow • Cflow • IPFIX • sFlow® Flow

17

immediatelyrecognizeundesireduploadsanddatabreaches

fromyournetwork.Thisallowsyoutostopaleakbeforeyouface

millions of dollars of cyber theft damages, lawsuits and public

embarrassment, or before your CEO, CSO or CIO lose their jobs

duetofaileddatasecuritypractices.

DATA BREACH DEFENSE

To spot potential data leaks or information security breaches,

youcananalyzenetworkflowrecordstoquicklyrecognizehosts

initiatingaconnection,receivingdataoutsideofnormalthresholds,

orexhibitingunexpectednetworkbehaviorpatterns.Withthe

propertools,flowdataanalysiscanhelpyoukeepsensitive

informationsafefromoutsideintrudersandunauthorizedinsiders.

Page 18: THE BIG BOOK OF NETWORK FLOWS FOR SECURITY4 Cisco’s NetFlow is just one of the many network flow protocols out there. Others include: • J-Flow • Cflow • IPFIX • sFlow® Flow

18

Successful DDoS triage and mitigation depend on two things:

speed of detection and accuracyofdetection.Usersconsidering

aDDoSsolutionoftenaskifitisbesttouseNetFloworsFlow.

To understand what is best, it’s important to understand the

differencesbetweenthetwotypesofflowdata.

NetFlow(andtheverycloselyrelatedcFlow,JFlow,andIPFIX)isa

summaryrecordformat,wherearouterorotherexportingdevice

tabulatesthestatisticsoneachflowofpacketsflyingby.Aflowis

typicallydefinedasthe5-tupleofsendingIPandport,receivingIP

andport,andtheprotocol.Eachpacketistabulatedandadded

totheappropriaterowinthetable:1morepacket,Xmorebytes.

Stateiskeptoneveryflowthattheexporterobserves,andwhen

aflowis60secondsold,therecordissenttothecollector,which

hasthetaskofdetectingthedenialofserviceattack.

sFlowtakesaslightlydifferentapproach,keepingnostateatall.

InsteadsFlowrandomlygrabsoneineveryNpacketsflyingby

andimmediatelysendsittothecollector.Althoughthisapproach

may appear somewhat less accurate than the NetFlow tabulation,

itisactuallyverygoodforfastDDoSdetection.Asthefloodor

amplificationattackstartstorampup,therateofpacketsflowing

bytheexporterstartstoincreaseveryrapidly.Thismeansthat

thenumberofpacketsamplesgoingtothecollector(whichis

responsiblefortheDDoSdetection)startstoincreaseimmediately.

WHICH NETWORK FLOW IS BEST FOR DDOS DETECTION AND MITIGATION?

Page 19: THE BIG BOOK OF NETWORK FLOWS FOR SECURITY4 Cisco’s NetFlow is just one of the many network flow protocols out there. Others include: • J-Flow • Cflow • IPFIX • sFlow® Flow

19

Although the NetFlow approach ensures that no packet is missed,

which is great for accurate network forensics, the nature of the

exporttimermayresultinmuchslowerdetections.IftheNetFlow

exporterhassufficientmemorytokeepstateonalltheattack

traffic,itmaytakeupto60secondsbeforethedetectingcollector

seesanyevidenceoftheattack!Thankfullythough,manynewer

NetFlow-capabledevicescanbetunedtoexportathigherrates,

resultinginimproveddetectiontimes.

Detectionaccuracyisanothermatter.SincebothsFlowand

NetFlowtransmitinformationonsendingandreceivingportand

bothtransmitinformationonflagcombinationsandIPaddresses,

itisreallyuptothecollectortomakeanaccuratedetection.

DistinguishingaDNSorNTPamplificationattackfromaSMURF,

or a FRAGGLE attack from a SynFlood, is key in performing

effectivemitigation.Mosttriagescenarios(whetherusinga

scrubbingdeviceormanuallymitigating)relyonknowingacouple

of key factors:

1. What are the targets being hit?

2. Arethebit/packetratessufficientlyhightoimpact

theservice?

3. Whatisthespecifictype(ortypes)ofattack?

BothNetFlowaswellassFlowprovidesufficientdetailto

accurately make that determination if the detection logic is present

inthecollectorsoftware.

Inpractice,bothsFlowandNetFlowvariantscanbedeployed

verysuccessfullyinDDoSdetection.AlthoughsomesFlow

deployments detect DDoS attacks in as little as 3 seconds, the

nature of the Internet has made it such that it can take some

timebeforetheattackreachesfullstrength.Combinedwiththe

Page 20: THE BIG BOOK OF NETWORK FLOWS FOR SECURITY4 Cisco’s NetFlow is just one of the many network flow protocols out there. Others include: • J-Flow • Cflow • IPFIX • sFlow® Flow

20

fasterNetFlowexportersthathaverecentlystartedtoreachthe

market,thespeedadvantageofsFlowisstartingtofade.Also,

keepinmindthatmanyenvironmentswillnothaveachoice,

as the exporting hardware is already in place, supporting only a

singletypeofexport.Sothechoicemaynotbeyours,andboth

approachescanbeusedtotuneandfinessefordetectionspeed

andaccuracy.Mostimportantly,ifyouhaveboth,thenuseboth.

Thebetterthevisibility,thebetteryourdefenses.

Page 21: THE BIG BOOK OF NETWORK FLOWS FOR SECURITY4 Cisco’s NetFlow is just one of the many network flow protocols out there. Others include: • J-Flow • Cflow • IPFIX • sFlow® Flow

21

Flowanalysiswillgetevenfasterinthefuture.Althoughpacket

volumesaregrowingexponentially,networkflowvolumesarenot

growingasquickly.Howisthispossible?Individualflowsaregetting

bigger(meaningmorepacketsperflow)andthisisbecausenetwork

sessionsareprimarilydrivenbyuserbehavior,suchasretrieving

email,browsingWebpages,andconsumingmedia.Forexample,

wecanonlywatchonemovieatatimeandreadoneemailata

time.Analysisspeedswillthereforegrowovertheyears,asthetime

toprocessasingleflowremainsconstant,regardlessofthepacket

sizeofthatflow.

Movingflowanalysisintotheclouddeliversadditionaladvantages.

We are able to harness the processing power to deal with transient,

high-volumethreats.Flowvolumesmayincreaseone-hundred-fold

duringaDDoSattack,whichcanputastrainonright-sizedin-house

flowprocessingequipment.Inthecloud,flowsareanalyzedona

muchlargerplatformthantheywouldbeinhouse;CPUresources

areavailabletoquicklydetect,analyze,andmitigatethesebehaviors

withoutresource-constrainedslowdowns.Somecraftyattackers

andverystealthybadbehaviorsonlybecomeapparentwhenviewed

overmanycustomernetworksatonce.Thefinalbenefitofanalysis

inthecloudisthemostexcitingofall—byanalyzingflowdata

fromthousandsofsourcesallovertheInternettogetherinasingle

window,emergentbehaviorsbecomevisiblethatwouldotherwise

behiddenfromviewwhenanalyzedinisolationinasinglenetwork.

Thispowerfulbenefitisunlockedbymovingflowdatatothecloud.

THE SCALABILITY OF NETWORK FLOWS

Page 22: THE BIG BOOK OF NETWORK FLOWS FOR SECURITY4 Cisco’s NetFlow is just one of the many network flow protocols out there. Others include: • J-Flow • Cflow • IPFIX • sFlow® Flow

22

Fortipsonconfiguringflowexportfromavarietyofroutersand

switches, check out the following resources:

• Flexible NetFlow export from Cisco routers

• Simple NetFlow export from recent Cisco routers

• NetFlow export from older Cisco routers

• J-Flow export from Juniper SRX Series routers

• NetFlow export on VMWare vCenter with ESXi

• NetFlow export on Open vSwitch SDN

TIPS ON CONFIGURING FLOW EXPORT

Page 23: THE BIG BOOK OF NETWORK FLOWS FOR SECURITY4 Cisco’s NetFlow is just one of the many network flow protocols out there. Others include: • J-Flow • Cflow • IPFIX • sFlow® Flow

23

ABOUT FLOWTRAQ

FlowTraq®providessoftwareandservicesfor

high-performance network monitoring, analytics,

security and forensics to detect a range of network

behaviors,includingdistributeddenialofservice

(DDoS),bruteforceattacks,botnets,worms,

networkscansandothernetworktrafficanomalies.

FlowTraqiscompatiblewithallthecommonflow

formats — including NetFlow, sFlow®,IPFIX,Cflow,

J-Flow,andPCAP—whethersampledornot.And

thankstoafull-fidelityparalleldatabase,FlowTraq

canrecalleveryindividualtrafficflowthatcrossed

yournetwork,nomatterhowlongago.Thisunlimited

filteringcapabilityisnecessarytofindanindividual

communication that is the source of the malware,

controlofthebotnet,ordestinationofexfiltrateddata.

TolearnmoreaboutFlowTraq,visitwww.flowtraq.com.

Experience FlowTraq for yourself with a free 14-day trial

(availabletoqualifiedorganizations).Torequestatrial,

visitwww.flowtraq.com/trial.

Page 24: THE BIG BOOK OF NETWORK FLOWS FOR SECURITY4 Cisco’s NetFlow is just one of the many network flow protocols out there. Others include: • J-Flow • Cflow • IPFIX • sFlow® Flow

16CavendishCourt,Lebanon,NewHampshire03766

Phone+1(603)727-4477|[email protected]

©2016FlowTraq,Inc.Allrightsreserved.

FlowTraqisaregisteredtrademarkofFlowTraq,Inc.