ProjectDeliverable

D8.2DataManagementPlan

ProjectNumber 700692ProjectTitle DiSIEM–Diversity-enhancementsforSIEMsProgramme H2020-DS-04-2015Deliverabletype ReportDisseminationlevel PUSubmissiondate 28.02.2017Responsiblepartner EDPEditor PedroDiasRodriguesRevision 0.11

The DiSIEM project has received funding from the European Union’s Horizon 2020researchandinnovationprogrammeundergrantagreementNo700692.

D8.2

2

EditorPedroDiasRodrigues,EDPContributorsAnaRespício,FFCULJoãoAlves,FFCULLuísMiguelFerreira,FFCULAlyssonBessani,FFCULPedroDiasRodrigues,EDPGonçaloSantosMartins,EDPIvoRosa,EDPSusanaGonzálezZarzosa,AtosMichaelKamp,FraunhoferZayaniDabbabi,AMADEUSIlirGashi,City

D8.2

3

ExecutiveSummaryThis document constitutes the Data Management Plan (DMP) of the DiSIEMproject, explaining how the project plans to manage datasets. It records andforeseestheactivitiesofallDiSIEMpartnersrelatedtotheproductionanduseofdatasets (for experimentation, measurement, demonstration or validationpurposes).Thedocumenthasbeencompiledasasummaryofaquestionnaire-basedsurveydistributedtoallthepartnersoftheDiSIEMconsortium.There are two key types of data being produced in the scope of the DiSIEMproject:rawdataoriginatedintheSIEMsystemsoperatedbyindustrypartners(Tier1)andprocesseddatageneratedbytheremainingpartners(Tier2).The most prevalent data format is Comma-Separated Values (CSV), a textualdescriptionofdatathatiscommonandwidelyusedinthebigdatacommunity.Severalpartners intend toshare theirdatasetspublicly,making themavailableforfurtherresearchoutsidetheproject.Since it is veryearly in theproject, little isknown in termsof sharing, volumeand archiving. The project is aware of these aspects and will tackle them byupdatingthepresentdocumentduringthedevelopmentof thespecificationsofthe experimentations. Therefore, information in this document is subject tochange.

D8.2

4

TableofContents1 Introduction................................................................................................................................71.1 OrganizationoftheDocument...................................................................................7

2 Methodology...............................................................................................................................82.1 DiSIEMPolicyforStorageandSharingofDatasets..........................................82.2 DataCollectionMethodology.....................................................................................8

3 DatasetFFCUL..........................................................................................................................103.1 DatasetDescription......................................................................................................103.2 Standardsandmetadata............................................................................................103.3 DataSharing....................................................................................................................103.4 Archivingandpresentation......................................................................................103.5 Datadetails......................................................................................................................10

4 DatasetCITY..............................................................................................................................124.1 DatasetDescription......................................................................................................124.2 Standardsandmetadata............................................................................................124.3 DataSharing....................................................................................................................124.4 Archivingandpresentation......................................................................................124.5 Datadetails......................................................................................................................12

5 DatasetEDP...............................................................................................................................145.1 DatasetDescription......................................................................................................145.2 Standardsandmetadata............................................................................................145.3 DataSharing....................................................................................................................145.4 Archivingandpresentation......................................................................................145.5 Datadetails......................................................................................................................14

6 DatasetAMADEUS..................................................................................................................166.1 DatasetDescription......................................................................................................166.2 Standardsandmetadata............................................................................................166.3 DataSharing....................................................................................................................166.4 Archivingandpresentation......................................................................................166.5 Datadetails......................................................................................................................16

7 DatasetDigitalMR...................................................................................................................237.1 DatasetDescription......................................................................................................237.2 Standardsandmetadata............................................................................................237.3 DataSharing....................................................................................................................237.4 Archivingandpresentation......................................................................................237.5 Datadetails......................................................................................................................23

8 DatasetFRAUNHOFER..........................................................................................................249 DatasetATOS............................................................................................................................259.1 DatasetDescription......................................................................................................259.2 Standardsandmetadata............................................................................................259.3 DataSharing....................................................................................................................259.4 Archivingandpresentation......................................................................................259.5 Datadetails......................................................................................................................25

10 SummaryandConclusions.............................................................................................27ListofAbbreviations.......................................................................................................................28References...........................................................................................................................................29Annex–Questionnairetemplate................................................................................................30

D8.2

5

ListofTablesTable1–Datadetails(EDP)........................................................................................................15Table2–Datadetails(AMADEUS)...........................................................................................16Table3–LSSASMlogs(AMADEUS).........................................................................................17Table4–HTTPaccesslogs(AMADEUS)................................................................................17Table5–SuricataIDS(AMADEUS)...........................................................................................18Table6–Ciscofirewalllogs(AMADEUS)...............................................................................19Table7–PaloAltoNetworks(AMADEUS)............................................................................19Table8–PaloAltoIDS(AMADEUS).........................................................................................20Table9–McAfeeePO(AMADEUS)...........................................................................................21Table10–BroIDS(AMADEUS).................................................................................................21Table11–CiscoVPN(AMADEUS)............................................................................................22Table12–Datadetails(Atos).....................................................................................................26

D8.2

6

RevisionHistoryVersion Date Author Notes0.1 29.12.2016 PedroDiasRodrigues

(EDP)Firstcontribution.

0.2 25.01.2017 PedroDiasRodrigues(EDP)

Reviewofdocumentcontents.Integrationofcommentsbyotherpartners.

0.3 07.02.2017 PedroDiasRodrigues(EDP)

AddedinputforEDPdataset

0.4 09.02.2017 MichaelKamp(Fraunhofer)

AddedinputforFraunhoferdataset

0.5 14.02.2017 SusanaGonzálezZarzosa(Atos)

AddedinputforAtosdataset

0.6 16.02.2017 IlirGashi(City) AddedinputforCitydataset0.7 19.02.2017 AlyssonNevesBessani

(FFCUL)AddedinputonFFCULdataset

0.8 20.02.2017 ZayaniDabbabi(AMADEUS)

AddedinputforAMADEUSdataset

0.9 24.02.2017 PedroDiasRodrigues(EDP)

Documentreview

0.10 27.02.2017 AlyssonNevesBessani(FFCUL)

Documentreview

0.11 28.02.2017 PedroDiasRodrigues(EDP)

Documentreview

1.0 02.03.2017 PedroDiasRodrigues(EDP)

Firstversionafterfinalreview

D8.2

7

1 IntroductionTheCommissionisrunningaflexiblepilotunderHorizon2020calledtheOpenResearchData(ORD)Pilot.TheORDpilotaimstoimproveandmaximizeaccesstoandre-useofresearchdatageneratedbyHorizon2020projectsandconsidersthe need to balance openness and protection of scientific information,commercialization and Intellectual Property Rights (IPR), privacy concerns,security,aswellasdatamanagementandpreservationaspects.As a participating project, DiSIEM is required to develop a Data ManagementPlan (DMP), identified as deliverable D8.2. The DMP is a key element of gooddatamanagement,describingthedatamanagement lifecycle forthedatatobecollected, processed and/or generated. The goal is to make research datafindable,accessible,interoperableandre-usable(FAIR).All partners have contributed to the document, completing a project-widequestionnaire thatwas then used to determine each partner’s role in creatingand/orprocessingdata.

1.1 OrganizationoftheDocumentSince each partner will generate and/or manipulate data, the document isorganizedwithonesectionperpartner(Sections3-9).Eachofthesesectionsisstructuredinfivesubsections:

1. Datasetdescriptioncontainsatextualdescriptionofthedataset.Itaimsatexplaining,inashortparagraph,whatthedatasetcontainsandwhatitsgoalis;

2. Standards and metadata focuses on explaining the internals of thedataset,namelyhowausercanfindsyntacticalandsemanticinformation;

3. Data sharing addresses the issues related to data access, and privacyconcerns, namely if the dataset is going to be indexed, and how and towhomitwillbemadeaccessible;

4. Archiving and presentation covers the aspects related to dataavailability, during and beyond the project, aswell as the actions takenandplannedtosupportavailability;

5. Datadetailsgoes intothespecificsofeachpartner’sdataset,describingitscontent.

Besides these per-partner sections, the document also contains a generaldescriptionofouroverallmethodologyintermsofdatacollectionandsharinginSection 2. The summary and conclusions of the DataManagement Plan are inSection10.Intheappendix,weincludedthequestionnaireeachpartnerfilledtopreparethedocument.

D8.2

8

2 MethodologyInthissection,weexplainsomegeneralpolicywedefinedtostoreandsharethedata sets produced during the project and the overall methodology used forproducingthisdocument.

2.1 DiSIEMPolicyforStorageandSharingofDatasetsOneofthemostimportantaspectsofthemethodologyishowdatasetsaretobestoredandusedduringtheproject.A first general concern is how the produced datasets are to be stored. Theconsortiumdecidedtodothatinthreeways,fordifferenttypesofdatasets:

• Forthepublicdatasets,i.e.,theoneswecanshareoutsidetheconsortium,we plan to publish them on the project webpage (or in another publicrepositorytobereferredbytheprojectwebpage).

• Forcontrolleddatasets,i.e.,theonesthatwillbeanonymizedandsharedwithintheconsortiumforenablingpartnerstodoexploratorystudies,wecreatedaspecialdirectoryintheprojectrepositoryforstoringthem.Theidea is to have a subdirectory for each dataset containing not only thedataset files but also a info.txt text file with a brief description andmetadataofthedataset.

• Forprivacy-sensitivedatasets,i.e.,thosethatcontaincriticalinformationfrompartnersand therefore require special care in sharing,wedecidedthatpartnersneedtoagreeonthespecificsofhowsharingcanbedone.This might include the signing of specific agreements and protocolsbetweentheinvolvedpartners.Inanycase,thisshouldbedonebetweenpartners,withoutanydirectinfluencefromtheconsortium.

Regarding the storage of controlled datasets, they will be kept in our projectrepository,which ismaintained in adedicatedKVMvirtualmachinehostedbyFCUL.ThisVMcanonlybedirectlyaccessedbyDI-FCULsystemadministratorsandisexternallyvisibleonlythroughthegitlabwebinterfaceandthroughthegitprotocol over SSL/TLS. All accesses require authentication using validcredentials and access control is enforced. Therefore, we believe an adequatelevelofprotectionisprovidedforthesedatasets.Aswillbeclear inthenextsections, thepreferredformatsfordatasetsareCSV(CommaSeparatedValues,asspecifiedinRFC4180[1])andJSON,sincebotharetext-basedandeasilyparsedbyanytoolorservicebeingusedwithintheproject.

2.2 DataCollectionMethodologyTo compile the data management plan, a questionnaire was first elaboratedcoveringthemainquestionsthatneedtobeansweredinthetemplateprovidedbytheEuropeanCommission[2].

D8.2

9

Inthesecondphase,eachprojectpartnerrespondedtothequestionnaire,fillingit with as much detail as possible at this stage of the project. Completedquestionnaires were stored for analysis and traceability in the project’s gitrepository.Inthethirdphase,theDataManagementPlanwascreatedasasynthesisofthequestionnaire results, attempting to take advantage of commonalities betweenresponsestoprovideasimpleviewofdatamanagementprocedureswithintheconsortium.Further revisions of the document will be based on updates to partnerquestionnaires.Therefore,theDMPwillbeupdatedatleastbythemid-termandfinalreports tobeable toaccommodateanynewdata formsandrequirementsthatcannotbeestimatedinthiscurrentstageoftheproject.

D8.2

10

3 DatasetFFCULFFCUL is an academic partner in the project therefore it is not expected tocontributewithdatasets aboutmonitored infrastructures.However, it plans tocontribute with some OSINT datasets that might be useful for evaluating thetoolsandtechniquesproposedforprocessingsuchkindsofdata.

3.1 DatasetDescriptionInprinciple,FFCULwillprovideacollectionof tweetsclassifiedas “relevantornot” for a given reference infrastructure, a list of operating systemsvulnerabilities collected from NVD and enriched with information from otherdatabases,andalistofcompromisedIPaddressescollectedfromseveralsecurityfeedsontheInternet.

3.2 StandardsandmetadataThe dataset will contain data formatted using the common Comma-SeparatedValues(CSV)standard.

3.3 DataSharingSinceall thesedatasetsarebeingcollectedfrompublic feedsfromtheInternet,FFCUL intends to make them publicly available, respecting possible dataprotectionlegislation.

3.4 ArchivingandpresentationThe dataset will be made available as companion papers exploring them arepublished.Theideaistohavepapersusingthedatasetsforvalidatingtoolsbuiltwithin theproject.Once thepapersaremadepublic, thedatasetswillbemadeavailableeitherthroughtheprojectwebpageorthroughDI-FCULwebpage.

3.5 DatadetailsFFCULwillprovide threedifferent typesofOSINTdatasets thatcanbeused tovalidatedifferentDiSIEMinnovations:

- A collection of tweets gathered from 80 cybersecurity-related accountssuchas sans_isc, e_kaspersky,alienvault, vuln_lab,etc.These tweetswillbemanually classifiedas relevantornot to somesyntheticorganizationinfrastructure;

- A list of operating systems vulnerabilities collected from NVD andenriched with information about exploits and patches obtained fromothervulnerabilitydatabasessuchasExploitDBandOSVDB;

- A list of compromised IPs collected frommore than a hundred securityfeedsorganizedbypublisheddateandsource.

D8.2

11

Noticethat“theoperatingsystemvulnerabilities”datasetissomewhatsimilartothedataofferedbythevepRisktoolfromCity(seenextsection).Inthefuture,wewilltrytointegratethesedatasetstoavoidduplicatingefforts.

D8.2

12

4 DatasetCITYCity,beinganAcademicpartnerintheproject,willbeprimarilyadataconsumerratherthanadataproducer.Weplantoanalysethedataprovidedbytheprojectpartnerstoevaluateandtestourextensionsandplug-insfordiversityanddatavisualisation.

4.1 DatasetDescriptionWedoplantoalsodeployourowntestbedtoevaluateandtesttheextensionswebuild for diversity and probabilistic modelling. The data will consist ofsyntheticallygeneratednetworkdata,aswellasdatacollectedfromaUniversityhoneypot.We are alsobuilding a tool that gatherspublic data on vulnerabilities, patchesandexploits.Thetoolismadeavailablefromthefollowingsite(theURLmaybeupdated and change in the future): http://veprisk.city.ac.uk/sample-apps/vepRisk/

4.2 StandardsandmetadataThedatafromourtestbedwillconsistofnetworktraffic, inthepcap format,aswell as the alerts of the IntrusionDetection Systems (IDS)wewill test: Snort,Suricata andBro.Thesewill be generated in the respective alert formatof thetoolvendors.ThedatafromthevepRisktoolcanbedownloadedfromthesiteinCSVformat.

4.3 DataSharingSynthetic data from our testbed will be shared with DiSIEM partners withoutrestriction.Datafromhoneynets,wouldneedtobeanonymizedfirsttoremovesensitive,confidentialand/orprivateinformation.DatafromvepRiskisavailablefromthepublicpageofthetool.

4.4 ArchivingandpresentationThedatasetwillbedisseminatedtotheconsortiumviatheGitrepository.

4.5 DatadetailsFor the vepRisk tool, the data is taken from the public databases onvulnerabilities, patches and exploits and the information on these data areavailable from the repositories where this data is collected namely, NVD1,Exploitdb2andvariouspatchdatabases(e.g.Microsoft3,Ubuntu4etc.)1 https://nvd.nist.gov/download.cfm2 https://www.exploit-db.com/searchsploit/3 https://technet.microsoft.com/en-us/security/bulletins.aspx4 https://www.ubuntu.com/usn/

D8.2

13

Regardingourtestbed,weexpectthedatawillincludenetworkflows(sourceanddestinationIPaddresses,sourceanddestinationports,networkprotocol,timestampetc.)andthealertsfromtheIDSplatforms.

D8.2

14

5 DatasetEDP

5.1 DatasetDescriptionHavinganoperatingSIEMplatformthatreceivesover10.000eventspersecond,EDP – Energias de Portugal, SA. has the capability to provide realistic andmeaningful data for analysis. The datasetwill consist of a significant subset ofreal events, comprisingdata frommultipleanddiverse sources, afteradequatepre-processing to ensure that no confidential information is wrongfullydistributed.

5.2 StandardsandmetadataThe dataset will contain data formatted using the common Comma-SeparatedValues(CSV)standard,asspecifiedinRFC4180[1].

5.3 DataSharingEDPwillmakedataavailablefortheprojectpartners.Thespecificinformationtobe shared depends on the need presented by the partners, as well as a riskassessmenttoguaranteelegalandbusinesspolicycompliance.ThefinaldatasetdetailswillbeindicatedinalaterreleaseoftheDMP.Information retrieved from EDP’s SIEM platform should not bemade publiclyavailableduetothecriticalnatureofthedataanduserprivacyconcerns.EDP is investigating tools to enable data masking and/or anonymization. Weidentified and started performing tests with two of such tools: Python Faker(http://blog.districtdatalabs.com/a-practical-guide-to-anonymizing-datasets-with-python-faker)andARX(http://arx.deidentifier.org/).

5.4 ArchivingandpresentationThedatasetwillbedisseminatedtotheconsortiumviatheofficialGitrepository.

5.5 DatadetailsThemostrelevantSIEMeventscollected inEDP’splatform,withasummaryoftherespectivefieldset,arepresentedinthefollowingtable.

D8.2

15

Eventsource

Field Firewall IPS Userauthentication

VPNaccess

Serveraccess Antivirus

Eventname √ √ √ √ √ √Sourceusername X X √ √ √ √Sourceaddress √ √ √ √ √ √Sourceport √ √ X X X XSourcegeocountry √ √ X √ √ XDestinationusername X X √ √ √ √

Destinationaddress X X √ √ √ √

Destinationport √ √ X X X XDestinationgeocountry √ √ X √ √ X

Applicationprotocol √ √ X X X X

Filename X X X X X √Policyname X X X X X √

Table1–Datadetails(EDP)

Fieldformat:Eventname:String(255-characterlimit);Sourceusername:String(255-characterlimit);Sourceaddress:IPAddress(IPv4);Sourcegeocountry:String(255-characterlimit);Destinationusername:String(255-characterlimit);Destinationaddress:IPAddress(IPv4);Destinationport:Integerfrom1to65535Destinationgeocountry:String(255-characterlimit);Applicationprotocol:String(255-characterlimit);Filename:String(255-characterlimit);Policyname:String(255-characterlimit).

D8.2

16

6 DatasetAMADEUS

6.1 DatasetDescriptionAmadeus can provide real datasets from different log sources: applications,Firewalls, OS syslog, Antiviruses, Proxy, VPN, IDS, DNS, etc. We need to pre-processandanonymisethedatabeforesharingitwithpartners.

6.2 StandardsandmetadataTwodataformatwillbeusedfortheshareddatasets:

1. Comma-SeparatedValues(CSV);2. JSON.

Adocumentationwillbeprovidedwitheachtypeofdatasettobesharedwiththepartners.

6.3 DataSharingAmadeusdatasetswillbesharedwithDiSIEMpartnersdependingontheneedspresented.However,partnersneedtoensurethatshareddatasetsshouldnotbemadepubliclyavailableinanycase,duetolegalandbusinesspolicyrestrictions.

6.4 ArchivingandpresentationThedatasetwillbedisseminatedtotheconsortiumviatheofficialGitrepository,oranysecurefilesharingmethod(inthecaseofprivacy-sensitivedata).

6.5 DatadetailsAsummaryofthedatasetstobesharedwithDiSIEMpartnerscanbefoundinthetablebelow:Source DescriptionLSSASMlogs An administration tool for an authentication

andaccesscontrolmanagementapplicationHTTPaccesslogs HTTPlogsfromane-commerceapplicationCisco,PaloAltoNetwork FirewalllogsMcAfee AntivirusSuricata,PaloAlto,Bro IDSCiscoVPN VPN

Table2–Datadetails(AMADEUS)

Thenextsectionsprovideadescriptionofthedatafieldsforeachdataset.

6.5.1 LSSASMlogsThe logs of an administration tool for an authentication and access controlmanagement application.Thedataset tobeprovided is a setofuser actions.A

D8.2

17

user session is a set of user actions with the same session id (PFX, see tablebelow):

Field DescriptionPFX SessionidOrga OrganisationAction TypeofactionperformeduserId UserissuingtheactionofficeId OfficefromwhichtheuserisconnectingCountry CountryCodeIP IPaddress*Browser Clientbrowserused*browserEngine ClientbrowserEngine*OS Clientoperatingsystem

*Thesefieldsarederivedfromtheuseragentstring.

Table3–LSSASMlogs(AMADEUS)

6.5.2 HTTPaccesslogsThisdatasetwillbeextractedfromawebserverofane-commerceapplication.ThefieldsarethedefaultHTTPrequestfieldswithsomeadditionalnestedfieldsextractedfromtheIPaddressandtheuseragentstring.Moredetailsinthetablebelow:

Field DescriptionDatetime TimestampMethod HTTPmethodUrlpath URIpathStatus HTTPstatuscodehttp_referrer HTTPreferrerUseragent UseragentStringAccespt_language AcceptLanguageintheHTTPheaderDuration RequestprocessingtimeHostname TargetHTTPhostnameReferrer_uri_proto ReferrerURIprotocolReferrer_hostname ReferrerHostnameReferrer_uri_path ReferrerURIpathReferrer_params ReferrerParametersUa NestedUseragentobjectremoteclientipaddress EndUserorCDNIPaddressclient_ip PrivateIPaddressofHTTPserverGeoip NestedGeocoordinatesobjectisp NestedISPobjectedge_proxy_cip EndUserorCDNIPaddressx_forwarded_for EndUserorCDNIPaddressJsessionid Thesessionidofagivenrequest

Table4–HTTPaccesslogs(AMADEUS)

D8.2

18

6.5.3 SuricataIDSThisdatasetisextractedfromtheOpensourceIDS/NSMengineSuricata.Abriefdescriptionofthemostrelevantfieldsisprovidedinthetablebelow:Field DescriptionCategory ThreatcategoryDest DestinationIPaddressSeverity ThreatseveritySignature ThreatSignatureSrc SourceIPaddressAnswer DNSserveranswerDate TimestampDest_nt_host DestinationIPorganizationDest_port DestinationportnumberDns NestedDNSresponseobjecthttp NestedHTTPrequestobjectEventtype SuricataeventtypeMessage_type Request/ReplyProto TransportLayerProtocolSrc_nt_host SameasDest_nt_hostSsl_issuer_common_name SSLcertificateissuernameSsl_issuer_organization SSLcertificateissuerorganizationSsl_publickkey SSLcertificatepublickeySsl_subject_common_name SSLsubjectnameSSL_subject_organization SSLsubjectorganizationSsl_version SSL/TLSversionTLS NestedTLSrequestsobject

Table5–SuricataIDS(AMADEUS)

6.5.4 CiscoFirewalllogsWithin the context of a security incident, administrators can use cisco syslogmessages to understand communication relationships, timing, and, in somecases,theattacker'smotivesand/ortools.Field Descriptionacl AccesscontrollistAction Thestatusoftheactions(e.g.allowed,blockedetc.)Cisco_ASA_action Thestatusof theCiscoAdaptiveSecurityAppliance

(e.g.allowed,blockedetc.)Cisco_ASA_message_id TheidoftheCiscomessageDescription TheDescriptionofthefirewalleventDest_category ThecategorydestinationoftheeventDest_dns DestinationDNSDest_mac ThephysicaladdressofthemacdestinationDest_nt_host DestinationnetworkhostDest_port Theportdestination

D8.2

19

Dest_zone TheserverdestinationoftheeventEventtype ThetypeoftheeventGroup ThegroupofserversMessage_id theIDofthemessageRule_name ThenameoftheruleSeverity_level Theseverityleveloftherule

Table6–Ciscofirewalllogs(AMADEUS)

6.5.5 Next-GenerationFirewall–PaloAltoNetworks(PAN)This next-generation firewall classifies all traffic, including encrypted traffic,basedonapplication,applicationfunction,userandcontent. Field DescriptionAction TheactiontakenbytheIDSApplication TheapplicationonwhichthealertwasraisedClient_ip TheIPoftheclientClient_location ThelocationoftheclientDate TimestampDest_asset_id TheassetdestinationIDDest_dns ThednsofthedestinationDest_interface ThedestinationnetworkinterfaceDest_ip TheIPofthedestinationDest_zone ThezoneofthedestinationDest_nt_host DestinationnetworkhostEventtype Thetypeofevent(e.g.allowed,blockedetc.)dstPort DestinationportProtocol ThecommunicationprotocolbeingusedRuleName ThenameoftheruleServer_IP TheIPoftheserver

Table7–PaloAltoNetworks(AMADEUS)

6.5.6 PaloAltoIDSThisdatasetisalsoextractedfromPaloAltoNetworksnext-generationfirewalls.Itcontainstheeventstaggedasthreats.Adescriptionofthemostrelevantfieldsisprovidedbelow:Field DescriptionAction ActiontakenbytheIDSApplication TheapplicationthatraisedthealertCategory CategoryoftheintrusionClient_ip ClientlocalIPaddressClient_location LocationoftheclientinthenetworkDate timestampDest_ip DestinationIPaddressDest_hostname DestinationhostnameDest_interface Destinationnetworkinterface

D8.2

20

Dest_nt_host DestinationIPorganizationDest_port DestinationPortnumberDestinationZone DestinationnetworkzoneIngressInterface IngressnetworkinterfaceProto TransportLayerprotocolSession_id CommunicationsessionidSeverity Severitylevel(1to5)Signature VulnerabilitysignatureSourceUser SourceUsernameSrc_bunit SourceuserbusinessunitSrc_category SourcecategorySrc_dns SourceDNSservernameSrc_mac SourceMACaddressSrc_nt_host SourceIPOrganizationSrc_owner SourceIPOwnerNameSrc_port SourcePortNumberSrc_zone SourceIPnetworkzoneThreat:category ThreatcategoryThreat:name ThreatNameUser UsernameUser_watchlist Boolean,trueifUserinwatchlist

Table8–PaloAltoIDS(AMADEUS)

6.5.7 McAfeeePOMcAfee ePolicy Orchestrator, a centralized security management software forantiviruses,isthesourceofthisdataset.Field DescriptionAction ActiontakenbyMcAfeeAntivirusCategory ThreatcategoryDate DateDest OfficeIDDest_bunit DestinationbusinessunitDest_ip DestinationIPaddressDest_mac DestinationMACaddressDest_nt_domain DestinationIPnetworkdomainDest_nt_host DestinationhostnameDest_owner DestinationUsernameDetection_method FirewalldetectionmethodDevent_description FirewalleventdescriptionFile_name SuspiciousfilenameFqdn FullyQualifieddomainnameIs_laptop Boolean,1ifLaptopusedLogon_user UsernameMcafee_epo_os OSnameOs_build OSbuildnumberOs_version OSversion

D8.2

21

Process ProcessnameProduct ComponentcreatingtheeventSeverity ThreatseveritylevelSeverity_id AnumbermappedtoseveritySrc SourceIPaddressSrc_bunit SourceIPbusinessunitSrc_category SourceIPcategorySrc_mac SourceMACaddressSrc_nt_host SourceIPnetworkzoneSrc_owner SourceIPownernameSrc_priority Sameasdest_priorityThreat_handled BooleanforwhetherthreatishandledThreat_type ThreatTypeUser_email Useremailaddress

Table9–McAfeeePO(AMADEUS)

6.5.8 BroIDSThisdatasetisextractedfromBro,anopensourcenetworkanalysisframework.BelowisadescriptionoftheBroeventsfields.Field DescriptionBody ThreatdescriptionCategory ThreatcategoryDate TimestampDest DestinationIPaddressDest_nt_host DestinationIPnetworkzoneDest_port DestinationportnumberEventtype BroeventtypeFile_desc SuspiciousfileO OrganizationSrc SourceIPaddressSrc_nt_host Sameasdest_nt_hostSrc_port SourcePortnumberTag::eventtype EventtypeUid UserID

Table10–BroIDS(AMADEUS)

6.5.9 CiscoVPNThis dataset contains events from a Cisco VPN server. A description of thedatasetfieldsisinthesummarybelow.Field DescriptionAssigned_ip PrivateIPassignedtotheusersessionCisco_ASA_user UsernameDate TimestampDuration VPNsessiondurationinseconds

D8.2

22

Eventtype CiscoVPNeventtypeGroup RemoteAccessGroupIP UserPublicIPaddressReason ConnectionLostreasonUser_email UseremailaddressUser_identity FullusernameUsername Username

Table11–CiscoVPN(AMADEUS)

D8.2

23

7 DatasetDigitalMRDigitalMR works with OSINT and has infrastructure to fetch information tocreatedatasets.Weintendto fetch informationfromsecurityrelatedblogsandtweetsforaspecifictimelineofinterest.Thesedatasetswillbeavailableduringtheproject.

7.1 DatasetDescriptionOur data consists of openly available content on the Internet from sourcesincludingblogs, forums,news, andsocialnetworks likeTwitter, InstagramandFacebook.Thisdataiseitherscrapedfromthesourcesusingourspeciallybuiltcrawlersor fetchedusing thebuilt-inAPIof thedata sources suchas theonesprovidedbyTwitterandFacebook.

7.2 StandardsandmetadataThe format of the data is in JSON which is widely supported by severalapplicationsand issemi-structured.Thesizeof thedatacanbeup to5millionpostsontheInternetdependingonthescopeoftheproject.

7.3 DataSharingGiventhatthecontentofthedatamightcontaininformationsuchasusernames,and privacy lawsmight vary between countries; it is the responsibility of theuserofthedatasettomakesurethattheapplicablelegislationsarerespected.

7.4 ArchivingandpresentationThe dataset will be shared to the consortium via the official Git repository inJSONandwillbeavailableforusebythepartners.

7.5 DatadetailsSomeofthecommonfieldsinthedataincludethefollowing:

• AuthorUsername• AuthorprofileURL• PostURL• ParenttweetURL(fortwittercontent)• Location• Content/Post(actualcontentofthedata)• Date• Tags(addedbyDigitalMR)• Relevance(addedbyDigitalMR)• Sentiment(addedbyDigitalMR)

D8.2

24

8 DatasetFRAUNHOFERFraunhofer does not plan to produce any dataset duringDiSIEM. Instead, dataprovidedfromtheprojectpartnerswillbeanalysedusingmachinelearningandvisual analytics methods. This may lead to the development of novelrepresentationsoftheeventdataproducedbytheSIEMplatforms,aswellasthediscoveryofuser-orsession-clusters.TheseresultscanbeusedtodevelopnovelvisualizationtoolsforSIEMdata.To represent event sequences, Fraunhofer evaluates the embedding, includingthe bag-of-words approach, event occurrence frequencies within a givensequence and the TF-IDF-score (term frequency multiplied with the inversedocument frequency) of events with respect to a given sequence database.Anotherapproachistodefineasimilaritymeasureforsequences.Tothatextend,Fraunhoferdevelopedanembeddingofevent types intoametric space,wherethedistancebetweeneventscorrespondtotheco-occurrencefrequencieswithina given sequence database. These feature representations of event sequenceswillbeused toembed thedata in2Dor3D forvisualization,aswellas to findclustersofsequencesandusersandtopredictwhetherasequenceisapotentialthreat.

9 DatasetATOS

9.1 DatasetDescriptionAtosdatasetwillbegeneratedinatestbedspecificallypreparedforDiSIEM.Thedatasetwillconsistof:• Events generated by applications or sensors installed in the testbed (e.g.

Snort,OSSec,netfilter,JBoss,linuxkernel,etc),oncenormalizedtotheeventformatusedbytheXL-SIEMcomponent;

• AlarmsgeneratedbyXL-SIEMcomponent.OSINTdataorIoCfromexternalfeedssuchasAlienVaultOpenThreatExchange(OTX)5couldalsobeusedbyXL-SIEMinAtostestbed.Sincedatawillbegenerated in the testbed,noconfidential informationwillbeprovidedinthedataset.

9.2 StandardsandmetadataCurrently,datageneratedinAtostestbedcanbeprovidedintwoformats:• Comma-SeparatedValues(CSV);• JSON.Nodocumentationormetadataisprovidedcurrentlywiththedataset.TheneedforsuchadditionaldocumentationwillbeanalysedforalaterreleaseoftheDMP.

9.3 DataSharingAtos will make data available for the remaining DiSIEM partners. Informationretrieved from Atos’ SIEM platform should not be made publicly availablewithoutpreviousauthorization.The specific information tobe shareddependsontheneedspresentedbythepartners,aswellasariskassessmenttoguaranteelegalandbusinesspolicycompliance.

9.4 ArchivingandpresentationThedatasetwillbedisseminatedtotheconsortiumviatheofficialGitrepository.Data generated in Atos testbed can be also shared to DiSIEM partners usingAdvancedMessageQueuingProtocol(AMQP)protocolsuchasRabbitMQServer.

9.5 DatadetailsSIEMeventstobecollected inAtostestbedandthe finaldatasetdetailswillbeindicatedinalaterreleaseoftheDMP.5https://otx.alienvault.com/

D8.2

Someeventsourcestobeconsideredare:

• Firewall;• Serveraccess;• NetworkIntrusionDetectionSystem.

Currently,SIEMeventscollected(oncenormalizedbythepluginsincludedintheXL-SIEMagentforeachspecificdatasource)havethefollowingfields:Field DescriptionType Typeofplugin:detectorormonitorDate Date(timestamp)onwhichtheeventisreceivedfromthesensor

Device IP address of the XL-SIEM agent generating the event in thenormalizedformat

Plugin_id IdentifierofthedatasourceofeventgeneratedPlugin_sid Typeofeventwithinthedatasourcespecifiedinplugin_idProtocol Protocol(TCP,UDP,ICMP…)

Src_ip IPwhich thesensorgenerating theoriginalevent identifiesas thesourceofthisevent

Src_port Sourceport

Dst_ip Ipwhich thesensorgenerating theoriginalevent identifiesas thedestinationofthisevent

Dst_port Destinationport

Log Eventdatathatthespecificpluginconsidersaspartofthelogandwhichisnotaccommodatedintheotherfields.

Data Raw event's payload, although the plugin may use this field foranythingelse.

Userdata1toUserdata9

Fieldsdefined in thenormalizedevent format toallocate relevantinformationfromthespecificevent'spayload.Theycancontainanyalphanumeric information, and on choosing one or another, thetypeofdisplaytheyhaveintheeventviewerwillchange.

OrganizationIdentifytheorganizationwheretheagentisdeployed.Table12–Datadetails(Atos)

D8.2

10 SummaryandConclusionsThe Data Management Plan of DiSIEM describes partners’ activity related todatasets. It contains a summaryof all the information available as of February28th, 2017. All (but one) partners intend to create datasets and make themavailablewithintheconsortium.Withrespecttodatasetdescriptions,mostofthedatamanipulatedbytheDiSIEMprojectisrelatedtosecurityeventscollectedfromSIEMsystemsandprocessedusingvariousexploratorymethods.Withrespecttostandardsandmetadata,themostprevalentformofdataformatis Comma-Separated Values (CSV), a textual description of data that is highlycommonandwidelyusedintheSIEMandbigdatacommunities.Thisformatisveryeasytomanipulate,particularlyadaptedtosharingovergit(astextfilesareeasilyversioned)andisunderstoodbyawiderangeoftools.Withrespecttosharing,severalpartnersintendtosharethedatasetsforfurtherresearchandpublication,atleastintheacademiccommunity.AcademicresearchandinnovationisthemainobjectiveofthedatamanagedintheDiSIEMproject.Allpartnersareawareofdata sharing limitationsdue toprivacy concernsandlegalobligations.Whennecessary,informationwillbeanonymizedortruncatedincompliancewiththeapplicablelegislation.With respect to archiving and presentation, partners plan to use internalresourcesandhavethemavailableatthetimeofwriting.Since it is very early in the project, this document only presents preliminaryproposals in terms of sharing, volume and archiving. The project is aware oftheseaspectsandwilltacklethembyupdatingthepresentdocumentduringthedevelopment of the specifications of the experimentations. Therefore,informationinthisdocumentissubjecttochange.

D8.2

ListofAbbreviationsAPI ApplicationProgrammingInterfaceCSV Comma-SeparatedValuesDMP DataManagementPlanDNS DomainNameSystemIDS IntrusionDetectionSystemIoC IndicatorsofCompromiseIPS IntrusionPreventionSystemJSON JavaScriptObjectNotationOSINT Open-sourceIntelligenceSIEM SecurityInformationandEventManagementVPN VirtualPrivateNetwork

D8.2

References[1] Y. Shafranovich. Comma Format and MIME Type for Comma-SeparatedValues(CSV)Files.RFC4180.October2005.https://tools.ietf.org/html/rfc4180[2] European Commission. Guidelines for FAIR Data Management in Horizon2020. Version 3.0. July 2016. http://ec.europa.eu/research/participants/data/ref/h2020/grants_manual/hi/oa_pilot/h2020-hi-oa-data-mgt_en.pdf

D8.2

Annex–Questionnairetemplate

DataManagementPlanQuestionnaire

TheDMPhastheobjectiveofdefininghowdatageneratedinthecontextoftheproject should be generated, stored, processed and made available. Thisdefinitionincludesbothinternaldatausedinsidethescopeoftheproject,limitedto thepartners,andprojectoutputs thataremadepublicso thatotherentitiescanbenefitfromtheinvestmentmadebytheEUCommission.

DescriptionofDataGive a brief description of the data, including any existing data or third-partysources that will be used, in each case noting its content, type and coverage.Outlineand justifyyour choiceof formatandconsider the implicationsofdataformatanddatavolumesintermsofstorage,backupandaccess.Willyougenerateanytypeofdata?(e.g.rawdatafromsystems,transformed/processedinformation,researchresults)

[PLEASEFILLIN]

Ifyes,whattype,formatandvolumeofdata? [PLEASEFILLIN]

Doyourchosenformatsandsoftwareenablesharingandlong-termaccesstothedata

[PLEASEFILLIN]

Atethereanyexistingdatathatexist/youcanreuse(link/information)?

[PLEASEFILLIN]

DataManagementDescribe the types of documentation that will accompany the data to helpsecondary users to understand and reuse it. This should at least include basicdetails that will help people to find the data, including who created orcontributedtothedata,itstitle,dateofcreationandunderwhatconditionsitcanbeaccessed.

D8.2

Documentationmayalsoincludedetailsonthemethodologyused,analyticalandprocedural information, definitions of variables, vocabularies, units ofmeasurement,anyassumptionsmade,and the formatand file typeof thedata.Considerhowyouwill capture this informationandwhere itwill be recorded.Whereverpossibleyoushouldidentifyanduseexistingcommunitystandards.Whocreated/contributed/ownsthedata? [PLEASEFILLIN]

Whatistheusedmethodology? [PLEASEFILLIN]Whatisthedata’sorigin?(e.g.application,systemorprocess) [PLEASEFILLIN]

Forwhom/enduseristhedatauseful?(e.g.university,researchorganization,scientificpublication)

[PLEASEFILLIN]

Doyouseeanypossibilitytointegrateorreusethedatainthefuture?Bywhom?

[PLEASEFILLIN]

Whatinformationisneededforthedatatobereadandinterpretedinthefuture?

[PLEASEFILLIN]

Howwillyoucapture/createthisdocumentationandmetadata? [PLEASEFILLIN]

Whatmetadatastandardswillyouuseandwhy? [PLEASEFILLIN]

ResearchDataIdentificationDiscoverable:Isthedataandassociatedsoftwareproducedand/orusedintheproject discoverable (and readily located) and identifiable by means of astandardidentificationmechanism?(e.g.DigitalObjectIdentifier).Accessible: Is the data and associated software produced and/or used in theproject accessible? If yes, in what modalities, scope and license models? (e.g.licencing framework for research and education, embargoperiods, commercialexploitation).Assessable and intelligible: Is the data and associated software producedand/or used in the project assessable for and intelligible to third parties incontexts such as scientific scrutiny and peer review? (e.g. are the minimaldatasetssenttogetherwithscientificpapersforpeerreview,isthedataprovidedinaway that judgementscanbemadeabout reliabilityand thecompetenceofthosewhocreatedthem).Useablebeyondtheoriginalpurposeforwhichitwascollected: Isthedataand associated software produced and/or used in the project usable by thirdparties? If yes, is there a validity for thisuse, or is thedatauseful evena longtimeafteritscollection?(e.g.isthedatasafelystoredincertifiedrepositoriesforlong term preservation and curation; is it stored together with the minimum

D8.2

software,metadataanddocumentation tomake ituseful; is thedatauseful forthewiderpublicneedsandusableforthelikelypurposesofnon-specialists).Interoperable to specific quality standards: Is the data and associatedsoftware produced and/or used in the project interoperable, allowing dataexchange between researchers, institutions, organisations and countries? (e.g.adhering to standards for data annotation, data exchange, compliant withavailable software applications, and allowing re-combinations with differentdatasetsfromdifferentorigins?)Isthedatadiscoverable? [PLEASEFILLIN]Isthedataaccessible? [PLEASEFILLIN]Isthedataassessableandintelligible? [PLEASEFILLIN]Is the data usable beyond the originalpurposeforwhichitwascollected? [PLEASEFILLIN]

Is the data interoperable to specificqualitystandards? [PLEASEFILLIN]

Accessibility–Datasharing,archivingandpreservationDescription of how datawill be shared, including access procedures, embargoperiods (if any), outlines of technical mechanisms for dissemination andnecessarysoftwareandothertoolsforenablingre-use,anddefinitionofwhetheraccesswillbewidelyopenorrestricted tospecificgroups. Identificationof therepositorywheredatawillbestored,ifalreadyexistingandidentified,indicatingthetypeofrepository(institutional,standardrepositoryforthediscipline,etc.)Descriptionoftheproceduresthatwillbeputinplaceforlong-termpreservationof the data. Indication of how long the data should be preserved, what is itsapproximated end volume, what the associated costs are and how these areplannedtobecovered.In case the dataset cannot be shared, the reason for this should bementioned(e.g. ethical, rules of personal data, intellectual property, commercial, privacy-related,security-related).Howwillpotentialusersfindoutaboutyourdata? [PLEASEFILLIN]

With whom will you share the data,andunderwhatconditions? [PLEASEFILLIN]

Will you share data via a repository,handlerequestsdirectlyoruseanothermechanism?

[PLEASEFILLIN]

Whenwillyoumakethedataavailable? [PLEASEFILLIN]Will you pursue getting a persistentidentifierforyourdata? [PLEASEFILLIN]

Whatdatamustberetained/destroyedfor contractual, legal, or regulatorypurposes?

[PLEASEFILLIN]

Howwillyoudecidewhatotherdatato [PLEASEFILLIN]

D8.2

keep?Whataretheforeseeableresearchusesforthedata? [PLEASEFILLIN]

Howlongwillthedataberetainedandpreserved? [PLEASEFILLIN]

Where e.g. in which repository orarchivewillthedatabeheld? [PLEASEFILLIN]

What costs if any will your selecteddatarepositoryorarchivecharge? [PLEASEFILLIN]

Howwillthesecostsbecovered? [PLEASEFILLIN]Have you costed in time and effort toprepare the data forsharing/preservation?

[PLEASEFILLIN]

Howwill thedatabesecure/Whatarethesecurity’smechanismsyouwillusetoprotectthedata?

[PLEASEFILLIN]

IntellectualPropertyRightsHowwillthedatabelicensedforreuse? [PLEASEFILLIN]

Arethereanyrestrictionsonthereuseofthird-partydata? [PLEASEFILLIN]

Willdatasharingbepostponed/restricted?e.g.topublishorseekpatents

[PLEASEFILLIN]

Whenwillthedatabelicensedforreuse? [PLEASEFILLIN]