facing the challenge of windows logs collection to ...€¦ · wef (windows event forwarding)...

© RadarServices // Classification: Public

.

Facing the challenge(s) of Windows logs collection to leverage valuable IOCs

Michel de Crevoisier

Security Analyst, Radar Cyber Security

15.10.2019, Berne


The five challenges


#1 High diversity of log sources

3

Built-in

Application

PowerShell

Security

System

[…]

Server roles

ADFS

Certification authority

DHCP server

DNS server

IIS web server

NPS Radius

Microsoft software

Advanced Threat Analytics (ATA)

Exchange

Skype

SQL Server

SYSMON

Defender

3rd party software

Ivanti software

Kaspersky

Veeam Backup

[…]


#2 Different log extensions

EVTX(standard Windows logs

in XML format)

ETL(analytical logs, like DNS

Server or PowerShell)

TXT(IIS, NPS, DHCP,

PowerShell Transcript, former DNS logs)

4


#3 Multiple architectural approaches

Access method / Protocol (MS-EVEN6, RPC, WMI,…)

Push vs Pull

Agent vs Agentless

Intermediate collector VS Direct sending to receiver

Central file store vs Shared folder

Managed agent VS Unmanaged agent

5


#4 Disabled and restrictive event logs

6

Valuable event logs disabled

• Protected users (if configured, on DCs only)

• LSA (Local Security Authority)

• IIS web server

• DNS client

Event logs with restrictive

access

• SMB server

• SMB client

• IIS web server

6


#5 Operational constraints

Security

• Avoid usage of high privileges

• Isolation between customer and security provider

Data exchange

• Data encryption

• Secured authentication method

Performance

• High availability

• Compression

Configuration

• Easy deployment

• Minimize configuration changes

• Low impact on operating system

Environment

• Cloud

• Domain VS Workgroup

• OT (Operational Technology)

7


Collecting standard Windows logs

© RadarServices // Classification: Public 9

WEF (Windows Event Forwarding)

Authentication and encryption through Kerberos in a domain or TLS certificates in a Workgroup

Data exchange over WinRM (push or pull)

XML-based language to control event IDs to collect or to suppress noisy events

Settings control over GPO

EPS control rate

WEC (Windows Event Collector)

Collects and stores all requested events from WEF clients according XML subscriptions

High availability capacities where clients send events to each WEC collector

Certain 3rd party software can also:

Emulate a WEC server by spoofing a WinRM listener (e.g.: SYSLOG-NG Premium, NXLog Enterprise, AlienVault USM > actually uses NXLog)

Manage multiple WEC servers with a central management console (e.g.: SuperCharger from Logbinder)

Unified & built-in solution to collect standard Windows logs

WEF/WEC introduction


HP/ArcSight, Australian Cyber Security, …

Who is publishing about WEF/WEC?

2013

2015

2017 & 2019

2017

10


Technical characteristics

Up to 4.000 source clients per collector (source: Microsoft)

Average logging is 5.000 EPS, can go up to 10.000 EPS (source: Microsoft)

Maximum recommended size per event log file: 4GB

Maximum recommended size for all Windows logs files: 16GB

Compression possible with event log size reduction

Limitations

All collected events are saved in Forwarded Events log file

All events are mixed without any tagging possibilities

Only standard event logs (EVTX) can be forwarded

Scaling out

WEF/WEC performance


The Palantir approach to the rescue

WEF/WEC advanced approach

• Different size and rotation strategy

• Channel can be tagged for SIEM ingestion

• Channel can be placed on different storage for better performance

Multiple event channels

• XML query to specify the events to collect

• Specify the event channel destination

Preconfigured subscriptions

12


A look in production on a WEC server

WEF/WEC advanced approach

SubscriptionsEvent channels

Deployment is not automatized

Requires several manual actions

Potential source of incorrect

configuration

13


PowerShell at the rescue

WEF/WEC deployment enhancement

Automated WEC server role setup

Automated Palantir toolset

deployment

Covers event channel and subscriptions

Adjusts log file size and location

Fixes SDDL permissions on WinRM service

Available on GitHub

https://github.com/rs-dev/windows-event-collector_auto-deploy

14


ArcSight agent

NXLog agent Community

RSYSLOG agent

Snare agent

Splunk UF agent

WinCollect agent

Winlogbeat agent

Injecting data with agent from the WEC server to your SIEM

WEF/WEC

15

Chosen agent software solution

Source clients WEC collector SIEM

Other target / External provider

JSON

CEF

Other target / External provider

/ Archiving solution


NXLog agent Enterprise

SYSLOG-NG Premium

Certificates are required

on each source client !

Injecting data without agent from the WEC server to your SIEM

WEF/WEC

16

Chosen software for WinRM server

listener emulation

Certificates pushed on hosts

SIEMSource clients


Collecting Windows DNS transaction logs


Technical possibilities overview

Collecting DNS transaction logs

DNS transactions

logs

Windows OS

DNS server logs

DNS debugging

ETW ETL

DNS client logs

DNS Event log

SYSMON (ID 22)

Linux/Unix OS

Bind, Unbound,

Dnsmasq, …

Passive DNS

Firewall or 3rd

party solutionNIDS solution

Mirrored traffic

Server 2012 R2

1 2 3

18

Disabled


Old school approach with Debugging DNS logs


Very simple access

High impact on performance

Only for debugging purpose

Not supported by MS for production

Does not include DNS

answer

Timestamp structure may

change

Delay before data is written

(>1min)No event ID

1

19


Event Tracing for Windows

Efficient kernel-level tracing facility that allows to save kernel or application-defined events

Allows to dynamically enable or disable logging in real time without any restart of the system

Great open source projects available:

About ETW

Performant C++ library to interact with ETW (https://github.com/Microsoft/krabsetw)

KrabsETW (Microsoft)

PowerShell module built around the KrabsETW APIs (https://github.com/zacbrown/PowerKrabsEtw)

PowerKrabsEtw

Splunk plugin to collect DNS events from ETW using "KrabsETW" (https://github.com/secops4thewin/TA-DNSETW)

TA-DNSETW

Flexible C# ETW wrapper running as a service - Blackhat 19 (https://github.com/fireeye/SilkETW)

SilkETW (FireEye)

Windows agent provided with a native ETW module (im_etw). Logs can be saved in a file and/or sent to a remote target

NXLog Community

2


Advanced approach with native ETW


System tools:

•Built-in: Logman, Perfmon, Netsh

• Installable: Xperf, Tracelog, NetMon, Microsoft MMA, Tracelogging

Splunk

•App “TA-DNSETW”: read ETW using the KrabsETW library from Microsoft

NXLog Community

•Built-in module to read and forward ETW logs

Solutions for production

Low impact on performance

Event ID provided

DNS answer is provided

(but encoded)

Not compatible with WEC

Requires agent or script

installationNo cache file

2

21


Event Tracing Logs

ETW trace session are saved into ETL log files

ETL files can be placed on a shared folder on each DNS server to be read remotely

Great open source tools available:

About ETL

PowerShell script that reads ETL logs and writes them into Windows Event Viewer (https://github.com/acalarch/ETL-to-EVTX)

ETL-to-EVTX

Executable which can decodes several types of ETL files (https://github.com/gcpartners/ETLParser)

ETLParser (GCPartners)

Python script that parses DNS ETL files (https://github.com/nerdiosity/DNSplice)

DNSplice

PowerShell script for Splunk UF that reads ETL logs (https://splunkbase.splunk.com/app/2937)

DNS Analytical App (Splunk)

Windows agent provided with a native ETL module. Logs can be saved in a file and/or sent to a remote target

NXLog Community

Read ETL file and convert it to JSON(https://github.com/microsoft/ETW2JSON)

ETW2JSON (Microsoft)

3


Advanced approach with ETL


Low impact on performance

Event ID provided

ETL file can be placed in a

shared folder

DNS answer is provided (but

encoded)

Not compatible with WEC per

default (*)

System tools:

•Built-in: Tracerpt

• Installable: Microsoft Message Analyzer (MMA)

Splunk

•App “DNS analytical”: PowerShell script that extracts ETL logs and send it to a remote listener

NXLog Community

•Built-in module to read and forward ETL logs (**)

Solutions for production

3

23

*ETL-to-EVTX script can convert ETL logs to EVTX log file **Currently in preview. Will be fully released in NXLog agent v5

according NXLog support


Steps and solutions overview


Overview of collecting methods

1: requires PowerShell script that extracts ETL content into EVTX log files

2: requires agent or plugin with ETL or ETW capacities

3: data in event log has no structure

4: not recommended, requires to query SCCM SQL Server database

5: requires SQL Server advanced configuration

6: pulling requires dealing with firewall, credentials and double NAT issues

7: only a limited set of logs are available. Per default, format and mapping are not

maintained. SCOM is not a SIEM.


Steps for a proper log collection

Download Palantir toolset

• https://github.com/palantir/windows-event-forwarding

Download and run the Radar deployment script

• https://github.com/rs-dev/windows-event-collector_auto-deploy

Configure clients to target your WEC server(s)

Install and configure your agent solution on your WEC server(s) to forward logs to your SIEM

Start gathering data in your SIEM

Configure advanced audit

policies

Enable PowerShell

auditing

Enable auditing for permission

changes (SACL)


.

Thank You

facing the challenge of windows logs collection to ...€¦ · wef (windows event forwarding)...

Documents