web content filtering and log data analysis with mikrotik ... · contents 1. the problem 2. content...
TRANSCRIPT
![Page 2: Web content filtering and log data analysis with MikroTik ... · Contents 1. The problem 2. Content filtering a. Methods (L7 for torrents, L7 for DNS, DNS poisoning) b. Pros and Cons](https://reader036.vdocuments.net/reader036/viewer/2022062311/5ecd116a4f13196b5d02d782/html5/thumbnails/2.jpg)
Contents
1. The problem2. Content filtering
a. Methods (L7 for torrents, L7 for DNS, DNS poisoning)b. Pros and Cons
3. Traffic analysisa. Methods (Netflow self managed, Netflow cloud)b. Pros and Cons
4. MikroTik Enforcer Portal by LucidView5. Thank you
![Page 3: Web content filtering and log data analysis with MikroTik ... · Contents 1. The problem 2. Content filtering a. Methods (L7 for torrents, L7 for DNS, DNS poisoning) b. Pros and Cons](https://reader036.vdocuments.net/reader036/viewer/2022062311/5ecd116a4f13196b5d02d782/html5/thumbnails/3.jpg)
The problem - Inappropriate content
![Page 4: Web content filtering and log data analysis with MikroTik ... · Contents 1. The problem 2. Content filtering a. Methods (L7 for torrents, L7 for DNS, DNS poisoning) b. Pros and Cons](https://reader036.vdocuments.net/reader036/viewer/2022062311/5ecd116a4f13196b5d02d782/html5/thumbnails/4.jpg)
The problem - Malware
![Page 5: Web content filtering and log data analysis with MikroTik ... · Contents 1. The problem 2. Content filtering a. Methods (L7 for torrents, L7 for DNS, DNS poisoning) b. Pros and Cons](https://reader036.vdocuments.net/reader036/viewer/2022062311/5ecd116a4f13196b5d02d782/html5/thumbnails/5.jpg)
The problem - Torrent management
![Page 6: Web content filtering and log data analysis with MikroTik ... · Contents 1. The problem 2. Content filtering a. Methods (L7 for torrents, L7 for DNS, DNS poisoning) b. Pros and Cons](https://reader036.vdocuments.net/reader036/viewer/2022062311/5ecd116a4f13196b5d02d782/html5/thumbnails/6.jpg)
Methods - MikroTik L7 Filtering
Layer 7 filtering for torrents on MikroTik RouterOS
L7 regular expression matches
![Page 7: Web content filtering and log data analysis with MikroTik ... · Contents 1. The problem 2. Content filtering a. Methods (L7 for torrents, L7 for DNS, DNS poisoning) b. Pros and Cons](https://reader036.vdocuments.net/reader036/viewer/2022062311/5ecd116a4f13196b5d02d782/html5/thumbnails/7.jpg)
Methods MikroTik L7 filteringFirewall rules for key words in L7 filter on MikroTik RouterOS
What is the problem with the above?
![Page 8: Web content filtering and log data analysis with MikroTik ... · Contents 1. The problem 2. Content filtering a. Methods (L7 for torrents, L7 for DNS, DNS poisoning) b. Pros and Cons](https://reader036.vdocuments.net/reader036/viewer/2022062311/5ecd116a4f13196b5d02d782/html5/thumbnails/8.jpg)
Methods - MikroTik L7 filtering for DNS
L7 DNS filtering on MikroTik RouterOS
Firewall rule to block DNS request
![Page 9: Web content filtering and log data analysis with MikroTik ... · Contents 1. The problem 2. Content filtering a. Methods (L7 for torrents, L7 for DNS, DNS poisoning) b. Pros and Cons](https://reader036.vdocuments.net/reader036/viewer/2022062311/5ecd116a4f13196b5d02d782/html5/thumbnails/9.jpg)
Methods - MikroTik L7 filtering for DNS
Result of L7 DNS filtering on MikroTik RouterOS - test on Ubuntu
Success! Blocked DNS does not resolve.
Other sites resolve successfully.
![Page 10: Web content filtering and log data analysis with MikroTik ... · Contents 1. The problem 2. Content filtering a. Methods (L7 for torrents, L7 for DNS, DNS poisoning) b. Pros and Cons](https://reader036.vdocuments.net/reader036/viewer/2022062311/5ecd116a4f13196b5d02d782/html5/thumbnails/10.jpg)
Pros of Layer 7 filtering on MikroTik RouterOS
● L7 simple to implement and very effective● Can block on keyword, i.e., Regex: xxx, or domain● Can block on payload content or DNS query● Can be done on RouterOS● Somewhat effective against host entries
![Page 11: Web content filtering and log data analysis with MikroTik ... · Contents 1. The problem 2. Content filtering a. Methods (L7 for torrents, L7 for DNS, DNS poisoning) b. Pros and Cons](https://reader036.vdocuments.net/reader036/viewer/2022062311/5ecd116a4f13196b5d02d782/html5/thumbnails/11.jpg)
Cons Layer 7 filtering
● “Almost all P2P traffic is encrypted, thus inspecting the content wouldn't help much.” - benefit of L7 is diminishing with torrents
● SSL - payload is encrypted● Gaming● Skype● Lists maintained on RouterOS● Lists limited by MikroTik resources (can impact small MikroTiks)
![Page 12: Web content filtering and log data analysis with MikroTik ... · Contents 1. The problem 2. Content filtering a. Methods (L7 for torrents, L7 for DNS, DNS poisoning) b. Pros and Cons](https://reader036.vdocuments.net/reader036/viewer/2022062311/5ecd116a4f13196b5d02d782/html5/thumbnails/12.jpg)
Methods DIY DNS poisoning
DIY Linux with Bind, PowerDNS or your favourite flavour.
Commercial or free category list, i.e., University Toulouse
● http://dsi.ut-capitole.fr/blacklists/index_en.php● http://www.squidblacklist.org/, ● http://squidguard.mesd.k12.or.us/blacklists.tgz ● http://www.shallalist.de/ ● http://urlblacklist.com
Implementation of this outside of the scope of RouterOS
Example of categories
![Page 13: Web content filtering and log data analysis with MikroTik ... · Contents 1. The problem 2. Content filtering a. Methods (L7 for torrents, L7 for DNS, DNS poisoning) b. Pros and Cons](https://reader036.vdocuments.net/reader036/viewer/2022062311/5ecd116a4f13196b5d02d782/html5/thumbnails/13.jpg)
DNS request
Poisoned response
Methods DNS poisoning
![Page 14: Web content filtering and log data analysis with MikroTik ... · Contents 1. The problem 2. Content filtering a. Methods (L7 for torrents, L7 for DNS, DNS poisoning) b. Pros and Cons](https://reader036.vdocuments.net/reader036/viewer/2022062311/5ecd116a4f13196b5d02d782/html5/thumbnails/14.jpg)
Methods - Commercial DNS
Example: Safe DNS - Any commercial or free DNS blocking service, OpenDNS (CISCO) etc.
Add IP address of MikroTik to DNS service portal
DNS Server addresses
![Page 15: Web content filtering and log data analysis with MikroTik ... · Contents 1. The problem 2. Content filtering a. Methods (L7 for torrents, L7 for DNS, DNS poisoning) b. Pros and Cons](https://reader036.vdocuments.net/reader036/viewer/2022062311/5ecd116a4f13196b5d02d782/html5/thumbnails/15.jpg)
Methods - Commercial DNS poisoning
Add address of DNS server on MikroTik RouterOS
Intercept all DNS requests and redirect to MikroTik
DNS blocking test on Ubuntu - convenient blocking page
![Page 16: Web content filtering and log data analysis with MikroTik ... · Contents 1. The problem 2. Content filtering a. Methods (L7 for torrents, L7 for DNS, DNS poisoning) b. Pros and Cons](https://reader036.vdocuments.net/reader036/viewer/2022062311/5ecd116a4f13196b5d02d782/html5/thumbnails/16.jpg)
Methods - Commercial DNS poisoningConvenient blocked error page
![Page 17: Web content filtering and log data analysis with MikroTik ... · Contents 1. The problem 2. Content filtering a. Methods (L7 for torrents, L7 for DNS, DNS poisoning) b. Pros and Cons](https://reader036.vdocuments.net/reader036/viewer/2022062311/5ecd116a4f13196b5d02d782/html5/thumbnails/17.jpg)
Pros DNS poisoning
● Commercial DNS offerings - lists maintained by third parties● Self managed DNS servers● Free Blocking lists● Blocking page
![Page 18: Web content filtering and log data analysis with MikroTik ... · Contents 1. The problem 2. Content filtering a. Methods (L7 for torrents, L7 for DNS, DNS poisoning) b. Pros and Cons](https://reader036.vdocuments.net/reader036/viewer/2022062311/5ecd116a4f13196b5d02d782/html5/thumbnails/18.jpg)
Cons DNS poisoning
● New sites not in lists● Some lists are old● Subscriptions expensive
![Page 19: Web content filtering and log data analysis with MikroTik ... · Contents 1. The problem 2. Content filtering a. Methods (L7 for torrents, L7 for DNS, DNS poisoning) b. Pros and Cons](https://reader036.vdocuments.net/reader036/viewer/2022062311/5ecd116a4f13196b5d02d782/html5/thumbnails/19.jpg)
Cons DNS poisoning
● Host entries, i.e.,
● Tor network
![Page 20: Web content filtering and log data analysis with MikroTik ... · Contents 1. The problem 2. Content filtering a. Methods (L7 for torrents, L7 for DNS, DNS poisoning) b. Pros and Cons](https://reader036.vdocuments.net/reader036/viewer/2022062311/5ecd116a4f13196b5d02d782/html5/thumbnails/20.jpg)
Netflow Traffic analysis - The problemGraphical traffic representation
![Page 21: Web content filtering and log data analysis with MikroTik ... · Contents 1. The problem 2. Content filtering a. Methods (L7 for torrents, L7 for DNS, DNS poisoning) b. Pros and Cons](https://reader036.vdocuments.net/reader036/viewer/2022062311/5ecd116a4f13196b5d02d782/html5/thumbnails/21.jpg)
Method - MikroTik Netflow log analysis
Enable Traffic flows on MikroTik RouterOS
![Page 22: Web content filtering and log data analysis with MikroTik ... · Contents 1. The problem 2. Content filtering a. Methods (L7 for torrents, L7 for DNS, DNS poisoning) b. Pros and Cons](https://reader036.vdocuments.net/reader036/viewer/2022062311/5ecd116a4f13196b5d02d782/html5/thumbnails/22.jpg)
Method - Netflow log analysis
Example: nftopng on Ubuntu
Installation of nftopng (free tier) - Ubuntu
$ wget http://apt.ntop.org/18.04/all/apt-ntop.deb
$ sudo dpkg -i apt-ntop.deb
$ sudo apt-get update -y
$ sudo apt-get install nprobe ntopng
![Page 23: Web content filtering and log data analysis with MikroTik ... · Contents 1. The problem 2. Content filtering a. Methods (L7 for torrents, L7 for DNS, DNS poisoning) b. Pros and Cons](https://reader036.vdocuments.net/reader036/viewer/2022062311/5ecd116a4f13196b5d02d782/html5/thumbnails/23.jpg)
Method - Netflow log analysis
Example: nftopng on Ubuntu
$ sudo nprobe -i none -n none -3 9995 --zmq tcp://127.0.0.1:5555
$ mkdir /tmp/ntopng/ ; ntopng -d /tmp/ntopng/ -i tcp://127.0.0.1:5555 -w 8080
![Page 24: Web content filtering and log data analysis with MikroTik ... · Contents 1. The problem 2. Content filtering a. Methods (L7 for torrents, L7 for DNS, DNS poisoning) b. Pros and Cons](https://reader036.vdocuments.net/reader036/viewer/2022062311/5ecd116a4f13196b5d02d782/html5/thumbnails/24.jpg)
Method - Netflow log analysis
![Page 25: Web content filtering and log data analysis with MikroTik ... · Contents 1. The problem 2. Content filtering a. Methods (L7 for torrents, L7 for DNS, DNS poisoning) b. Pros and Cons](https://reader036.vdocuments.net/reader036/viewer/2022062311/5ecd116a4f13196b5d02d782/html5/thumbnails/25.jpg)
Pros
● Visibility ● Own infrastructure● Analysis in house● Configurable● Can be free or subscription
![Page 26: Web content filtering and log data analysis with MikroTik ... · Contents 1. The problem 2. Content filtering a. Methods (L7 for torrents, L7 for DNS, DNS poisoning) b. Pros and Cons](https://reader036.vdocuments.net/reader036/viewer/2022062311/5ecd116a4f13196b5d02d782/html5/thumbnails/26.jpg)
Cons
● Can be expensive● Maintenance of software and hardware● Skilled technical resources● Lots of manual configuration required● Does not scale
![Page 27: Web content filtering and log data analysis with MikroTik ... · Contents 1. The problem 2. Content filtering a. Methods (L7 for torrents, L7 for DNS, DNS poisoning) b. Pros and Cons](https://reader036.vdocuments.net/reader036/viewer/2022062311/5ecd116a4f13196b5d02d782/html5/thumbnails/27.jpg)
LucidView’s MikroTik Enforcer Portal
![Page 28: Web content filtering and log data analysis with MikroTik ... · Contents 1. The problem 2. Content filtering a. Methods (L7 for torrents, L7 for DNS, DNS poisoning) b. Pros and Cons](https://reader036.vdocuments.net/reader036/viewer/2022062311/5ecd116a4f13196b5d02d782/html5/thumbnails/28.jpg)
MikroTik Enforcer Portal
Netflow and DNS in cloud
![Page 29: Web content filtering and log data analysis with MikroTik ... · Contents 1. The problem 2. Content filtering a. Methods (L7 for torrents, L7 for DNS, DNS poisoning) b. Pros and Cons](https://reader036.vdocuments.net/reader036/viewer/2022062311/5ecd116a4f13196b5d02d782/html5/thumbnails/29.jpg)
MikroTik Enforcer Portal - RouterOS scriptsScript generated on MikroTik Enforcer Portal
![Page 30: Web content filtering and log data analysis with MikroTik ... · Contents 1. The problem 2. Content filtering a. Methods (L7 for torrents, L7 for DNS, DNS poisoning) b. Pros and Cons](https://reader036.vdocuments.net/reader036/viewer/2022062311/5ecd116a4f13196b5d02d782/html5/thumbnails/30.jpg)
MikroTik Enforcer Portal - Technical
Features (script to follow)
● VPN to LucidView Cloud for kill lists● Traffic flow to Cloud● DNS via VPN (and syslog)● DNS failover● DNS Intercept● Firewall Kill list● Category filtering● Reporting
![Page 31: Web content filtering and log data analysis with MikroTik ... · Contents 1. The problem 2. Content filtering a. Methods (L7 for torrents, L7 for DNS, DNS poisoning) b. Pros and Cons](https://reader036.vdocuments.net/reader036/viewer/2022062311/5ecd116a4f13196b5d02d782/html5/thumbnails/31.jpg)
MikroTik Enforcer Portal - Content filter
![Page 32: Web content filtering and log data analysis with MikroTik ... · Contents 1. The problem 2. Content filtering a. Methods (L7 for torrents, L7 for DNS, DNS poisoning) b. Pros and Cons](https://reader036.vdocuments.net/reader036/viewer/2022062311/5ecd116a4f13196b5d02d782/html5/thumbnails/32.jpg)
MikroTik Enforcer Portal - Dashboard
![Page 33: Web content filtering and log data analysis with MikroTik ... · Contents 1. The problem 2. Content filtering a. Methods (L7 for torrents, L7 for DNS, DNS poisoning) b. Pros and Cons](https://reader036.vdocuments.net/reader036/viewer/2022062311/5ecd116a4f13196b5d02d782/html5/thumbnails/33.jpg)
MikroTik Enforcer Portal - Dashboard
![Page 34: Web content filtering and log data analysis with MikroTik ... · Contents 1. The problem 2. Content filtering a. Methods (L7 for torrents, L7 for DNS, DNS poisoning) b. Pros and Cons](https://reader036.vdocuments.net/reader036/viewer/2022062311/5ecd116a4f13196b5d02d782/html5/thumbnails/34.jpg)
MikroTik Enforcer Portal - Dashboard
![Page 35: Web content filtering and log data analysis with MikroTik ... · Contents 1. The problem 2. Content filtering a. Methods (L7 for torrents, L7 for DNS, DNS poisoning) b. Pros and Cons](https://reader036.vdocuments.net/reader036/viewer/2022062311/5ecd116a4f13196b5d02d782/html5/thumbnails/35.jpg)
MikroTik Enforcer Portal - Reports
![Page 36: Web content filtering and log data analysis with MikroTik ... · Contents 1. The problem 2. Content filtering a. Methods (L7 for torrents, L7 for DNS, DNS poisoning) b. Pros and Cons](https://reader036.vdocuments.net/reader036/viewer/2022062311/5ecd116a4f13196b5d02d782/html5/thumbnails/36.jpg)
MikroTik Enforcer RouterOS script
Variables
![Page 37: Web content filtering and log data analysis with MikroTik ... · Contents 1. The problem 2. Content filtering a. Methods (L7 for torrents, L7 for DNS, DNS poisoning) b. Pros and Cons](https://reader036.vdocuments.net/reader036/viewer/2022062311/5ecd116a4f13196b5d02d782/html5/thumbnails/37.jpg)
MikroTik Enforcer RouterOS scriptVPN and cloud access
![Page 38: Web content filtering and log data analysis with MikroTik ... · Contents 1. The problem 2. Content filtering a. Methods (L7 for torrents, L7 for DNS, DNS poisoning) b. Pros and Cons](https://reader036.vdocuments.net/reader036/viewer/2022062311/5ecd116a4f13196b5d02d782/html5/thumbnails/38.jpg)
MikroTik Enforcer RouterOS scriptFirewall kill lists and DNS Intercept
![Page 39: Web content filtering and log data analysis with MikroTik ... · Contents 1. The problem 2. Content filtering a. Methods (L7 for torrents, L7 for DNS, DNS poisoning) b. Pros and Cons](https://reader036.vdocuments.net/reader036/viewer/2022062311/5ecd116a4f13196b5d02d782/html5/thumbnails/39.jpg)
MikroTik Enforcer RouterOS script
Log all DNS requests to Syslog server and enable Traffic flow
![Page 40: Web content filtering and log data analysis with MikroTik ... · Contents 1. The problem 2. Content filtering a. Methods (L7 for torrents, L7 for DNS, DNS poisoning) b. Pros and Cons](https://reader036.vdocuments.net/reader036/viewer/2022062311/5ecd116a4f13196b5d02d782/html5/thumbnails/40.jpg)
MikroTik Enforcer RouterOS scriptDNS failover in case of cloud accessibility problem
![Page 41: Web content filtering and log data analysis with MikroTik ... · Contents 1. The problem 2. Content filtering a. Methods (L7 for torrents, L7 for DNS, DNS poisoning) b. Pros and Cons](https://reader036.vdocuments.net/reader036/viewer/2022062311/5ecd116a4f13196b5d02d782/html5/thumbnails/41.jpg)
MikroTik Enforcer Portal Pros
● Scales● Affordable● DNS and firewall blocking● Simple to add (download complete script and modify to suit application)● Detailed reporting● Automated reporting (i.e., security reports to your inbox)● Customised branding● Youtube and Google safe search● Torrent and Suspect blocking● Time based rules
![Page 42: Web content filtering and log data analysis with MikroTik ... · Contents 1. The problem 2. Content filtering a. Methods (L7 for torrents, L7 for DNS, DNS poisoning) b. Pros and Cons](https://reader036.vdocuments.net/reader036/viewer/2022062311/5ecd116a4f13196b5d02d782/html5/thumbnails/42.jpg)
MikroTik Enforcer Portal Cons
Leaves you with too much time on your hands.
www.lucidview.net