dnsflow- enabling netflow- like telemetry for dns · intro • an analogy to (gasp) telecom...
TRANSCRIPT
DNSFlow- Enabling netflow-like telemetry for DNS
Kyle Creyts, Manish Karir Merit Network Inc.
Joe Eggleston, Craig Labovitz
DeepField Networks
Intro • An analogy to (gasp) telecom networks:
• Data plane – scalable telemetry via Netflow – Operationally incredibly useful – Embedded everywhere – Well-developed analysis ecosystem
• Control (signaling) plane – DNS – Potential for broad operational visibility (beyond dns
system itself) – Embedded nowhere (some syslog support) – Nascent analysis ecosystem
• DNS + Netflow… – Lends DNS-based context to analysis of data trafffic
DNSFlow
• DNSFlow – protocol/reference implementation for efficient export of DNS responses
• Primary operational insight from recursive responses – Also compelling security applications for iterative
responses (see ISC Passive DNS)
• Goal – Open access to data – Vendor-neutral, open, standards-based, cheap
(free), efficient, lightweight, flexible deployment
Challenges
0
5,000
10,000
15,000
20,000
New responses/hour
• Scale of data • But, most queries
redundant – 1:20 compression ratio
possible – Even more with smart
sampling
Status
• Open source, version 1 – Simple (hundreds of lines of ANSI C) – Flow like output – NMSG (Passive DNS) interoperability layer
• Upcoming version IPFIX – Easy integration with existing tools.
• Support for compression, sampling and built-in algorithms identifying redundant queries
• Deployed in several production networks today
• https://github.com/deepfield/dnsflow
Deployment
• Runs on any *nix. • Anywhere with
visibility of recursive responses. – Recursive resolver – SPAN/mirror port of
resolver • Discussions towards
embedding in open source and commercial DNS Servers
Recursive DNS
Data Traffic
Users
DNSServer
DNSFlow from tap or
server plugin
Collection/Analysis
Commodity PCor VM
Iterative DNS
Using DNSFlow Data – Phishing
• Notification of phishing activity – used in conjunction with DNS blacklists such as
phishtank, malwaredomains.com, or SURBL – when a user is successfully phished (opens a link) with a
domain in the blacklist, it will incur a DNS lookup – that DNS answer will appear in DNSflow – can query DNSflow data, see which users (client IPs) were
phished, and which domain(s)-IP pair they were victim to
alert_id set_id IP f/ response
client IP
timestamp BL source name from DNS
7907823 4819173 xxx.235.133.xxx
10.1.1.2 2012-04-09 00:57:57.379013
malwaredomains www.sexy-screen-savers.com.
7924562 4829607 xxx.233.142.xxx
10.1.1.1 2012-04-09 05:45:08.815553
malwaredomains www.meb.gov.tr.
Building DNSMapDB • Goal: To characterize the DNS-IP mappings as observed at a regional ISP • Two different types of entries in DNSMapDB – largely-static and largely-
dynamic • After first 7 days we continue to learn new pairs but at a constant rate of
roughly 1000pairs/hour
0
2,000
4,000
6,000
8,000
10,000
12,000
14,000
16,000
18,000
20,000
New pairs/hour (7 days)
Characterizing DNSMAPdb
• There are two different contributors to DNSMapDB – Mostly static infrastructure – Largely dynamic mappings
which originate from CDNs and massive web hosting – dynamic infrastructure
• Shortest name 2 character and longest 235, but average is 27 characters.
• Total of 308 unique TLDs • Top 5 TLDs account for
almost 90% of the DNS names observed
com, 62% net, 15%
org, 7%
edu, 2%
us, 2%
Second level domain distribution • How big is a given second level domain – most have very few
name-IP mappings < 60 second level domains have more 10 or more mappings
• Looking at the other end of the spectrum very few have large number of entries but how big do these get? – Atleast 40 have more than 1000 entries each - top 8 have 10K or more each
1
10
100
1,000
10,000
100,000
1 10 100 1,000 10,000 100,000
2nd
-Lev
el D
om
ain
s
Name/IP Pairs
IP Address distribution by /8
• Distribution of 200K unique IPs in DNSMapDB by slash8
50
64
173 174
184 208
216
0
1000
2000
3000
4000
5000
6000
7000
8000
9000
10000
0 8 14
19
31
39
46
54
59
64
69
75
80
85
90
95
100
107
112
117
122
127
132
137
142
147
152
157
162
167
172
177
183
189
195
200
205
210
215
220
230
Slash 8
Nu
mb
er o
f IPs
in D
NSM
APd
b *.cloudfront.net
*.amazonaws.com *.tumblr.com
DNSMapDB Entries by /8
17
66
74
127
192
205
209
216
0
10000
20000
30000
40000
50000
60000
70000
80000
90000
0 5 10
15
20
25
30
35
40
45
50
55
60
65
70
75
80
85
90
95
100
105
110
115
120
125
130
135
140
145
150
155
160
165
170
175
180
185
190
195
200
205
210
215
220
225
230
235
240
245
250
255
Cloudfront – 52K
Blogspot – 12K Google – 6K Google – 27K
2o7 – 9K
Cloudfront – 12K
Mcafee – 13K Spamhaus – 9K Mail-abuse – 3K Edgesuite – 4K
Akamai – 4K Apple – 12K Akadns – 10K
• Primary contributors to DNSMapDB entries
