netflow sflow proxy collection
DESCRIPTION
With the proliferation of DMZ’s (so-called De-Militarized Zones or firewall protected areas) and extranets today, network managers are increasingly faced with the problem of collecting data from NetFlow and sFlow enabled equipment when security policy prevents UDP to cross the firewall from these segregated areas. This paper discusses three solutions for NetFlow and sFlow collection from the DMZ: 1) add collectors into DMZ, 2) use a separate network management network, or 3) add proxy collectors into DMZ. This paper discusses the problem the security policy creates and three solutions for this problem.TRANSCRIPT
NetFlow, sFlow Proxy Collection for DMZ
Anthony V. Edwards Tavve Software Co.
Tavve Software Co. One Copley Plaza
Suite 480 Morrisville, NC 27560
+1 919-460-1789 www.tavve.com
Copyright © 2005 Tavve Software Co. 2
NetFlow, sFlow Proxy Collection for DMZ
Anthony V. Edwards Tavve Software Co.
Executive Summary With the proliferation of DMZ’s (so-called De-Militarized Zones or firewall protected areas) and extranets today, network managers are increasingly faced with the problem of collecting data from NetFlow and sFlow enabled equipment when security policy prevents UDP to cross the firewall from these segregated areas. This paper discusses three solutions for NetFlow and sFlow collection from the DMZ: 1) add collectors into DMZ, 2) use a separate network management network, or 3) add proxy collectors into DMZ. This paper discusses the problem the security policy creates and three solutions for this problem. NetFlow vs. sFlow vs. IPFIX Please note that a discussion of NetFlow and sFlow is outside the scope of this white paper. However, resources are listed at the end of this paper for your further reading. Briefly, NetFlow is a protocol for packets output by routers containing statistical data of particular network traffic flow. SFlow is similar, but frequently found in switches. IPFIX is a recent open standard based on NetFlow. All three of these protocols are UDP based, unidirectionally communicated from a router or switch to a server known as a collector. The Collector Model sFlow.org summarizes the usefulness of sFlow stating, “By statistically sampling network traffic, network administrators gain a system-wide view of the traffic, network security and application traffic sources throughout the network.”2 Whether you are collecting from NetFlow enabled products from Juniper, Cisco Systems, NetScout, Hewlett-Packard, or others; or whether you are collecting from sFlow enabled products from Extreme Networks, Force10, Foundry, or others; the collection model is the same: applications and users generate traffic through networking equipment which is converted to data flows to collectors to a server to end-user applications. Cisco Systems diagrams this process as shown in Figure 1. The NetFlow and sFlow packets are the result of packet sampling. The amount of NetFlow and sFlow traffic varies, according to site settings. For a flow, the agent may cache a session of data, aggregating before transmitting, delaying as much as 30 minutes. The amount of NetFlow or sFlow traffic to the collector depends on network activity and amount of caching. The collector stores all the data into a database, which later synchronizes with the server. Once the data is on the server, it is available to end-user applications which includes network planning, accounting/billing, traffic profiling, application profiling, user profiling, analysis, traffic visualization, and more.
Copyright © 2005 Tavve Software Co. 3
A variety of applications are available today to process collected NetFlow or sFlow data from the hardware vendors and from NetScout, Hewlett-Packard, Hitachi, InMon, QoSmetrics, and others. Many network managers depend upon this data to properly manage their networks. Those managers depend upon the information from all locations in the network, even remote or off-limits networks such as DMZ subnets, extranets, or remote sites. When Security Becomes an Obstacle NetFlow and sFlow data can be collected as long as the network path allows the UDP traffic flow. Increasingly, security policies are prohibiting the passing of UDP traffic.
Figure 1. NetFlow Infrastructure1
Enabled Device
nGenius
Firewall
Collector
No NetFlow / sFlow Data
agent
NetFlow / sFlow Data
X
X
Figure 2. The Firewall Blocks NetFlow & sFlow data.
Copyright © 2005 Tavve Software Co. 4
When the firewall will no longer pass UDP traffic, the sFlow and NetFlow traffic is blocked as well. The firewall becomes an obstacle, preventing this data from reaching the collector. For example, see Figure 2 depicting a NetFlow enabled device failing to transmit NetFlow to its nGenius collector. Clearly, if the firewall blocks NetFlow or sFlow traffic from the Collector, it is time to re-architect this part of the network management design. In the remainder of this paper, we will look at three possible solutions: (1) adding a collector on the DMZ side, (2) using a separate management network, or (3) using a proxy collector. Solution 1: Add More Collectors A solution to the security policy issue of no UDP through the firewall is to deploy collectors in the DMZ. Instead of the NetFlow or sFlow UDP traffic going through the firewall, it instead goes to the DMZ collector. The collector then communicates to the NetFlow or sFlow server using TCP through the firewall to update the database on the server. This requires at least one firewall rule per collector to allow the collector to update the database on the server. See Figure 3 below. To implement this solution, you will need a collector in every isolated DMZ that contains NetFlow or sFlow enabled devices, even if that DMZ only contains a few devices. For each collector, you need to
1. Acquire and install a computer as the hardware for the collector 2. Purchase and install the collector software 3. Add a firewall rule (or more) so the collector can communicate with the server
Enabled Device
nGeniusServer
Firewall
Collector
Database synch/update
agent
NetFlow / sFlow Data
Figure 3. A Collector per DMZ Design
nGenius CollectorDMZ#1
Server
Admin
agent
nGenius CollectorDMZ#2
Copyright © 2005 Tavve Software Co. 5
4. Add a firewall rule (or more) so the system and database administrator can access the collector system
5. Configure the NetFlow or sFlow devices to use the new collector The advantage of this solution is:
1. That it requires no more expertise to setup collection in a DMZ than ordinarily needed in the enterprise.
The disadvantages of this solution are:
1. The extra cost required for multiple collectors 2. The additional hardware cost involved with those new collectors 3. The added labor cost involved with OS, software, and database setup 4. The delays or justifications with security to negotiate all those new firewall rules 5. The ongoing labor cost involved with OS, software, and database maintenance
Solution 2: Use Separate Management Network Another solution to the security policy issue of no UDP through the firewall is to deploy a parallel network management network. All devices generating network management traffic need an interface dedicated to a network management traffic only subnet. The collector is deployed as dual-homed with one interface on the corporate network and one interface on the network management network. NetFlow and sFlow traffic are carried over the network management network to the collector. The collector communicates to the server over the corporate network.
X
Figure 4. A separate network management subnet bypasses firewall
Enabled Device
nGeniusCollector
Firewall
Collector
NetFlow data
agent
NetFlow data
nGeniusServer
Server
Database synch/update
agent
NetFlow data
Net Mgmt Subnet
X X
Admin
Copyright © 2005 Tavve Software Co. 6
To implement this solution, you will need a dedicated interface per device: 1. Dedicate an Ethernet interface on each router to the network management
network 2. Connect your switch’s network management LAN port to the network
management network. If your switch does not have a dedicated management port, then it cannot be part of this solution.
3. Add an extra network interface to the collector host 4. Configure all these dedicated interfaces to be on the same subnet
The advantages of this solution are:
1. This is a very low cost approach, presuming the dedicated interface cost is low. 2. There is no need for a dedicated collector 3. No proxy software or appliance is needed
The disadvantages of this solution are:
1. It may be difficult to get approval from your security group. Security groups consider the scenario where a DMZ device becomes compromised. In such a scenario, security may be concerned that attacking traffic may infiltrate the network management network. In turn, the attacking traffic could infiltrate the collector host before ultimately attacking the corporate network. Such an attack would circumvent the protection of the firewall, which is undesired.
2. Some network equipment (such as switches) do not have a dedicated network management interface, rendering this solution incomplete.
Solution 3: Add a Proxy Collector Another solution to the security policy issue of no UDP through the firewall is to deploy proxy collectors in the DMZ. Instead of the NetFlow or sFlow UDP traffic going through the firewall, it instead goes to the proxy collector. The proxy collector then communicates to proxy software, which can be co-located with the collector. The proxy software generates the NetFlow or sFlow data as UDP for the collector. This requires only one firewall rule to allow the proxy collector to forward the data. One such proxy collector is Tavve’s ZoneRanger network management appliance. It fits into the solution as shown in Figure 5 below. To implement this solution, you will need a proxy collector in every isolated DMZ that contains NetFlow or sFlow enabled devices. Several proxy collectors can report their data to the same collector. For each proxy collector, you need to
1. Acquire and install a proxy collector appliance (ZoneRanger) 2. Add one firewall rule so the proxy collector can communicate with the collector 3. Configure the NetFlow or sFlow devices to use the new collector
Copyright © 2005 Tavve Software Co. 7
If the proxy collector is an appliance, such as ZoneRanger, you do not need to configure rules for system administrator or database administrator access through the firewall to it. The one firewall rule that allows the ZoneRanger to communicate with the complementary Ranger Gateway software also allows configuration by web browsing to a Ranger Gateway port. The advantages of this solution are:
1. The ZoneRanger appliance does not require system or database administrators to setup or maintain – the network management support team configures it
2. No more additional collectors required than usual 3. Only one firewall rule (per proxy collector) needed in firewall (easier to negotiate
with security) 4. Aside from proxying NetFlow or sFlow data, it proxies other network
management traffic (SNMP, traps, syslogs)
Figure 5. Proxy Collectors Sharing One Collector Design
Enabled Device
nGeniusCollector
Firewall
Collector
Proxy Software
Ranger Gateway
NetFlow/sFlow data
ZoneRangerDMZ#1
agent
Proxy Collector
NetFlow/sFlow data
nGeniusServer
Server
Database synch/update
ZoneRanger DMZ#2
agent
NetFlow/sFlow data
Admin
Copyright © 2005 Tavve Software Co. 8
The disadvantage of this solution is: 1. It is a new tool, so some learning is required
Conclusion When the security policy of your enterprise prevents you from sending your NetFlow or sFlow data to your enterprise-side collector, you can choose one of three solutions: (a) add a collector into each isolated DMZ, (b) use a separate network management network, or (c) add a proxy collector into each isolated DMZ. Depending upon product and ongoing administration costs, one solution may be more favorable than the other for your enterprise. References / Further Reading
1. "NetFlow Services and Applications" White Paper by Cisco Systems, Inc. Copyright © 1992--2002 Cisco Systems, Inc. http://www.cisco.com/warp/public/cc/pd/iosw/ioft/neflct/tech/napps_wp.htm
2. "Traffic Monitoring using sFlow" by sFlow.org. Copyright © 2003 sFlow.org http://www.sFlow.org/sFlowOverview.pdf
3. "Juniper Networks Solutions for Network Accounting" White Paper by Chuck Semeria and Hannes Gredler. Copyright © 2001 Juniper Networks. http://www.juniper.net/solutions/literature/white_papers/200010.pdf
4. "IPFIX fine-tines traffic analysis" article from 8/11/03 NetworkWorldFusion by Paul Kohler and Benoit Claise. Copyright © 2003 Network World, Inc. http://www.nwfusion.com/news/tech/2003/0811techupdate.html with diagram http://www.nwfusion.com/graphics/2003/0811tu.gif
5. "Resource Links / Encyclopedia / S / sFlow" from NetworkWorldFusion Resource Encyclopedia. Copyright © 1994-2005 Network World, Inc. http://www.nwfusion.com/details/6416.html?def
6. "Using sFlow" by sFlow.org Copyright © 2003-2004 sFlow.org. http://sflow.org/using_sflow/index.php