SHARKFEST ‘10 | Stanford University | June 14–17, 2010
The Shark Distributed MonitoringSystem: Distributing Wireshark DeepPacket Analysis to LAN/WANInfrastructuresJune 16, 2010
Loris DegioanniCTO | CACE Technologies
SHARKFEST ‘10Stanford UniversityJune 14-17, 2010
SHARKFEST ‘10 | Stanford University | June 14–17, 2010
Quotes from the conference
“The fundamentals are the same, but the scale of network troubleshooting is much bigger today.”
Hansang Bae, Citi
“One of the main reasons why you need to capture is to get the smoking gun.”
Tim Chung, Google
“No matter what, any company needs to capture all its network traffic. It’s the only final proof when something goes wrong. What’s interesting is how the captured data is used.”
Tim Belcher, NetWitness
SHARKFEST ‘10 | Stanford University | June 14–17, 2010
Components
• Probe– 24/7 packet collection– Live metrics– Watches/Alerts– Support for multiple consoles– Ajax control interface– Basic HTML interface for capture settings– Sold as
• Appliance 1G/10G• Kit (1G/10G card + software)• Software only (Q4)
• Console– Windows .net application– Charting– Reporting– Send to wireshark
• Controller (Q4)– Centralized user and license management– Centralized way of pushing views and watches– Centralized alert/event collection and correlation– Automatic reporting– Automatic search for specific IPs/ports/… across multiple probes
SHARKFEST ‘10 | Stanford University | June 14–17, 2010
Appliances
• 8 TB Storage• 3 Gbps sustained to disk
• 4 TB Storage• 1 Gbps sustained to disk
• 16 TB Storage• 7 Gbps sustained to disk
SHARKFEST ‘10 | Stanford University | June 14–17, 2010
Announcing 10 Gbit Appliances and Kit
• Full rate Capture and analysis• 7+ Gbps sustained to disk
SHARKFEST ‘10 | Stanford University | June 14–17, 2010
Collection: Capture jobs
• Capture and rotate the packets coming from the network• Uses a dedicated file system on the RAID array• Can run multiple capture ports at the same time.
– E.g. Different capture ports
• A single capture job can capture from multiple ports
SHARKFEST ‘10 | Stanford University | June 14–17, 2010
Now what?
I can capture a lot of packets. Now what?
SHARKFEST ‘10 | Stanford University | June 14–17, 2010
Indexing
• Writing packets uses a lot of the disk bandwidth• Even if I stop capturing, since I was writing at full speed,
reading the data is going to take around the same time of writing it– Read needs to be localized– I need high level visibility to reach the point I need
SHARKFEST ‘10 | Stanford University | June 14–17, 2010
Indexing
• Happens while capturing, you can enable it when you create a capture job
• Contains summary of the network traffic– Volume, talkers and protocol information– “Netflow on steroids”
• Designed to be extremely efficient in terms of disk usage• Coordinated with the packet store
SHARKFEST ‘10 | Stanford University | June 14–17, 2010
Packet capture and indexing architecture
Capture threadMoves packets from the live interface into the memory buffer.
Memory buffer
Write-to-disk threadMoves packets from the memory buffer into the packet storage system. Create time indices for fast packet retrieval.
RAID level 0Packet Storage
System
Live network traffic
Content Indexing ThreadCaptures packets off the live interface and creates an index that speeds up “index-friendly” Views.
Index Storage
Pilot Live Views ThreadsPerform live network analysis, including trending and alerting.
View SQL DB
Pilot Offline Views ThreadsPerform retrospective analysis on stored packets.
View SQL DB
Pilot Indexed Views ThreadsReturn summary information about terabytes of traffic in a matter of seconds.
View SQL DB