presentation title presentation date - wireshark · •worked example –get into some packets ......

SharkFest'17 US • Carnegie Mellon University • June 19-22, 2017

SharkFest'17 USWorkflow-based Analysis of Wireshark Traces

Paul OffordProject Leader, TribeLab

Now we can all be experts


Agenda

• Recurring Gray Problems

• Principles and objectives of our approach

• Performance problem scenarios

• Performance analysis concepts

• Analysis strategy

• Worked example – get into some packets

• Q&A


The Issue

Recurring

Gray

Problem

It keeps happening

The causing technology is unknown

PerformanceError

Incorrect output

“It must be

the network”


The Challenge

Recurring

Gray

Problem

Ongoing

Problem

• Static cause

• Recreate

• Short capture

• Easy analysis

• Transient cause

• Can’t recreate

• Long-term capture

• Challenging analysis


Analysis Principles

We look at one symptom at a time

We study a specific instance of that symptom

We capture live problems

Produce irrefutable evidence of the cause


Analysis Objective

WAN Server

Server

Server

AA

PCWAN

Accel

WAN

Accel

A

A

1.0s

0.2s 0.2s0.3s 0.1s

0.4s

12.8s

User

experiences

15s

response time

APacket capture units

(dumpcap, NetShark, etc.)

The problem

is here


Incre

asin

g T

ime o

f Day

Process-to-Process Communication

Web ServerBrowser

Network

Application

Request

Application

Response

Serv

ice

Tim

e

AP

DU

Response

Tim

eNB: Application

messages

(APDUs) not

packets


Client-Service Chains

Web Server

Fileserver

Browser

NetworkC CS

S

DatabaseC

S


Slow Response – Scenario 1

Web ServerBrowser

Network

Database

9.8 s

9.3 s

10 s

APDU Req

APDU Rsp

APDU Req

APDU Rsp

APDU Req

APDU Rsp

Mouse Click

Page

Rendered

0.1 s 0.4 s 9.4 s

0.1 s

9.9 s

0.1 s



Web ServerBrowser

Network

Database

9.8 s10 s

APDU Req

APDU Rsp

APDU Req

APDU Rsp

APDU Req

APDU Rsp

Mouse Click

Page

Rendered

0.1 s 9.6 s 0.2 s

0.1 s

9.9 s

0.1 s

0.1 s


Slow Response – Scenario 2b

Web ServerBrowser

NetworkData

base

9.8 s 9.2 s10 s

APDU Req

APDU Rsp

APDU Req

APDU Rsp

APDU Req

APDU Rsp

Mouse Click

Page

Rendered

0.1 s 0.4 s

0.2 s

0.1 s

9.9 s

0.1 s

0.1 s

APDU Req

APDU Rsp

Web

Svc

9.2 s



Web ServerBrowser

Network

Database

0.6 s

10 s

APDU Req

APDU Rsp

APDU Req

APDU Rsp

APDU Req

APDU Rsp

Mouse Click

Page

Rendered

9.3 s 0.2 s 0.4 s

0.1 s

0.7 s

0.1 s

0.1 s



Web ServerBrowser

Network

Database

0.6 s

10 s

APDU Req

APDU Rsp

APDU Req

APDU Rsp

APDU Req

APDU Rsp

Mouse Click

Page

Rendered

0.1 s 0.2 s 0.4 s

0.1 s

9.9 s

9.3 s

0.1 s

Retransmissions


Multi-tier Correlation


APDU vs. PacketsRequest APDU

Response APDU

tcp.port==80 && tcp.len>0


Response Time Element (RTE) Model

Web ServerBrowser

Network

Database

Mouse Click

Page

Rendered

ClientTime Service Time

Request Spread

Response Spread

APDU Response Time

APDU Req – Part aAPDU Req – Part b

APDU Req – Part cAPDU Req – Part d

APDU Rsp – Part αAPDU Rsp – Part β

APDU Rsp – Part γ

Data Packetstcp.len>0


TRANSUM and RTE

• TRANSUM is a Wireshark plugin

• Standard in Wireshark 2.4 (mid-late July)

• TribeLab Plugin for Wireshark 2.2


RTE Data

TRANSUM

adds a new

decode section


RTE Summary

• APDU Response Time• Client perspective

• Big number = There‘s a problem

• Service Time• Service perspective

• Big number = There‘s a problem with the service

• Spread Time• Network perspective

• Big number = There‘s a problem with the network


Simple Analysis Strategy

• No high APDU Response Times• Problem in the client program

• High Service Time• Confirm with matching traces

• Translate data into something meaningful to service owner

• Hit the service owner with the facts

• High Spread Time• Focus in the problem time frame

• Use techniques a la Betty, Chris, Hansang, Jasper, Kary, Laura, etc.


Before we continue…

Questions?


Introducing

Workbench


Workbench

•Any data

•Any tool

•Expert guidance

• In dev - Free trial

•FOC Community

Edition

•Large volume

filtering

•Marker finder

•Transformation

•Packet matching

•Side-by-side

analysis


Workbench

Main Features


Workflows

• Model work practices

• Adds context

• Adapts


Sharing Workflows

Library

Community

TribeLab

3rd Parties

• Private

• Private and shared with colleagues

• Public

In

Development


Now for the Main Course

Use Case


Video

The Symptom


Configuration

Database

Browser

IE

MS-SQL

Network App/Web

Server

192.168.10.81

192.168.3.78

IIS

File ServerSMB2

192.168.3.79


And now…

Analyze the Data


tcp.len = 588 bytes

tcp.len = 1,460 bytes

tcp.len = 5,120

TCP Segmentation Offload (TSO)

PC or

Server

Process

EthernetSwitch PC or Server

NIC

TCP

IPEth




tcp.len = 740 bytes


tcp.len = 588 bytes

TCP

IPDrv

Without

Large Receive Offload

(LRO)


Summary• Recurring gray problems – “It’s the network”

• Analyzing the packet volumes is challenging

• Markers help a lot – see Matthew York, SF16

• Most applications - Request-Response pattern

• TRANSUM allows top-down performance analysis

• Workflow approach - fast, accurate & consistent

• Workbench platform allows sharing of workflows


Contact

• Paul Offord

• P +44 (0) 1279 211 668

• E [email protected]

• W www.tribelab.com

• T @paulofforda7

Contact

mailto:[email protected]

http://www.tribelab.com/

presentation title presentation date - wireshark · •worked example –get into some packets ......

Documents