September 2008 - 1

RTC-MonRTC-MonEnabling High-Speed and Extensible Real-Time

Communications Monitoring

Diego Costantini, Felipe Huici[costantini|huici]@nw.neclab.eu

September 2008 - 2

Motivation

• Internet as medium for real-time communications like VoIP has grown significantly over the past years

• However best-effort model not well-suited to these sort of applications

• Need real-time monitoring in order to – ensure good service quality– take corrective action should service degradation occur

September 2008 - 3

Monitoring Difficulties

• Real-time monitoring is difficult– System often has to correlate signaling and media streams– Connections tend to use dynamic ports– Monitoring in real-time results in non-trivial performance

requirements In the case of VoIP, the abundance of small packets exacerbates

this problem

September 2008 - 4

State of the Art

• Current software solutions insufficient– Limited monitoring capabilities, not easy to extend– Long development time– Not efficient enough to track traffic in real-time

• Hardware solutions are inflexible and hard to extend• Need a general monitoring framework that

– provides basic VoIP monitoring capabilities in order to reduce development time

– is easily extensible– is efficient enough to cope with demands of real-time traffic

monitoring

September 2008 - 5

RTC-Mon: The Real-Time Communications Monitor Framework

•Implemented over Linux•Consists of:

•Extended PF_RING with plugin architecture•SIP and RTP plugins•Event-based user-level library

•Also implemented VoIP Console•Proof-of-concept monitoring application•Small, quick to develop (800 lines of code)•Can track numerous VoIP statistics

extendedPF_RING

SIPplugin

RTPplugin

libpfring

libvoip

customplugin

VoIPconsole

customapplication

network interfaces

RT

C-M

ON

FR

AM

EW

OR

KA

PP

S

US

ER

SP

AC

EK

ER

NE

L S

PA

CE

September 2008 - 6

Extended PF_RING

• PF_RING– Linux kernel module providing new type of socket– Yields high performance– Can be used with any network card

• Extended PF_RING– Has plugin architecture – Includes IP defragmentation

extendedPF_RING

SIPplugi

n

RTPplugi

n

libpfring

libvoip

customplugin

VoIPconsole

customapplication

network interfaces

RT

C-M

ON

FR

AM

EW

OR

KA

PP

S

US

ER

SP

AC

EK

ER

NE

L S

PA

CE

September 2008 - 7

Extended PF_RING

• Each PF_RING socket has set of rules– Processed sequentially

• Each rule has– a filter– a plugin ID to send packets to– an action ID which decides what to do after plugin processes packet

• Filter can be hash or wild-card based– Hash: can be used to track flows (RTP plugin)– Wildcard: more flexible (match all UDP packets to one port)

extendedPF_RING

SIPplugi

n

RTPplugi

n

libpfring

libvoip

customplugin

VoIPconsole

customapplication

network interfaces

RT

C-M

ON

FR

AM

EW

OR

KA

PP

S

US

ER

SP

AC

EK

ER

NE

L S

PA

CE

September 2008 - 8

Libvoip

• User-level, event-based library• Extensible, can support other, non-VoIP applications• Made of a dispatcher and trackers

– Dispatcher receives packets from plugins and decides which trackers to

send them to

– TrackersCombine information from several plugins to monitor a specific

characteristic of the trafficCan perform further analysis, parsing and can keep stateLibrary is extended by writing custom trackers

extendedPF_RING

SIPplugi

n

RTPplugi

n

libpfring

libvoip

customplugin

VoIPconsole

customapplication

network interfaces

RT

C-M

ON

FR

AM

EW

OR

KA

PP

S

US

ER

SP

AC

EK

ER

NE

L S

PA

CE

September 2008 - 9

Libvoip

• Implemented two trackers– UserTracker, uses SIP information to keep track of SIP users– CallTracker, combines SIP/RTP data to monitor call information

extendedPF_RING

SIPplugi

n

RTPplugi

n

libpfring

libvoip

customplugin

VoIPconsole

customapplication

network interfaces

RT

C-M

ON

FR

AM

EW

OR

KA

PP

S

US

ER

SP

AC

EK

ER

NE

L S

PA

CE

September 2008 - 10

VoIP Console

• Proof-of-concept monitoring application– Only 800 lines of code, short development time– Uses UserTracker, CallTracker and SIP/RTP plugins

• Despite small size, can still track large number of statistics

extendedPF_RING

SIPplugi

n

RTPplugi

n

libpfring

libvoip

customplugin

VoIPconsole

customapplication

network interfaces

RT

C-M

ON

FR

AM

EW

OR

KA

PP

S

US

ER

SP

AC

EK

ER

NE

L S

PA

CE

September 2008 - 11

Evaluation: Testbed

• IXIA 400 generates up to 50,000 concurrent trash UDP flows• VoIP traffic generator replays VoIP trace

– Trace has 1,000 calls each lasting 30 seconds

extendedPF_RING

SIPplugi

n

RTPplugi

n

libpfring

libvoip

customplugin

VoIPconsole

customapplication

network interfaces

RT

C-M

ON

FR

AM

EW

OR

KA

PP

S

US

ER

SP

AC

EK

ER

NE

L S

PA

CE

CodecSample

Size(bytes)

Sample Rate (ms)

Max Num Flows

(Gb link)

G.711 80 10 11,468

G.726 20 5 18,116

G.726 15 5 21,186

G.728 10 5 31,780

G.729 10 10 32,051

iLBC 38 20 36,101

G.723.1 24 30 45,732

G.723.1 20 30 48,077

September 2008 - 12

Performance

• Focus is on RTP traffic– puts larger strain on system than SIP

• Measure improvement in CPU usage from performing data analysis in kernel

extendedPF_RING

SIPplugi

n

RTPplugi

n

libpfring

libvoip

customplugin

VoIPconsole

customapplication

network interfaces

RT

C-M

ON

FR

AM

EW

OR

KA

PP

S

US

ER

SP

AC

EK

ER

NE

L S

PA

CE

September 2008 - 13

Performance

• Measure monitoring performance for 50,000 flows and varying number of RTP rules (flows)

extendedPF_RING

SIPplugi

n

RTPplugi

n

libpfring

libvoip

customplugin

VoIPconsole

customapplication

network interfaces

RT

C-M

ON

FR

AM

EW

OR

KA

PP

S

US

ER

SP

AC

EK

ER

NE

L S

PA

CE

September 2008 - 14

Performance

• Rule reconfiguration timesextendedPF_RING

SIPplugi

n

RTPplugi

n

libpfring

libvoip

customplugin

VoIPconsole

customapplication

network interfaces

RT

C-M

ON

FR

AM

EW

OR

KA

PP

S

US

ER

SP

AC

EK

ER

NE

L S

PA

CE

# Rules Avg. Ins.(usec)

Max. Ins.(usec)

Avg. Del.(usec)

Max. Del.(usec)

10,000 20.9 98 4.7 81

20,000 22.3 130 5.3 95

30,000 25.2 210 7.9 150

40,000 27.9 379 12.7 22.5

50,000 47.1 2037 19.1 52.7

• In worst-case (2 msec) a single G.711 or G.729 packet would go untracked

September 2008 - 15

Conclusions

• Presented RTC-Mon– Framework for quickly developing real-time monitoring

applications– Easily extensible, can support future applications– Performs well even under heavy loads and small packet sizes

• Future work– Additional plugins (RTSP, RTCP, MPEG-7?)– Distributed monitoring

Track SIP/RTP flows traversing different paths