Testing SIP Services Over IP

Agenda

SIP testing – advanced scenarios SIP testing - Real Life Examples

Test Equipment Requirements

SIP Testing – the basics

Call simulation Call monitoring

Signaling analysis Media analysis

Conformance testing Load (stress) generation

Advanced SIP Testing Advanced testing

Benchmarking Performance Media degradation Envelope determination

In terms of a large network…

For complex scenarios: Many users Many call flows Different codecs Different networking

configuration NAT, Firewall…

Stress Testing??

Aimed at determining the operational envelope: Network elements Service Certain concept / design

Why stress test: Vendors: want to determine performance figures

before shipping Service Providers: want to find the problems in the

lab

Stress Testing!

Not just “number of calls”

Emulate a network wide environment: Large number of users Many network elements (IPs) Emulate NAT

Excessive call load Many profiles / scenarios Many different services

What are we looking for

Predicting the element behavior in real life: How does the UUT (network element /

service) behave in real life (=stress…) Where is the “breaking point” What happens at the “breaking point”

How are new / existing calls affected? Is the media degraded?

Stress Factors

Basic call flows (signaling) RTP streams Media type (Video, Audio) Advanced call flows SIP services

Signaling Stress Requirements

Generate and respond to SIP service requests

Simulate User Agent Client / Server communication procedures (RFC 3261)

Emulate behavior of one or many SIP phones different signaling and media parameters for

each

Registration Rate

True emulation of user behaviour Generate calls only when registered

Mega-blast generation of registration events Full configuration of SIP registration

procedures Mimic real user behaviour

Advanced Testing Requirements (1)

Multiple users 100,000’s UACs Configurable groups each with different behavior profiles

Multiple IPs 1000s of IPs - both media and signaling

Multiple MACs

Multiple VLANs

Advanced Testing Requirements -- 2

Full SIP message customization Video generation DTMF generation Different call flows:

Hold, Consultation Hold, Music on Hold, 3-way Conference, Call redirect, Call reject…

Erroneous flows

Key Traffic Generation Measurements

Session success rates and GoS

SIP method count and response code

Registration history

Presence and IM message statistics

Incoming and outgoing RTP stream counters

Real Life Examples

Common pitfalls

Registration properties Tendency to forget the credentials

Registration rates Initially takes too long Server will not withstand load

Non-standard flows Test signaling & media simultaneously

Network Model: Simple Test Lab

Test Environment:•Multi-service solution•Multi-technology: VoIP & PSTN

Core 2 Core 1Call

Analysis

IP / ATM / FR

SIP Stress Generator

UUT UUT

Call Analysis

SIP Stress Generator

Analog call generator

Network Model: Simple Test Lab (cont.) SIP Stress Generator:

Generate different levels of stress including RTP Voice and video simulation

Call Analysis: Signaling and media analysis Voice quality (intrusive, non-intrusive and subjective) RTP transport streams

Analog Call Generator Connect to PSTN interfaces Measure voice quality, voice distortion

Testing Options

Performance - under different levels of stress: Signaling and media quality evaluation Call handling Media degradation

Interoperability Different codec models PSTN and VoIP devices

Reliability Quality evaluation Stability testing

Firewall, SBC, CSCF Testing

Load scenarios BHCA – 1,000,000s per hour CPS – hundreds to thousands Concurrent calls > 100,000s

Public IP

Soft-Switch

Session Boarder

Controller

Session Boarder

Controller

Soft-Switch

SP1 SP2

SIP Stress Generator SIP Stress

Generator

SIP Stress Generator

Call Analysis

Call Analysis

Test bed 1 Test bed 2

Firewall, SBC & CSCF: Test Lab

SIP Stress Generation requirements: Different levels of stress Different services Different media Different call flows

QoS analysis: Voice quality Transport

Firewall, SBC & CSCF: Testing Needs

QoS Delays, Jitter, Packet loss, MOS

Stability (network elements / services) Benchmarking and comparisons

Simultaneous test bed operation

Test Bed 1

Test Bed 2

Test Bed 3

Traffic Simulator

Traffic Simulator

Traffic Simulator

Traffic Simulator

Traffic Simulator

Traffic Simulator

Elegant solution

Traffic Generator

VLAN switch

Traffic Generator

VLAN1

VLAN2

VLAN3

Test Bed 1

Test Bed 2

Test Bed 3