1 © nokia streaming qos/musa aldarawsheh, april 8 th 2005 author: musa aldarawsheh senior i&v...

42
1 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8 th 2005 Author: Musa Aldarawsheh Senior I&V Engineer/ Nokia Networks Supervisor: Prof. Sven-Gustav Häggman Instructor: M.Sc. Karl Tigerstedt Streaming Performance over Third Generation Cellular Networks

Upload: sybil-sparks

Post on 01-Jan-2016

216 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: 1 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Author: Musa Aldarawsheh Senior I&V Engineer/ Nokia Networks Supervisor: Prof. Sven-Gustav Häggman

1 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005

Author: Musa Aldarawsheh

Senior I&V Engineer/ Nokia Networks

Supervisor: Prof. Sven-Gustav Häggman

Instructor: M.Sc. Karl Tigerstedt

Streaming Performance over Third Generation Cellular Networks

Page 2: 1 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Author: Musa Aldarawsheh Senior I&V Engineer/ Nokia Networks Supervisor: Prof. Sven-Gustav Häggman

2 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005

Contents

IntroductionIntroduction

PS streaming QoS theory

QoS Key Performance Indicators

Environment Description

Measurements Tools

Preliminary Streaming QoS Tests Streaming Measurements Using Nokia Internal Streaming Server Streaming Measurements Using Helix Universal Server-Mobile

Streaming QoS Measurements over Real Time Bearers in Unloaded Cell

Streaming QoS Measurements over Non Real Time Bearers in Unloaded Cell

Streaming in Highly Loaded Cell With Conversational Class Users

Streaming Performance Measurement in Mobile Situation

GPRS Attach and PDP Context Activation Times

Summary & Conclusions

References

Page 3: 1 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Author: Musa Aldarawsheh Senior I&V Engineer/ Nokia Networks Supervisor: Prof. Sven-Gustav Häggman

3 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005

Introduction

• This work is aimed to verify the performance of multimedia streaming service over a WCDMA network

from end-user perspectives. The measurements were carried out over streaming and interactive QoS class

in stationary and mobile situation, the mobility measurements include streaming during soft handover.

• The streaming QoS will be verified through a set of Key Performance Indicators (KPI). These includes call

setup and call quality KPIs.

• Multimedia streaming verification was carried out in Nokia WCDMA Test Network, in Espoo (Finland) during

weeks 45 - 52 2004.

Page 4: 1 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Author: Musa Aldarawsheh Senior I&V Engineer/ Nokia Networks Supervisor: Prof. Sven-Gustav Häggman

4 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005

PS Streaming QoS Theory

• Multimedia streaming is a technology that allows compressed live or pre-recorded video and audio content to be delivered to the end user [Lai_p398]. Streaming service considered as real time service because the user plays back the content while downloading. Examples of streaming services are video-on-demand ,web-radio, and news streams.

• Media can be divided into two types continuous and discrete media,The first type is only handled in this report

To establish a streaming session there are different protocols involved to set-up the connection between the streaming server and the streaming client, They are :

• RTSP (Real Time Streaming Protocol) is used to for session set-up and session control• RTP (Real Time Protocol) is used to transport media data (video/audio) over unreliable

connection

(UDP/IP). • RTCP (Real Time Control Protocol) used to carry information on the participants and

session quality.

RLC transfer modes for PS Streaming

• RLC operates in one of three modes. These are Acknowledge mode (AM), Unacknowledged mode (UM) and transparent mode(TM).

• The possible RLC transfer modes for PS Streaming services are either UM-Unacknowledged mode or AM-Acknowledged mode

Token Bucket Algorithm:Token bucket algorithm is a reference algorithm to conform the definition of bit rate, see [23107].

Page 5: 1 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Author: Musa Aldarawsheh Senior I&V Engineer/ Nokia Networks Supervisor: Prof. Sven-Gustav Häggman

5 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005

QoS Key Performance Indicators

• QoS is the ability of the network to provide a service with assured service level to the user of the network

• QoE (Quality of user Experience) is how an end user perceives the usability of a service when on use

(how satisfied he/she is with the service usage)

• The following QoS KPIs has been defined to measure the performance of multimedia streaming over WCDMA networks.

• Session setup time :This is the time period when the streaming client requests media content from the streaming server until the time when the session setup between client and server is ready and the media start to flow.

• Initial Buffering time :This is the time elapsed between the completion of session set-up and the time that media starts playing at the client's device. This time is dependent on client memory storage.

• Total set-up time :

This is the time when the user request a media content form the streaming server until the time the media is starting to be displayed in the streaming client.

• Delay Experienced by the End User:

This is the time taken to establish a certain service from the point when the user request the service until the user start to receive information once the service is established. The delay experienced by a streaming service user is the sum of GPRS attach time, PDP context activation and total setup times.

Page 6: 1 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Author: Musa Aldarawsheh Senior I&V Engineer/ Nokia Networks Supervisor: Prof. Sven-Gustav Häggman

6 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005

QoS Key Performance Indicators• Packet Delay Variation (Jitter): This is the delay variation of arrived packets. • Packet losses: The number of media packets (video/audio) lost during the streaming session.

• Throughput: The number of received bits per unit time. Total throughput includes (media + headers) throughputs.

• Number and length of rebufferings: This measure is of great importance to the user because the rebuffering operation can stop the audio/video playback. In this thesis only the rebuffering periods that cause a pause in video/audio playback are taken into account.

• Subjective Audio/Video quality: The following simple rating scale is defined through streaming QoS measurements

• Excellent• Good• Fair• Poor• Bad

• Lip synchronisation: This metrics is used to measure the synchronization between audio and video. The metrics can be subjective and objective. The former can be based on the following rating scale.

• subjective Lip synchronization

– In synch– Just noticeably out-of-synch– Unacceptably out-of-synch

• The later can be calculated in the streaming client as the ratio between displayed frames to decoded frames (Lip synchronization index)

Page 7: 1 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Author: Musa Aldarawsheh Senior I&V Engineer/ Nokia Networks Supervisor: Prof. Sven-Gustav Häggman

7 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005

Environment description

• The measurements were done in the NTN (Nokia Test Network), in Leppävaara area during weeks 45-52 , 2004

15 ©NOKIA E2E Perfomance Optimisation.ppt

GGSNBackbone3G -SGSNHelix Universal Server Mobileas streaming server

RANIu PS

Gi

RNC

Backbone

GGSN3G-SGSNDial up connection

to requestingstreaming class with AT commands

Streaming application

Page 8: 1 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Author: Musa Aldarawsheh Senior I&V Engineer/ Nokia Networks Supervisor: Prof. Sven-Gustav Häggman

8 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005

Measurements Tools

• Nokia OSS system for logging cell load information • Nemo Outdoor • Global Positioning System (GPS) • Nokia 3G Test Mobile “ Nokia 7600 model”• Nokia Traffic Generator Tool • Helix Mobile Producer • Commview Packet Analyser • Ethereal Packet Analyser • Matlab Tool • Excel • Windows Grep • Nokia Internal Streaming Client • Nokia Internal Streaming Server • Helix Universal Server -Mobile

Page 9: 1 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Author: Musa Aldarawsheh Senior I&V Engineer/ Nokia Networks Supervisor: Prof. Sven-Gustav Häggman

9 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005

Preliminary Streaming QoS Tests

Objectives: Preliminary evaluation of multimedia streaming over Nokia WCDMA Test Network with different

client –server streaming solutions.

Test Description:

 

Four tests were executed by using two different streaming client–server solutions, two tests for each solution.

The client-server streaming solutions are:

1. Nokia Internal Streaming Client and Nokia Internal Streaming Server

2. Nokia Internal Streaming Client and Helix Universal Server-Mobile

 

The test was done in stationary situation near to the WCDMA base station where the carrier to interference

ratio Ec/No > -6 dB. Two different RT PS RABs were used, 64/64 and 64/128 UL/DL. Multimedia files were

used with bit rates 51 and 115 kbps over 64/64 and 64/128 bearers respectively.

Page 10: 1 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Author: Musa Aldarawsheh Senior I&V Engineer/ Nokia Networks Supervisor: Prof. Sven-Gustav Häggman

10 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005

Preliminary Streaming QoS Tests

Video Audio

Subjective QoS (MOS) Good Good

Packet Loss Ratio, Number of expected packets to be received

0.5%, 589 0.0%, 750

Preliminary test 1:Multimedia streaming over 64/64 kbps UL/DL RT PS RAB, BLER target 1%

Environment: Nokia internal streaming client + Nokia internal streaming

The test results are presented in Table 1. Average video/audio packets sizes are 450 and 150 byte respectively

Table 1 Streaming performance over 64/64 RT RAB

The results show a good performance of multimedia streaming when using Nokia Streaming Client –Server. The MOS was “Good” for both audio and video.

Page 11: 1 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Author: Musa Aldarawsheh Senior I&V Engineer/ Nokia Networks Supervisor: Prof. Sven-Gustav Häggman

11 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005

Preliminary Streaming QoS Tests

Preliminary test 2: Multimedia streaming over 64/128 kbps UL/DL RT PS RAB, BLER target 1%

Environment: Nokia internal streaming client + Nokia internal streaming

The test results are presented in Table 2. Average video/audio packets sizes are 381 and 313

byte respectively

Table 2 Streaming performance over 64/128 RT RAB

Video Audio

Subjective QoS (MOS) Fair Fair

Packet Loss Ratio, Number of expected packets to be received

1.9%, 2000 0.3%, 300

The results show a satisfactory performance of multimedia streaming when using Nokia Streaming Client –Server solution. The MOS was “Fair” for both audio and video. Video packet losses were in acceptable level.

Page 12: 1 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Author: Musa Aldarawsheh Senior I&V Engineer/ Nokia Networks Supervisor: Prof. Sven-Gustav Häggman

12 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005

Preliminary Streaming QoS Tests

Figure 1 Helix Server throughput over 64/64 RT bearer

Streaming Server sending bit rate more than guaranteed channel bit rate

Preliminary test 3: Multimedia streaming over 64/64 kbps UL/DL RT PS RAB, BLER target 1%

Environment: Nokia internal streaming client + Helix Server

The test results are presented in Table 3. Average video/audio packets sizes are 485 and 150 byte respectively.

The results show a slightly high video packet losses and low audio packet losses, high video packet losses explain

the poor perceived video quality

Video Audio

Subjective QoS (MOS)

Poor Fair

Packet Loss Ratio, Number of expected packets to be received

6.6%, 634 0.3%, 299

Table 3 Streaming performance over 64/64RT RAB

Page 13: 1 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Author: Musa Aldarawsheh Senior I&V Engineer/ Nokia Networks Supervisor: Prof. Sven-Gustav Häggman

13 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005

Preliminary Streaming QoS Tests

Figure 2 Helix Server throughput over 64/128 RT bearer

Preliminary test 4: Multimedia streaming over 64/128 kbps UL/DL RT PS RAB, BLER target 1%

Environment: Nokia internal streaming client + Helix Server

The test results are presented in Table 4. Average video/audio packets sizes are 333 and 150 byte respectively.

The results show a very high video/audio packet loss, which explain the poor perceived video/audio quality. Streaming Server sending bit rate more

than guaranteed channel bit rate

Video Audio

Subjective QoS (MOS)

Bad Bad

Packet Loss Ratio, Number of expected packets to be received

29.6 %, 2136 31.1 %, 301

Table 4 Streaming performance over 64/128RT RAB

Page 14: 1 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Author: Musa Aldarawsheh Senior I&V Engineer/ Nokia Networks Supervisor: Prof. Sven-Gustav Häggman

14 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005

Preliminary Streaming QoS Tests

• The streaming performance over 64/64 and 64/128 UL/DL real time bearers was bad when using Helix server.

• The streaming performance over 64/64 and 64/128 UL/DL real time bearers was satisfactory when using

Nokia internal streaming server.

The tests had shown that streaming server was very often sending media packets with short delays and the

data rate exceeded the channel maximum bit rate. Data rate that exceed the maximum channel rate will be

discarded according to token bucket algorithm. The current implementation of token bucket algorithm is

done according to 3GPP specification [TS23.107 Annex B], which is not handling appropriately the burst

traffic situation. As a result of these findings the token bucket algorithm was modified to be able to tolerate

traffic burstiness conditions. This modification allows streaming class to use traffic shaping functionality,

which smooth the traffic pattern and decrease the burstiness of the network.

All the measurements that follow were done with the modified token bucket algorithm except SHO

measurements.

Page 15: 1 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Author: Musa Aldarawsheh Senior I&V Engineer/ Nokia Networks Supervisor: Prof. Sven-Gustav Häggman

15 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005

Streaming QoS Measurements over Real Time Bearers in Unloaded Cell

Objectives:

To verify the performance of multimedia streaming over Nokia WCDMA Test Network in a stationary situation

Test 1: Multimedia streaming over 64/64 kbps UL/DL RT PS, BLER target 1%

Environment: Nokia internal streaming client + Helix Server

AT command used :at+cgeqreq=1,1,64,64,64,64,0,1500,"1E3",,0,1000

Total number of video and audio packets sent by streaming server is 589 and 750 respectively. Average video

and packet sizes are 485 and 150 byte respectively.

Table 5 Call setup QoS KPIs over 64/64 streaming bearer, BLER 1%

Trial Setup time

msec

Buffering Time

msec

Total Setup time

msec

1 6539 5017 11556

2 6498 5017 11515

3 6522 5057 11579

The end user will experience the following delays before start to see the movie e.g in trial 1 the end user experience the following delaysAttach time+PDP context time + setup time + buffering time = 6.7+6.53+5=18.23 secSee GPRS attach and PDP context measurements.

Page 16: 1 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Author: Musa Aldarawsheh Senior I&V Engineer/ Nokia Networks Supervisor: Prof. Sven-Gustav Häggman

16 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005

Streaming QoS Measurements over Real Time Bearers in Unloaded Cell

Total throughput kbit/s (1 sec. window)

0

10

20

30

40

50

60

70

1 11 21 31 41 51

sec.

kbit

/s Measured

Reference

Trial Average Throughput

kbps

PLR %

audio

PLR%

video

Packet Delay video

Min,max,avg

msec

Packet Delay Audio

Min,max,avg

msec

MOS

video

MOS

audio

Lip Sync

index

Subjective lip synchronization

Number and Length of rebufferings

 (msec)

1 58.29 0.33% 1.26% 0,521,96.7 30,671,200.5 Good Good 0.99 In synch 0 / 0

2 57.15 1.67% 2.05% 0,333,97 10,681,204.6 Fair Good 0.69 In synch 0 / 0

3 57.99 1.67% 0.33% 0,301,96 40,681,204.2 Good Good 0.85 In synch 0 / 0

Table 6 End-to-End and objective/subjective video/audio QoS KPIs over 64/64 streaming bearer, BLER target 1%

Figure 3 shows the total throughput (media + header throughputs) received by the streaming client. The received throughput is within the channel bandwidth in most of the times.

Figure 3 Streaming client total throughput over 64/64 RT bearer

Page 17: 1 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Author: Musa Aldarawsheh Senior I&V Engineer/ Nokia Networks Supervisor: Prof. Sven-Gustav Häggman

17 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005

Streaming QoS Measurements over Real Time Bearers in Unloaded Cell

Objectives:

To verify the performance of multimedia streaming over Nokia WCDMA Test Network in a stationary situation

Test 2: Multimedia streaming over 64/128 kbps UL/DL RT PS, BLER target 1%

Environment: Nokia internal streaming client + Helix Server

AT command used :at+cgeqreq=1,1,64,128,64,128,0,1500,"1E3",,0,1000

Total number of video and audio packets sent by streaming server is 2136 and 310 respectively. Average

video and packet sizes are 333 and 150 byte respectively.

Table 7 Call setup QoS KPIs over 64/128 RT streaming bearer, BLER 1%

Trial Setup time

msec

Buffering Time

msec

Total Setup time

msec

1 11141 5017 16158

2 7048 5017 12065

Page 18: 1 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Author: Musa Aldarawsheh Senior I&V Engineer/ Nokia Networks Supervisor: Prof. Sven-Gustav Häggman

18 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005

Streaming QoS Measurements over Real Time Bearers in Unloaded Cell

Packet losses are in acceptable level Subjective QoS was good Lip sync index shows a satisfactory valuesNo pause and rebufferings occurred during thestreaming session.The received throughput is within the channel bandwidth in most of the times.

Trial Average Throughput

kbps

PLR %

audio

PLR%

video

Packet Delay video

Min,max,avg

msec

Packet Delay Audio

Min,max,avg

msec

MOS

video

MOS

audio

Lip Sync

index

Subjective lip synchronization

Number and Length of rebufferings

 (msec)

1 119.44 0.3%

0.33 %

0,171,28.4 10,621,202

Good Good

0.57 Just noticeably out of synch

0 / 0

2 119.32 0.3%

0.23% 0,160,28 20,601,201

Good Good

0.97 In Synch 0 / 0

Table 8 End-to-End and objective/subjective video/audio QoS KPIs over 64/128 streaming bearer, BLER target 1%

Total throughput kbit/s (1 sec. window)

0

20

40

60

80

100

120

140

1 11 21 31 41 51

sec.

Measured

Reference

Figure 4 Streaming client total throughput over 64/128 RT bearer

Page 19: 1 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Author: Musa Aldarawsheh Senior I&V Engineer/ Nokia Networks Supervisor: Prof. Sven-Gustav Häggman

19 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005

Streaming QoS Measurements over Real Time Bearers in Unloaded Cell

Objectives:

To verify the performance of multimedia streaming over Nokia WCDMA Test Network in a stationary situation

Test 3: Multimedia streaming over 64/256 kbps UL/DL RT PS, BLER target 1%

Environment: Nokia internal streaming client + Helix Server

AT command used :at+cgeqreq=1,1,64,256,64,256,0,1500,"1E3",,0,1000

Total number of video and audio packets sent by streaming server is 4066 and 297 respectively. Average

video and packet sizes are 341 and 321 byte respectively.

Table 9 Call setup QoS KPIs over 64/256 streaming bearer, BLER 1 %Trial Setup time

msec

Buffering Time

msec

Total Setup time

msec

1 10006 5017 15023

2 9020 5017 14037

3 10851 5017 15868

Page 20: 1 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Author: Musa Aldarawsheh Senior I&V Engineer/ Nokia Networks Supervisor: Prof. Sven-Gustav Häggman

20 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005

Streaming QoS Measurements over Real Time Bearers in Unloaded Cell

Trial Average Throughput

kbps

PLR %

audio

PLR%

video

Packet Delay video

Min,max,avg

msec

Packet Delay Audio

Min,max,avg

msec

MOS

audio

MOS

video

Lip Sync

index

Subjective lip synchronization

Number and length of rebufferings

msec

1 203.73 0.9% 0.93 %

0,821,16.9 10,991,207 Fair Poor 0.46 Unacceptably out-of-synch

5 / 5860

2 177 1% 1 % 0,4707,17 10,1192,206

Fair Bad 0.34 Unacceptably out-of-synch

11 / 5947

3 187 1.3% 0.8% 0,3345,17.6 0,1192,209 Fair Bad 0.38 Unacceptably out-of-synch

5 / 5860

Figure 5 shows the total throughput received by the streaming client. The total throughput was exceeding the channel bandwidth in most of the times .Subjective video/audio quality was not totally satisfactory Lip sync index shows a very low values which indicates that the video and audio are out of syncMany rebufferings occurred during the video.

Table 10 End-to-End and objective/subjective video/audio QoS KPIs over 64/256 streaming bearer, BLER target 1%

Total throughput kbit/s (1 sec. window)

0

50

100

150

200

250

300

350

400

1 11 21 31 41 51

sec.

kbit

/s Measured

Reference

Figure 5 Streaming client total throughput over 64/128 RT bearer

Page 21: 1 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Author: Musa Aldarawsheh Senior I&V Engineer/ Nokia Networks Supervisor: Prof. Sven-Gustav Häggman

21 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005

Streaming QoS Measurements over Non Real Time Bearers in Unloaded Cell

Trial Setup Time

msec

Buffering Time

msec

Total Setup time

msec

1 6839 5017 11846

2 6778 5017 11795

3 6922 5057 11979

Objectives:

Total number of video and audio packets sent by streaming server is 2136 and 300 To verify the performance of multimedia streaming over NRT bearer in a stationary situation

Test 4: Multimedia streaming over 64/64 kbps UL/DL NRT PS, BLER target 1%

Environment: Nokia internal streaming client + Helix Server

respectively. Average video and packet sizes are 333 and 321 byte respectively.

Table 11 Call setup QoS KPIs over 64/64 NRT streaming bearer, BLER 1 %

Page 22: 1 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Author: Musa Aldarawsheh Senior I&V Engineer/ Nokia Networks Supervisor: Prof. Sven-Gustav Häggman

22 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005

Streaming QoS Measurements over Non Real Time Bearers in Unloaded Cell

Trial

Average Throughput

kbps

PLR %

audio

PLR%

video

Packet Delay video

Min,max,avg

msec

Packet Delay Audio

Min,max,avg

msec

MOS

video

MOS

audio

Lip Sync

index

Subjective lip synchronization

Number and Length of rebufferings

 (msec)

1 58.29 0.0% 0.0 % 0,555,96.7 30,671,200 Good Good 0.86 In Sync 0 / 0

2 57.15 0.0% 0.0% 0,340,97 10,681,204 Good Good 0.76 In Sync 0 / 0

3 57.99 0.0% 0.0% 0,370,96 40,680,203 Good Good 0.82 In Sync 0 / 0

Table 12 End-to-End and objective/subjective video/audio QoS KPIs over 64/64 NRT bearer, BLER target 1%

No Packet losses Subjective video and audio QoS was good Lip sync index shows a good values No pause and rebuffering occurred during thestreaming session.

Page 23: 1 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Author: Musa Aldarawsheh Senior I&V Engineer/ Nokia Networks Supervisor: Prof. Sven-Gustav Häggman

23 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005

Streaming QoS Measurements over Non Real Time Bearers in Unloaded Cell

Trial Setup Time

msec

Buffering Time

msec

Total Setup time

msec

1 10230 5017 15247

2 8921 13938 11338

Objectives:

To verify the performance of multimedia streaming over NRT bearer in a stationary situation

Test 5: Multimedia streaming over 64/128 kbps UL/DL NRT PS, BLER target 1%

Environment: Nokia internal streaming client + Helix Server

Total number of video and audio packets sent by streaming server is 2136 and 300 respectively. Average

video and packet sizes are 333 and 321 byte respectively.

Table 13 Call setup QoS KPIs over 64/128 NRT streaming bearer, BLER 1 %

Page 24: 1 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Author: Musa Aldarawsheh Senior I&V Engineer/ Nokia Networks Supervisor: Prof. Sven-Gustav Häggman

24 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005

Streaming QoS Measurements over Non Real Time Bearers in Unloaded Cell

Trial Average Throughput

kbps

PLR %

audio

PLR%

video

Packet Delay video

Min,max,avg

msec

Packet Delay Audio

Min,max,avg

msec

MOS

video

MOS

audio

Lip Sync

index

Subjective lip synchronization

Number and length of rebufferings

msec

1 119.98 0.0% 0.0 %

0,231,28.3 20,590,200 Good

Good 0.62 In Sync 0 / 0

2 119.96 0.0% 0.0% 0,230,28.4 10,661,202 Good

Good 0.81 In Sync 0 / 0

No Packet losses Subjective video and audio QoS was good Lip sync index shows a good values No pause and rebuffering occurred during thestreaming session.The received throughput is exceeding the channel Bandwidth sometimes

Total throughput kbit/s (1 sec. window)

0

50

100

150

200

250

1 11 21 31 41 51

sec.

kbit

/s Measured

Reference

Table 14 End-to-End and objective/subjective video/audio QoS KPIs over 64/128 NRT bearer, BLER target 1%

Figure 6 Streaming client total throughput over 64/128 NRT bearer

Page 25: 1 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Author: Musa Aldarawsheh Senior I&V Engineer/ Nokia Networks Supervisor: Prof. Sven-Gustav Häggman

25 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005

Streaming QoS Measurements over Non Real Time Bearers in Unloaded Cell

Trial Setup Time

msec

Buffering Time

msec

Total setup time

msec

1 9813 5018 14831

2 12071 5017 17071

Objectives:

To verify the performance of multimedia streaming over NRT bearer in a stationary situation

Test 6: Multimedia streaming over 64/384 kbps UL/DL NRT PS, BLER target 1%

Environment: Nokia internal streaming client + Helix Server

Total number of video and audio packets sent by streaming server is 4066 and 301 respectively. Average

video and packet sizes are 333 and 321 byte respectively.

Table 15 Call setup QoS KPIs over 64/384 NRT streaming bearer, BLER 1 %

Page 26: 1 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Author: Musa Aldarawsheh Senior I&V Engineer/ Nokia Networks Supervisor: Prof. Sven-Gustav Häggman

26 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005

Streaming QoS Measurements over Non Real Time Bearers in Unloaded Cell

Trial Average Throughput

kbps

PLR %

audio

PLR%

video

Packet Delay video

Min,max,avg

msec

Packet Delay Audio

Min,max,avg

msec

MOS

audio

MOS

video

Lip Sync

index

Subjective lip synchronization

Number and length of rebufferings

msec

1 213.94 0.8 % 1.1 % 0,390,15.3 0,1041,204 Good Fair 0.45 Just noticeably out-of-synch

2 / 5743

2 219.4 0.0% 0.1% 0,491,14.9 0,931,203 Good Fair 0.5 Just noticeably out-of-synch

2/ 5944

Packet losses ~ 1% Subjective audio/video QoS was satisfactoryLip sync index shows a relatively low values The received throughput is exceeding the channel Bandwidth very often

Total throughput kbit/s (1 sec. window)

0

50

100

150

200

250

300

350

400

1 11 21 31 41 51

sec.

kbit

/s Measured

Reference

Table 16 End-to-End and objective/subjective video/audio QoS KPIs over 64/384 NRT bearer, BLER target 1%

Figure 7 Streaming client total throughput over 64/384 NRT bearer

Page 27: 1 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Author: Musa Aldarawsheh Senior I&V Engineer/ Nokia Networks Supervisor: Prof. Sven-Gustav Häggman

27 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005

Streaming in Highly Loaded Cell With Conversational Class Users

Objective: To verify that the guaranteed bit rate of a streaming class user is kept in a highly loaded cell.

Test Description:

The test was executed by starting a single video streaming session with a predefined data rate and loading

the cell with conversational class users (voice calls). The throughput of streaming user was monitored.

Environment:

Increasing common channels power by 11 dB

Page 28: 1 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Author: Musa Aldarawsheh Senior I&V Engineer/ Nokia Networks Supervisor: Prof. Sven-Gustav Häggman

28 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005

Streaming in Highly Loaded Cell With Conversational Class Users Test 7: Streaming over 64/64 RT bearer

Total # of send packets =1389Total # of packets lost = 15Packet losses ~1%

Throughput decreased significantly due to packet losses

High BLERDL BLER

Throughput PPP Layer

CPICH RSCP

UE Tx Pwr

CPICH Ec/No

Page 29: 1 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Author: Musa Aldarawsheh Senior I&V Engineer/ Nokia Networks Supervisor: Prof. Sven-Gustav Häggman

29 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005

Streaming in Highly Loaded Cell With Conversational Class Users Test 7: Streaming over 64/64 RT bearer

PtxTotal DL=~ Ptxtarget DL

UL RT Load

New AMR calls arenot admitted after 33 existing AMR calls

Page 30: 1 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Author: Musa Aldarawsheh Senior I&V Engineer/ Nokia Networks Supervisor: Prof. Sven-Gustav Häggman

30 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005

Streaming in Highly Loaded Cell With Conversational Class Users Test 8: Streaming over 64/128 RT bearer

Total # of send packets =8118Total # of packets lost = 36Packet losses 0.4%

Throughput is not affected by cell congestion

Throughput PPP Layer

DL BLER

CPICH RSCP

UE Tx Pwr

CPICH Ec/No

New AMR calls arenot admitted after 37 existing AMR calls

Page 31: 1 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Author: Musa Aldarawsheh Senior I&V Engineer/ Nokia Networks Supervisor: Prof. Sven-Gustav Häggman

31 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005

Streaming in Highly Loaded Cell With Conversational Class Users Test 9: Streaming over 64/256 RT bearer

Total # of send packets =8247Total # of packets lost = 34Packet losses 0.4%

CPICH EcNo

UE TX Power

CPICH RSCP

Throughput PPP Layer

DL BLER

New AMR calls arenot admitted after 37 existing AMR calls

Page 32: 1 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Author: Musa Aldarawsheh Senior I&V Engineer/ Nokia Networks Supervisor: Prof. Sven-Gustav Häggman

32 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005

Streaming Performance Measurement in Mobile Situation

Softer handover

Soft handover

• Objective: To verify the performance of multimedia streaming over real time bearer during soft handover in Nokia WCDMA Test Network.

The description of tests is as follow:

• Two tests were executed over two streaming class bearers, one test per bearer. The bearer bit rates are 64/64 and 64/128 kbps UL/DL. BLER target value was set to 1% for all tests. Multimedia files were used with bit rates 103 and 213 kbps over 64/128 and 64/256 RT PS RABs respectively. The tests were conducted by using Nokia internal streaming client and Nokia streaming server. Streaming performance was verified during soft handover. The carrier to interference ration was “Ec/No > -6” dB.

Every test is repeated 2-3 times to

ensure reliability.

This test was executed with original

token bucket algorithm

Page 33: 1 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Author: Musa Aldarawsheh Senior I&V Engineer/ Nokia Networks Supervisor: Prof. Sven-Gustav Häggman

33 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005

Streaming Performance Measurement in Mobile Situation Streaming over 64/128 RT bearer  

Objectives:

To verify the performance of multimedia streaming during SHO

Test 10: Multimedia streaming during SHO over 64/128 kbps UL/DL RT PS, BLER target 1%

Environment: Nokia internal streaming client + Nokia internal streaming server

AT command used :at+cgeqreq=1,1,64,128,64,128,0,1500,"1E3",,0,1000

Total number of video and audio packets sent by streaming server is 1880 and 296 respectively. Average

video and packet sizes are 382 and 313 byte respectively.

Table 17 Call setup QoS KPIs over 64/128 streaming bearer, BLER 1 %

Trial Setup time

msec

Buffering Time

msec

Total Setup time

msec

1 4196 5017 9213

2 4126 5017 9143

2 3935 5017 8952

Page 34: 1 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Author: Musa Aldarawsheh Senior I&V Engineer/ Nokia Networks Supervisor: Prof. Sven-Gustav Häggman

34 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005

Streaming Performance Measurement in Mobile Situation Streaming over 64/128 RT bearer   

Video packet losses are high due to token packet

algorithm problemLip sync index shows a satisfactory values No rebufferings occurred during the streaming session.

Total throughput kbit/s (1 sec. window)

0

20

40

60

80

100

120

140

1 11 21 31 41 51

sec.

kbit

/s Measured

Reference

Trial Average Throughput

kbps

PLR %

audio

PLR%

video

Packet Delay video

Min,max,avg

msec

Packet Delay Audio

Min,max,avg

msec

MOS

video

MOS

audio

Lip Sync

index

Subjective lip synchronization

Number and Length of rebufferings

 (msec)

1 112.8 6% 7.34%

0,221,33 10,651,212 Fair Good 0.58 Just noticeably out-of-synch

0 / 0

2 112.8 5.4% 7.5 % 0,341,34 60,631,210 Fair Good 0.56 Just noticeably out-of-synch

0 / 0

3 112.9 7% 7.3% 0,251,33 20,570,214 Fair Good 0.55 Just noticeably out-of-synch

0 / 0

Table 18 End-to-End and objective/subjective video/audio QoS KPIs over 64/128 RT bearer during SHO, BLER target 1%

Figure 8 Streaming client total throughput over 64/128 RT bearer

Page 35: 1 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Author: Musa Aldarawsheh Senior I&V Engineer/ Nokia Networks Supervisor: Prof. Sven-Gustav Häggman

35 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005

Streaming Performance Measurement in Mobile Situation Streaming over 64/128 RT bearer   

Soft HOSofter HO

Soft handover takes place between different base stations sites within one RNC (intra-RNC soft handover) or

within separate controlled RNCs (inter-RNC soft handover). In softer handover the UE is controlled by at least

two cells belonging to the same base station site.

Page 36: 1 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Author: Musa Aldarawsheh Senior I&V Engineer/ Nokia Networks Supervisor: Prof. Sven-Gustav Häggman

36 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005

Streaming Performance Measurement in Mobile Situation Streaming over 64/256 RT bearer

Objectives:

To verify the performance of multimedia streaming during SHO

Test 11: Multimedia streaming during SHO over 64/256 kbps UL/DL RT PS, BLER target 1%

Environment: Nokia internal streaming client + Nokia internal streaming server

AT command used :at+cgeqreq=1,1,64,256,64,256,0,1500,"1E3",,0,1000

Total number of video and audio packets sent by streaming server is 4066 and 297 respectively. Average

video and packet sizes are 341 and 321 byte respectively.

Table 19 Call setup QoS KPIs over 64/256 streaming bearer, BLER 1 %

Trial Setup time

msec

Buffering Time

msec

Total Setup time

msec

1 3675 5017 8692

2 3856 5017 8873

2 3846 5017 8863

Page 37: 1 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Author: Musa Aldarawsheh Senior I&V Engineer/ Nokia Networks Supervisor: Prof. Sven-Gustav Häggman

37 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005

Streaming Performance Measurement in Mobile Situation Streaming over 64/256 RT bearer

Trial Average Throughput

kbps

PLR %

audio

PLR%

video

Packet Delay video

Min,max,avg

msec

Packet Delay Audio

Min,max,avg

msec

MOS

audio

MOS

video

Lip Sync

index

Subjective lip synchronization

Number and length of rebufferings msec

1 220 2.3% 7.25% 0,180,22 60,460,204 Good Fair 0.56 Just noticeably out-of-synch

0 / 0

2 221 1.3% 6.95% 0,170,21 10,445,202 Good Fair 0.56 Just noticeably out-of-synch

0 / 0

3 220 3.3% 7.05% 0,330,21 30,511,207 Good Fair 0.55 Just noticeably out-of-synch

0 / 0

Video packet losses are high due to token packet

algorithm problemSubjective audio/video QoS was satisfactoryLip sync index shows satisfactory values No rebufferings occurred during the streaming session.

Total throughput kbit/s (1 sec. window)

0

50

100

150

200

250

300

1 11 21 31 41 51

sec.

kbit

/s Measured

Reference

Table 20 End-to-End and objective/subjective video/audio QoS KPIs over 64/256 RT bearer during SHO, BLER target 1%

Figure 9 Streaming client total throughput over 64/256 RT bearer

Page 38: 1 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Author: Musa Aldarawsheh Senior I&V Engineer/ Nokia Networks Supervisor: Prof. Sven-Gustav Häggman

38 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005

Streaming Performance Measurement in Mobile Situation Streaming over 64/256 RT bearer

Soft HOSofter HO

Page 39: 1 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Author: Musa Aldarawsheh Senior I&V Engineer/ Nokia Networks Supervisor: Prof. Sven-Gustav Häggman

39 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005

GPRS Attach and PDP Context Activation Times

Objectives: To measure the GPRS attach and PDP context activation times over streaming class bearers.

Three different RAB bit rates are used, 64/64, 64/128, and 64/256 kbps UL/DL. BLER target value was 1% for

all tests. Every test was repeated 15 times to ensure reliability.

Attach Times were calculated as the difference time between ATTACH_REQ and ATTACH_COMPLETE messages.

PDP context activation time was calculated as a time difference between PDP_context_create_req and

PDP_context_activate_ind messages

Table 33 shows that average PDP context + GPRS attach time for 64/64, 64/128, and 64/256 are 6.71, 7.09, and 7.28 seconds.

Table 21 Average GPRS attach+ PDP Context Activation.

Real Time Bearer PDP context + GPRS attach

Average Time (sec)

64/64 6.71

64/128 7.09

64/256 7.28

Page 40: 1 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Author: Musa Aldarawsheh Senior I&V Engineer/ Nokia Networks Supervisor: Prof. Sven-Gustav Häggman

40 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005

Summary & Conclusions

• The preliminary evaluation tests using Nokia internal streaming server had shown a good performance of multimedia streaming over 64/64 UL/DL real time bearer and a satisfactory performance over 64/128 UL/DL real time bearer.

• The preliminary evaluation tests using Helix streaming server had shown a bad performance of multimedia streaming over 64/64 and 64/128 UL/DL real time bearers.

The following conclusions are drawn from the preliminary evaluation tests:

• Nokia streaming server has shown a better performance than Helix streaming server.

• The bad performance of multimedia streaming over streaming class was due to the large number of discarded packets by the token bucket algorithm. The current implementation of token bucket algorithm, which is according to 3GPP specification was not suitable to control the traffic flow of streaming servers.

• The results obtained after token bucket modification have shown a good performance of multimedia streaming over 64/64 and 64/128 real time bearers and the performance over 64/256 UL/DL RT bearer was not totally satisfactory.

• The performance of video streaming over 64/64 and 64/128 NRT bearers was good. Video and audio quality was good and video/audio packet losses were 0.0% in all trials.

Page 41: 1 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Author: Musa Aldarawsheh Senior I&V Engineer/ Nokia Networks Supervisor: Prof. Sven-Gustav Häggman

41 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005

Summary & Conclusions

• The performance of video streaming over 64/384 UL/DL NRT bearer was satisfactory.

• The performance of multimedia streaming over 64/128 and 64/256 UL/DL RT during SHO was satisfactory. (original token bucket algorithm + Nokia streaming server)

The following conclusions are drawn from the streaming verification tests after token bucket algorithm modifications.

• The performance of video streaming over real time bearer was significantly improved after token bucket algorithm modification.

• Multimedia streaming over NRT bearers in unloaded cell show a slightly better performance than streaming over RT bearers.

• The streaming service user will experience a delay of ~20 seconds before the media contents starts to be displayed in the streaming client. This delay includes attach, context, and total setup times.

Page 42: 1 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Author: Musa Aldarawsheh Senior I&V Engineer/ Nokia Networks Supervisor: Prof. Sven-Gustav Häggman

42 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005

References

[Hol] Harri Holma and Antti Toskala, ”’WCDMA for UMTS Radio Access For Third Generation Mobile Communications”, Second Edition, John Wiley & Sons, LTD, 2002

[Lai] Jaana Laiho, Achim Wacker and Tomas Novosad, “ Radio Network Planning and Optimisation for UMTS”, John Wiley & Sons, LTD, 2002, 484 pages.

[22105] 3GGP TS 22.105 Service aspects; Services and Service Capabilities, version 4.1.0, January 2001, 31 pages.

[23107] 3GPP Technical Specification 23.107: QoS concept and Architecture, version 4.6.0, December 2002, 38 pages