1 © nokia streaming qos/musa aldarawsheh, april 8 th 2005 author: musa aldarawsheh senior i&v...
TRANSCRIPT
1 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005
Author: Musa Aldarawsheh
Senior I&V Engineer/ Nokia Networks
Supervisor: Prof. Sven-Gustav Häggman
Instructor: M.Sc. Karl Tigerstedt
Streaming Performance over Third Generation Cellular Networks
2 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005
Contents
IntroductionIntroduction
PS streaming QoS theory
QoS Key Performance Indicators
Environment Description
Measurements Tools
Preliminary Streaming QoS Tests Streaming Measurements Using Nokia Internal Streaming Server Streaming Measurements Using Helix Universal Server-Mobile
Streaming QoS Measurements over Real Time Bearers in Unloaded Cell
Streaming QoS Measurements over Non Real Time Bearers in Unloaded Cell
Streaming in Highly Loaded Cell With Conversational Class Users
Streaming Performance Measurement in Mobile Situation
GPRS Attach and PDP Context Activation Times
Summary & Conclusions
References
3 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005
Introduction
• This work is aimed to verify the performance of multimedia streaming service over a WCDMA network
from end-user perspectives. The measurements were carried out over streaming and interactive QoS class
in stationary and mobile situation, the mobility measurements include streaming during soft handover.
• The streaming QoS will be verified through a set of Key Performance Indicators (KPI). These includes call
setup and call quality KPIs.
• Multimedia streaming verification was carried out in Nokia WCDMA Test Network, in Espoo (Finland) during
weeks 45 - 52 2004.
4 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005
PS Streaming QoS Theory
• Multimedia streaming is a technology that allows compressed live or pre-recorded video and audio content to be delivered to the end user [Lai_p398]. Streaming service considered as real time service because the user plays back the content while downloading. Examples of streaming services are video-on-demand ,web-radio, and news streams.
• Media can be divided into two types continuous and discrete media,The first type is only handled in this report
To establish a streaming session there are different protocols involved to set-up the connection between the streaming server and the streaming client, They are :
• RTSP (Real Time Streaming Protocol) is used to for session set-up and session control• RTP (Real Time Protocol) is used to transport media data (video/audio) over unreliable
connection
(UDP/IP). • RTCP (Real Time Control Protocol) used to carry information on the participants and
session quality.
RLC transfer modes for PS Streaming
• RLC operates in one of three modes. These are Acknowledge mode (AM), Unacknowledged mode (UM) and transparent mode(TM).
• The possible RLC transfer modes for PS Streaming services are either UM-Unacknowledged mode or AM-Acknowledged mode
Token Bucket Algorithm:Token bucket algorithm is a reference algorithm to conform the definition of bit rate, see [23107].
5 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005
QoS Key Performance Indicators
• QoS is the ability of the network to provide a service with assured service level to the user of the network
• QoE (Quality of user Experience) is how an end user perceives the usability of a service when on use
(how satisfied he/she is with the service usage)
• The following QoS KPIs has been defined to measure the performance of multimedia streaming over WCDMA networks.
• Session setup time :This is the time period when the streaming client requests media content from the streaming server until the time when the session setup between client and server is ready and the media start to flow.
• Initial Buffering time :This is the time elapsed between the completion of session set-up and the time that media starts playing at the client's device. This time is dependent on client memory storage.
• Total set-up time :
This is the time when the user request a media content form the streaming server until the time the media is starting to be displayed in the streaming client.
• Delay Experienced by the End User:
This is the time taken to establish a certain service from the point when the user request the service until the user start to receive information once the service is established. The delay experienced by a streaming service user is the sum of GPRS attach time, PDP context activation and total setup times.
6 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005
QoS Key Performance Indicators• Packet Delay Variation (Jitter): This is the delay variation of arrived packets. • Packet losses: The number of media packets (video/audio) lost during the streaming session.
• Throughput: The number of received bits per unit time. Total throughput includes (media + headers) throughputs.
• Number and length of rebufferings: This measure is of great importance to the user because the rebuffering operation can stop the audio/video playback. In this thesis only the rebuffering periods that cause a pause in video/audio playback are taken into account.
• Subjective Audio/Video quality: The following simple rating scale is defined through streaming QoS measurements
• Excellent• Good• Fair• Poor• Bad
• Lip synchronisation: This metrics is used to measure the synchronization between audio and video. The metrics can be subjective and objective. The former can be based on the following rating scale.
• subjective Lip synchronization
– In synch– Just noticeably out-of-synch– Unacceptably out-of-synch
• The later can be calculated in the streaming client as the ratio between displayed frames to decoded frames (Lip synchronization index)
7 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005
Environment description
• The measurements were done in the NTN (Nokia Test Network), in Leppävaara area during weeks 45-52 , 2004
15 ©NOKIA E2E Perfomance Optimisation.ppt
GGSNBackbone3G -SGSNHelix Universal Server Mobileas streaming server
RANIu PS
Gi
RNC
Backbone
GGSN3G-SGSNDial up connection
to requestingstreaming class with AT commands
Streaming application
8 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005
Measurements Tools
• Nokia OSS system for logging cell load information • Nemo Outdoor • Global Positioning System (GPS) • Nokia 3G Test Mobile “ Nokia 7600 model”• Nokia Traffic Generator Tool • Helix Mobile Producer • Commview Packet Analyser • Ethereal Packet Analyser • Matlab Tool • Excel • Windows Grep • Nokia Internal Streaming Client • Nokia Internal Streaming Server • Helix Universal Server -Mobile
9 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005
Preliminary Streaming QoS Tests
Objectives: Preliminary evaluation of multimedia streaming over Nokia WCDMA Test Network with different
client –server streaming solutions.
Test Description:
Four tests were executed by using two different streaming client–server solutions, two tests for each solution.
The client-server streaming solutions are:
1. Nokia Internal Streaming Client and Nokia Internal Streaming Server
2. Nokia Internal Streaming Client and Helix Universal Server-Mobile
The test was done in stationary situation near to the WCDMA base station where the carrier to interference
ratio Ec/No > -6 dB. Two different RT PS RABs were used, 64/64 and 64/128 UL/DL. Multimedia files were
used with bit rates 51 and 115 kbps over 64/64 and 64/128 bearers respectively.
10 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005
Preliminary Streaming QoS Tests
Video Audio
Subjective QoS (MOS) Good Good
Packet Loss Ratio, Number of expected packets to be received
0.5%, 589 0.0%, 750
Preliminary test 1:Multimedia streaming over 64/64 kbps UL/DL RT PS RAB, BLER target 1%
Environment: Nokia internal streaming client + Nokia internal streaming
The test results are presented in Table 1. Average video/audio packets sizes are 450 and 150 byte respectively
Table 1 Streaming performance over 64/64 RT RAB
The results show a good performance of multimedia streaming when using Nokia Streaming Client –Server. The MOS was “Good” for both audio and video.
11 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005
Preliminary Streaming QoS Tests
Preliminary test 2: Multimedia streaming over 64/128 kbps UL/DL RT PS RAB, BLER target 1%
Environment: Nokia internal streaming client + Nokia internal streaming
The test results are presented in Table 2. Average video/audio packets sizes are 381 and 313
byte respectively
Table 2 Streaming performance over 64/128 RT RAB
Video Audio
Subjective QoS (MOS) Fair Fair
Packet Loss Ratio, Number of expected packets to be received
1.9%, 2000 0.3%, 300
The results show a satisfactory performance of multimedia streaming when using Nokia Streaming Client –Server solution. The MOS was “Fair” for both audio and video. Video packet losses were in acceptable level.
12 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005
Preliminary Streaming QoS Tests
Figure 1 Helix Server throughput over 64/64 RT bearer
Streaming Server sending bit rate more than guaranteed channel bit rate
Preliminary test 3: Multimedia streaming over 64/64 kbps UL/DL RT PS RAB, BLER target 1%
Environment: Nokia internal streaming client + Helix Server
The test results are presented in Table 3. Average video/audio packets sizes are 485 and 150 byte respectively.
The results show a slightly high video packet losses and low audio packet losses, high video packet losses explain
the poor perceived video quality
Video Audio
Subjective QoS (MOS)
Poor Fair
Packet Loss Ratio, Number of expected packets to be received
6.6%, 634 0.3%, 299
Table 3 Streaming performance over 64/64RT RAB
13 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005
Preliminary Streaming QoS Tests
Figure 2 Helix Server throughput over 64/128 RT bearer
Preliminary test 4: Multimedia streaming over 64/128 kbps UL/DL RT PS RAB, BLER target 1%
Environment: Nokia internal streaming client + Helix Server
The test results are presented in Table 4. Average video/audio packets sizes are 333 and 150 byte respectively.
The results show a very high video/audio packet loss, which explain the poor perceived video/audio quality. Streaming Server sending bit rate more
than guaranteed channel bit rate
Video Audio
Subjective QoS (MOS)
Bad Bad
Packet Loss Ratio, Number of expected packets to be received
29.6 %, 2136 31.1 %, 301
Table 4 Streaming performance over 64/128RT RAB
14 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005
Preliminary Streaming QoS Tests
• The streaming performance over 64/64 and 64/128 UL/DL real time bearers was bad when using Helix server.
• The streaming performance over 64/64 and 64/128 UL/DL real time bearers was satisfactory when using
Nokia internal streaming server.
The tests had shown that streaming server was very often sending media packets with short delays and the
data rate exceeded the channel maximum bit rate. Data rate that exceed the maximum channel rate will be
discarded according to token bucket algorithm. The current implementation of token bucket algorithm is
done according to 3GPP specification [TS23.107 Annex B], which is not handling appropriately the burst
traffic situation. As a result of these findings the token bucket algorithm was modified to be able to tolerate
traffic burstiness conditions. This modification allows streaming class to use traffic shaping functionality,
which smooth the traffic pattern and decrease the burstiness of the network.
All the measurements that follow were done with the modified token bucket algorithm except SHO
measurements.
15 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005
Streaming QoS Measurements over Real Time Bearers in Unloaded Cell
Objectives:
To verify the performance of multimedia streaming over Nokia WCDMA Test Network in a stationary situation
Test 1: Multimedia streaming over 64/64 kbps UL/DL RT PS, BLER target 1%
Environment: Nokia internal streaming client + Helix Server
AT command used :at+cgeqreq=1,1,64,64,64,64,0,1500,"1E3",,0,1000
Total number of video and audio packets sent by streaming server is 589 and 750 respectively. Average video
and packet sizes are 485 and 150 byte respectively.
Table 5 Call setup QoS KPIs over 64/64 streaming bearer, BLER 1%
Trial Setup time
msec
Buffering Time
msec
Total Setup time
msec
1 6539 5017 11556
2 6498 5017 11515
3 6522 5057 11579
The end user will experience the following delays before start to see the movie e.g in trial 1 the end user experience the following delaysAttach time+PDP context time + setup time + buffering time = 6.7+6.53+5=18.23 secSee GPRS attach and PDP context measurements.
16 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005
Streaming QoS Measurements over Real Time Bearers in Unloaded Cell
Total throughput kbit/s (1 sec. window)
0
10
20
30
40
50
60
70
1 11 21 31 41 51
sec.
kbit
/s Measured
Reference
Trial Average Throughput
kbps
PLR %
audio
PLR%
video
Packet Delay video
Min,max,avg
msec
Packet Delay Audio
Min,max,avg
msec
MOS
video
MOS
audio
Lip Sync
index
Subjective lip synchronization
Number and Length of rebufferings
(msec)
1 58.29 0.33% 1.26% 0,521,96.7 30,671,200.5 Good Good 0.99 In synch 0 / 0
2 57.15 1.67% 2.05% 0,333,97 10,681,204.6 Fair Good 0.69 In synch 0 / 0
3 57.99 1.67% 0.33% 0,301,96 40,681,204.2 Good Good 0.85 In synch 0 / 0
Table 6 End-to-End and objective/subjective video/audio QoS KPIs over 64/64 streaming bearer, BLER target 1%
Figure 3 shows the total throughput (media + header throughputs) received by the streaming client. The received throughput is within the channel bandwidth in most of the times.
Figure 3 Streaming client total throughput over 64/64 RT bearer
17 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005
Streaming QoS Measurements over Real Time Bearers in Unloaded Cell
Objectives:
To verify the performance of multimedia streaming over Nokia WCDMA Test Network in a stationary situation
Test 2: Multimedia streaming over 64/128 kbps UL/DL RT PS, BLER target 1%
Environment: Nokia internal streaming client + Helix Server
AT command used :at+cgeqreq=1,1,64,128,64,128,0,1500,"1E3",,0,1000
Total number of video and audio packets sent by streaming server is 2136 and 310 respectively. Average
video and packet sizes are 333 and 150 byte respectively.
Table 7 Call setup QoS KPIs over 64/128 RT streaming bearer, BLER 1%
Trial Setup time
msec
Buffering Time
msec
Total Setup time
msec
1 11141 5017 16158
2 7048 5017 12065
18 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005
Streaming QoS Measurements over Real Time Bearers in Unloaded Cell
Packet losses are in acceptable level Subjective QoS was good Lip sync index shows a satisfactory valuesNo pause and rebufferings occurred during thestreaming session.The received throughput is within the channel bandwidth in most of the times.
Trial Average Throughput
kbps
PLR %
audio
PLR%
video
Packet Delay video
Min,max,avg
msec
Packet Delay Audio
Min,max,avg
msec
MOS
video
MOS
audio
Lip Sync
index
Subjective lip synchronization
Number and Length of rebufferings
(msec)
1 119.44 0.3%
0.33 %
0,171,28.4 10,621,202
Good Good
0.57 Just noticeably out of synch
0 / 0
2 119.32 0.3%
0.23% 0,160,28 20,601,201
Good Good
0.97 In Synch 0 / 0
Table 8 End-to-End and objective/subjective video/audio QoS KPIs over 64/128 streaming bearer, BLER target 1%
Total throughput kbit/s (1 sec. window)
0
20
40
60
80
100
120
140
1 11 21 31 41 51
sec.
Measured
Reference
Figure 4 Streaming client total throughput over 64/128 RT bearer
19 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005
Streaming QoS Measurements over Real Time Bearers in Unloaded Cell
Objectives:
To verify the performance of multimedia streaming over Nokia WCDMA Test Network in a stationary situation
Test 3: Multimedia streaming over 64/256 kbps UL/DL RT PS, BLER target 1%
Environment: Nokia internal streaming client + Helix Server
AT command used :at+cgeqreq=1,1,64,256,64,256,0,1500,"1E3",,0,1000
Total number of video and audio packets sent by streaming server is 4066 and 297 respectively. Average
video and packet sizes are 341 and 321 byte respectively.
Table 9 Call setup QoS KPIs over 64/256 streaming bearer, BLER 1 %Trial Setup time
msec
Buffering Time
msec
Total Setup time
msec
1 10006 5017 15023
2 9020 5017 14037
3 10851 5017 15868
20 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005
Streaming QoS Measurements over Real Time Bearers in Unloaded Cell
Trial Average Throughput
kbps
PLR %
audio
PLR%
video
Packet Delay video
Min,max,avg
msec
Packet Delay Audio
Min,max,avg
msec
MOS
audio
MOS
video
Lip Sync
index
Subjective lip synchronization
Number and length of rebufferings
msec
1 203.73 0.9% 0.93 %
0,821,16.9 10,991,207 Fair Poor 0.46 Unacceptably out-of-synch
5 / 5860
2 177 1% 1 % 0,4707,17 10,1192,206
Fair Bad 0.34 Unacceptably out-of-synch
11 / 5947
3 187 1.3% 0.8% 0,3345,17.6 0,1192,209 Fair Bad 0.38 Unacceptably out-of-synch
5 / 5860
Figure 5 shows the total throughput received by the streaming client. The total throughput was exceeding the channel bandwidth in most of the times .Subjective video/audio quality was not totally satisfactory Lip sync index shows a very low values which indicates that the video and audio are out of syncMany rebufferings occurred during the video.
Table 10 End-to-End and objective/subjective video/audio QoS KPIs over 64/256 streaming bearer, BLER target 1%
Total throughput kbit/s (1 sec. window)
0
50
100
150
200
250
300
350
400
1 11 21 31 41 51
sec.
kbit
/s Measured
Reference
Figure 5 Streaming client total throughput over 64/128 RT bearer
21 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005
Streaming QoS Measurements over Non Real Time Bearers in Unloaded Cell
Trial Setup Time
msec
Buffering Time
msec
Total Setup time
msec
1 6839 5017 11846
2 6778 5017 11795
3 6922 5057 11979
Objectives:
Total number of video and audio packets sent by streaming server is 2136 and 300 To verify the performance of multimedia streaming over NRT bearer in a stationary situation
Test 4: Multimedia streaming over 64/64 kbps UL/DL NRT PS, BLER target 1%
Environment: Nokia internal streaming client + Helix Server
respectively. Average video and packet sizes are 333 and 321 byte respectively.
Table 11 Call setup QoS KPIs over 64/64 NRT streaming bearer, BLER 1 %
22 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005
Streaming QoS Measurements over Non Real Time Bearers in Unloaded Cell
Trial
Average Throughput
kbps
PLR %
audio
PLR%
video
Packet Delay video
Min,max,avg
msec
Packet Delay Audio
Min,max,avg
msec
MOS
video
MOS
audio
Lip Sync
index
Subjective lip synchronization
Number and Length of rebufferings
(msec)
1 58.29 0.0% 0.0 % 0,555,96.7 30,671,200 Good Good 0.86 In Sync 0 / 0
2 57.15 0.0% 0.0% 0,340,97 10,681,204 Good Good 0.76 In Sync 0 / 0
3 57.99 0.0% 0.0% 0,370,96 40,680,203 Good Good 0.82 In Sync 0 / 0
Table 12 End-to-End and objective/subjective video/audio QoS KPIs over 64/64 NRT bearer, BLER target 1%
No Packet losses Subjective video and audio QoS was good Lip sync index shows a good values No pause and rebuffering occurred during thestreaming session.
23 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005
Streaming QoS Measurements over Non Real Time Bearers in Unloaded Cell
Trial Setup Time
msec
Buffering Time
msec
Total Setup time
msec
1 10230 5017 15247
2 8921 13938 11338
Objectives:
To verify the performance of multimedia streaming over NRT bearer in a stationary situation
Test 5: Multimedia streaming over 64/128 kbps UL/DL NRT PS, BLER target 1%
Environment: Nokia internal streaming client + Helix Server
Total number of video and audio packets sent by streaming server is 2136 and 300 respectively. Average
video and packet sizes are 333 and 321 byte respectively.
Table 13 Call setup QoS KPIs over 64/128 NRT streaming bearer, BLER 1 %
24 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005
Streaming QoS Measurements over Non Real Time Bearers in Unloaded Cell
Trial Average Throughput
kbps
PLR %
audio
PLR%
video
Packet Delay video
Min,max,avg
msec
Packet Delay Audio
Min,max,avg
msec
MOS
video
MOS
audio
Lip Sync
index
Subjective lip synchronization
Number and length of rebufferings
msec
1 119.98 0.0% 0.0 %
0,231,28.3 20,590,200 Good
Good 0.62 In Sync 0 / 0
2 119.96 0.0% 0.0% 0,230,28.4 10,661,202 Good
Good 0.81 In Sync 0 / 0
No Packet losses Subjective video and audio QoS was good Lip sync index shows a good values No pause and rebuffering occurred during thestreaming session.The received throughput is exceeding the channel Bandwidth sometimes
Total throughput kbit/s (1 sec. window)
0
50
100
150
200
250
1 11 21 31 41 51
sec.
kbit
/s Measured
Reference
Table 14 End-to-End and objective/subjective video/audio QoS KPIs over 64/128 NRT bearer, BLER target 1%
Figure 6 Streaming client total throughput over 64/128 NRT bearer
25 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005
Streaming QoS Measurements over Non Real Time Bearers in Unloaded Cell
Trial Setup Time
msec
Buffering Time
msec
Total setup time
msec
1 9813 5018 14831
2 12071 5017 17071
Objectives:
To verify the performance of multimedia streaming over NRT bearer in a stationary situation
Test 6: Multimedia streaming over 64/384 kbps UL/DL NRT PS, BLER target 1%
Environment: Nokia internal streaming client + Helix Server
Total number of video and audio packets sent by streaming server is 4066 and 301 respectively. Average
video and packet sizes are 333 and 321 byte respectively.
Table 15 Call setup QoS KPIs over 64/384 NRT streaming bearer, BLER 1 %
26 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005
Streaming QoS Measurements over Non Real Time Bearers in Unloaded Cell
Trial Average Throughput
kbps
PLR %
audio
PLR%
video
Packet Delay video
Min,max,avg
msec
Packet Delay Audio
Min,max,avg
msec
MOS
audio
MOS
video
Lip Sync
index
Subjective lip synchronization
Number and length of rebufferings
msec
1 213.94 0.8 % 1.1 % 0,390,15.3 0,1041,204 Good Fair 0.45 Just noticeably out-of-synch
2 / 5743
2 219.4 0.0% 0.1% 0,491,14.9 0,931,203 Good Fair 0.5 Just noticeably out-of-synch
2/ 5944
Packet losses ~ 1% Subjective audio/video QoS was satisfactoryLip sync index shows a relatively low values The received throughput is exceeding the channel Bandwidth very often
Total throughput kbit/s (1 sec. window)
0
50
100
150
200
250
300
350
400
1 11 21 31 41 51
sec.
kbit
/s Measured
Reference
Table 16 End-to-End and objective/subjective video/audio QoS KPIs over 64/384 NRT bearer, BLER target 1%
Figure 7 Streaming client total throughput over 64/384 NRT bearer
27 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005
Streaming in Highly Loaded Cell With Conversational Class Users
Objective: To verify that the guaranteed bit rate of a streaming class user is kept in a highly loaded cell.
Test Description:
The test was executed by starting a single video streaming session with a predefined data rate and loading
the cell with conversational class users (voice calls). The throughput of streaming user was monitored.
Environment:
Increasing common channels power by 11 dB
28 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005
Streaming in Highly Loaded Cell With Conversational Class Users Test 7: Streaming over 64/64 RT bearer
Total # of send packets =1389Total # of packets lost = 15Packet losses ~1%
Throughput decreased significantly due to packet losses
High BLERDL BLER
Throughput PPP Layer
CPICH RSCP
UE Tx Pwr
CPICH Ec/No
29 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005
Streaming in Highly Loaded Cell With Conversational Class Users Test 7: Streaming over 64/64 RT bearer
PtxTotal DL=~ Ptxtarget DL
UL RT Load
New AMR calls arenot admitted after 33 existing AMR calls
30 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005
Streaming in Highly Loaded Cell With Conversational Class Users Test 8: Streaming over 64/128 RT bearer
Total # of send packets =8118Total # of packets lost = 36Packet losses 0.4%
Throughput is not affected by cell congestion
Throughput PPP Layer
DL BLER
CPICH RSCP
UE Tx Pwr
CPICH Ec/No
New AMR calls arenot admitted after 37 existing AMR calls
31 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005
Streaming in Highly Loaded Cell With Conversational Class Users Test 9: Streaming over 64/256 RT bearer
Total # of send packets =8247Total # of packets lost = 34Packet losses 0.4%
CPICH EcNo
UE TX Power
CPICH RSCP
Throughput PPP Layer
DL BLER
New AMR calls arenot admitted after 37 existing AMR calls
32 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005
Streaming Performance Measurement in Mobile Situation
Softer handover
Soft handover
• Objective: To verify the performance of multimedia streaming over real time bearer during soft handover in Nokia WCDMA Test Network.
The description of tests is as follow:
• Two tests were executed over two streaming class bearers, one test per bearer. The bearer bit rates are 64/64 and 64/128 kbps UL/DL. BLER target value was set to 1% for all tests. Multimedia files were used with bit rates 103 and 213 kbps over 64/128 and 64/256 RT PS RABs respectively. The tests were conducted by using Nokia internal streaming client and Nokia streaming server. Streaming performance was verified during soft handover. The carrier to interference ration was “Ec/No > -6” dB.
Every test is repeated 2-3 times to
ensure reliability.
This test was executed with original
token bucket algorithm
33 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005
Streaming Performance Measurement in Mobile Situation Streaming over 64/128 RT bearer
Objectives:
To verify the performance of multimedia streaming during SHO
Test 10: Multimedia streaming during SHO over 64/128 kbps UL/DL RT PS, BLER target 1%
Environment: Nokia internal streaming client + Nokia internal streaming server
AT command used :at+cgeqreq=1,1,64,128,64,128,0,1500,"1E3",,0,1000
Total number of video and audio packets sent by streaming server is 1880 and 296 respectively. Average
video and packet sizes are 382 and 313 byte respectively.
Table 17 Call setup QoS KPIs over 64/128 streaming bearer, BLER 1 %
Trial Setup time
msec
Buffering Time
msec
Total Setup time
msec
1 4196 5017 9213
2 4126 5017 9143
2 3935 5017 8952
34 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005
Streaming Performance Measurement in Mobile Situation Streaming over 64/128 RT bearer
Video packet losses are high due to token packet
algorithm problemLip sync index shows a satisfactory values No rebufferings occurred during the streaming session.
Total throughput kbit/s (1 sec. window)
0
20
40
60
80
100
120
140
1 11 21 31 41 51
sec.
kbit
/s Measured
Reference
Trial Average Throughput
kbps
PLR %
audio
PLR%
video
Packet Delay video
Min,max,avg
msec
Packet Delay Audio
Min,max,avg
msec
MOS
video
MOS
audio
Lip Sync
index
Subjective lip synchronization
Number and Length of rebufferings
(msec)
1 112.8 6% 7.34%
0,221,33 10,651,212 Fair Good 0.58 Just noticeably out-of-synch
0 / 0
2 112.8 5.4% 7.5 % 0,341,34 60,631,210 Fair Good 0.56 Just noticeably out-of-synch
0 / 0
3 112.9 7% 7.3% 0,251,33 20,570,214 Fair Good 0.55 Just noticeably out-of-synch
0 / 0
Table 18 End-to-End and objective/subjective video/audio QoS KPIs over 64/128 RT bearer during SHO, BLER target 1%
Figure 8 Streaming client total throughput over 64/128 RT bearer
35 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005
Streaming Performance Measurement in Mobile Situation Streaming over 64/128 RT bearer
Soft HOSofter HO
Soft handover takes place between different base stations sites within one RNC (intra-RNC soft handover) or
within separate controlled RNCs (inter-RNC soft handover). In softer handover the UE is controlled by at least
two cells belonging to the same base station site.
36 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005
Streaming Performance Measurement in Mobile Situation Streaming over 64/256 RT bearer
Objectives:
To verify the performance of multimedia streaming during SHO
Test 11: Multimedia streaming during SHO over 64/256 kbps UL/DL RT PS, BLER target 1%
Environment: Nokia internal streaming client + Nokia internal streaming server
AT command used :at+cgeqreq=1,1,64,256,64,256,0,1500,"1E3",,0,1000
Total number of video and audio packets sent by streaming server is 4066 and 297 respectively. Average
video and packet sizes are 341 and 321 byte respectively.
Table 19 Call setup QoS KPIs over 64/256 streaming bearer, BLER 1 %
Trial Setup time
msec
Buffering Time
msec
Total Setup time
msec
1 3675 5017 8692
2 3856 5017 8873
2 3846 5017 8863
37 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005
Streaming Performance Measurement in Mobile Situation Streaming over 64/256 RT bearer
Trial Average Throughput
kbps
PLR %
audio
PLR%
video
Packet Delay video
Min,max,avg
msec
Packet Delay Audio
Min,max,avg
msec
MOS
audio
MOS
video
Lip Sync
index
Subjective lip synchronization
Number and length of rebufferings msec
1 220 2.3% 7.25% 0,180,22 60,460,204 Good Fair 0.56 Just noticeably out-of-synch
0 / 0
2 221 1.3% 6.95% 0,170,21 10,445,202 Good Fair 0.56 Just noticeably out-of-synch
0 / 0
3 220 3.3% 7.05% 0,330,21 30,511,207 Good Fair 0.55 Just noticeably out-of-synch
0 / 0
Video packet losses are high due to token packet
algorithm problemSubjective audio/video QoS was satisfactoryLip sync index shows satisfactory values No rebufferings occurred during the streaming session.
Total throughput kbit/s (1 sec. window)
0
50
100
150
200
250
300
1 11 21 31 41 51
sec.
kbit
/s Measured
Reference
Table 20 End-to-End and objective/subjective video/audio QoS KPIs over 64/256 RT bearer during SHO, BLER target 1%
Figure 9 Streaming client total throughput over 64/256 RT bearer
38 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005
Streaming Performance Measurement in Mobile Situation Streaming over 64/256 RT bearer
Soft HOSofter HO
39 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005
GPRS Attach and PDP Context Activation Times
Objectives: To measure the GPRS attach and PDP context activation times over streaming class bearers.
Three different RAB bit rates are used, 64/64, 64/128, and 64/256 kbps UL/DL. BLER target value was 1% for
all tests. Every test was repeated 15 times to ensure reliability.
Attach Times were calculated as the difference time between ATTACH_REQ and ATTACH_COMPLETE messages.
PDP context activation time was calculated as a time difference between PDP_context_create_req and
PDP_context_activate_ind messages
Table 33 shows that average PDP context + GPRS attach time for 64/64, 64/128, and 64/256 are 6.71, 7.09, and 7.28 seconds.
Table 21 Average GPRS attach+ PDP Context Activation.
Real Time Bearer PDP context + GPRS attach
Average Time (sec)
64/64 6.71
64/128 7.09
64/256 7.28
40 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005
Summary & Conclusions
• The preliminary evaluation tests using Nokia internal streaming server had shown a good performance of multimedia streaming over 64/64 UL/DL real time bearer and a satisfactory performance over 64/128 UL/DL real time bearer.
• The preliminary evaluation tests using Helix streaming server had shown a bad performance of multimedia streaming over 64/64 and 64/128 UL/DL real time bearers.
The following conclusions are drawn from the preliminary evaluation tests:
• Nokia streaming server has shown a better performance than Helix streaming server.
• The bad performance of multimedia streaming over streaming class was due to the large number of discarded packets by the token bucket algorithm. The current implementation of token bucket algorithm, which is according to 3GPP specification was not suitable to control the traffic flow of streaming servers.
• The results obtained after token bucket modification have shown a good performance of multimedia streaming over 64/64 and 64/128 real time bearers and the performance over 64/256 UL/DL RT bearer was not totally satisfactory.
• The performance of video streaming over 64/64 and 64/128 NRT bearers was good. Video and audio quality was good and video/audio packet losses were 0.0% in all trials.
41 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005
Summary & Conclusions
• The performance of video streaming over 64/384 UL/DL NRT bearer was satisfactory.
• The performance of multimedia streaming over 64/128 and 64/256 UL/DL RT during SHO was satisfactory. (original token bucket algorithm + Nokia streaming server)
The following conclusions are drawn from the streaming verification tests after token bucket algorithm modifications.
• The performance of video streaming over real time bearer was significantly improved after token bucket algorithm modification.
• Multimedia streaming over NRT bearers in unloaded cell show a slightly better performance than streaming over RT bearers.
• The streaming service user will experience a delay of ~20 seconds before the media contents starts to be displayed in the streaming client. This delay includes attach, context, and total setup times.
42 © NOKIA Streaming QoS/Musa Aldarawsheh , April 8th 2005
References
[Hol] Harri Holma and Antti Toskala, ”’WCDMA for UMTS Radio Access For Third Generation Mobile Communications”, Second Edition, John Wiley & Sons, LTD, 2002
[Lai] Jaana Laiho, Achim Wacker and Tomas Novosad, “ Radio Network Planning and Optimisation for UMTS”, John Wiley & Sons, LTD, 2002, 484 pages.
[22105] 3GGP TS 22.105 Service aspects; Services and Service Capabilities, version 4.1.0, January 2001, 31 pages.
[23107] 3GPP Technical Specification 23.107: QoS concept and Architecture, version 4.6.0, December 2002, 38 pages