on losses, pauses and jumps and the wideband e-model
Post on 09-Jun-2015
124 Views
Preview:
DESCRIPTION
TRANSCRIPT
Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experimental
ON LOSSES, PAUSES AND JUMPS AND THE
WIDEBAND E-MODEL
Adil Raja Anna Jagodzinska Vincent Barriac
France Telecom R&D,TECH/OPERA/MOV
Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experimental
OUTLINE
1 BACKGROUND AND MOTIVATION
2 VOIP SIMULATION
3 METHODOLOGY
4 PREPARATION OF THE TEST MATERIAL
5 INTRODUCTION TO GP
6 EXPERIMENTAL SETUP
7 RESULTS AND ANALYSIS
Comparison With Multiple Linear RegressionComparison With E-ModelPerformance Evaluation Against Data From Auditory Tests
8 CONCLUSIONS
Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experimental
OUTLINE
1 BACKGROUND AND MOTIVATION
2 VOIP SIMULATION
3 METHODOLOGY
4 PREPARATION OF THE TEST MATERIAL
5 INTRODUCTION TO GP
6 EXPERIMENTAL SETUP
7 RESULTS AND ANALYSIS
Comparison With Multiple Linear RegressionComparison With E-ModelPerformance Evaluation Against Data From Auditory Tests
8 CONCLUSIONS
Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experimental
OUTLINE
1 BACKGROUND AND MOTIVATION
2 VOIP SIMULATION
3 METHODOLOGY
4 PREPARATION OF THE TEST MATERIAL
5 INTRODUCTION TO GP
6 EXPERIMENTAL SETUP
7 RESULTS AND ANALYSIS
Comparison With Multiple Linear RegressionComparison With E-ModelPerformance Evaluation Against Data From Auditory Tests
8 CONCLUSIONS
Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experimental
OUTLINE
1 BACKGROUND AND MOTIVATION
2 VOIP SIMULATION
3 METHODOLOGY
4 PREPARATION OF THE TEST MATERIAL
5 INTRODUCTION TO GP
6 EXPERIMENTAL SETUP
7 RESULTS AND ANALYSIS
Comparison With Multiple Linear RegressionComparison With E-ModelPerformance Evaluation Against Data From Auditory Tests
8 CONCLUSIONS
Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experimental
OUTLINE
1 BACKGROUND AND MOTIVATION
2 VOIP SIMULATION
3 METHODOLOGY
4 PREPARATION OF THE TEST MATERIAL
5 INTRODUCTION TO GP
6 EXPERIMENTAL SETUP
7 RESULTS AND ANALYSIS
Comparison With Multiple Linear RegressionComparison With E-ModelPerformance Evaluation Against Data From Auditory Tests
8 CONCLUSIONS
Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experimental
OUTLINE
1 BACKGROUND AND MOTIVATION
2 VOIP SIMULATION
3 METHODOLOGY
4 PREPARATION OF THE TEST MATERIAL
5 INTRODUCTION TO GP
6 EXPERIMENTAL SETUP
7 RESULTS AND ANALYSIS
Comparison With Multiple Linear RegressionComparison With E-ModelPerformance Evaluation Against Data From Auditory Tests
8 CONCLUSIONS
Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experimental
OUTLINE
1 BACKGROUND AND MOTIVATION
2 VOIP SIMULATION
3 METHODOLOGY
4 PREPARATION OF THE TEST MATERIAL
5 INTRODUCTION TO GP
6 EXPERIMENTAL SETUP
7 RESULTS AND ANALYSIS
Comparison With Multiple Linear RegressionComparison With E-ModelPerformance Evaluation Against Data From Auditory Tests
8 CONCLUSIONS
Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experimental
OUTLINE
1 BACKGROUND AND MOTIVATION
2 VOIP SIMULATION
3 METHODOLOGY
4 PREPARATION OF THE TEST MATERIAL
5 INTRODUCTION TO GP
6 EXPERIMENTAL SETUP
7 RESULTS AND ANALYSIS
Comparison With Multiple Linear RegressionComparison With E-ModelPerformance Evaluation Against Data From Auditory Tests
8 CONCLUSIONS
Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experimental
BACKGROUND
VoIP listening quality is not only distorted by packet lossand codec related impairments only.
Temporal discontinuities such as pauses and jumps(packet discards) also play a role. (S. Voran, 03)
Packet loss happens due to network congestion.
Jumps and Pauses happen due to the jitter/jitter bufferinteraction.
Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experimental
BACKGROUNDLOSS, PAUSE AND JUMPS
0 100 200 300 400 500−6
−4
−2
0
2
4
6
8
10x 10
−3
1 32
FIGURE: A sequence of 3 frames
Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experimental
BACKGROUNDLOSS, PAUSE AND JUMPS
0 100 200 300 400 500−6
−4
−2
0
2
4
6
8
10x 10
−3
1 E 3
FIGURE: Loss
Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experimental
BACKGROUNDLOSS, PAUSE AND JUMPS
0 100 200 300 400 500−6
−4
−2
0
2
4
6
8
10x 10
−3
1 E 2
FIGURE: Pause
Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experimental
BACKGROUNDLOSS, PAUSE AND JUMPS
0 100 200 300 400 500−6
−4
−2
0
2
4
6
8
10x 10
−3
1 3 4
FIGURE: Jump
Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experimental
VOIP SIMULATION
Sender ReceiverJitter Loss
FIGURE: VoIP Simulation
Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experimental
WEIBULL DISTRIBUTION
VoIP jitter is a self-similar phenomenon that can be modeled bya heavy tailed distribution.Notable distributions are:
Weibull (✓), Pareto, Exponential.
Weibull distribution is characterized by: A shape parameter(A), a scale (B) parameter, and a location parameter.
Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experimental
WEIBULL DISTRIBUTION
0 50 100 150 200 250 300 3500
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2x 10
4
FIGURE: A=1
Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experimental
WEIBULL DISTRIBUTION
0 10 20 30 40 50 60 70 80 900
500
1000
1500
2000
2500
3000
3500
4000
4500
5000
FIGURE: A=2
Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experimental
WEIBULL DISTRIBUTION
0 10 20 30 40 50 60 70 800
500
1000
1500
2000
2500
3000
3500
4000
4500
5000
FIGURE: A=2.5
Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experimental
PACKET LOSS
1 (NO LOSS)
0 (LOSS)
p 1-q
q
1-p
FIGURE:
Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experimental
LOSS, PAUSE AND JUMP STATE MODEL
A loss, pause jump state model can be learned from anetwork trace analysis or the network emulation.
with state transition probabilities.
Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experimental
LOSS, PAUSE AND JUMP STATE MODEL
Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experimental
LOSS, PAUSE AND JUMP STATE MODEL
Conversely the state model can be used to generaterealistic loss, pause and jump patterns given realisticvalues for mean loss, pause and jump rates.
For instance:
n2l = l2n ×(mlr)
1−mlr .
l2n = 1 − clp.
Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experimental
VOIP SIMULATION SYSTEM
FIGURE: Simulation system for derivation of Ie,WB,eff
Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experimental
NETWORK TRAFFIC PARAMETERS
TABLE: Various Network Traffic Parameters
No. Variable Abbreviation1 mean loss rate mlr2 mean burst length – loss mbl_loss3 mean pause rate mpr4 mean burst length – pause mbl_pause5 mean jump rate mjr6 mean burst length – jump mbl_jump7 mean impairment rate mir=mlr+mpr+mjr8 mean burst length impairments mbl_impairment9 equipment impairment factor Ie,WB
10 gradient of the Ie,WB,eff grad
Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experimental
Ie,WB,eff VS mir
0 5 10 15 20 250
10
20
30
40
50
60
70
80
MIR
I e,W
B,e
ff
G.722G.729G.711
FIGURE: Ie,WB,eff vs mir for ITU-T G.711, G.729 and G.722
Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experimental
VARIOUS TEMPORAL DISCONTINUITY RATES AND THE
RESPECTIVE BURST LENGTHS
TABLE: Various Temporal Discontinuity Rates and The RespectiveBurst Lengths
Temporal Discontinuity Rate Burst Length0 0
0.005 1, 20.01 1, 2, 40.015 1, 2, 30.02 1, 2, 40.025 1, 2, 50.03 1, 2, 30.035 1, 2, 70.04 1, 4, 8
Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experimental
INTRODUCTION TO GENETIC PROGRAMMING (GP)
Genetic Programming is a coarse emulation of DarwinianEvolution.
The search space is composed of all the possiblecomputer programs.GP Life Cycle:
1 Create an initial population of computer programs.2 Evaluation.3 Selection.4 Reproduction.5 Evaluation.6 Replacement.7 Continue from 3.
Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experimental
A TYPICAL GP BREEDING CYCLE
FIGURE:
Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experimental
TABLE: Common GP Parameters among all experiments
Parameter ValueInitial Population Size 300Initial Tree Depth 6Selection LPPTournament Size 2Genetic Operators Crossover and Subtree MutationOperators Probability Type AdaptiveInitial Operator probabilities 0.5 eachSurvival Half ElitismGeneration Gap 1Function Set plus, minus, multiply, divide,sin,
cos, log2, log10, loge, sqrt,power, if
Terminal Set Random real-valued numbersbetween 0.0 and 1.0. Integers (2–10) andNetwork traffic parameters from Table 1.
Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experimental
RESULTS AND ANALYSIS
TABLE: Statistical analysis of the GP experiments and derived models
(a) RMSE Statistics for Best Individuals of 50 Runs for Experiments 1, 2 & 3Experiment 1 Experiment 2 Experiment 3
Stats RMSEtr RMSEte Size RMSEtr RMSEte Size RMSEtr RMSEte SizeMean 5.5482 98.9506 26.2800 5.55 18.94 29.34 5.34 13.90 28.18Std.Dev. 0.3514 152.59 11.4661 0.39 70.27 12.3612 0.4612 47.19 9.73Max. 5.97 500 68 6.0084 494.52 73 5.89 333.60 59Min. 4.5409 4.82 11 4.41 4.37 6 4.40 4.38 14
(b) Results of Mann-Whitney-Wilcoxon Significance TestExperiment 1 Experiment 2 Experiment 3
RMSEtr RMSEte Size RMSEtr RMSEte Size RMSEtr RMSEte SizeExperiment 1 0 0 0 0 1 0 1 1 0Experiment 2 0 1 0 0 0 0 1 0 1Experiment 3 1 1 0 1 0 1 0 0 0
Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experimental
RESULTS AND ANALYSIS
TABLE: Performance Statistics of the Proposed Models
Training TestModel RMSEsMOS RMSEs Ie,WB,eff σ Ie,WB,eff RMSEs MOS RMSEs Ie,WB,eff σ Ie,WB,effEquation (1) 0.1763 4.8185 0.8840 0.1759 4.8182 0.8805Equation (2) 0.1602 4.4108 0.9038 0.1596 4.3708 0.9028Equation (3) 0.1619 4.4021 0.9042 0.1611 4.3808 0.9023Equation (4) 0.1692 4.6026 0.8948 0.1679 4.5460 0.8944Equation (5) 0.1764 4.8644 0.8816 0.1948 5.4231 0.8781
Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experimental
THE PROPOSED MODELS
Ie,WB,eff = (1){
mir × cos(Ie,WB) +
√
mbl_imp
Ie,WB− mir 1/4 − mir
}
×(−163.87)− 9.35
Ie,WB,eff = (2)
√
√
log10(grad)
mir + 9+
(sin(mir ) + mir )log10(grad)
4 −
√ √mpr
mbl_loss+9
√
log10(grad)−
√
log10(grad)mbl_jump+9
×(−0.0933) + 87.1174
Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experimental
THE PROPOSED MODELS
Ie,WB,eff = (3)
log10
(
0.54grad + 3 × mir
)
+log10
(
0.74grad +2×mir
)
3
7 × log10
(
0.54grad + 2 × mir + 6.56 −
√
mbl_imp)
+ mir
×(270.37) + 102.40
Ie,WB,eff = (4)
(sin(grad × mir ))
√40×mbl_imp
Ie,WB × 107.43 − 5.94
Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experimental
SCATTER PLOTS
0 10 20 30 40 50 60 70 80 900
10
20
30
40
50
60
70
80
90
Ie,WB,eff
−− GP
I e,W
B,e
ff −−
WB
−P
ES
Q
(a)
0 10 20 30 40 50 60 70 80 900
10
20
30
40
50
60
70
80
90
Ie,WB,eff
−− GP
I e,W
B,e
ff −−
WB
−P
ES
Q(b)
FIGURE: Ie,WB,eff predicted by equation (3) vs target Ie,WB,eff for: (a)training data (b) test data
Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experimental
PARAMETER SIGNIFICANCE ANALYSIS
Ie,wb grad mlr mbl_loss mjr mbl_jump mpr mbl_pause mir mbl_impairment0
10
20
30
40
50
60
70
80
90
100
Experiment 1Experiment 2
FIGURE: Percentage of the best individuals employing various inputparameters in acceptable runs of each of the two experiments.
Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experimental
COMPARISON WITH MULTIPLE LINEAR REGRESSION
Ie,WB,eff = (5)
0.35 × Ie,WB − 0.006 × grad + 383.62 × mir − 1.18 ×
mbl_imp + 34.65
Has inferior performance as opposed to proposed models.
Results are reported in Table 5
Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experimental
COMPARISON WITH E-MODEL
Ie,WB,eff = Ie,WB + (129 − Ie,WB)×Ppl
PplBurstR + Bpl
(6)
TABLE: Comparison between the Prediction Accuracies of theE-Model and the Proposed Model
E-Model Equation (3)Codec RMSE RMSE RMSE RMSE(kbps) Bpl train test train testG.711 (64) 22.39 6.7971 6.7003 4.6748 4.5626G.729 (8) 30.50 4.0824 3.8701 3.0513 3.1362G.722 (64) 19.8053 8.1087 8.1510 5.6865 5.6093
Average – 6.3294 6.2405 4.4709 4.4360% PG – – – 29.36 28.92
Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experimental
PERFORMANCE EVALUATION AGAINST DATA FROM
AUDITORY TESTS
TABLE: Target Network Impairment Conditions for the Auditory Tests
Condition mlr mbl_loss mjr mbl_jump mpr mbl_pause mir mbl_impairment1 0 0 0 0 0 0 0 02 0.03 1.0 0 0 0 0 0.03 1.03 0.03 4.0 0 0 0 0 0.03 4.04 0 0 0.03 1.0 0 0 0.03 1.05 0 0 0.03 4.0 0 0 0.03 4.06 0 0 0 0 0.03 1.0 0.03 1.07 0 0 0 0 0.03 4.0 0.03 4.08 0.06 1.0 0 0 0 0 0.06 1.09 0.06 4.0 0 0 0 0 0.06 4.010 0 0 0.06 1.0 0 0 0.06 1.011 0 0 0.06 4.0 0 0 0.06 4.012 0 0 0 0 0.06 1.0 0.06 1.013 0 0 0 0 0.06 4.0 0.06 4.014 0.09 1.0 0 0 0 0 0.09 1.015 0.09 4.0 0 0 0 0 0.09 4.016 0 0 0.09 1.0 0 0 0.09 1.017 0 0 0.09 4.0 0 0 0.09 4.018 0 0 0 0 0.09 1.0 0.09 1.019 0 0 0 0 0.09 4.0 0.09 4.020 0.04 4.0 0.04 4.0 0.04 4.0 0.12 12.0
Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experimental
PERFORMANCE EVALUATION AGAINST DATA FROM
AUDITORY TESTS
TABLE: Comparison Between the Results of Auditory Tests andWB-PESQ
RMSE MOS σ MOS0.4475 0.8399
TABLE: Comparison between the Prediction Accuracies of theE-Model and the Proposed Model Against Data From Auditory Tests
E-Model Equation (3)Codec (kbps) Bpl Ie,WB RMSE σ RMSE σ
G.711 (64) 25.1 36 18.2549 0.7827 11.8532 0.8182G.729 (8) 19.0 47 34.9341 0.8249 9.4212 0.9309G.722 (64) 7.1 13 46.6618 0.8124 11.8840 0.8966
Overall – – 24.8994 0.3852 11.1129 0.8758
Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experimental
PERFORMANCE EVALUATION AGAINST DATA FROM
AUDITORY TESTS
0 0.06 0.12 0.06 0.12 0.06 0.12−50
0
50
100
150
200
MIR
I e,W
B,e
ff
AuditoryE−ModelProposedG.711
G.722
G.729
FIGURE: Ie,WB,eff vs mir derived from auditory tests, E-Model andequation (3) are plotted for G.711, G.722 and G.729.
Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experimental
CONCLUSIONS
We have proposed a new methodology that employs GP toderive novel equipment impairment factors.
The poposed models outperform the existing E-Modelformulation.
We have taken into account additional sources ofimpairments; pauses and jumps.
We have also proposed a 4-state loss, pause and jumpMarkov model to characterize VoIP traffic.
The methodology is general and may be augmented withthe results of auditory tests.
top related