perceptual evaluation of speech quality (pesq)

11 TAC2000/2000.7

802.16 IP Telephony Lab

Perceptual Evaluation of Speech Quality (PESQ)

Speaker: Wen-Jen LinSpeaker: Wen-Jen LinDate: Dec. 3 2009Date: Dec. 3 2009

22 TAC2000/2000.7

802.16 IP Telephony Lab

Outline IntroductionIntroduction ITU-T P.862ITU-T P.862 P.862.3P.862.3 P.862.2P.862.2 P.862.1P.862.1 P.862 - Overview of the basic philosophy used in P.862 - Overview of the basic philosophy used in

PESQPESQ ConclusionConclusion ExperienceExperience Demo PESQ programDemo PESQ program ReferenceReference

33 TAC2000/2000.7

802.16 IP Telephony Lab

IntroductionClassic Quality MeasurementClassic Quality Measurement

Noise RatioNoise RatioFrequency Response Functions etc.Frequency Response Functions etc.Speech coding ( ETSI GSM EFR/AMR, ITU-T Speech coding ( ETSI GSM EFR/AMR, ITU-T

G.728/729/723.1 etc)G.728/729/723.1 etc)New types of distortionsNew types of distortions

Voice over IP (packet loss and variable delay)Voice over IP (packet loss and variable delay)Voice over ATM (cell loss)Voice over ATM (cell loss)Voice over mobile (GSM, UMTS, frame repeat, Voice over mobile (GSM, UMTS, frame repeat,

front end clipping, comfort noise generation)front end clipping, comfort noise generation)

44 TAC2000/2000.7

802.16 IP Telephony Lab

Introduction (cont.)PSQM (Perceptual Speech Qualify MeasurPSQM (Perceptual Speech Qualify Measur

e)e)P.861 - The first international standard for the P.861 - The first international standard for the

perceptual quality measurement of telephone-baperceptual quality measurement of telephone-band (300-3400 Hz) speech signals.nd (300-3400 Hz) speech signals.

The scope of recommendation P.861 was limiteThe scope of recommendation P.861 was limited to the assessment of telephone-band speech cd to the assessment of telephone-band speech codecs only.odecs only.

55 TAC2000/2000.7

802.16 IP Telephony Lab

ITU-T P.862 P.862P.862

Perceptual evaluation of speech quality (PESQ): An objective methoPerceptual evaluation of speech quality (PESQ): An objective method for end-to-end speech quality assessment of d for end-to-end speech quality assessment of narrow-bandnarrow-band telephon telephone networks and speech codecse networks and speech codecs

Amendment 2Amendment 2 P.862.1P.862.1

Mapping functionMapping function for transforming P.862 raw result scores to MOS- for transforming P.862 raw result scores to MOS-LQOLQO

P.862.2P.862.2 Wideband Wideband extension to Recommendation P.862 for the assessment oextension to Recommendation P.862 for the assessment o

f wideband telephone networks and speech codecsf wideband telephone networks and speech codecs P.862.3P.862.3

Application guideApplication guide for objective quality measurement based on Reco for objective quality measurement based on Recommendations P.862, P.862.1 and P.862.2mmendations P.862, P.862.1 and P.862.2

66 TAC2000/2000.7

802.16 IP Telephony Lab

ApplicationP.862.3 – Example of measurement set-up P.862.3 – Example of measurement set-up

and terminologyand terminology

77 TAC2000/2000.7

802.16 IP Telephony Lab

P.862.1MOS-LQO (P.800.1)MOS-LQO (P.800.1)ITU-T Rec. P.862 provides raw scores in ITU-T Rec. P.862 provides raw scores in

the range –0.5 to 4.5.the range –0.5 to 4.5.The mapping function:The mapping function:

88 TAC2000/2000.7

802.16 IP Telephony Lab

Figure of Mapping Function

99 TAC2000/2000.7

802.16 IP Telephony Lab

P.862 - Overview of the basic philosophy used in PESQ

1010 TAC2000/2000.7

802.16 IP Telephony Lab

P.862 - Overview of the basic philosophy used in PESQ (cont.)

1111 TAC2000/2000.7

802.16 IP Telephony Lab

Addition of background noise

1212 TAC2000/2000.7

802.16 IP Telephony Lab

Factors for which PESQ had demonstrated acceptable accuracy

Speech input levels to a codecSpeech input levels to a codecErrors in the transmission channel Errors in the transmission channel

between an encoder and a decoderbetween an encoder and a decoderBit rates if a codec has more than oBit rates if a codec has more than o

ne bit-rate modene bit-rate modeTranscodingsTranscodingsEnvironmental noise in the sending Environmental noise in the sending

sideside

1313 TAC2000/2000.7

802.16 IP Telephony Lab

PESQ is not intended to be used to assess

Effect of listening levelEffect of listening levelConversational delayConversational delayTalker echo, where a subjects hears his own Talker echo, where a subjects hears his own

voice delayedvoice delayedTalker sidetone, where a subjects may hear iTalker sidetone, where a subjects may hear i

ts own voice distortedts own voice distortedNon-intrusive measurements, where only outNon-intrusive measurements, where only out

put signals are available from the system muput signals are available from the system musicsic

1414 TAC2000/2000.7

802.16 IP Telephony Lab

which PESQ has not currently been validated

1515 TAC2000/2000.7

802.16 IP Telephony Lab

ConclusionPESQ has been evaluated on a very wide raPESQ has been evaluated on a very wide ra

nge of speech codecs and telephone networnge of speech codecs and telephone network tests.k tests.

It has been found to produce accurate predicIt has been found to produce accurate predictions of quality in the presence of diverse entions of quality in the presence of diverse end-to-end network behaviors.d-to-end network behaviors.

PESQ represents a significant step forward iPESQ represents a significant step forward in the accuracy and range of applicability of n the accuracy and range of applicability of objective speech quality assessment methodobjective speech quality assessment methods.s.

1616 TAC2000/2000.7

802.16 IP Telephony Lab

Head and Torso Simulator(HATS)

1717 TAC2000/2000.7

802.16 IP Telephony Lab

Spirent Abacus

1818 TAC2000/2000.7

802.16 IP Telephony Lab

Abacus Control GUI

1919 TAC2000/2000.7

802.16 IP Telephony Lab

Testbed

DSLAIIHandset Interface

ControllerMalden

2020 TAC2000/2000.7

802.16 IP Telephony Lab

RTP Flow

2121 TAC2000/2000.7

802.16 IP Telephony Lab

Demo PESQ program

2222 TAC2000/2000.7

802.16 IP Telephony Lab

Reference HATS - HATS - http://www.bksv.com/doc/bp2240http://www.bksv.com/doc/bp2240.pdf.pdf ITU-T Rec. P.862, “Perceptual Evaluation of Speech QualiITU-T Rec. P.862, “Perceptual Evaluation of Speech Quali

ty (PESQ), an Objective Method for End-to-end Speech Qty (PESQ), an Objective Method for End-to-end Speech Quality Assessment of Narrowband Telephone Networks anuality Assessment of Narrowband Telephone Networks and Speech Codecs”, International Telecommunication Uniod Speech Codecs”, International Telecommunication Union, Geneva, Switzerland (2001 Feb.)n, Geneva, Switzerland (2001 Feb.)

Malden - http://www.malden.co.uk/dsla.htmMalden - http://www.malden.co.uk/dsla.htm Perceptual Evaluation of Speech Quality (PESQ), the new Perceptual Evaluation of Speech Quality (PESQ), the new

ITU standard for end-to-end speech quality assessment. PaITU standard for end-to-end speech quality assessment. Part II – Psychoacoustic model - rt II – Psychoacoustic model - http://www.mp3-tech.org/programmer/docs/2001-P03b.pdfhttp://www.mp3-tech.org/programmer/docs/2001-P03b.pdf

Spirent - http://www.spirent.com/Spirent - http://www.spirent.com/