voice quality and testing eit-03 alan percy director of business development audiocodes
DESCRIPTION
Voice Quality and Testing EIT-03 Alan Percy Director of Business Development AudioCodes. Who is AudioCodes?. Leading OEM Manufacturer of Enabling VoIP equipment Gateways Media Servers Boards, Chips and Modules Key IPR holder for G.723.1 11 th Year in Business $82.8M Revenue for FY04 - PowerPoint PPT PresentationTRANSCRIPT
Voice Quality and TestingEIT-03
Alan PercyDirector of Business
DevelopmentAudioCodes
Who is AudioCodes?• Leading OEM Manufacturer
of Enabling VoIP equipmento Gatewayso Media Serverso Boards, Chips and Modules
• Key IPR holder for G.723.1• 11th Year in Business• $82.8M Revenue for FY04• NASDAQ: AUDC• Over 440 Employees• Worldwide Company:
o Headquarters in Israelo US offices in San Jose, Boston, Chicago, Dallas,
RTP, and Somerset NJo Offices in Mexico, France, UK, China, and Japan
Background
• Customer demand is building
• VoIP carriers are proliferating
• What will differentiate the carriers?
Consumer VoIP Subscribers
0
2
4
6
8
10
'03 '04 '05 '06 '07 '08
Year
in m
illio
ns
Source: In-Stat
New Carriers, New Challenges
A few years ago, we equipment providers thought we had this figured out.
Then:• Private networks• Intra-Enterprise• Carefully controlled
(over-engineered) networks
New Carriers, New Challenges
Today: New Frontier• Depending on existing broadband
o Cableo DSLo Fixed wireless
• New Wireless Serviceso 3Go WiMaxo WiFio SAT
What have we learned?
You can’t always control the networko To make the business model work, you
sometimes need to leverage the existing infrastructure
What have we learned?
The last mile is the hardest mile!
What have we learned?
• Murphy’s law is true!(Poor voice quality will only affect the CEO
during an important investor call)
• Poor voice quality will get you kicked out
• Huge variation in voice quality between equipment vendors(Especially when network conditions are
not “ideal”)
Is voice quality important?
“Voice Quality is the most sensitive issue for business customers” Greg Schreiber, VP of Product Development - Vaspian
What affects voice quality?
Choosing the Right Coder
Vocoder Factors
• Single speaker, no packet loss, no acoustic noise
• Vocoder quality given non speech audio (music, background noise)
• Bit rate • Algorithmic delay = Packet size + look
ahead • Audio bandwidth • Low bit rate algorithm type • Complexity (# MIPS)• Intrinsic robustness to packet loss
Vocoder Factors
Vocoder implementation issues:• Not all vocoders standards have bit exact
test vectors• And test vectors don’t cover every angle …• Little known fact - even long approved and
deployed vocoder standards still have bugs – let alone new ones …
o Audiocodes has found and reported bugs in numerous standard vocoders
• Hence the importance of Speech Quality Testing and deployment experience
Vocoder Tradeoffs
• Complexity is always a constraint• Bit rate and algorithm type affect speech
and non-speech vocoding quality • Larger packet size enables lower bit rate
but increases delay (e.g. G.723.1 vs. G.729A)
• Higher bit rate enables better intrinsic robustness to packet loss
• Higher audio bandwidth requires higher bit rate
Echo Canceller Factors
• Imperfect EC performance could result in:
o Residual echoo Clippingo Distorted speecho Annoying variation in background noise
• Longer round trip delay makes echo cancellation more critical
Echo Canceller Factors
• ECs are challenged most during Double Talk (DT) o Double Talk happen occasionally during
full duplex conversationso During Double Talk
• EC may diverge • The Non Linear Processor (NLP) could clip,
attenuate or distort the near-end speech• Therefore it is crucial to disable the NLP during
Double Talk
o Timely and reliable detection of Double Talk could be tricky
Echo Canceller Factors
• Background noise continuity (level, spectrum) should be maintained
• The variety of real life echo paths poses an algorithmic and testing challengeo field experience crucial
• EC standards (e.g. G.168) specify testing methods - not standard algorithms o proprietary EC algorithms could vary
significantly in qualityo Quality testing is essential
Echo Canceller Tradeoffs
• Attenuation vs. tail length vs. adaptation speed vs. steady state coefficients noise and EC stability
• Echo Attenuation in Single Talk vs. Double Talk performance
• Complexity (# MIPS)
Packet Loss Concealment
• Vocoder type (CELP vs. ADPCM vs. PCM)
• All vocoders (including G.711) have PLC standardized to some extent
• Vendor distinction depends on Jitter Buffer quality and PLC enhancements
• Trade offso Latency vs. Jitter buffer size vs. %
frame erasure rateo Redundancy schemes vs. bit rate
Condition Interaction Effect
Effect:• Customers tell us that G.723.1 many
times sounds better than other’s G.711
Why?• Lower bit rate reduces congestion and
results in fewer dropped packets.• G.723.1 also includes packet loss
concealment, smoothing over the gaps.
Improving VoIP Quality
Techniques vendors use to maximize quality:• Dynamic Jitter Buffer
o Adjusting to ever-changing network conditions
• Packet loss concealmento All coders, including G.711
• Echo cancellation algorithms• VAD and CNG algorithms• Extensive field and lab testing
Speech Quality Testing
• Methods• Events• Results
Speech Quality Testing Methods
• MOSo Traditional subjective method, uses a panel of
listeners, accurate but expensive
• PESQ - Perceptual Evaluation of Speech Quality o objective method based on human hearing model, ITU
P.862 standardo Computer driven
• G.168o ITU standard for echo canceller testing, uses artificial
speech signals• P.501 / P.502
o ITU standard for complex test signals used to test echo cancellers and other modules.
o These signals were designed to provide more information to the designer about the weakness points.
ETSI Speech Quality Test Event
• Sponsored by ETSI - Done by HEAD Acoustics Labs • Has been held several times over the past three
years• VoIP equipment testing in controlled conditions • Receive an objective and comparative analysis • The tests include a range of aspects, including:
o Speech sound quality o Echo measurements o Double Talk performance o Transmission quality in the presence of background
noise
• Anonymous - Only your results are identified
ETSI Test Setup
G a t e w a y
I P
E 1
N I S T N e t
P a c k e t M O N I T O R
E 1
P a c k e t L o s s D e l a y
O u t p u t
H E A D a c o u s t i c s T e s t S y s t e m AA CC QQ UU AA
I n p u t
G a t e w a y I P I P
I S D N D S S 1 I S D N D S S 1
P B X
ETSI results “Pie Chart”• Six conversational and six listening tests
• Ten gateways participated in 3rd SQTE
• The results for each gateway are summarized in a Pie Chart with a slice per each category
• The performance in each category was represented by the radius of the slice and by a color code
o Red – the amount of which the results were under the requirement
o Yellow – result was below the requirement if accompanied with red or OK if no red
o Green – result was above the requirement
ETSI SQTE Test Results
ETSI SQTE Test Results
AudioCodes
How can you use this information?
• Ask for ETSI results when shopping• Read and evaluate performance
needs against test results.• Justify partner selection• Help sell your solution
Where to learn more:
Full ETSI SQTE Test Report:http://www.etsi.org/plugtests/History/2004SQTE.htm
AudioCodes ETSI SQTE Results:http://www.audiocodes.com/Content.aspx?voip=2189
Let’s talk! Stop by Booth #405