mobile multimedia application over dora - cdma2000 · mobile multimedia application over dora ......

Mobile Multimedia Application over DOrA

How to Ensure Positive End-user Experience?

© 2006 Spirent Communications.

Agenda

°Why DOrA?

°Multimedia Applications over DOrA

°Multimedia Applications testing requirements

°Overview of current video analysis methodologies and algorithms

°Spirent’s unique approach to mobile multimedia applications testing


Drivers for EV-DO Rev A …

FasterFaster

0

500

1000

1500

2000

2500

3000

3500

IS-95 1xRTT EV-DO Rev. 0 EV-DO Rev. A

Commercial Service ?

Dat

a S

pee

d ?

CheaperCheaper BetterBetter

(1995) (2000) (2002) (2007)


Applications Enabled by EV-DO Rev A

°° Higher Data RatesHigher Data Rates

°° Lower LatencyLower Latency

°° EndEnd--toto--End QoSEnd QoS

°° Video StreamingVideo Streaming

°° Video TelephonyVideo Telephony

°° Mobil IPTVMobil IPTV

°° VoiceVoice--overover--IPIP (VoIP)

°° Push To TalkPush To Talk

°° Instant MessagingInstant Messaging

Enables


Mobile Multimedia ApplicationsWhat Makes Them Feasible Now°° Transport TechnologiesTransport Technologies

± Higher data rates, lower latency, higher QoS

°° DevicesDevices

± Higher processing power, more memory, high-end displays

°° Improved Content Improved Content

± Higher sources of content quantity and quality

Air InterfaceAir Interface

CDMA2000

CDMA2000

EVDO

EVDO

Release

0

Release

0 EVDO

EVDO

Release

A

Release

AUMTS

UMTS

Release

99

Release

99

HSDPA with

HSDPA with

QPSK only

QPSK only

Mo

bile

M

ob

ile

Ap

plic

atio

ns

Ap

plic

atio

ns

153 kbps153 kbps

2.4 Mbps2.4 Mbps3.1 Mbps3.1 Mbps

384 kbps384 kbps1.8 Mbps1.8 Mbps

Layer 1 Layer 1 Throughput Throughput RatesRates

SMSSMS

MMSMMS

File File

SharingSharing

Image Image

SharingSharing

Short Video

Short Video

burstburst

Streaming Video

Streaming Video


Key Wireless ApplicationsMarket Growth Indicators

° Many operators offering high speed data services

° More multimedia capable devices

° Better content

° Increased Number of Repeat Applications Purchases

° Increased Purchase of New Applications

Mobile ApplicationsMobile Applications present a significant opportunity for higher ARPUhigher ARPU

Repeat PurchaseRepeat Purchase & Increased UsageIncreased Usage are key measures of Operators SuccessOperators Success


Mobile Multimedia ApplicationsMarket Drivers

°° New Areas of OpportunitiesNew Areas of Opportunities

± Potential for being the next “killer app”

± Service Differentiation

± Increased Revenue

~$700 million to $2.3 billion in revenue

~4.3 million to 14 million Mobile TV subscribers by 2009

°° Growing Demand for New ServicesGrowing Demand for New Services

± Video On-Demand

± Video Streaming

± Video Telephony

± Mobile TV


Mobile Multimedia ApplicationsTesting Challenges

Current Testing methodology limited± No content-based objective methodology

available – testing limited to subjective testing

Mobile Video Testing Standards unavailable

± Mid-late 2007 (VQEG, IT)

Rapidly Changing Requirements± New applications, technologies &

physical design changes

± QoS± Perceptual Quality of Content± Overall satisfaction

Ensuring Positive Quality of End User Experience


SubjectiveSubjective MOSMOS

CaptureCapture

Full ReferenceFull Reference

NonNon--ReferenceReference

ObjectiveObjective

AnalysisAnalysis MOS MOS

+ +

Cause AnalysisCause Analysis

MOS MOS

+ +

Cause AnalysisCause Analysis

QualityQuality

AssessmentAssessment Objective Objective methodology can methodology can

provide more provide more detail data in detail data in

addition to the addition to the quality MOSquality MOS

Video/Voice Quality AssessmentTesting Methodologies


Quality Assessment ToolsSubjective Testing Benefits and Limitations

qq Mean Opinion Score (MOS)Mean Opinion Score (MOS)

° Listening or Video Quality are defined as a measure of listener/viewer’s satisfaction and are generally expressed as a Quality MOS Score

Excellent 5

Good 4

Fair 3

Poor 2

Bad 1

q Advantages

° Reflects the true end user experience based upon their

°Psycho visual quality experience

q Disadvantages

° Usually involves organizing large study groups

° Time consuming

° Cost prohibitive

° Studies are not repeatable

Objective methodology offers costObjective methodology offers cost--effective, repeatable tests in a much shorter timeeffective, repeatable tests in a much shorter time


Objective Multimedia Analysis Methodologies Challenges of Capturing Video from End User’s Perspective

° Invasive methods° Raw data stream

± Captured with device and decoded on PC with media player

± Doesn’t account for device codec and display

° Decoded video extracted from device

± No standard video output interface available

± Device-resident capturing application impacts performance

° Non-invasive methods° Camera solutions – allows replicating true end use experience

° No impact on device performance

° Device-independent

Invasive Methods do not capture the exact endInvasive Methods do not capture the exact end--user experienceuser experience


q Non-Reference model makes use only of the output signal

Multimedia Analysis Methodologies & Tools – Full Reference & Non-reference Models

q Full-Reference objective models collect the information from both sides of the network

q Advantages of using both models

° Having the two models will provide a coverage for all testing scenarios - in the absence and the presence of a reference test signal

Experience Expectation Semantics

Intrusive (full reference) model

Non-Intrusive (no reference) model

Subjective assessment

Source Output

VMon P.563

Network

under test

VQuad SQuad-LQ

Learn fromSubjects


APEX Wireless Multimedia Analysis (WMA) Module Offering Unique Value Propositions

Powerful Powerful objectiveobjective & & technology independent technology independent multimedia analysis toolmultimedia analysis tool

In-house subject matter expertiseto quickly respond to changing

requirements & help drive standards

Industry leadingIndustry leading quality quality analysis video/audio analysis video/audio

algorithmsalgorithms

Uniquely flexibleUniquely flexible video capture video capture & device fixture with& device fixture with

High resolution camera & High resolution camera & universal fixtureuniversal fixture


Cellular Network

Video Test Clips

Controller PC

APEX Test Manager

Media Server

APEX Wireless Multimedia Analysis ModuleSystem Architecture

Algorithms

Wireless Multimedia FixtureMobile TV Broadcast

Video Streaming

Video Telephony

Captured Video


° Full Reference Video model

±± VQuadVQuad

° Non-Reference Video model

±± VMonVMon

° Full Reference Voice model

±± SQUAD LQ/NS/AEC/RTTSQUAD LQ/NS/AEC/RTT

° Non-Reference Voice model

±± NiNA+/P.563NiNA+/P.563

Industry Leading SwissQual Video/Voice Quality Algorithms

°Advanced Video testing solutions

±Lip-Sync (audio & video synch)

±Video Telephony

±Video Streaming

±Mobile TV

Quality Video/Voice Analysis Requires Comprehensive Tools


Quality Video Analysis Requires Comprehensive ToolsVQuad – A Full Reference Quality Assessment Model

§ Intrusive Algorithm

§ Human vision

§ Results

§ MOS

§ PSNR

§ Block distortion

§ Blurring

§ Packet Loss

§ Jitter

§ Jerkiness

§ Level

§ Audio Video synch

Full Reference Test (VQuad)Full Reference Test (VQuad)

Distortion Evaluation

Perceptual Modeling

Contrast and Gain Alignment

Format Conversion

Network

Reference Reference VideoVideo

Captured Captured VideoVideo

Video Quality MeasureVideo Quality Measure


Quality Video Analysis Requires Comprehensive ToolsVMon – A Non-Reference Quality Assessment Model

§ A Non-Intrusive Algorithm

§ Human vision

§ Results

§ MOS

§ Repeated Frames

§ Fps Calculated

§ Inter-frame Difference

§ Colorfulness

§ Black Frames

§ Skipping with Pausing

§ Blurring

§ Blocking

NonNon--Reference Test (VMon)Reference Test (VMon)

Captured VideoCaptured Video

Distortion Evaluation

Perceptual Modeling

Temporal & Spatial Analysis

Format Conversion

Network

Video Quality Measure

Source VideoSource Video


How to optimize the video quality assessment Process?

° Providing root causes analysis of video helps

± Reduce development time

± Improve efficiency of testing process

± Accelerate analysis and improvement of positive end user experience

Root Cause Root Cause ExamplesExamples

BlurrinessBlurriness

Block DistortionBlock Distortion

Tiling & SlicingTiling & Slicing

ColorfulnessColorfulness

Frame FreezingFrame Freezing


ITU International Telecommunication Union

ETSI European Telecommunications Standards Institute

VQEG Video Quality Expert Group

STQ Speech Quality Group

JRG-MM Joint Raporteurs Group for MultiMedia

MMQA MultiMedia Quality Assessment

Provides the actual Provides the actual Audio/Video quality Audio/Video quality

measurement methodsmeasurement methods

ITU-D

ITU

ITU-R ITU-T

JRG-MMQAReporterReporter

Provides framework for drive Provides framework for drive testing and measuring testing and measuring Audio/Video QualityAudio/Video Quality

Standard Recommendations

HeadHead

Perceptual Quality Measures Group

in ITUITU--T SG12T SG12

ETSI

STQ-mobile

VQEG

UniversitiesIndustries

MMQA

EditorEditor

Spirent-SwissQual Leading the Way in Standards


APEX Test Manager is an openopen--architecture, scalablearchitecture, scalable test system designed to meet the rapidly evolving needs of Applications testing.

Customers Demand for More Flexibility and Control

üüEasyEasy--toto--useuse üüCustomizableCustomizable üüAutomatedAutomated

APEX Platform üüOpenOpen

Tests DataDataPerformancePerformance

WMAWMA App App xx App App yyüüModularModular

Reporting üüCustomizableCustomizable

“Echo” Test creation üüIntelligent, realIntelligent, real--time time OneOne--step recording of test scores used for customized test creation!step recording of test scores used for customized test creation!

Multiple Tests üüSingle Environment Single Environment Field Tests & LabField Tests & Lab--based tests all in One Environment!based tests all in One Environment!


Wireless Multimedia Analysis ModuleFeature Summary

°° ObjectiveObjective, repeatablerepeatable and reliablereliable multimedia analysis

° Quality video and audio algorithm

± MOS with cause analysiscause analysis (Blurring, Blocking, Noise, etc.)

±± LipLip--SyncSync measurements – with full reference

°° EnhancedEnhanced customizable, automatedcustomizable, automated software management through Spirent Test ManagerTest Manager

°° OpenOpen & flexibleflexible architecture - with ability to create and change test cases with ease & efficiency with APEX platform

°° BearerBearer--independentindependent solution works with EV-DO RevA, MediaFLO, Mobile TV, VCAST etc.


Summary Delivering Value to Customers & End-users

°Multimedia technology enablers are already here

° Success will be measured by the incremental revenue they generate

° The end user’s positive experience is critical to attracting new customers and keeping repeat customers

° Spirent provides the necessary solutions to verify a positive experience throughout the multimedia application testing process

Spirent: Helping you get it right the first time!Spirent: Helping you get it right the first time!

Thank You!

mobile multimedia application over dora - cdma2000 · mobile multimedia application over dora ......

Documents