no-reference metrics for video streaming applications
DESCRIPTION
No-Reference Metrics For Video Streaming Applications. International Packet Video Workshop (PV 2004) Presented by : Bhavana CPSC 538 February 21, 2004. Video Quality Assessment. What’s Quality ? - Implies Comparison => reference Three Techniques : - Full-Reference eg. MSE, PSNR - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: No-Reference Metrics For Video Streaming Applications](https://reader036.vdocuments.net/reader036/viewer/2022062423/56814511550346895db1d1cf/html5/thumbnails/1.jpg)
No-Reference Metrics For No-Reference Metrics For Video Streaming ApplicationsVideo Streaming Applications
International Packet Video Workshop (PV 2004) International Packet Video Workshop (PV 2004)
Presented by : BhavanaPresented by : Bhavana
CPSC 538CPSC 538
February 21, 2004February 21, 2004
![Page 2: No-Reference Metrics For Video Streaming Applications](https://reader036.vdocuments.net/reader036/viewer/2022062423/56814511550346895db1d1cf/html5/thumbnails/2.jpg)
Video Quality AssessmentVideo Quality Assessment
What’s Quality ?What’s Quality ?
- Implies Comparison => reference- Implies Comparison => reference
Three Techniques :Three Techniques :
- Full-Reference eg. MSE, PSNR- Full-Reference eg. MSE, PSNR
- Reduced Reference- Reduced Reference
- No Reference- No Reference
![Page 3: No-Reference Metrics For Video Streaming Applications](https://reader036.vdocuments.net/reader036/viewer/2022062423/56814511550346895db1d1cf/html5/thumbnails/3.jpg)
What is a No-Reference metric ?What is a No-Reference metric ?
Estimating end-user’s QoE of a Estimating end-user’s QoE of a multimedia stream without using an multimedia stream without using an original stream as a reference.original stream as a reference.
In other words :In other words :
“ “ Quantify quality via blind distortion Quantify quality via blind distortion measurement”measurement”
![Page 4: No-Reference Metrics For Video Streaming Applications](https://reader036.vdocuments.net/reader036/viewer/2022062423/56814511550346895db1d1cf/html5/thumbnails/4.jpg)
PurposePurpose
To evaluate two types of distortions in To evaluate two types of distortions in streaming of compressed video over streaming of compressed video over packet-switched networkspacket-switched networks
- Compression related : block-edge - Compression related : block-edge impairment impairment
- Transmission related : packet-loss - Transmission related : packet-loss impairment impairment
![Page 5: No-Reference Metrics For Video Streaming Applications](https://reader036.vdocuments.net/reader036/viewer/2022062423/56814511550346895db1d1cf/html5/thumbnails/5.jpg)
Where Can It Be Used ?Where Can It Be Used ?
For real time monitoring .For real time monitoring .
Reference unavailable or expensive to sendReference unavailable or expensive to send
Feedback to Streaming Server .Feedback to Streaming Server .
Evaluation of Compression AlgorithmsEvaluation of Compression Algorithms
![Page 6: No-Reference Metrics For Video Streaming Applications](https://reader036.vdocuments.net/reader036/viewer/2022062423/56814511550346895db1d1cf/html5/thumbnails/6.jpg)
What are Block-Based Codecs ?What are Block-Based Codecs ?
Process several pixels of video together Process several pixels of video together in blocksin blocks At high compression rates, strong At high compression rates, strong discontinuities called block edges come discontinuities called block edges come up.up.What’s blockiness ? What’s blockiness ?
“ “Distortion of image characterized by Distortion of image characterized by appearance of underlying block encoding appearance of underlying block encoding structurestructure
![Page 7: No-Reference Metrics For Video Streaming Applications](https://reader036.vdocuments.net/reader036/viewer/2022062423/56814511550346895db1d1cf/html5/thumbnails/7.jpg)
![Page 8: No-Reference Metrics For Video Streaming Applications](https://reader036.vdocuments.net/reader036/viewer/2022062423/56814511550346895db1d1cf/html5/thumbnails/8.jpg)
Block – based DistortionBlock – based Distortion
Idea Idea : A block-edge gradient can be : A block-edge gradient can be masked by a region of high spatial activity masked by a region of high spatial activity around it .around it .
Measure two things :Measure two things :
- spatial activity around block edges : - spatial activity around block edges : σσ
- block-edge gradient : - block-edge gradient : ΔΔ
![Page 9: No-Reference Metrics For Video Streaming Applications](https://reader036.vdocuments.net/reader036/viewer/2022062423/56814511550346895db1d1cf/html5/thumbnails/9.jpg)
Calculation of NR Blockiness MetricCalculation of NR Blockiness Metric
For each 8 x 8 Block BFor each 8 x 8 Block Bij ij , ,
For each edge IFor each edge Ik k of Bof Bijij ,,
divide edge into 3 segments adivide edge into 3 segments aklkl
For each segment of aFor each segment of akl kl
calculate calculate σσklkl
calculate calculate ΔΔklkl
![Page 10: No-Reference Metrics For Video Streaming Applications](https://reader036.vdocuments.net/reader036/viewer/2022062423/56814511550346895db1d1cf/html5/thumbnails/10.jpg)
I4
I1
E2E4
E3
I2
E1
I3
Bij
0 1 2 3 4 5 6 7
a1
a2
a3
An 8 x 8 block and its edges
Three segments akl of a block edge
![Page 11: No-Reference Metrics For Video Streaming Applications](https://reader036.vdocuments.net/reader036/viewer/2022062423/56814511550346895db1d1cf/html5/thumbnails/11.jpg)
NR Blockiness Metric contd’NR Blockiness Metric contd’
CCB B = No. of Blocks for which at least one edge = No. of Blocks for which at least one edge
satisfies :satisfies :
σσkl kl < < εε where where εε = 0.1 = 0.1
ΔΔkl kl > > ττ where where ττ = 2.0 = 2.0
εε = min. spatial activity required to mask gradient = min. spatial activity required to mask gradient
ττ = max. gradient which is imperceivable.= max. gradient which is imperceivable.
ββFF = = CCB B / Total no. of blocks in the frame/ Total no. of blocks in the frame
![Page 12: No-Reference Metrics For Video Streaming Applications](https://reader036.vdocuments.net/reader036/viewer/2022062423/56814511550346895db1d1cf/html5/thumbnails/12.jpg)
Simulation Setup For NR Simulation Setup For NR Blockiness MetricBlockiness Metric
Aim : to measure how well the NR Aim : to measure how well the NR Blockiness metric conveys QoEBlockiness metric conveys QoE
Codec : MPEG -4 , GOP = 30 framesCodec : MPEG -4 , GOP = 30 frames
Bit Rate => compression levelBit Rate => compression level
![Page 13: No-Reference Metrics For Video Streaming Applications](https://reader036.vdocuments.net/reader036/viewer/2022062423/56814511550346895db1d1cf/html5/thumbnails/13.jpg)
![Page 14: No-Reference Metrics For Video Streaming Applications](https://reader036.vdocuments.net/reader036/viewer/2022062423/56814511550346895db1d1cf/html5/thumbnails/14.jpg)
![Page 15: No-Reference Metrics For Video Streaming Applications](https://reader036.vdocuments.net/reader036/viewer/2022062423/56814511550346895db1d1cf/html5/thumbnails/15.jpg)
![Page 16: No-Reference Metrics For Video Streaming Applications](https://reader036.vdocuments.net/reader036/viewer/2022062423/56814511550346895db1d1cf/html5/thumbnails/16.jpg)
NR Packet Loss MetricNR Packet Loss Metric
Error Concealment : Replace Error Concealment : Replace damaged/lost macroblock with damaged/lost macroblock with corresponding macroblock from previous corresponding macroblock from previous frame.frame.
Idea Idea : Use length of artifact to estimate : Use length of artifact to estimate amount of distortion caused by packet lossamount of distortion caused by packet loss
![Page 17: No-Reference Metrics For Video Streaming Applications](https://reader036.vdocuments.net/reader036/viewer/2022062423/56814511550346895db1d1cf/html5/thumbnails/17.jpg)
Calculation of NR Packet Loss Calculation of NR Packet Loss MetricMetric
For a m x n frameFor a m x n frame
For each 16 x 16 macroblockFor each 16 x 16 macroblock
Calculate :Calculate :
ÊÊjj = strength vector across macroblock = strength vector across macroblock
edgeedge
ÊÊ΄́jj = strength vector within macroblock = strength vector within macroblock
near the edgenear the edge
![Page 18: No-Reference Metrics For Video Streaming Applications](https://reader036.vdocuments.net/reader036/viewer/2022062423/56814511550346895db1d1cf/html5/thumbnails/18.jpg)
Macroblock 1
Macroblock 2
Figure : Calculating Strength vector across and within a macroblock
![Page 19: No-Reference Metrics For Video Streaming Applications](https://reader036.vdocuments.net/reader036/viewer/2022062423/56814511550346895db1d1cf/html5/thumbnails/19.jpg)
Convert strength vectors into binary Convert strength vectors into binary vectorsvectors
EEjj(k) = 1 if (k) = 1 if ÊÊj j > > ττ
= 0 otherwise= 0 otherwise
EE΄́jj(k) = 1 if (k) = 1 if ÊÊ΄́j j > > ττ
= 0 otherwise= 0 otherwise
where where ττ = 15 = 15
![Page 20: No-Reference Metrics For Video Streaming Applications](https://reader036.vdocuments.net/reader036/viewer/2022062423/56814511550346895db1d1cf/html5/thumbnails/20.jpg)
If the sum of differences between the two binary If the sum of differences between the two binary edge vectors is substantial , then there is edge vectors is substantial , then there is distortiondistortion
Packet loss metric for jPacket loss metric for j thth macroblock macroblock
HHj j = = ∑ | ∑ | EEjj(k) - E(k) - E΄́jj(k) | if (k) | if ∑ | ∑ | EEjj(k) - E(k) - E΄́jj(k) | > (k) | > ζζ
= 0 otherwise= 0 otherwise
where where ζζ = 10% of frame width (n) = 10% of frame width (n)
Packet loss metric for whole frame Packet loss metric for whole frame
F = ∑ HF = ∑ Hjj22
![Page 21: No-Reference Metrics For Video Streaming Applications](https://reader036.vdocuments.net/reader036/viewer/2022062423/56814511550346895db1d1cf/html5/thumbnails/21.jpg)
Simulation Setup for NR Packet Simulation Setup for NR Packet Loss MetricLoss Metric
Bit Rate = 1.5 MbpsBit Rate = 1.5 Mbps
Frame Rate = 30 fpsFrame Rate = 30 fps
Frame Size = 352 x 240Frame Size = 352 x 240
Used NTT DoCoMo packet loss Used NTT DoCoMo packet loss generating software .generating software .
![Page 22: No-Reference Metrics For Video Streaming Applications](https://reader036.vdocuments.net/reader036/viewer/2022062423/56814511550346895db1d1cf/html5/thumbnails/22.jpg)
![Page 23: No-Reference Metrics For Video Streaming Applications](https://reader036.vdocuments.net/reader036/viewer/2022062423/56814511550346895db1d1cf/html5/thumbnails/23.jpg)
![Page 24: No-Reference Metrics For Video Streaming Applications](https://reader036.vdocuments.net/reader036/viewer/2022062423/56814511550346895db1d1cf/html5/thumbnails/24.jpg)
Limitations Of NR-metricsLimitations Of NR-metrics
Blockiness metric might fail in the Blockiness metric might fail in the presence of strong de-blocking filters presence of strong de-blocking filters which might otherwise introduce blurwhich might otherwise introduce blur
Metric predictions lose meaning in Metric predictions lose meaning in presence of other distortions like blur, presence of other distortions like blur, noise etc.noise etc.
![Page 25: No-Reference Metrics For Video Streaming Applications](https://reader036.vdocuments.net/reader036/viewer/2022062423/56814511550346895db1d1cf/html5/thumbnails/25.jpg)
Future DirectionsFuture Directions
VQEG standardization effortsVQEG standardization efforts HVS based approachesHVS based approaches Statistical models for natural scenesStatistical models for natural scenes NR QA schemes for NR QA schemes for
- Non-block based compression schemes - Non-block based compression schemes such Wavelet-based such Wavelet-based
-Targeting full range of artifacts -Targeting full range of artifacts
![Page 26: No-Reference Metrics For Video Streaming Applications](https://reader036.vdocuments.net/reader036/viewer/2022062423/56814511550346895db1d1cf/html5/thumbnails/26.jpg)
ReferencesReferences
No Reference Image and Video Quality No Reference Image and Video Quality AssessmentAssessment http://http://live.ece.utexas.edu/research/quality/nrqa.htmlive.ece.utexas.edu/research/quality/nrqa.htm Objective video Quality Assessment Objective video Quality Assessment http://www.cns.nyu.edu/~zwang/files/papers/QAhttp://www.cns.nyu.edu/~zwang/files/papers/QA_hvd_bookchapter.pdf_hvd_bookchapter.pdfPerceptual Video Quality and Blockiness Metrics Perceptual Video Quality and Blockiness Metrics for Multimedia Streaming Applicationsfor Multimedia Streaming Applicationswww.stefan.winkler.net/Publications/www.stefan.winkler.net/Publications/wpmc2001.pdf wpmc2001.pdf