lec13, video iii (video coding standards),...
TRANSCRIPT
Multimedia SystemsMultimedia Systems
Video IIIVideo III
(Video Coding Standards)(Video Coding Standards)
Course PresentationCourse Presentation
(Video Coding Standards)(Video Coding Standards)
Mahdi Amiri
April 2014
Sharif University of Technology
Video Coding StandardsStandardization OrganizationsStandardization Organizations
Two organizations have dominated video compression
standardization.
ITU-T Video Coding Experts Group (VCEG)
International Telecommunications Union –Telecommunications
Standardization Sector (ITU-T, a United Nations Organization,
Multimedia Systems, Mahdi Amiri, Video IIIPage 1
Standardization Sector (ITU-T, a United Nations Organization,
formerly CCITT), Study Group 16, Question 6.
ISO/IEC Moving Picture Experts Group (MPEG)
International Standardization Organization and International
Electrotechnical Commission, Joint Technical Committee Number
1, Subcommittee 29, Working Group 11.
Video Coding StandardsDynamicsDynamics
VCEG is older and more focused on conventional (esp. low-delay)
video coding goals (e.g. good compression and packet-loss/error
resilience)
MPEG is larger and takes on more ambitious goals (e.g. “object
oriented video”, “synthetic-natural hybrid coding”, and digital
Multimedia Systems, Mahdi Amiri, Video IIIPage 2
oriented video”, “synthetic-natural hybrid coding”, and digital
cinema)
Sometimes the major organizations team up (e.g. ISO, IEC and ITU
teamed up for both MPEG-2 and JPEG)
Relatively little industry consortium activity (DV and organizations
that tweak the video coding standards in minor ways, such as DVD,
3GPP, 3GPP2, SMPTE, IETF, etc.)
Growing activity for internet streaming media outside of formal
standardization (e.g., Microsoft, Real Networks, Quicktime)
Video Coding StandardsThe Scope of Picture and Video Coding StandardizationThe Scope of Picture and Video Coding Standardization
A Video standard specifically do not define an encoder; rather, they
define the output that an encoder should produce.
A decoding method is defined in each standard (only the Bitstream
Syntax and Decoding Process are standardized):
e.g. use IDCT, but not how to implement the IDCT.
Multimedia Systems, Mahdi Amiri, Video IIIPage 3
e.g. use IDCT, but not how to implement the IDCT.
Permits optimization beyond the obvious.
Permits complexity reduction for implementability.
Provides no guarantees of Quality - only interoperability.
Ensuring Ensuring interoperabilityinteroperability::
Enabling communication between Enabling communication between
devices made by different devices made by different
manufacturersmanufacturers
Video Quality EvaluationObjectiveObjective
A computer algorithm judges the distortion between
videos
Attempts to model a human observer
There is currently no standard method
Multimedia Systems, Mahdi Amiri, Video IIIPage 4
There is currently no standard method
Video Quality EvaluationObjective Metrics: PSNRObjective Metrics: PSNR
Peak Signal-To-Noise Ratio (PSNR)
Used widely in evaluating coding performance
Purely mathematical difference
Can be tricked quite easily
Multimedia Systems, Mahdi Amiri, Video IIIPage 5
Can be tricked quite easily
Root Mean Squared Root Mean Squared
Error (RMSE)Error (RMSE)
255 = 2^n 255 = 2^n –– 11
n: the number of bits per image samplen: the number of bits per image sample
Video Quality EvaluationPSNR, ExamplePSNR, Example
Multimedia Systems, Mahdi Amiri, Video IIIPage 6
OriginalOriginal PSNR 35.4 [dB]PSNR 35.4 [dB] PSNR 29.0 [dB]PSNR 29.0 [dB]
ABCABAB
OriginalOriginal PSNR 45.53 [dB]PSNR 45.53 [dB]
Multimedia Systems, Mahdi Amiri, Video IIIPage 7
PSNR 36.81 [dB]PSNR 36.81 [dB] PSNR 31.45 [dB]PSNR 31.45 [dB]
Video Quality EvaluationMahalanobisMahalanobis DistanceDistance
PrasantaPrasanta Chandra Chandra MahalanobisMahalanobis
18931893--19721972
Euclid, Euclid, FloruitFloruit 300 BC300 BC
(Statue of Euclid in the Oxford University (Statue of Euclid in the Oxford University
Museum of Natural History)Museum of Natural History)
The The MahalanobisMahalanobis distance differs from Euclidean distance differs from Euclidean
distance in that it takes into account the distance in that it takes into account the
correlations of the data set and is scalecorrelations of the data set and is scale--invariant.invariant.
Multimedia Systems, Mahdi Amiri, Video IIIPage 8
( ) ( )1T −− −x µ Σ x µRef.: www.aiaccess.netRef.: www.aiaccess.net( ) ( )
T
− −x µ x µ
Euclidian distance (Squared)Euclidian distance (Squared) MahalanobisMahalanobis distance (Squared)distance (Squared)
Video Quality EvaluationObjective Metrics: PSNRObjective Metrics: PSNR
How to trick PSNR
Take a natural image
Give more bits to areas you look at more
Give less bits to areas you look at less
Multimedia Systems, Mahdi Amiri, Video IIIPage 9
Give less bits to areas you look at less
Subjective rating will be high, PSNR low
OriginalOriginal Attention Map ExampleAttention Map Example TestTest(High subjective rating, low PSNR)(High subjective rating, low PSNR)
Video Quality EvaluationSubjective: MOSSubjective: MOS
Mean Opinion Score (MOS)
A numerical indication of the perceived quality of the media received
after being transmitted and eventually compressed using codecs. MOS is
expressed in one number, from 1 to 5, 1 being the worst and 5 the best.
A human “subject” rates the video on a scale.A human “subject” rates the video on a scale.
Multimedia Systems, Mahdi Amiri, Video IIIPage 10
Video Quality EvaluationSubjectiveSubjective
In the ITU recommendations, there are many subjective
quality test methods.
Absolute Category Rating (ACR)
Degradation Category Rating (DCR)
Multimedia Systems, Mahdi Amiri, Video IIIPage 11
Degradation Category Rating (DCR)
The Double-Stimulus Continuous Quality-Scale
method (DSCQS)
Video Quality EvaluationSubjective: ACR and DCRSubjective: ACR and DCR
Absolute Category Rating (ACR)
Subjects are asked to rate the quality of the
presentation based on the level of the quality they have
in their opinion for it after viewing or listening it
(Single Stimulus).
No reference sequence.
Multimedia Systems, Mahdi Amiri, Video IIIPage 12
(Single Stimulus).
Degradation Category Rating (DCR)
Test sequences are presented in pairs. The first
stimulus presented in each pair is always the source
reference without any impairments (Double Stimulus).
Ref.: www.irisa.fr/armor/lesmembres/MohamedRef.: www.irisa.fr/armor/lesmembres/Mohamed
/Thesis/node147.html/Thesis/node147.html
Known reference sequence.
Video Quality EvaluationSubjective: DSCQSSubjective: DSCQS
Double Stimulus Continuous Quality Scale Method (DSCQS)
Multimedia Systems, Mahdi Amiri, Video IIIPage 13
Ref.: www.irisa.fr/armor/lesmembres/MohamedRef.: www.irisa.fr/armor/lesmembres/Mohamed
/Thesis/node147.html/Thesis/node147.html
For having fidelity test the observers are
not told which is the reference sequence.
Unknown reference sequence.
R-D Curve of Video CodecsABCABC
Multimedia Systems, Mahdi Amiri, Video IIIPage 14
R-D Curve of Video CodecsABCABC
ReminderReminder
CIFCIF--size image size image
352352××288288
Multimedia Systems, Mahdi Amiri, Video IIIPage 15
R-D Curve of Video CodecsABCABC
Multimedia Systems, Mahdi Amiri, Video IIIPage 16
R-D Curve of Video CodecsABCABCR-D Performance of MPEG Codecs
44
46
48
50
Multimedia Systems, Mahdi Amiri, Video IIIPage 17
32
34
36
38
40
42
350 450 550 650 750 850 950 1050
Bit rate (kbps)
PSNR (Y)
MPEG-1 MPEG-2 MPEG-4 H.264
R-D Curve of Video CodecsABCABC
Multimedia Systems, Mahdi Amiri, Video IIIPage 18
Video Coding StandardsGary J. Sullivan, Ph.D.
Gary J. Sullivan, Ph.D.
Rapporteur
Rapporteur| C
hair
| Chair
Rapporteur
Rapporteur| Co
| Co--Chair
Chair
Rapporteur
Rapporteur| C
o| C
o--Chair
Chair
Multimedia Systems, Mahdi Amiri, Video IIIPage 19
Gary J. Sullivan, Ph.D.
Gary J. Sullivan, Ph.D.
ITU
ITU--T VCEG
T VCEG Rapporteur
Rapporteur
ISO/IEC M
PEG Video
ISO/IEC M
PEG Video Rapporteur
Rapporteur
ITU/ISO/IEC JVT
ITU/ISO/IEC JVT Rapporteur
Rapporteur
Video Coding StandardsHistoryHistory
Multimedia Systems, Mahdi Amiri, Video IIIPage 20
en.wikipedia.org/wiki/Data_compressionThe Society of Motion Picture and Television Engineers, SMPTE (pron. simpti) is an internationally
recognized standards organizations founded in 1916 (en.wikipedia.org/wiki/SMPTE).
Video Coding StandardsMajor Video Compression ApplicationsMajor Video Compression Applications
Multimedia Systems, Mahdi Amiri, Video IIIPage 21
Video Coding StandardsMoving Picture Experts Group (MPEG)Moving Picture Experts Group (MPEG)
A working group of ISO/IEC in charge of the
development of standards for coded
representation of digital audio and video and
related data.
Multimedia Systems, Mahdi Amiri, Video IIIPage 22
related data.
Established in 1988
23 years of activity
The number of independent standards: more than
125
Video Coding StandardsMPEG-1 The standard on which such products as Video CD and MP3 are based
MPEG-2 The standard on which such products as Digital Television set top boxes and DVD are
based;
MPEG-4 The standard for multimedia for the fixed and mobile web;
MPEG-7 The standard for description and search of audio and visual content;
MPEG-21 The Multimedia Framework;
MPEG-A The standard providing application-specific formats by integrating multiple MPEG
technologies;
MPEG-B A collection of Systems specific standards
MPEG-C A collection of Video specific standards
Multimedia Systems, Mahdi Amiri, Video IIIPage 23
MPEG-C A collection of Video specific standards
MPEG-D A collection of Audio specific standards
MPEG-E A standard (M3W) providing support to download and execution of multimedia
applications
MPEG-H A standard (HEVC) providing a significantly increased video compression performance
MPEG-M A standard (MXM) for packaging and reusability of MPEG technologies
MPEG-U A standard for rich-media user interface
MPEG-V A standard for interchange with virtual worlds
Video Coding StandardsVideo Coding Experts Group (VCEG)Video Coding Experts Group (VCEG)
Part of study group 16 (Multimedia coding, systems and
applications) of the ITU-T. Established in 1984
H.120 The first digital video coding standard
H.261 Was the first practical digital video coding standard.
Multimedia Systems, Mahdi Amiri, Video IIIPage 24
H.261 Was the first practical digital video coding standard.
H.262 It is identical in content to the video part of the ISO/IEC MPEG-2 standard.
H.263 Provided a suitable replacement for H.261 at all bitrates.
H.263v2 Also known as H.263+, Enhanced robustness against data loss in the transmission channel.
H.264 The ITU-T H.264 standard and the ISO/IEC MPEG-4 Part 10 standard (formally, ISO/IEC 14496-
10) are technically identical.
H.265 Not yet developed; expected 2012 or later.
H.271 Video back channel messages for conveyance of status information and requests from a video
receiver to a video sender.
Video Coding StandardsH.120H.120
The First Digital Video Coding Standard
ITU-T (ex-CCITT) Rec. H.120: 1984
v1 (1984) had conditional replenishment, DPCM, scalar
quantization, variable-length coding, switch for quincunx sampling
v2 (1988) added motion compensation and background prediction
Multimedia Systems, Mahdi Amiri, Video IIIPage 25
v2 (1988) added motion compensation and background prediction
Operated at 1.544 (NTSC) and 2.048 (PAL) Mbits/s
Few units made, essentially not in use today
Conditional ReplenishmentConditional Replenishment: Can signal to leave a block area of the image : Can signal to leave a block area of the image
unchanged, or replace it with new data (using a threshold value).unchanged, or replace it with new data (using a threshold value).
Quincunx samplingQuincunx sampling: In a digital video system, a sampling structure with : In a digital video system, a sampling structure with
an array of samples where alternate rows of pixel samples are displaced an array of samples where alternate rows of pixel samples are displaced
horizontally in the grid by half of the pitch of the pixel samples along the horizontally in the grid by half of the pitch of the pixel samples along the
remaining rows.remaining rows.
Video Coding StandardsH.261H.261
ITU-T, completed in 1990, The first widespread
practical success
Video telephony and teleconferencing over ISDN
(Integrated Services Digital Network)
Multimedia Systems, Mahdi Amiri, Video IIIPage 26
Embodying typical structure that dominates today
Combination of DPCM and DCT
Motion Compensation
p x 64kbps (64-2048 kbps)
Still in use, although mostly as a backward-compatibility
feature – overtaken by H.263
Video Coding StandardsMPEGMPEG--11
MPEG-1: “Coding of moving pictures and associated audio for
digital storage media” (1992)
Group of Pictures, Motion Estimation and Motion Compensation,
Differential Coding, DCT, Quantization, Entropy Coding
Video on digital storage media (CD-ROM)
Multimedia Systems, Mahdi Amiri, Video IIIPage 27
Video on digital storage media (CD-ROM)
Target was VHS Quality at 1.5MBits/s (at 352x240 resolution)
Basis of Video-CD
MP3 (MPEG-1 Layer 3)
16 bits
Sampling rate - 32, 44.1, or 48 kHz
Bitrate – 32 to 320 kbps
De facto - 44.1 kHz sample rate, 192 kbps bitrate
Video Coding StandardsMPEGMPEG--11
Only supports progressive pictures.
Adds bi-directional motion prediction to H.261 design.
Adds half-pixel motion estimation (See next slide).
Slice-structured coding.
Multimedia Systems, Mahdi Amiri, Video IIIPage 28
Slice-structured coding.
DC-only “D” pictures.
Superior quality to H.261 when operated a higher bit
rates ( > 1 Mbps for CIF 352x288 resolution).
Now mostly overtaken by MPEG-2.
Video Coding StandardsMPEGMPEG--1, Half1, Half--Pixel MEPixel ME
Half-Pixel (coarse-fine) Motion Estimation Algorithm
1) Coarse step: Perform integer motion estimation on blocks; find best integer-pixel MV
2) Fine step: Refine estimate to find best half-pixel MV
a) Spatially interpolate the selected region in reference frame
b) Compare current block to interpolated reference frame block
c) Choose the integer or half-pixel offset that provides best matchTypically, bilinear Typically, bilinear
Multimedia Systems, Mahdi Amiri, Video IIIPage 29
Typically, bilinear Typically, bilinear
interpolation is used for interpolation is used for
spatial interpolationspatial interpolation
Video Coding StandardsMPEGMPEG--22
MPEG-2: “Generic coding of Moving Pictures and
Associated Audio”.
Broadcasting and storage.
Satellite TV, DVD, Digital TV.
Multimedia Systems, Mahdi Amiri, Video IIIPage 30
Ubiquity in hardware implies that it will be here for a
long time.
Transition to HDTV has taken over 10 years and is not finished yet
Different profiles and levels allow for quality control.
Bitrates: Typ. 4-9 MBits/s (Not especially useful below 4 Mbps,
normal range of use 5-30 Mbps).
Video Coding StandardsMPEGMPEG--22
Support for interlaced scan, various picture
sampling formats, user defined quantization
matrix.
Essentially same as MPEG-1 for progressive-
Multimedia Systems, Mahdi Amiri, Video IIIPage 31
Essentially same as MPEG-1 for progressive-
scan pictures.
Various forms of scalability (SNR, Spatial,
Temporal and hybrid).
Base Layer: Basic quality requirement, For SDTV.
Enhanced Layer: High quality service, For HDTV.
Video Coding StandardsMPEGMPEG--2 Profiles and Levels2 Profiles and Levels
Goal: To enable more efficient implementations for different
applications (interoperability points)
Profile: Subset of the tools applicable for a family of applications
Level: Bounds on the complexity for any profile
Multimedia Systems, Mahdi Amiri, Video IIIPage 32
Video Coding StandardsBitrateBitrate allocationallocation
CBR – Constant BitRate
Streaming media uses this.
Easier to implement.
VBR – Variable BitRate
Multimedia Systems, Mahdi Amiri, Video IIIPage 33
VBR – Variable BitRate
DVD’s use this.
Allocate more bits for complex scenes.
Usually requires 2-pass coding.
This is worth it, because you assume that you encode
once, decode many times.
Video Coding StandardsMPEG Container FormatMPEG Container Format
Container format is a file format that can
contain data compressed by standard codecs.
2 types for MPEG
Program Stream (PS) – Designed for
Multimedia Systems, Mahdi Amiri, Video IIIPage 34
Program Stream (PS) – Designed for
reasonably reliable media, such as disks.
Transport Stream (TS) – Designed for lossy
links, such as networks or broadcast antennas.
Video Coding StandardsMPEGMPEG--3 ?3 ?
Originally developed for HDTV, but abandoned when
MPEG-2 was determined to be sufficient.
Multimedia Systems, Mahdi Amiri, Video IIIPage 35
Video Coding StandardsH.263H.263
ITU-T Rec. H.263 (v1: 1995): The next generation of
video coding performance, developed by ITU-T – the
current premier ITU-T video standard (has overtaken
H.261 as dominant videoconferencing codec)
Video telephony over PSTN (public switched telephone
Multimedia Systems, Mahdi Amiri, Video IIIPage 36
Video telephony over PSTN (public switched telephone
network)
Wins by a factor of two at very low rates
Version 2 (late 1997 / early 1998) & version 3 (2000)
later developed with a large number of new features
H.263+ & H.263++ (Extensions to H.263)
Video Coding StandardsMPEGMPEG--44
MPEG-4: “Coding of audio-visual objects”.
Started as very low-bitrate project.
Contains the H.263 baseline design and adds many
creative new extras:
Multimedia Systems, Mahdi Amiri, Video IIIPage 37
creative new extras:
Coding of media objects (Segmented coding of shapes).
Bitrate: variable.
Synthetic/Semi-synthetic objects.
XMT: Like HTML, but to build videos.
First standard with Intellectual Property Management.
Video Coding StandardsMPEGMPEG--44
Part Number Title Description
Part 1 ISO/IEC 14496-1 SystemsDescribes synchronization and multiplexing of video and audio. For example
Transport stream.
Part 2 ISO/IEC 14496-2 VisualA compression codec for visual data (video, still textures, synthetic images, etc.).
One of the many "profiles" in Part 2 is the Advanced Simple Profile (ASP).
Part 3 ISO/IEC 14496-3 Audio
A set of compression codecs for perceptual coding of audio signals, including some
variations of Advanced Audio Coding (AAC) as well as other audio/speech coding
tools.
Part 4 ISO/IEC 14496-4 Conformance Describes procedures for testing conformance to other parts of the standard.
Part 5 ISO/IEC 14496-5 Reference Software Provides software for demonstrating and clarifying the other parts of the standard.
Delivery Multimedia
Multimedia Systems, Mahdi Amiri, Video IIIPage 38
Part 6 ISO/IEC 14496-6
Delivery Multimedia
Integration Framework
(DMIF).
Part 7 ISO/IEC 14496-7Optimized Reference
Software
Provides examples of how to make improved implementations (e.g., in relation to
Part 5).
Part 8 ISO/IEC 14496-8 Carriage on IP
networksSpecifies a method to carry MPEG-4 content on IP networks.
Part 9 ISO/IEC 14496-9 Reference HardwareProvides hardware designs for demonstrating how to implement the other parts of
the standard.
Part 10 ISO/IEC 14496-
10
Advanced Video
Coding (AVC)A codec for video signals which is technically identical to the ITU-T H.264 standard.
http://en.wikipedia.org/wiki/MPEGhttp://en.wikipedia.org/wiki/MPEG--44
Video Coding StandardsMPEGMPEG--4, Object Based Coding4, Object Based Coding
Extension of MPEG-1/2-type algorithms to code arbitrarily shaped objects
Multimedia Systems, Mahdi Amiri, Video IIIPage 39
Basic Idea: Extend BlockBasic Idea: Extend Block--DCT and BlockDCT and Block--ME/MCME/MC--
prediction to code prediction to code arbitrarily shaped arbitrarily shaped objects.objects.
[MPEG Committee][MPEG Committee]
Video Coding StandardsMPEGMPEG--4, Sprite Coding4, Sprite Coding
Sprite: Large background image.
Hypothesis: Same background exists for many
frames, changes resulting from camera motion and
occlusions.
Multimedia Systems, Mahdi Amiri, Video IIIPage 40
One possible coding strategy:
1. Code & transmit entire sprite once.
2. Only transmit camera motion parameters for each
subsequent frame.
Significant coding gain for some scenes.
Video Coding StandardsMPEGMPEG--4, Sprite Coding4, Sprite Coding
Multimedia Systems, Mahdi Amiri, Video IIIPage 41
[MPEG Committee][MPEG Committee]
Video Coding StandardsH.264 or MPEGH.264 or MPEG--4 Part 10 (AVC)4 Part 10 (AVC)
MPEG-4 Part 10: Advanced Video Coding / H.264.
Designed by a Joint MPEG and VCEG.
Claims 50% bitrate savings to MPEG-2, 30% over MPEG-4!
Bitrate: 10’s to 100’s kb/s.
Variable Block Size, Multiple Reference Frames, Integer Transform,
Multimedia Systems, Mahdi Amiri, Video IIIPage 42
Variable Block Size, Multiple Reference Frames, Integer Transform,
Intra Prediction, In-loop Deblocking Filtering, 1/4-pel Resolution
Motion Estimation, ASO (Arbitrary Slice Ordering), FMO (Flexible
Macroblock Ordering).
Enhanced entropy coding.CAVLC (Context Adaptive Variable Length Coding)
CABAC (Context Adaptive Binary Arithmetic Codes)
Increased complexity relative to prior standards.
Video Coding Standards, H.264Integer TransformInteger Transform
MPEG-2, MPEG-4
Multimedia Systems, Mahdi Amiri, Video IIIPage 43
MPEG-4 AVC
Find out more about integer
transform by looking for
"lifting scheme"
Video Coding Standards, H.264Variable Block SizeVariable Block Size
The fixed block size may not be suitable for all motion objects
Improve the flexibility of comparison
Reduce the error of comparison
7 types of blocks for selection
Multimedia Systems, Mahdi Amiri, Video IIIPage 44
00
10
2 3
0 11
16 x 16 16 x 8 8 x 16 8 x 8
00
10
2 3
0 11
8 x 8 8 x 4 4 x 8 4 x 4
Video Coding Standards, H.264Variable Block SizeVariable Block Size
Residual (without MC) showing optimum choice of partitions
Multimedia Systems, Mahdi Amiri, Video IIIPage 45
Video Coding Standards, H.264Multiple Reference FramesMultiple Reference Frames
The neighboring frames are not the most similar in some cases.
The B-frame can be reference frame.
B-frame is close to the target frame in many situations.
Multimedia Systems, Mahdi Amiri, Video IIIPage 46
Video Coding Standards, H.264DeblockingDeblocking FilterFilter
There are severe blocking artifacts.
4*4 transforms and block-based motion compensation.
Result in bit rate savings of around 6~9%.
Improve subjective quality and PSNR of the decoded picture.
Multimedia Systems, Mahdi Amiri, Video IIIPage 47
Without Filter With AVC Deblocking Filter
Video Coding Standards, H.264FMO (Flexible FMO (Flexible MacroblockMacroblock Ordering)Ordering)
Slice (composed in FMO) ���� Enhance Robustness to data loss
Multimedia Systems, Mahdi Amiri, Video IIIPage 48
Subdivision of a picture into Subdivision of a picture into
slices when slices when not using FMOnot using FMO
Subdivision of a QCIF frame into Subdivision of a QCIF frame into
slices when slices when utilizing FMOutilizing FMO
H.264, ProfilesABCABC
ABC
http://en.wikipedia.org/wiki/MPEG
http://en.wikipedia.org/wiki/MPEG--4_AVC.htm
4_AVC.htm
Multimedia Systems, Mahdi Amiri, Video IIIPage 49
http://en.wikipedia.org/wiki/MPEG
http://en.wikipedia.org/wiki/MPEG
H.264, ProfilesABCABC
ABC
Multimedia Systems, Mahdi Amiri, Video IIIPage 50
Video Coding Standards, H.265H.265 or HEVCH.265 or HEVC
High Efficiency Video Coding (HEVC) / H.265.
Under joint development by MPEG and VCEG.
Claims 50% bitrate savings to H.264.
Two of the key features where HEVC was improved compared to H.264/MPEG-4
AVC was:
Support for higher resolution video.
MP: Main Profile
HP: High Profiel
ASP: Advanced Simple Profile
HLP: High Latency Profile
Multimedia Systems, Mahdi Amiri, Video IIIPage 51
Support for higher resolution video.
Improved parallel processing methods. en.wikipedia.org/wiki/High_Efficiency_Video_Coding
Scalable Video CodingMotivationMotivation
Basic situation:
1. Diverse receivers may request the same video
Different bandwidths, spatial resolutions, frame rates, computational capabilities
2. Heterogeneous networks and a priori unknown network conditions
Wired and wireless links, time-varying bandwidths
When you originally code the video you don’t know which client or network situation
Multimedia Systems, Mahdi Amiri, Video IIIPage 52
When you originally code the video you don’t know which client or network situation
will exist in the future.
Probably have multiple different situations, each requiring a different compressed
bitstream.
Need a different compressed video matched to each situation.
Possible solutions:
1. Compress & store MANY different versions of the same video.
2. Real-time transcoding (e.g. decode/re-encode).
3. Scalable coding.
Scalable Video CodingType of ScalabilityType of Scalability
Multimedia Systems, Mahdi Amiri, Video IIIPage 53
The basic types of scalability in video codingThe basic types of scalability in video coding
Scalable Video CodingTemporal ScalabilityTemporal Scalability
Based on the use of B-frames to refine the temporal resolution.
B-frames are dependent on other frames.
However, no other frame depends on a B-frame.
Each B-frame may be discarded without affecting other frames.
Multimedia Systems, Mahdi Amiri, Video IIIPage 54
Scalable Video CodingSpatial ScalabilitySpatial Scalability
Based on refining the spatial resolution
Base layer is low resolution version of video.
Enhanced (Enh1) contains coded difference between
upsampled base layer and original video.
Also called: Pyramid coding.
Multimedia Systems, Mahdi Amiri, Video IIIPage 55
Scalable Video CodingQuality ScalabilityQuality Scalability
Based on refining the amplitude resolution.
Base layer uses a coarse quantizer.
Enh1applies a finer quantizer to the difference between the original DCT
coefficients and the coarsely quantized base layer coefficients.
Also called: SNR Scalability.
Multimedia Systems, Mahdi Amiri, Video IIIPage 56
Note: Base & enhancement layers Note: Base & enhancement layers
are at the are at the samesame spatial resolution.spatial resolution.
Video Coding StandardsIndependent playersIndependent players
Efforts from independent companies in video
compression.
Main competitors:RealNetworks (www.real.com)
Multimedia Systems, Mahdi Amiri, Video IIIPage 57
RealNetworks (www.real.com)
Microsoft Windows Media (WM) (www.microsoft.com)
Apple Quicktime (www.apple.com)
On2 (www.on2.com) (now acquired by Google)
MPEG & Multimedia ApplicationMPEGMPEG--77
MPEG-7: “Multimedia Content Description
Interface” (2001)
Describing audio/video
Applications:
It uses XML to store metadata, and can
be attached to timecode in order to tag
particular events, or synchronize lyrics
Multimedia Systems, Mahdi Amiri, Video IIIPage 58
Applications:
Indexing of video databases.
Search & Retrieval.
Browsing.
Independence between description and content.
particular events, or synchronize lyrics
to a song, for example.
MPEGMPEG--2121
Aims at defining an open framework for multimedia
applications.
MPEG-21 is based on two essential concepts:
Definition of a Digital Item (a fundamental unit of distribution
and transaction).
MPEG & Multimedia Application
Multimedia Systems, Mahdi Amiri, Video IIIPage 59
and transaction).
Users interacting with Digital Items.
Main objective of the MPEG-21:
Is to define the technology needed to support users to exchange,
access, consume, trade or manipulate Digital Items in an efficient
and transparent way.
MPEGMPEG--21 and MPEG21 and MPEG--7 Example7 Example
Metadata and Identifiers
within an MPEG-21 Music
Album
MPEG & Multimedia Application
Multimedia Systems, Mahdi Amiri, Video IIIPage 60
MPEGMPEG--21, DIDL21, DIDL
Digital Item Declaration Language (DIDL)
Example:
<ITEM>
<DESCRIPTOR>
MPEG & Multimedia Application
Multimedia Systems, Mahdi Amiri, Video IIIPage 61
<DESCRIPTOR>
<STATEMENT TYPE=“text/plain”>
A Personal Photo
</STATEMENT>
</DESCRIPTOR>
<COMPONENT>
<RESOURCE
REF=“file:///pix/dsc1234.REF=“file:///pix/d
sc1234.jpg”/>
</COMPONENT>
</ITEM>
The Digital Item Declaration
Model describes a set of
abstract terms and concepts
to form a useful model for
defining Digital Items.
Thank You
Multimedia SystemsMultimedia Systems
Video III (Video Coding Standards)Video III (Video Coding Standards)
Multimedia Systems, Mahdi Amiri, Video IIIPage 62
Thank You
1. http://ce.sharif.edu/~m_amiri/
2. http://www.dml.ir/
FIND OUT MORE AT...
Next Session: Multimedia Networks INext Session: Multimedia Networks I