cmpt365 multimedia systems 1 media compression - video coding standards spring 2015 cmpt 365...

CMPT365 Multimedia Systems 1

Media Compression- Video Coding Standards

Spring 2015

CMPT 365 Multimedia Systems


Video Coding Standards

H.264/AVC


Coding Rate and Standards

8 16 64 384 1.5 5 20

kbit/s Mbit/s

Very low bitrate Low bitrate Medium bitrate High bitrate

Mobilevideophone

Videophoneover PSTN

ISDNvideophone

Digital TV HDTVVideo CD

MPEG-4 MPEG-1 MPEG-2H.261H.263


Standardization Organizations

ITU-T VCEG (Video Coding Experts Group)

standards for advanced moving image coding methods appropriate for conversational and non-conversational audio/visual applications.

ISO/IEC MPEG (Moving Picture Experts Group)

standards for compression and coding, decompression, processing, and coded representation of moving pictures, audio, and their combination

Relation ITU-T H.262~ISO/IEC 13818-2(mpeg2) Generic Coding of Moving Pictures and

Associated Audio. ITU-T H.263~ISO/IEC 14496-2(mpeg4)

WG - work groupSG – sub group ISO/IEC JTC 1/SC 29/WG 1 Coding of Still Pictures

ISO/IEC JTC 1/SC 29/WG 11


Introduction

H.261 MPEG-1 MPEG-2 H.263 MPEG-4 H.264


H.261

Earliest DCT-based video standard: 1990 ITU Recommendation for videoconferencing and

videophones over ISDN Targeted bit rate: p x 64 kbps (p=1, …, 30)

Videophone: low rate, e.g., 64kbps Videoconferencing: high rate, e.g., 384kbps (p=6) Max: 1.92Mbps (p=30)

Picture format: CIF (Common Intermediate Format, 352 x 288) QCIF (Quarter CIF): 176 x 144.

Max delay: 150 ms (for bidirectional interactivity) Sequential search Amenable to low-cost VLSI implementation No B mode


MPEG-1

Committee formed in 1988 Finalized in 1991 Used for VCD Random access, fast forward/reverse search Delay: 1 sec (for unidirectional video access) 1/2-pixel ME/MC No deblocking filter B frames Software-only decoding is possible MPEG-1 Audio coding:

3 layers of encoding:• Layer 1: 4 : 1 compression ratio with CD quality• Layer 2: 6 : 1 to 8 : 1• Layer 3 (MP3): 10 : 1 to 12 : 1


MPEG-2

Completed in 1994 Extension of MPEG-1 Standard for DVD, SDTV, HDTV Support interlaced inputs Support scalable coding Flexible frame size Low delay Support a wide range of applications Source format: 4:4:4:, 4:2:2, 4:2:0 1/2-pixel ME/MC (bilinear interpolation) B frames MPEG-2 Audio:

Support 5.1 channels AAC: 30% fewer bits than MP3


Profiles and Levels

Defined to manage the large number of coding tools and the broad range of formats and bit rates supported

Profiles and levels define a set of conformance points, each targeting a class applications

Maximize interoperability and limiting the complexity Profile: a subset of the entire bit stream syntax Levels: a specified set of constraints imposed on values

of the syntax elements in the bit stream (maximum bit rate, buffer size, pic. resolution)


MPEG-2 Levels

Level Max Pixels Max Lines Max Frame/s

Low 352 288 30

Main 720 576 30

High 1440 1440 1152 60

High 1920 1152 60


H.263 Derived from H.261 Intended for very low bit-rate application

Better quality at 18-24kbps than H.261 at 64 kbps Used in MS NetMeeting, Messenger …

Can handle high resolution (up to 16CIF: 1408 x 1152) No loop filter 1/2-pixel ME/MC Optional coding modes (defined in 8 Annexes):

Unrestricted motion vector (Annex D):• MV can point outside of picture boundary by extrapolating the

boundary pixels (repeat padding is usually used)• MV range: [-31.5, 31.5]

Arithmetic coding Advanced prediction (Annex F):

• Overlapped block motion compensation• 4MV: 1 for each 8x8 block


H.263+ and H.263++

H.263+: Second version of H.263 Some further optional features: Annex I to T. Annex J: in-loop deblocking filter H.263++: three more optional modes (2000)

Annex V: Data partitioned slice mode • For enhanced resilience to transmission error


MPEG-4

Based on H.263 A new concept rather than an improved algorithm Deal with a variety of multimedia contents: audio, visual

, image, graphic. Part 2: Visual

Based on H.263 Object-based coding Coding of animated objects Scalability: Fine Granular Scalability (FGS) Texture coding: wavelet-based

Part 10: Advanced Video Coding H.264


Video Objects (VO)

MPEG-4 treats a video sequence as a collection of video objects

Each scene is decomposed into multiple objects The segmentation method is not part of the standard

Each object is specified by shape, motion, and texture. Natural visual Objects:

Image, video, sprite (background) Synthetic visual object:

Face and body 2-D mesh 3-D mesh

The decoder can compose different scenes by using different number of decoded objects


Scene Composition

The decoder can compose different scenes by using different number of decoded objects


MPEG-4 Structure

A/Vobject

Decoder

MUX

Com

posito

r

Bitstream Audio/Video scene

A/Vobject

Decoder

A/Vobject

Decoder


A video frame

Background VOP

VOP

VOP

More MPEG-4 Example

Instead of ”frames”: Video Object Planes Shape Adaptive DCT

Alpha map

SA DCT


Object 2

Object 1

Object 3

Object 4

Example

Problems, comments?


Example


Status

Microsoft, RealVideo, QuickTime, ... But only recentagular

frame based H.264 = MPEG-4 part 10

(2003)


H.264

1998: Call for proposal for H.26L issued by ITU-T VCEG (Video Coding Expert Group)

Objective: 50% bit rate savings compared to MPEG-2 High quality video at both low and high bit rates More error resilience tools

Oct. 1999: First draft design Dec. 2001: VCEG and MPEG formed the Joint

Video Team (JVT) Approved in 2003:

ITU-T H.264 and ISO/IEC MPEG-4 Part 10 Advanced Video Coding (AVC)


H. 264 Applications Bit rate: 64kbps to 240Mbps Broadcast over cable, satellite, DSL … Interactive/serial storage on optical/magnetic devices, DVD … Conversational services over network Video on demand, streaming media over network Multimedia messaging service over network

Three Profiles: Baseline, Main, and Extended 15 levels Four new profiles in Fidelity Range Extenstions (FRExt):

High, High 10, High 4:2:2, High 4:4:4


H. 264 Slice coding

Slice 1

Slice 2

Slice 3

Slices can have different shapes and sizes Each slice is self-contained

Can be decoded without knowing data other slices Useful for:

Error resilience and concealment Parallel processing


Summary of StandardsStandard Digitisation

formatCompressed rate Example applications

H. 261 CIF/ QCIF X 64 kbps Video conferencing over LANs

H. 263 S-QCIF/ QCIF <64kbps Video conferencing over low bits rate channels

MPEG 1 SIF <1.5Mbps VHS quality video storage

MPEG 2LowMain

High 1440

High

SIF4:2:04:2:24:2:04:2:24:2:04:2:0

<4Mbps<15Mbps<20Mbps<60Mbps<80Mbps<80Mbps<100Mbps

VHS quality video recordingDigital video broadcasting

High definition TV (4/3)

High definition TV (16/9)

MPEG 4 Various 5kbps – tens Mbps Versatile multimedia coding standard

H.264 Various Various Various

SIF: Standard Interchange Format, 352x240 pixels at 30 Hz.


Beyond H.264

H.265 50% goal (bitrate reduction) Start from 2010 February 2012: Committee Draft (complete draft of standard) July 2012: Draft International Standard January 2013: Final Draft International Standard (ready to be ratified as a

Standard) July 7: Formal release

Scalable video coding Multiview video/3D video 4K UHD

cmpt365 multimedia systems 1 media compression - video coding standards spring 2015 cmpt 365...

Documents