overview: image and video coding standards...bernd girod: ee398b image communication ii video coding...

18
Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 1 Overview: Video Coding Standards Video coding standards: applications and common structure Relevant standards organizations ITU-T Rec. H.261 ITU-T Rec. H.263 ISO/IEC MPEG-1 ISO/IEC MPEG-2 ISO/IEC MPEG-4 Recent progress: H.264/AVC

Upload: others

Post on 05-May-2020

5 views

Category:

Documents


1 download

TRANSCRIPT

Page 1: Overview: image and video coding standards...Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 4 MPEG-1 Encoder Pre-processing DCT Weighting Quantization

Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 1

Overview: Video Coding Standards

Video coding standards: applications and common structureRelevant standards organizationsITU-T Rec. H.261 ITU-T Rec. H.263ISO/IEC MPEG-1 ISO/IEC MPEG-2ISO/IEC MPEG-4Recent progress: H.264/AVC

Page 2: Overview: image and video coding standards...Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 4 MPEG-1 Encoder Pre-processing DCT Weighting Quantization

Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 2

ISO/IEC MPEG

MPEG-1 Standard (1991) (ISO/IEC 11172)Target bit-rate about 1.5 MbpsTypical image format CIF, no interlaceFrame rate 24 ... 30 fpsMain application: video storage for multimedia (e.g., on CD-ROM)

MPEG-2 Standard (1994) (ISO/IEC 13818)Extension for interlace, optimized for TV resolution (NTSC: 704 x 480 Pixel)Image quality similar to NTSC, PAL, SECAM at 4 - 8 MbpsHDTV at 20 Mbps

MPEG-4 Standard (1999) (ISO/IEC 14496)Object based codingWide-range of applications, with choices of interactivity, scalability, error resilience, etc.

Page 3: Overview: image and video coding standards...Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 4 MPEG-1 Encoder Pre-processing DCT Weighting Quantization

Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 3

MPEG-1/2: GOP Structure

"Group of Pictures" = “GOP“, GOP structure is very flexible

I-Picture P-Picture P-Picture

B-Pictures

time

1 3 4 2 6 7 8 5

Page 4: Overview: image and video coding standards...Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 4 MPEG-1 Encoder Pre-processing DCT Weighting Quantization

Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 4

MPEG-1 Encoder

Pre-processing DCT Weighting Quantization VLC Video

multiplex BufferPicture reordering

Inverse quantization

Inverse weighting

Inverse DCT

Picture store 1

Picture store 2

Motion compensation+

-

+

1/2

zero

Motion vectors, macroblock info, start codesVideo in

Bitstream

Page 5: Overview: image and video coding standards...Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 4 MPEG-1 Encoder Pre-processing DCT Weighting Quantization

Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 5

MPEG-1: coding of I-pictures

I-pictures: intraframe coded8x8 DCTArbitrary weighting matrix for coefficientsDifferential coding of DC-coefficientsUniform quantizationZig-zag-scan, run-level-codingEntropy codingUnfortunately, not quite JPEG

Page 6: Overview: image and video coding standards...Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 4 MPEG-1 Encoder Pre-processing DCT Weighting Quantization

Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 6

MPEG-1: coding of P-pictures

Motion-compensated prediction from an encoded I-picture or P-picture (DPCM)Half-pel accuracy of motion compensation, bilinear interpolationOne displacement vector per macroblockDifferential coding of displacement vectorsCoding of prediction error with 8x8-DCT, uniform threshold quantization, zig-zag-scan as in I-pictures

Page 7: Overview: image and video coding standards...Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 4 MPEG-1 Encoder Pre-processing DCT Weighting Quantization

Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 7

MPEG-1: coding of B-pictures

Motion-compensated prediction from two consecutive P- or I-pictures

either• only forward prediction (1 vector/macroblock)

or• only backward prediction (1 vector/macroblock)

or• Average of forward and backward prediction = interpolation (2

vectors/macroblock)

Half-pel accuracy of motion compensation, bilinear interpolationCoding of prediction error with 8x8-DCT, uniform quantization, zig-zag-scan as in I-pictures

Page 8: Overview: image and video coding standards...Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 4 MPEG-1 Encoder Pre-processing DCT Weighting Quantization

Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 8

MPEG-2 vs. MPEG-1

Efficiently compress interlaced digital video at broadcast quality

Field/frame picturesChroma samplingNew prediction modesField/frame DCTAdditional scan patterns for DCT coefficientsMotion compensation with blocks of size 16x8 pels

Improved coding efficiency by different quantization, VLC tablesVarious scalability modes

Page 9: Overview: image and video coding standards...Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 4 MPEG-1 Encoder Pre-processing DCT Weighting Quantization

Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 9

Coding of Interlaced Video (1)

Frame and field picture structures

Page 10: Overview: image and video coding standards...Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 4 MPEG-1 Encoder Pre-processing DCT Weighting Quantization

Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 10

Coding of Interlaced Video (2)

Field prediction for field pictures

Page 11: Overview: image and video coding standards...Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 4 MPEG-1 Encoder Pre-processing DCT Weighting Quantization

Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 11

Coding of Interlaced Video (3)

Field prediction for frame pictures

16

16

16

8

8

Page 12: Overview: image and video coding standards...Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 4 MPEG-1 Encoder Pre-processing DCT Weighting Quantization

Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 12

Coding of Interlaced Video (4)

Dual prime for P pictures

Page 13: Overview: image and video coding standards...Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 4 MPEG-1 Encoder Pre-processing DCT Weighting Quantization

Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 13

Coding of Interlaced Video (5)

Field/frame DCTAlternate Scan

Page 14: Overview: image and video coding standards...Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 4 MPEG-1 Encoder Pre-processing DCT Weighting Quantization

Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 14

MPEG-4

Support highly interactive multimedia applications as well as traditional applicationsAdvanced functionalities: interactivity, scalability, error resilience…Coding of natural and synthetic audio and video, as well as graphicsEnable the multiplexing of audiovisual objects and composition in a scene

Video on LANs, Internet videoWireless videoVideo databaseInteractive home shoppingVideo e-mail, home moviesVirtual reality games, flight simulation, multi-viewpoint training

‘TV/film’AV-data

‘Computer’Interactivity

‘Telecom’Wireless

Page 15: Overview: image and video coding standards...Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 4 MPEG-1 Encoder Pre-processing DCT Weighting Quantization

Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 15

MPEG-4: Scene with audiovisual objects

Page 16: Overview: image and video coding standards...Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 4 MPEG-1 Encoder Pre-processing DCT Weighting Quantization

Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 16

MPEG-4: Video coding

Basic video codingDefinition of Video Object (VO), Video Object Layer (VOL), VideoObject Plane (VOP) Improved coding efficiency vs. MPEG-1/2

• Based on H.263 baseline• Global motion compensation• Sprites• Quarter pixel motion compensation

Page 17: Overview: image and video coding standards...Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 4 MPEG-1 Encoder Pre-processing DCT Weighting Quantization

Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 17

MPEG-4: Video coding

Object-based video codingBinary shape coding

α-map shape codingPadding for block-based DCT of textureShape-adaptive DCT

DWT for still texture codingMesh animation, face and body animation

Page 18: Overview: image and video coding standards...Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 4 MPEG-1 Encoder Pre-processing DCT Weighting Quantization

Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 18

Shape Adaptive DCT