MPEG-5 Essential Video Coding (EVC) StandardSamsung Research

Kiho Choi

Introduction

1

2

Background for a development of new video coding standards

• It's estimated that by 2022, 82 percent of the global internettraffic will come from video streaming and downloads (Cisco, 2019)

• COVID-19 pandemic makes the following situation:

• “Video use up 4x from six months ago” (RCRwireless, 29 June 2020, Limelight)

• “YouTube is reducing the quality of videos for the next month, and it's because increased traffic amid the coronavirus outbreak is straining internet bandwidth” (Business Insider, 25 March 2020)

• “Netflix to cut streaming quality in Europe for 30 days” (BBC, 19 March 2020)

Rationale

3

Next Generation Video Coding Standards

• Historically, standard body produces a video coding standard with every 10 year period

• Versatile Video Coding Standard is being made under a joint team of ITU-T and MPEG

• MPEG-5 Essential Video Coding Standard is being made under MPEG

History

ITU-TStandards

H.261 H.263 H.263+ H.263++

Joint ITU-T/MPEG

H.262/MPEG2 H.264/AVC HEVC

MPEGStandard

MPEG-1 MPEG-4

VVC

MPEG-5EVC

1984 1988 1992 1996 2003 2004 2008 2013 2016 2020

4

MPEG-5 Part 1 Essential Video Coding (EVC) standard

• Developed by MPEG

• Novel process targeting

• Business requirements

• Licensing

• Technical requirements

• Facilitate rapid deployment and widespread use of the standard throughout the media industry

Overall Goal of MPEG-5 EVC• The new video coding standard should provide a video compression solution which combines to Meet

Business and Technical Requirements

• Timely availability of published licensing terms to allow reliable business plans to be created

• Coding efficiency at least as good as HEVC

• Complexity suitable for real time encoding

• the ability to address existing and emerging use cases, including

• offline encoding for streaming VOD

• live OTT streaming

EVC Project started in Jan 2019 after a CfP was completed• Four companies responded: Huawei, Qualcomm and Samsung [+Divideon] and they contributed to the

drafting process

EVC Project Information

5

Timeline

• WD in January 2019

• CD in July 2019

• DIS in October 2019

• FDIS in April 2020

• IS in October 2020

Goal & Timeline

EVC2019 / 2020

Jan. Feb. Mar. Apr. May Jun. Jul. Aug. Sep. Oct. Nov. Dec. Jan. Feb. Mar. Apr.

WD

CD

DIS

FDIS

6

EVC Profiles• Baseline profile: which contains tools that were made public more than 20 years ago or for which only Type 1

declarations have been received in a previous standard, and this profile is used for a video coding.

• Main profile: which contains a small number of additional tools, each providing a significant gain, and this profile is used for a video coding.

• Baseline Still Picture profile: which contains tools that are used in Baseline profile in order to support a picture coding.

• Main Still Picture profile: Still Picture profile: which contains tools that are used in Main profile in order to support a picture coding.

Structure of EVC profiles• EVC baseline profile is a basis of EVC codec that is not overlapped with HEVC technology.

• EVC Main profiles contains a small number of tools on top of baseline profile not using HEVC for the basis.

• When turning off a tool in Main profile, a baseline tool is used for a default tool for the same functionality.

EVC Standard structure

Main

Baseline

7

Tool Summary

*Switchable tools*Fallback to Baseline from Main

8

Tool Summary (cont’d)

Tool enabling flag in SPS Baseline Main Description

sps_btt_flag 0 0 or 1 0: QT, 1: BTT

sps_suco_flag 0 0 or 1 0: off, 1: on

sps_admvp_flag 0 0 or 1 0: base skip, direct, 6-tap interpolation, 1: merge, 8-tap interpolation, memory b/w, selective MVD signaling, MV compression

sps_affine_flag n/a 0 or 1 0: off, 1: on

sps_amvr_flag n/a 0 or 1 0: off, 1: on

sps_dmvr_flag n/a 0 or 1 0: off, 1: on

sps_mmvd_flag n/a 0 or 1 0: off, 1: on

sps_hmvp_flag n/a 0 or 1 0: off, 1: on

sps_eipd_flag 0 0 or 1 0: 5 modes, no chroma mode signaling, 1: 33 modes, 5 chroma modes, MPM, reference sample filling

sps_ibc_flag n/a 0 or 1 0: off, 1: on

sps_cm_init_flag 0 0 or 1 0: no context modeling and init., 1: context modeling and init.

sps_adcc_flag n/a 0 or 1 0: run-level-last coefficient coding, 1: advanced coefficient coding

sps_iqt_flag 0 0 or 1 0: H.264 chroma QP mapping, 1: HEVC chroma QP mapping, consideration of internal bit-depth

sps_ats_flag n/a 0 or 1 0: off, 1: on

sps_addb_flag 0 0 or 1 0: H.263 like DBF, 1: HEVC like DBF

sps_alf_flag 0 0 or 1 0: off, 1: on

sps_htdf_flag 0 0 or 1 0: off, 1: on

sps_rpl_flag 0 0 or 1 0: GOP size signaling, 1: RPL

sps_pocs_flag 0 0 or 1 0: implicit POC, 1: POC LSB signaling

sps_dquant_flag 0 0 or 1 0: CU based delta QP, 1: region based delta QP

sps_dra_flag 0 0 or 1 0: off, 1: on

9

Tool SummaryCategory Tool in Baseline Profile Tool in Main Profile

Partitioning • Tile & Slice: no tile, no slice• CTU size: 64• Block partitioning: QT

• Tile & Slice: support• CTU size: 128, 64, 32• Block Partitioning: BTT• Pipeline unit size: 64x64• SUCO• Local dual tree (for small chroma intra blocks)

Intra • DC and 4 angular modes• Ref. sample filling: middle value of bit-depth

• Mode coding: adaptive code word assignment• Chroma mode: Direct mode only

• Constrained intra prediction

• DC/Bi-linear/Plane and 30 angular modes• Ref. sample filling: extrapolation from top-left ref. sample• Interpolation filter: 4-tap Gaussian filter• Mode coding: MPM + PIMS• Chroma mode: Direct mode + DC/Bi-linear/Hor./Ver.• Intra block copy (IBC)• Constrained intra prediction

Inter • Skip mode: two indices signaling for both lists• Temporal direct mode• 6-tap MC interpolation (wiener filter)

• Skip mode: the same w/ merge• Merge mode• 8-tap MC interpolation (DCT-IF filter)• Affine, AMVR, DMVR, MMVD, HMVP• Memory B/W considerations: no 4x4 inter, uni 8x4/4x8• MV compression in unit of 8x8• Selective MVD signaling (a.k.a. ABP)

TransformQuantization

• DCT2 (64x64 - 2x2)

• CU-level delta QP: CU based• Chroma QP mapping table (from H.264)

• DCT2 (64x64 – 2x2) w/ internal bit-depth consideration• ATS (Intra: a.k.a. MTS, Inter: a.k.a. SBT)• CU-level delta QP: region based• Chroma QP mapping table (from HEVC)• Chroma QP mapping table signaling in SPS

In-loop filtering • DBF: H.263 + adaptive boundary strength • DBF: HEVC w/ minor modification• ALF, HTDF (post-reconstruction filtering)• Tile boundary filtering on/off• DRA for HDR

Entropy coding • CABAC w/o context modeling & initialization• Coefficient coding: Run/Level/Last coding

• CABAC• Coefficient coding: Advanced coefficient coding

High level syntax • POC: no signaling• Picture size: multiple of 8• Reference picture management: using GOP information• NAL: 2-byte header, IDR/Non-IDR/SPS/PPS/APS/SEI/FD• Tool set indicator

• POC: LSB signaling• Picture size: multiple of 8• Reference picture management: Reference picture list (RPL)• NAL: 2-byte header, IDR/Non-IDR/SPS/PPS/APS/SEI/FD• Tool set indicator

Testing result

10

11

Configuration

• Baseline tool set: Base tool set contains tools that were made public more than 20 years ago or for which only Type 1 declarations have been received in a previous standard

Coding performance

• Baseline profile (ETM6.0)

• Anchor: AVC/H.264 (JM19.0 High10)

Coding performance of MPEG-5 Test model in CTC

Y U V EncT DecT

4K seqence -38% -33% -38% 46% 91%

2K seqence -25% -27% -26% 39% 94%

Overall -31% -30% -32% 42% 93%

Random Access

Over JM19.0

12

Configuration

• Main profile tool set: Enhanced tools set contains a small number of additional tools

Coding performance

• Main profile (ETM6.0)

• Anchor: HEVC/H.265 (HM16.6)

Coding performance of MPEG-5 Test model in CTC

Y U V EncT DecT

Class A -30% -27% -26% 436% 168%

Class B -23% -24% -21% 540% 147%

Overall -26% -25% -24% 486% 157%

Random Access

Over HM16.16

13

Configuration

• Random access (RA): IDR picture period 1.1 second or less.

• Low Delay (LD): No picture reordering between decoder processing and output

Test sequence

Test sets

• AVC/H.264 (JM19.0 High10) vs EVC Baseline profile

• HEVC (HM16.20 Main10) vs EVC main profile

Preliminary subjective testing result for SDR

Class Sequence name Resolution Frame count Frame rate Chroma format Bit depth

SDR-A

BarScene 4096×2160p 600 60 4:2:0 10DrivingPOV 4096×2160p 600 60 4:2:0 10

Dancers 4096×2160p 600 60 4:2:0 10CatRobot 3840×2160p 600 60 4:2:0 10

SDR-B

BarScene 1920×1080p 600 60 4:2:0 10DrivingPOV 1920×1080p 600 60 4:2:0 10

Metro 1920×1080p 600 60 4:2:0 10RushHour 1920×1080p 300 30 4:2:0 10

14

MOS BD-rate for Baseline profile

Preliminary subjective testing result for SDR

SDR, Baseline, RA, 4KJM19.0 ETM6.0 BD-rate (piecewise cubic)

QPISlice bits MOS QPISlice bits MOS bits reduction

Class A

CatRobot

23 18249728 8.22 25 12647710 8.81

-44.0%27 9427642 7.81 29 6650028 8.2532 5238860 5.81 35 3126513 5.9439 2413263 2.38 43 1359661 2.66

BarScene

25 10638455 8.19 27 5376868 8.34

-47.9%27 5769534 7.66 29 3181024 7.6932 2565515 5.72 34 1480618 6.0639 1256031 1.91 41 687569 2.22

Dancers

24 10627865 7.34 26 5251676 8.28

-52.7%26 5292067 6.75 28 2775577 7.1930 2485826 4.97 32 1358234 5.0636 1301353 1.78 38 695513 3.44

DrivingPov

22 62531140 8.66 24 44390447 8.72

-55.9%26 32484781 8.09 28 20275946 8.4731 15621178 5.75 34 8194473 6.2837 7318298 3.09 40 3734092 4.00

Average -50.1%

SDR, Baseline, LB, 2KJM19.0 ETM6.0 BD-rate (piecewise cubic)

QPISlice bits MOS QPISlice bits MOS bits reduction

Class B

BarScene

21 5159374 8.44 22 3494251 8.44

-47.9%25 2430629 7.06 26 1615057 7.5030 1281545 4.81 31 797000 5.6937 611865 1.25 38 370352 1.75

DrivingPOV

23 19820597 7.31 24 15471625 7.50

-35.5%27 10461258 4.88 28 7598756 5.1932 4936486 3.13 33 3422388 3.4437 2617603 1.25 37 1899191 1.88

Metro

19 15337648 8.38 20 12343012 8.50

-32.7%22 10271261 7.19 23 8144720 7.2527 5334311 5.44 28 3890474 5.7533 2416243 2.19 34 1713629 3.25

RushHour

19 13767828 9.25 21 10782626 9.19

-21.5%23 8535295 8.25 24 7091102 8.4429 4387800 6.44 30 3297792 6.2534 2568422 4.38 34 2016885 4.38

Average -34.4%

15

MOS BD-rate for Main profile

Preliminary subjective testing result for SDR

SDR, Main, RA, 4KHM16.20 ETM6.0 BD-rate (piecewise cubic)

QPISlice bits MOS QPISlice bits MOS bits reduction

Class A

CatRobot

22 23098443 8.66 23 11693305 9.16

-46.2%27 6932117 8.44 29 4398668 8.4732 3505155 7.56 34 2203761 7.9137 1899506 5.75 39 1160625 5.97

BarScene

27 4732382 8.38 29 2603829 8.47

-47.9%32 1511056 6.88 35 896521 7.5038 719676 4.38 41 434653 4.6344 378746 1.44 47 244750 2.03

Dancers

22 26218368 8.41 24 11753572 8.72

-60.2%27 3085503 7.03 29 1280342 7.6632 1036903 5.91 34 686214 6.5637 560204 3.06 39 371672 3.47

DrivingPov

24 39860381 9.09 26 23448296 8.59

-45.5%28 17680603 8.28 30 10855738 8.4732 9049443 7.47 34 5707010 7.6638 3732064 4.19 40 2302969 4.97

Average -50.0%

SDR, Main, LB, 2KJM19.0 ETM6.0 BD-rate (piecewise cubic)

QPISlice bits MOS QPISlice bits MOS bits reduction

Class B

BarScene

21 4244292 8.38 22 2681083 8.75

-43.43%26 1425755 7.06 28 866037 7.4432 603112 4.00 33 431622 4.5638 283253 1.94 39 203966 2.31

DrivingPOV

23 17689497 7.56 24 11913780 8.13

-44.80%26 10134741 6.50 27 6749032 7.2530 5211334 4.88 31 3404732 5.0035 2328864 3.13 36 1545882 3.56

Metro

22 9385266 8.19 24 5857578 8.38

-35.16%27 4513628 6.50 28 3245318 6.5632 2256883 4.00 33 1577111 4.3837 1128646 1.63 38 789186 2.00

RushHour

20 10716039 8.94 22 7542164 9.38

-33.17%26 4944466 8.13 28 3258581 8.1331 2647473 6.38 33 1719232 6.1337 1318001 3.38 38 943607 3.88

Average -39.1%

16

Configuration

• Random access (RA): IDR picture period 1.1 second or less.

Test sequence

Test sets

• HEVC (HM16.20 Main10) vs EVC main profile

Subjective testing result for HDR

Class Sequence name Resolution Frame count Frame rate

HDR-AChimera5_4k 4096×2160p 600 60Chimera7_4k 4096×2160p 600 60Meridian2_4k 4096×2160p 600 60

HDR-BHurdles 1920×1080p 500 50

Chimera5_2k 1920×1080p 600 60Chimera7_2k 1920×1080p 600 60

17

MOS BD-rate for Main profile

Subjective testing result for HDR

HDR, Main, RA

HM16.20 ETM6.0 BD-rate (piecewise cubic)

QPISlice bits MOS QPISlice bits MOS bits reduction

HDR-A

ChimeraHDR5

22 54488219 8.79 23 28928371 9.11

-45.8%27 19311801 8.00 28 11658852 8.2932 8903308 6.36 33 5401148 6.3937 4405775 4.93 39 2326584 4.86

ChimeraHDR7

27 3326652 9.21 28 2127680 9.18

-24.7%32 1711685 8.39 33 1095575 7.8237 936040 6.50 38 603392 6.0041 619571 4.54 42 395069 3.96

Meridian2

22 8987798 9.04 23 4695802 8.39

-37.9%27 2980003 7.21 29 1587915 7.2132 1343550 6.14 33 839590 6.2537 640978 3.39 38 411606 2.79

HDR-B

ChimeraHDR5

22 19632155 8.39 23 10733191 8.89

-51.7%27 7226635 6.57 28 4401733 7.0032 3297088 4.89 33 2019439 5.4637 1599786 3.00 38 986845 3.46

ChimeraHDR7

22 3540874 9.11 23 2066647 9.32

-21.5%27 1403573 8.46 28 923356 8.5432 705344 7.00 33 474195 5.9637 394319 4.36 38 274733 3.50

Hurdles

27 4380346 8.71 28 3075951 8.93

-31.4%34 1710449 6.79 35 1144671 6.7537 1180312 5.43 38 806299 5.0039 923333 4.07 40 639654 4.39

Average 4K -36.1%

Average 2K -34.9%

18

Visual quality comparison at the same bitrate

Visual test

19

Visual quality comparison at the half bitrate for 8K sequence

Visual test

Conclusion

20

21

The main goal of the EVC standard is to provide a significantly improved compression capability over existing video coding standards with timely publication of terms.

The EVC standard is able to deliver the same subjective quality as the references, while only requiring about a half of the bit rate used by the previous standards (i.e., HEVC for EVC Main and AVC for EVC Baseline)

The MPEG-5 EVC standard is expected to be a great video codec for emerging delivery protocols and networks, such as 5G, enabling the delivery of high-quality video services to an ever-growing audience by providing improved coding performance.

Conclusion

22

Thank you