media file formats for broadcasters

Media File Formats for the Media File Formats for the BroadcasterBroadcaster

UnravelingUnraveling the the mysterymystery

Tom Beckers Matthias De Geyter

Wim Ermens Luk Overmeire

VRT Medialab - 16/10/2008

VRT-medialab: onderzoek en innovatie2

Agenda

Part One - Luk Overmeire & Matthias De Geyter

Introduction

Image and advanced compression formats for HD

File formats for HD production: MXF and AAF

--- pause ---

Part Two - Tom Beckers & Wim Ermens

Visual and technical quality control for HD

Demos


DistributionProduction infrastructure

Information management

Infrastructure(storage and network)

VRT Medialab4 research domains

IngestMedia

Asset Mgnt

Editing

Playout

Mastering

Manufacturing

EngineeringDistribution

Automation

Product

Development

Information front-end

Research Archiving

broadcaster

informationessence

customer

Internet

Mobile

Wireless andwired networks

Satellite

Personalization

User interaction

User generatedcontent

Social platforms


Production infrastructure

Infrastructure(storage and network)

VRT Medialab4 research domains

IngestMedia

Asset Mgnt

Editing

Playout

Mastering

Information management

Manufacturing

EngineeringDistribution

Automation

Product

Development

Research Archiving

Distribution

Internet

Mobile

Wireless andwired networks

Satellite

Information front-end

Personalization

User interaction

User generatedcontent

Social platforms


Context: tape-based mediaproduction


essence

information

Context: file-based mediaproduction with central warehouse

essence

information


Central storage

INGEST

Media

Asset Mgmt

IP network

Central

Storage

Draft Processing

Concurrent

engineering

Browse Clients Processing

during

ingest/transfer

POSTProduction

Craft

editors

Subtitling

Archiving

Analysis

Annotation

Follow-upcontinuity Editor-in-Chief

Post-sonorisation

PLAY-OUT

File-based media production


Media production formats

image formats

! resolution, frame rate, raster

compression formats

! high & low resolution

file/wrapper formats

essence

information

=essence

information

+


Typical broadcast chain

production

contribution

archive

distributionBlu-ray DVD

linear

Web

I

N

G

E

S

T

P

L

A

Y

O

U

T

central

warehous

e

post

production

Studio acquisition

File-based camera


Typical broadcast chainFormats for Standard Definition

production

contribution

archive

distributionDVD

linear

Web

I

N

G

E

S

T

P

L

A

Y

O

U

T

central

warehous

e

post

production

Studio acquisition

File-based camera

MPEG-2

long GOP

4:2:0

8-10 Mbps

MPEG-2

long GOP

4:2:2

10-25 Mpbs

DV25

IMX-30

IMX-50

576 lines, interlaced

25 frames/sec

MPEG-1

VC-1

MXF OP1a

DV25

IMX-30

IMX-50

MXF/AAFMXFH.263/VC-1

MPEG-2

long GOP


Typical broadcast chainFormats for High Definition

production

contribution

archive

distributionBlu-ray DVD

linear

Web

I

N

G

E

S

T

P

L

A

Y

O

U

T

central

warehous

e

post

production

Studio acquisition

File-based camera

H.264 4:2:0

bit rate ?MPEG-2 ->H.264

4:2:0, 4:2:2

bit rate ?

XDCAM HD,

AVC-Intra,

DNxHD,

ProRes,

JPEG2000

720p50, 1080i25,

1080p25, 1080p50

H.264 (1),

H.264 (2),

…

MXF OP1a

MXF/AAF,

QuicktimeMXF

XDCAM HD,

AVC-Intra,

DNxHD,

ProRes,

JPEG2000

uncompressed,

lossless compressed ?

H.264/VC-1

H.264/VC-1

long GOP


Proliferation of media formats

HD image formats

! 720p50,1080i25, 1080p25

=> 1080p50

HD compression formats

! legacy versus new formats

! contribution, camera, production, archive, distribution

proxy formats (low resolution)

wrapper formats

! MXF flavors, Quicktime


The HDTV format jungle

Physikalischer Transport

IT-basiert: File Format

Kompressionsformat

Abtastformat



Kompressionsformat

Abtastformat



Kompressionsformat

Abtastformat



Kompressionsformat

Abtastformat

Physical Transport(HD-SDI, IP, Tape, ..)

File Formats

(MXF, QT, ..)

Compression formats(MPEG-2, H.264, JPEG2000, ..)

Sampling formats(1080i/25, 720p/50, 1920/1440 pixels,

4:2:0, 4:4:4, ... )

Production

(more granular variants for news, features, etc.)

Distribution

Contribution („live“ and/or „File “)

Presentation (Displays)

Lossy

conversions

HDTV increases

variety of formats

HDTV increases

variety of options

„A format“

Further formats (e.g. consumer)

HDTV increases need

for efficient wrapping


Production areas

high-end production

! landmarks, drama

mainstream production

! soap series, documentaries, shows

News and sports

video journalism

! consumer cameras

" single or multiple scanning formats ?

" single or multiple compression formats ?

" 720p/50 a/o 1080i/25

" 4:2:2 a/o further

subsampling

" 10 Bit a/o 8 Bit

" I-frame only a/o long GOP

" Headroom a/o low data rate

" … a/o …

Mainstream HDTV platform

Image formats for HD(and beyond)


EBU

Tech 3299

HDTV System 1-4

Today’s and future image formats

Future

Today

System 2 – 1080p25

resolution 1920 x 1080

25 frames/sec

progressive

3 Gbit/s

1,5 Gbit/s

System 3 – 1080i25


50 fields/sec

interlaced

System 1 – 720p50


50 frames/sec

progressive

System 4 – 1080p50


50 frames/sec

progressive

Super Hi-Vision (U-HDTV)


60 frames/sec, progressive

audio: 22.2 channels

Far future 24 Gbit/s

?


EBU – Tech 3299

2,0736104:2:25010801920System 41080p/50

1,0368104:2:22510801920System 31080p/25

1,0368104:2:22510801920System 21080i/25

0,9216104:2:2507201280System 1720p/50

Net image BitRate [Gbit/s]

Sub-sampling /Quantisation [Bit]

Framerate

Activelines

Horizontalsamples

HDTVSystems

EBU 3299

Interface BitRate [Gbit/s]

1,5

1,5

1,5

3,0

0,270,207104:2:225576720SDTV

576i/25


The resolution revolution

Maximal viewing distance:

SD: d < 5.h

HD: d < 3.h

U-HDTV: d < 0.75.h

d = viewing distance

h = display height

now

+ 5y

+ 15y?


Good HDTV – More than anumbers game

1080 (i) > 720 (p) => “More is better” ?


How can smaller be better ? (1)

interlace factor

! interlaced scanning: 50 fields/sec (1 field = ! frame)

! 540 lines < dynamic vertical resolution < 1080 lines

Studies (Bell Laboratories, NHK): interlace factor ! 0.6 => 648 lines < 720 lines



interlace factor

interlace artefacts (motion rendition)

! e.g. serrated edges



interlace factor

interlace artefacts

resolution capacity of human eye (" 1 arc minute)

Required horizontal sampling for different screen sizes, viewed at 2.7m

720x576


How can smaller be better?

interlace factor

interlace artefacts

resolution capacity of human eye

compression artefacts

! better compression performance for progressive

! compression of interlaced media generally involves de-interlacing


Progressive versus interlaced

far better suited for moving pictures (e.g.sports)

easier to encode

plasma, LCD are progressive displays

=> end-to-end progressive workflow is possible

same resolution visibility from “viewingdistance”

rescaling is easier than deinterlacing

the future is progressive => evolution to1080p50 is more straightforward

legacy format

! as of today, generally bettersupported throughout the workflow

– camera, set-top-box, postproduction, central MAM, …

! existing program material

! current offering is predominantly

1080i25 content

compatible with 1080p25 through 1080psf

Pro

Contra

full support in end-to-end workflows to becarefully tested

! camera (quality), central MAM, postproduction, Dolby E surround sound

! time labeling ! (editing)

! distribution (Telenet, Belgacom, …)

! contribution (productiehuizen)

higher complexity

! down conversion requires de- and re-interlacing

! de-interlacing is very hard to do well

legacy format - “wet van de remmende voorsprong”

less suited for slow motion and chroma keying


720p

Different scenarios for imageformats

1080i 1080i

Acquisition

1080ito

1080p

Production

1080i

Distribution

1080i 1080p

Full HDDisplay

1080i 1080i 1080i 720p1080i

to720p

720pto

1080p

1080p

Deinterlacingconverter

Rescalingconverter

Scenario 1

Scenario 2

1080i1080i

to720p

720p 720p720p

to1080p

1080pScenario 3

Scenario 4 720p 720p 720p 720p720p

to1080p

1080p

Distribution

format

Production

format

Best quality

What is effect

of conversion

on quality?

Best supported


Broadcast production chain –Points of attention for 720p50 workflows

Studio camera

Cross-convertorSet-top-box

Real-time infrastructure

File-based production environment

Cross-convertor

MXF

Central MAM

Post productionSurround sound

p50 timing

End-to-end workflow should be carefully designed !

1

Today’s content is mostly 1080i

2

Motion compensation algorithm

5

Support (legacy systems)

3

Light sensitivity, S/N ratio

4 Full tested support (essence, TC, …)

7 Dolby E compatibility with p50

6 TC support (legacy formats)

8 EBU Joint Taskforce on

Timing & Synchronisation

9EICTA label - HDTV


EBU/SMPTE Joint Taskforce onTime Labelling and Synchronization

twofold goal

! new time labelling method allowing higher frame rates (50 fpsand higher)

! new synchronization mechanism

! current standards (SMPTE 12M, colour black) are 30 years old

=> workable but limited

status

! Request For Technology (RFT) – February 2008

– responses from Sony, Harris, TGV, …

! draft Request For Standardisation (RFS)

– time-related labeling


Scenarios for 1080p25

1080p25 1080p25

Acquisition

1080psfto

1080p

Production

1080p25

Distribution

1080psf(1080i)

1080p

Full HDDisplay

1080p25 720p1080p25

to720p

720pto

1080p

1080p

Rescalingconverterwith loss of resolution

Rescalingconverter

Scenario 1

Scenario 2 1080p25 1080p25

psf = progressive with segmented frames

! frames are segmented into two interlaced fields with equal timeindex => compatible with 1080i distribution

! easy deinterlacer (simple “weave”)

1080p25to

1080psf

Simplereordening


What about 1080p50?

no one already broadcasts in 1080p50

! but some have plans …

1080p50 in production

! not around the corner, but plausible within 3-5 years

! very good master for both 720p50 and 1080i25

! better quality (less artifacts due to smaller block sizes andhigher coding efficiency)

1080p50 in distribution

! commercial factors will likely be the driving factor

Compressionformats for HD


Basics of video coding

What is video compression?

! Video compression is a technique that reduces the bit rate torepresent video images by exploiting signal redundancies whilepreserving the original quality as much as possible

basic types of redundancies

! statistical redundancies

– spatial and temporal interpixel redundancies

– coding redundancy

! psycho-visual redundancies

– human visual system is less sensitive to higher frequenciesand color


Common techniques for videocompression

source

encoderre-

quantizationentropy

encoding

input

videocompressed

video

subsampling transform

horizontal

subsampling

color

subsampling

DCT

wavelet

prediction

Intra-frame

Interframe

lossless

vs lossy

rate control

CBR/VBR


Color subsampling

source

encoderquantization

entropy

encoding

input

videocompressed

video

subsampling transform

horizontal

subsampling

color

subsampling

DCT

wavelet

prediction

Intra-frame

Interframe

lossless

vs lossy

rate control

CBR/VBR

color

subsampling


Typical compression ratios

3/4color subsampling 4:2:0

4/5Quantization = 8 bits

3/4horizontal subsampling

50-200 Mbps

35-50 Mbps

1/5-1/20

1/20-1/30

Intra-frame compression vs

Interframe compression

500 Mbps1/2Lossless encoding

1,5 Gbps

(net " 1 Gbps)

1HD-SDI (4:2:2 subsampling)

Video bit rateFactorCoding technique

Reference = HD-SDI 720p50/1080i25 (10 bit)


Mainstream HD productionplatform – codec dilemmas

horizontal subsampling?

color subsampling: 4:2:2 or 4:2:0?

8 or 10 bit quantization?

intra-frame or interframe coding?

scalable or single-layer compression?

single or multiple compression formats?

required bit rate?

720p or 1080i?


Legacy HD production formats

Inter4:2:0; 8 bit

Horizontal

subsampling

20/25HDV

Inter4:2:0; 8 bit

Horizontal

subsampling

35XDCAM HD 35

Intra4:2:2; 8 bit

Horizontal

subsampling

100DVCProHD

Intra4:2:2; 8 bit

Horizontal

subsampling

140 (*)HDCAM

Intra4:2:2 or 4:4:4

10 bit

440HDCAM SR

Intra/interframe

coding

Subsampling

Quantization

Bit rate (Mbps)Compression

algorithm

(*): net video rate = 117Mbps


Advanced HD compression formats formainstream production

XDCAM HD

MPEG-2 Long-GOP

4:2:2, 8 Bit, data rate 50 Mbps, VBR

P2 HD

AVC Intra

4:2:2/4:2:0, 10/8 Bit, data rate 100 / 50 Mbps, CBR

EditcamHD

DNxHD

4:2:2, 8 Bit, Data rate 120 Mbit/s (185 Mbps), CBR

Infinity

JPEG2000

4:2:2, 10 Bit, Data rate 100 / 75 / 50

Mbit/s

Camera formats (optimized for storage and power

consumption)


Advanced HD compression formats formainstream production

post-production formats (optimized for editing)

185 / 120Net video data rate[Mbit/s]

4:2:2Colour sub-sampling

8 / 10 BitQuantisation [Bit]

SMPTE VC-3Standard

185 / 120Net video data rate[Mbit/s]

4:2:2Colour sub-sampling

8 / 10 BitQuantisation [Bit]

---Standard

DNxHD ProRes"

“intermediate codecs”


MPEG-2 long GOP

aka XDCAM HD422

advantages

! proven and optimized codec

! same bit rate as for Standard Definition (50Mbps)

! in general: good quality for first generation

! early support for end-to-end broadcast chain

disadvantages

! interframe is less suited for editing purposes

– smart rendering techniques necessary

– fade-to-black, dissolve effects

– artefacts affect multiple frames

! quality loss in multi-generation scenarios

successor = H.264 long GOP?


H.264 - AVC Intra

H.264 = MPEG-4 AVC = MPEG-4 Part 10

! joint project of ITU and MPEG

addition of High Profiles for HD resolution

! FRExt (Fidelity Range Extension)

– higher bit depths (10), 4:2:2/4:4:4 colour formats

! High 10 Intra (50 Mbps) and High 4:2:2 Intra (100 Mbps)

advanced compression tools

! Intra prediction: use of correlation between adjacent pixels

! entropy encoding improvement

– CABAC: Context Adaptive Binary Arithmetic Code

point of attention

! higher computational complexity


JPEG 2000

available bit rates: 50/75/100 Mbps

advantages

! scalable codec: no need for proxy transcoding

! wavelet-based coding: no blocking artefacts

! industry standard for Digital Cinema

! low latency

! license free

disadvantages

! not widely supported in end-to-end broadcast chain at thismoment


Dirac codec family

developed by BBC R&D (2000-now)

! license-free, open source/well documented

wavelet-based codec

wide range of applications

! Dirac (long GOP): low bit rate distribution (mobile, Web)

! Dirac Pro (intra-frame): low latency, high-quality production(HD, DC, Super Hi-Vision)

– SMPTE VC-2

– transport mapping to SDI (720p50/1080i25) and HD-SDI(1080p50)

! high flexibility: lossy/lossless, variable bit depths and coloursample schemes, …

not yet supported, less mature than other HD codecs


Scalable video coding (SVC)

extension of H.264

! recently standardized (2007)

! base layer (H.264) + enhancement layers

scalability options

! spatial, temporal, SNR/quality, combined

possible applications

! alternative for simulcast: SD/HD

! HD TV distribution for 720p@50fps or 1080i@25fps with1080p@50fps enhancement

! broadcast production for different resolutions

targeted profiles for broadcasting


Choice of HD compressionformat

key parameters

! quality (desired, floor level)

! storage and bandwidth requirements

! workflow

– recording medium

– editing requirements: layers, editing easiness, …

– support by different systems in HD production architecture

– end-to-end performance (transcoding, rendering, …)

! when?


Mainstream HD productionplatform – codec dilemmas

horizontal subsampling?

color subsampling: 4:2:2 or 4:2:0?

8 or 10 bit quantization?

intra-frame or interframe coding?

scalable or single-layer compression?

single or multiple compression formats?

required bit rate?

720p or 1080i?

no

4:2:2

8/10

50/100 Mbps

media file formats for broadcasters

Technology

hd file formats

media file formats

onderzoek en innovatie

new formats

compression formats

hd production

multiple compression

1080p50 hd compression