iee 5037 multimedia communications lecture 15:...

IEE 5037 Multimedia CommunicationsLecture 15: MPEG System and Digital Right Management: from MPEG-2 to

MPEG-7 and MPEG-21

Dept. Electronics Engineering,N

ational Chiao T

ung University Adapted from Prof. Hang’s slides and many other’s slides

1

Institute of Electronics,National C

hiao Tung U

niversity

Overview: Beyond the Compression

• System level problems– How to packet and composite these A/V contents?

• Buffer management• A/V synchronization• Program information• A/V composition• Defined in MPEG-2 system and MPEG-4 system

– How to management these A/V contents• Content management: search by metadata descriptions

– MPEG-7• Digit right management: framework for delivery and

consumption– MPEG-21

2


hiao Tung U

niversity

MPEG-2 System Standard

• MPEG Standards: System, Video, and Audio• MPEG-2 system applications -- Broadcast digital

TV, Cable digital TV, Interactive TV, etc.• MPEG-2 System specifications:

– Provide timing information for buffer management, audio/video synchronization, ...

– Multiplex audio, video, data and control bit streams

– Support access control (security), program specific information, digital storage control,

3


hiao Tung U

niversity

MPEG-2 System Model

Pack /

Mux

Videoencoding Buffer

Audioencoding Buffer

VideodecodingBuffer

ChannelUnpack /

DemuxAudio

decodingBuffer

Video

in

Audio

in

Buffer Buffer

Video

out

Audio

out

Data

outData

in

Elementary Stream (ES): One type of data, e.g., coded video, coded audio, etc.Packetized Elementary Stream (PES): With stream_id and PES header --- different headers for different contents (e.g., A/V: PES_priority, DSM_trick_play, PTS …)

4


hiao Tung U

niversity

5


hiao Tung U

niversity

MPEG-4 Systems Goals

• AVO (Media Object) -- object-based representation of audiovisual content.

• Describe the composition of the AVOs to create compound objects that form a Scene.

• Multiplex and synchronize AVO streams for transportation over network with QoS.

• Interact with the Scene generated at the receiver end.

6


hiao Tung U

niversity

Architecture of MPEG-4

MPEG-4 Systems

DMIF

Visual AudioOther

Visual / AudioStandards

7


hiao Tung U

niversity

Architecture of MPEG-4

DAI

ESI

Delivery aware, media unaware :14496-6 DMIF14496-1 FlexMux

Media aware, delivery unaware :14496-2/3 Audio and Visual14496-1 Scene Description and Object Descriptor Protocol

Media and Delivery unaware :14496-1 Systems, adaptation of the MPEG-2 traditional activities

DeliveryLayer

FlexMuxTool

Sync.Layer

CompressionLayer

Composition

8


hiao Tung U

niversity

MPEG-4 DMIF

• DMIF = “Delivery Multimedia Integration Framework”

D M I FThe multimedia content delivery integration framework

Cable,Satellite,

etc.

Broadcast Technology

Internet, ATM,

etc.

InteractiveNetwork Technology

CD,DVD,etc.

Storage Technology

9


hiao Tung U

niversity

Terminologies

• Access Unit– -- the smallest data entities, e.g., video frame

• Elementary Stream– -- composed by access units

• SL-Packetized Stream (SPS)– -- Synchronization Layer Packetized Stream

• FlexMux Stream– -- composed by one or more SPS

10


hiao Tung U

niversity

MPEG-4 DMIF (cont.)

• DAI = DMIF Application Interface• DNI = DMIF Network Interface

SL SL SL

TransMux Layer

FlexMux

TransMux Streams

FlexMux Channel

TransMux Channel FlexMux Streams DMIF Network Interface

DMIF Application Interface

Elementary Stream Interface

SL-Packetized Streams

Elementary Streams

FlexMux

Sync Layer

DMIF Layer

SL SLSL

FlexMux

SL

(RTP)UDP

IP

(PES)MPEG2

TS

AAL2ATM

H223PSTN

....

....

....DABMux

File Broad-cast

Inter-active (not specified in MPEG-4)

Del

iver

y La

yer

11


hiao Tung U

niversity

MPEG-4 Terminal

End user PresentationEnd user Presentation

Composition & Rendering

Composition & Rendering

Hierarchical, Interactive, Audiovisual Scene

Network

Layer

Network

Layer

DecompressionDecompression

Primitive AV Objects

...

DemultiplexDemultiplex

...

Elementary Stream

Scene Description

Scene Description

Upstream DataUser Events, Class Request, ...

12


hiao Tung U

niversity

Scene Description -- BIFS

• Coded scene description in a compact binary form Called “Binary Format for Scene -- BIFS”

• BIFS-Command – -- To modify a set of scene properties at a given time

instance.• BIFS-Animation

– -- To continuously update certain fields in the scene graph.

13


hiao Tung U

niversity

Application Example

-- Ping-pong Game• Green Bar and Blue Bar controlled

by Player1 & Player2 with keyboard• We can change the speed of the

ball (square)• The ball contains an H.263 video

sequence• As ball moves, we sendEventIn to

update the position in BIFS

14


hiao Tung U

niversity

15


hiao Tung U

niversity

Motivation of MPEG-7 and MPEG-21

16


hiao Tung U

niversity

MPEG7

• MPEG7 為因應日漸繁多的多媒體資訊，期望能夠快速且有效的檢索出用戶所需的不同類型的多媒體資料。

• 於1998年10月提出，於2001年公佈。• MPEG7 可針對各種不同類型的多媒體資訊進行標準化的描述，並將該描述與所描述的內容相聯繫，以實現快速有效的檢索。

17


hiao Tung U

niversity

MPEG-7 (ISO/IEC 15938)

• Content Management– Increasing availability of multimedia content– Finding, selecting, filtering desired material difficult– Desire for information about the content

• Objective– Standardized content-based descriptions for multiple types of

audio-visual information– Enable rapid and efficient content identification– Suitability for large range of applications

• MPEG-1, -2 and -4 represent the content “the bits”

• MPEG-7 represents information about the content“the bits about the bits”

18


hiao Tung U

niversity

Multimedia Semantic Framework

19


hiao Tung U

niversity

Multimedia Semantic Framework

20


hiao Tung U

niversity

Towards Knowledge Management andTransaction Enrichment for Digital Media

21


hiao Tung U

niversity

MPEG-7 OverviewXML Metadata for Multimedia Content Description

22


hiao Tung U

niversity

MPEG-7 Standard Scope

23


hiao Tung U

niversity

Annotation Example

24


hiao Tung U

niversity

Example: Semantic Concept Detection

25


hiao Tung U

niversity

Example: Personalization and Summarization

26


hiao Tung U

niversity

27


hiao Tung U

niversity

MPEG21

• 多媒體版權的管理• 定義各種可供傳播與交易的數位元件基本單位（Digital Item），並且讓各種使用者（包括製作、傳播與消費者）擁有同樣取得數位元件的權利。MPEG組織希望MPEG 21能夠有效釐清數位多媒體專利權的歸屬議題，未來的使用者也將有機會在任何時間、地點、設備上（Any time，any place，any device）取得數位多媒體資訊。

28


hiao Tung U

niversity

MPEG-21 (ISO/IEC 21000)

• What?– Multimedia Framework for multimedia delivery and consumption– Content creator and content consumer as focal points

• Why?– Many elements (standards) exist for delivery and consumption of

multimedia contents– Absence of 'big picture” to describe how elements relate to each

other– Increase interoperability to allow existing components to be used

together by filling gaps• Why now?

– HW building blocks and infrastructure in place– Compression, transmission, description standards are ready

29


hiao Tung U

niversity

30


hiao Tung U

niversity

MPEG-21 Standards

31


hiao Tung U

niversity

32


hiao Tung U

niversity

AuthorAuthorAgentAgent

PublisherPublisherAggregatorAggregator

DistributorDistributorWholesalerWholesaler eTailer ConsumerConsumer

Authoring Tools Publishing S/WAsset ManagementContent PackagingContent Hosting

WWW. Store FrontRights Clearing User Interface

Digital Rights

Meta Data

Digital Content Products & ServicesDigital Content Products & Services

Digital IdentificationDigital Identification

Digital Supply Chain

33


hiao Tung U

niversity

34


hiao Tung U

niversity

Multiple Flows and Transactions

• Between any two points there are flows:• Information about content• Authentication between peer entities• Content• Content usage rights• Technology usage rights• Delivery usage rights• Money

35


hiao Tung U

niversity

36


hiao Tung U

niversity

Current Practice

• Today’s media contain implicit or explicit rules– A book can be read and resold– A CD may be played, but may not be copied– A video cassette is for private consumption, can be

rented but may not be copied– A public broadcast can be watched because license fee

has been paid– A commercial broadcast can be watched because one

undertakes to watch commercials– A pay TV broadcast can be watched because one has

paid a monthly subscription

37


hiao Tung U

niversity

Future Practice?

• The Multimedia Framework gives unlimited flexibility:

• One can buy an MPEG-21 Digital Item and:– Copy it to a portable device once only– Rent it for 24 hours– Rent it for 10 playbacks– Distribute it to 10 friends and get a copy for free– Access it for free at a low quality, and for a fee at high

quality– ??????

38


hiao Tung U

niversity

MPEG-21 Objectives

• Vision– To define a multimedia framework to enable transparent use of

multimedia resources across a wide range of networks and devices used by different communities

• Purpose– Enable electronic creation, delivery, trade of digital multimedia

content• Goals

– Provide access to information and services from almost anywhere at anytime with ubiquitous terminals and networks

– Identify, describe, manage, and protect multimedia content to support delivery chain of content creation, production, delivery, and consumption

39


hiao Tung U

niversity

Fundamental Concept: The Digital Item

• A structured digital object with a standard representation, identification and meta-data

• The fundamental unit of distribution and transaction in the MPEG-21 framework

• Digital Item = resource + metadata + structure• Resource: individual asset, e.g., MPEG-2 video• Metadata: descriptive information, e.g., MPEG-7• Structure: relationships among parts of the item

40


hiao Tung U

niversity

Resources

MPEG-1

MPEG-2

MPEG-4

MPEG-7

Metadata New Metadata& Resource

Forms

Structure

MPEG-21

Digital Item

41


hiao Tung U

niversity

Benefits of Digital Item

• Tangibility– content is more than files on a disk

• Deliverability– more automated, less end-user involvement

• Configurability– express options/augmentations for specific users,

groups, locales, prices, etc.

42


hiao Tung U

niversity

ISO/IEC 21000-1Technical Report (TR)

• Published Sept 2001• Outlines objectives and vision of 7 elements for

the MPEG-21 – Digital Item Declaration– Digital Item Representation– Digital Item Identification and Description– Digital Item Management and Usage– Intellectual Property Management and Protection– Terminals and Networks– Event Reporting

43


hiao Tung U

niversity

Digital Item Declaration ISO/IEC 21000-2

• Reached FDIS in May 2002• Purpose & Conceptual Model

– Declare a Digital Item’s makeup and structure– Composite whole composed of various parts

• Syntactic Representation – XML-based Digital Item Declaration Language (DIDL)

• Features– Hierarchical, generic structure– Flexible meta-data expression– Reusable and configurable elements

FDIS Final Draft Information Standard

44


hiao Tung U

niversity

Digital Item Identification ISO/IEC 21000-3

• Provides means to uniquely identify and describe Digital Items

• Defines the relationship between Digital Items and existing ID systems – This part does not specify any new ID systems

• Identification– syntax: URI of the form

urn:mpeg:mpeg21:diid:sss:nnn• sss denotes the identification system • nnn denotes a unique identifier within that identification system

FCD – Final Committee Document

45


hiao Tung U

niversity

IP Management & ProtectionISO/IEC 21000-4

• Multimedia Digital Rights Management (DRM) – Enables users to express their rights, interest, and

agreements related to Digital Items (DI’s)– Enables users to derive appropriate levels of assurance

that those rights, interests and agreements will be persistently and reliably managed and protectedacross a wide range of networks and devices

46


hiao Tung U

niversity

Intellectual Property Management & Protection (IPMP)

• Management and protection – MPEG-4 audio-visual objects– MPEG-7 descriptors and description schemes – Other DI types including personal data and rights to its

use• Three parts to the standard underway:

– IPMP Framework– Rights Expression Language (REL)– Rights Data Dictionary (RDD)

47


hiao Tung U

niversity

Rights Expression Language ISO/IEC 21000-5

• Targets for Standardization– Machine-readable language

• Declare rights and permissions using the terms as defined in the Rights Data Dictionary

– RDD-REL are intended to provide flexible, interoperablemechanisms to support transparent and augmented use of digital resources

– REL supports guaranteed end-to-end interoperability, consistencyand reliability between different systems and services

CD Committee Draft

48


hiao Tung U

niversity

Rights Data Dictionary ISO/IEC 21000-6

• Targets for Standardization– Model– Core set of terms– The relation between the RDD and the core set of

descriptive terms in the REL

49


hiao Tung U

niversity

Digital Item Adaptation ISO/IEC 21000-7• Targets for standardization

– Usage Environment Description Tools– Resource Adaptability Tools– Systems Related Tools

50


hiao Tung U

niversity

51


hiao Tung U

niversity

52


hiao Tung U

niversity

53


hiao Tung U

niversity

54


hiao Tung U

niversity

55


hiao Tung U

niversity

iee 5037 multimedia communications lecture 15:...

Documents