multimedia transport and delivery - institut mines-t©l©com

Institut Mines-Télécom

GPAC Team

Seminaire RT4

Cyril Concolato

08/04/2013


People

+ PhD Students, Post-Docs, S/W Engineers

• N. Bouzakaria, R. Bouazizi, V. Gros, M. Sarkis

• J. Gorin, A. Shafiei, V. Nguyen

• & many former students/engineers

Part of the TPT / TSI / Multimedia Group

2

C. Concolato J.-C. Dufourd J. Le Feuvre J.-C. Moissinac


Activities

Current Research Themes

• Multimedia Adaptation and Multimodal Interfaces

• Multimedia Transport and Delivery

• Multimedia Services and Transmedia

Teaching

• Multimedia Courses (Master, Mastère, Lifelong

learning)

Our specificities

• Open Source Software: GPAC

• Standardization: MPEG, W3C, …

3


Current Research Projects

ANR HybRadio

FUI Austral

FUI 4Ever

FUI OptisaT2

FSN M44G

ANR ILOT

EU COLTRAM

EU KIC ICT ANSFM

Industrial partnerships

4


Multimedia Adaptation and Multimodal

Interfaces

Problems • Adapt the multimedia consumption to the user context

(preferences, network, device constraints)

• Find representations and architectures for adaptations

Latest work (in progress) • Usage of semantic information for adaptation and content

generation

• V. Gros’s PhD – “Digital Book Publishing”

Some other results • W3C recommendation: "Multimodal Architecture and

Interfaces"

• Multimodality: B. Rodriguez’s PhD.

• Adaptation: B. Pellan’s PhD

5



Interfaces: Ontologies and Semantic

- Interactions Description

- Ontologies to describe

- How/Where/Which device

you interact with,

- Tight relationship with

W3C MMI group

- Description of

multimedia adaptation

engines

6



Interfaces: Adaptation Engine Ontologies

WSML

example

Displayed with

Web Services

Modeling

Toolkit

Now:

RDF+OWL+OWL-S

7


Multimedia Transport and Delivery:

Context

Internet Video Growth

• Cisco’s forecast

• Netflix Example: 30% of US traffic during peak hours

Mobile Video

• >= 50% of traffic

8



Requirements

Basic constraints are satisfied

• Audio/Video Synchronization

• Handles Loss and Corruption

• Fast start-up

• Limited rebuffering

• Compatible with Live, VoD, trick modes, …

Additional constraints

• Adaptive to varying bandwidth conditions (seamlessly,

granularity)

• IP-compatibility (proxies, firewalls, web servers,

caches)

• Efficiency

9



« HTTP Streaming » Principles

Use a playlist

• Continuously download small media files via HTTP

─ Selection btw. alternative files based on bandwidth

─ Constant duration files, cacheable

Properties

• Time continuity btw. files across quality to guarantee seamless playback and switching

• Compatible with live and VoD

Some Limitations

• Latency for live events

• Some overhead (e.g. MPEG-2 TS / HTTP)

Proprietary Solutions: Apple, Microsoft, Adobe

Standards: MPEG, 3GP, OIPF, W3C, …

10



HTTP Streaming Research Aspects

Adaptation strategies and Quality of Experience

Cache optimizations

Energy efficient streaming (mobile)

Low latency live streaming

Scalable media streaming

Hybrid Broadcast Broadband

Multipath streaming

…

PhD Student: N. Bouzakaria

11



Application Integration

How to repurpose broadcast data?

How to design HTML5 streaming-aware applications?

How to deliver new media data (HEVC, subtitling …)?

How to synchronize broadcast/broadband events?

Audio

Data carousel (500 ms)

Live data

Data carousel (2 s)

Concolato et al., Usages of DASH

for rich media services. MMSys

'11.

DOI=10.1145/1943552.1943587

12


Multimedia Services and Transmedia:

Research Topics

New types of services

• Interactive Multimedia

─ Digital TV (HBBTV, …)

─ Digital Radio

─ Web (HTML5, SVG)

• Stereoscopic Displays

• Second Screens

13



Second Screens

User Interfaces are evolving

• Integrate advanced rich media

─ 2D/3D, animations, video/audio clips

• Possible to have a common

user interfaces on many devices

─ TV, Phone, Desktop, Web page

Designed around

dedicated small applications

Widgets

14



Transmedia & Widgets

1) Discovery

3) UPnP Control

messages

widget

2) Download

& activation

4) UPnP Notification

messages

Widget Web Server

DLNA Blu-Ray Player

DLNA TV

Internet

Home Network

4) Data exchange

16



Our Approach

MPEG-U (Rich Media User Interface)

• Lead by Telecom ParisTech

─ Collaboration with Samsung R&D HQ

• MPEG Media Type

• Simplified representation

• Widget Communication

COLTRAM EU Project

• Collaboration with Fraunhofer FOKUS

• Use of Web Technologies

─ PhD Student: R. Bouazizi

• Authoring of Migratable Web Apps

─ PhD Student: M. Sarkis

17

Communicating and migratable interactive

multimedia documents, MTAP 2011


Standardization activities

MPEG

• Since 1996

• Mainly MPEG Systems activities and standards

─ Interactivity: MPEG-4 BIFS, MPEG-4 LASeR

─ File Format: MP4, AVC, SVC, HEVC

─ Streaming: MPEG-2 TS, MPEG DASH

• Editors of several standards (+ conformance

sequences, reference software, utility software)

W3C

• Since 2004

• Working Groups: SVG, HTML, MMI, WebApps, …

18


The GPAC Open Source Project

GPAC (http://gpac.wp.institut-telecom.fr)

• Multimedia player, encoder, and delivery tools

• Distributed Open Source under LGPL License

Goals

• Platform for research and academic purposes

• Dissemination of multimedia standards

Development Context

• Within our research activities

• In sync with industry standards (MPEG, W3C, DVB, ...)

• Using academic resources (student projects,

internships)

19

http://gpac.wp.institut-telecom.fr/




The GPAC Open Source Project:

Highlights

13 years-old project!

~ 500 000 lines of code

• Cross-platform, ~ANSI C

• CPU architectures:

─ x86, ppc, arm, arc, sh4 / 32-/64-bits

• OS:

─ Windows (desktop and mobile), Linux, Mac OS X, various UNIXes (BeOS, Solaris, …), Symbian, iOS, Android.

Good visibility/community

• 5000 downloads per month

• Research community

─ 2nd Best OSS @ ACM Multimedia 2007

• End-User community

─ Mostly uses MP4Box

20



MP4Box

Swiss-army knife for the ISO Base Media File Format and derivatives: • MP4,3GP,OMA,MJ2K,MP21, PSP...

• Major MP4 packager on the web

Media track manipulations • Importers and exporters

─ audio, video, subtitling in different formats

• Grouping, disabling, tagging ...

Delivery preparation: offline/live • Interleaving for HTTP download

• RTP Hinting for 3GP/OMA streaming servers

• DASH (segmentation, MPD generation)

• MPEG-2 TS generation

Encryption/Decryption

File analysis

Scene Description encoding

21



Basic Playback Features

Classic Audio/Video player • Similar to Quicktime, VLC

• Integrated with all browsers (IE, Safari*, Chrome, Firefox, Opera)

• Available with different UI: command-line, GUI (Windows, SDL, Qt, ...)

Various media codecs • Including HEVC, H.264|AVC, SVC, AAC …

• Based on third party libraries

Various network stacks • Web protocols: HTTP, IceCast, DASH

• Streaming protocols: RTP, RTSP

• File Formats: MP4, MPEG-2 TS...

• Home Networking protocols: UPnP/DLNA, ...

Extensible through plug-in architecture

22



Advanced Playback Features

Enhanced Multimedia Player • Very similar to a Web Browser

• Support for several document formats

Synthetic Graphics • 2D: SVG, MPEG-4 BIFS, Flash

• 3D: VRML, X3D

• With some acceleration (OpenGL / OpenGL ES / TinyGL)

Advanced Composition • Mixing of languages

• Animation & Synchronization

• Composition & Synchronization of multiple media elements

Interactivity • Support for JavaScript, including AJAX

23


Merci!

24

multimedia transport and delivery - institut mines-t©l©com

Documents