3d video: from stereo to multi-view
TRANSCRIPT
![Page 1: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/1.jpg)
Fraunhofer Institute for TelecommunicationsHeinrich-Hertz-Institut Berlin Einsteinufer 3710587 BerlinGermany
3D Video: From Stereo to Multi-View
+49 30 310 02 – [email protected] http://www.hhi.fraunhofer.de
Karsten Müller
TEWI - Kolloquium,Universität Klagenfurt
![Page 2: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/2.jpg)
Kars
ten
Mül
ler
Overview
• Introduction, scenarios, requirements
• 3D Video: Stereo Video + MVC
• 3D Video: 3D Video Formats & Coding
• International 3D media research and development
• Summary
18.03.2010 Universität Klagenfurt
Slide 2
![Page 3: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/3.jpg)
Kars
ten
Mül
ler
3D Video Introduction
• Provision of left and right eye view• Added depth sensation • Resolved 2D viewing
ambiguity• Additional features
(e.g. free viewpoint, depth-controlled object insertion)
3D Video is more than adding 1 dimension to 2D!
Slide 3
18.03.2010 Universität Klagenfurt
![Page 4: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/4.jpg)
3D Video Scenarios3D cinema• Increasing number of 3D productions• First studios start to release all productions also in 3D• Technology: stereoscopic 3D, glasses based
3D home entertainment• Different types of displays available: stereoscopic, auto-stereoscopic
with 2 … N views• Various technologies, input formats and display sizes• Glasses based systems may not be acceptable
3D mobile• auto-stereoscopic 2 view display with fixed viewing position• Good 3D viewing in spite of small display sizes
Kars
ten
Mül
ler
Slide 4
18.03.2010 Universität Klagenfurt
![Page 5: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/5.jpg)
18.03.2010© Fraunhofer HHI
Folie 5
Kars
ten
Mül
ler
Stereo Video Solution
• Original Cameras = Views for Display
• Production format specification (e.g. SMPTE)
• Rectification, color correction, format conversion
• Conventional Stereo Format, e.g. Side-by-side
• Multi-view video coding (MPEG-4 MVC/H.264)
• High-resolution glasses-based stereo display
Stereo-Capture
Post-processing
MVC-Coding
MVC-Decoding
ProductionFormat
TransportFormat
Transport Format
Stereo-Display
Display Format
![Page 6: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/6.jpg)
3D format: Conventional Stereo video (CSV)Ka
rste
n M
ülle
r
Slide 6
18.03.2010 Universität Klagenfurt
Original Image Source: Interactive Visual Media Groupof Microsoft Research
![Page 7: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/7.jpg)
Kars
ten
Mül
ler
Current Coding Standards for Stereo Video
• Advanced Video Coding (MPEG-4 AVC / H.264)– Simulcast– Stereo SEI (Supplementary Enhancement Information)– Auxiliary Picture Syntax
• MPEG-C part 3 (container format for V+D, e.g. with AVC coding)
• Multi-view Video Coding (MPEG-4 MVC / H.264)
Slide 7
18.03.2010 Universität Klagenfurt
![Page 8: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/8.jpg)
Kars
ten
Mül
ler
Multi-view Video Coding
• Joint Coding of camera views with similar content
Slide 8
18.03.2010 Universität Klagenfurt
![Page 9: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/9.jpg)
Kars
ten
Mül
ler
MVC Coding Method
• Coding approach for exploiting temporal and inter-view dependencies using MPEG4-AVC/H.264
• Usage of hierarchical B pictures in temporal direction
• Usage of P or hierarchical B pictures in inter-view direction
• Frame reordering to optimize memory usage
No H.264 changes required for multi-view coding
• Standardized in July 2008• Adopted for 3D Blu Ray in Dec. 2009
Slide 9
18.03.2010 Universität Klagenfurt
![Page 10: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/10.jpg)
Kars
ten
Mül
ler
MVC Coding Structure- Picture Reordering
Slide 10
18.03.2010 Universität Klagenfurt
![Page 11: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/11.jpg)
Kars
ten
Mül
ler
MVC Coding Structure
Slide 11
18.03.2010 Universität Klagenfurt
![Page 12: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/12.jpg)
Kars
ten
Mül
ler
MVC Camera Setting Adjustment - Star
Slide 12
18.03.2010 Universität Klagenfurt
![Page 13: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/13.jpg)
Kars
ten
Mül
ler
MVC Camera Setting Adjustment – 2D Array
Slide 13
18.03.2010 Universität Klagenfurt
![Page 14: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/14.jpg)
Kars
ten
Mül
ler
MVC Coding Results
• Coding structure allows AVC coder, to select the best inter-view-temporal neighbors within the MVC sequence
• Coding gain obtained by usage of hierarchical B-frames and by exploitation of inter-view dependencies
Slide 14
18.03.2010 Universität Klagenfurt
![Page 15: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/15.jpg)
Kars
ten
Mül
ler
MVC Coding Results – delta_PSNR
Slide 15
18.03.2010 Universität Klagenfurt
![Page 16: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/16.jpg)
Broadcasters currently concentrate on stereo and will not invest in new infrastructuresThey will adopt a system, which they can feed through existing pipes: e.g. anaglyph or multiplexed stereo:
checkerboardside-by-sideabove-below etc.
Will Stereo TV be the long term solution?
anaglyph polarized shutter
Kars
ten
Mül
ler
Slide 16
18.03.2010 Universität Klagenfurt
Short Term 3DTV will be Stereo TV
![Page 17: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/17.jpg)
The Answer is Given by Our ChancellorKa
rste
n M
ülle
r
Slide 17
18.03.2010 Universität Klagenfurt
![Page 18: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/18.jpg)
3DTV viewing conditions are more complex than in 3D cinema:• Glasses are generally not accepted in the living room • There are many different 3D display technologies • Varying viewing conditions (distance, display size)• User preferences have to be considered• Not all users can see stereo (≈10%)• Not all users want to see stereo• Users might not want to see all programmes in stereo
Kars
ten
Mül
ler
Slide 18
18.03.2010 Universität Klagenfurt
Special Requirements for 3DTV
![Page 19: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/19.jpg)
Kars
ten
Mül
ler
Multi-view Displays
• The most common form of multi-view displays utilise either lenticularscreens or parallax barriers.
• Lenticular screens consist of a series of vertically aligned or slanted cylindrical lenses and parallax barriers vertically aligned apertures.
• Light is guided in the appropriate directions by either focussing it or by blocking unwanted rays.
VIEW 1 VIEW 2 VIEW 3
LENTICULAR
VIEW 1 VIEW 2 VIEW 3
PARALLAX BARRIER23 1
1 4 44 11
SCREENSlide 19
18.03.2010 Image Source: De Montfort University, UK
![Page 20: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/20.jpg)
Kars
ten
Mül
ler
3D Video Color only: Assumption 1
• Multi-view Displays require N views (e.g. 5,9,22,50)• Why not transmitting N color views with MVC?
Slide 20
18.03.2010 Universität Klagenfurt
![Page 21: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/21.jpg)
Kars
ten
Mül
ler
MVV and MVC Restrictions
• Pure coding of multi-view video
• Still linear dependency of bit rate from number of views
• Number and position of views fixed over the whole 3D chain
• No baseline adaptation for different displays
• Number of Views fixed in MVV/MVC
Slide 21
18.03.2010 Universität Klagenfurt
![Page 22: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/22.jpg)
Kars
ten
Mül
ler
MVC Restrictions
• Coding experiments on camera density
Original camera distance
Slide 22
18.03.2010 Universität Klagenfurt
![Page 23: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/23.jpg)
Kars
ten
Mül
ler
MVC Restrictions (2)
• Coding experiments on camera density with Rena test sequence
average rate average rate per cameraSlide 23
18.03.2010 Universität Klagenfurt
![Page 24: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/24.jpg)
Kars
ten
Mül
ler
3D Video Color only: Assumption 2
• Multi-view Displays require N views (e.g. 5,9,22,50)• Why not transmitting 2 or 3 color views with MVC
and reconstructing N views at the receiver?
Slide 24
18.03.2010 Universität Klagenfurt
![Page 25: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/25.jpg)
Color Only Intermediate ViewKa
rste
n M
ülle
r
Slide 25
18.03.2010
Original Image Source: Interactive Visual Media Group of Microsoft Research
![Page 26: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/26.jpg)
Depth mapKa
rste
n M
ülle
r
Slide 26
Color information 256 depth layers
18.03.2010 Universität Klagenfurt
Original Image Source: Interactive Visual Media Groupof Microsoft Research
![Page 27: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/27.jpg)
Example Reconstruction
Pure image blending Depth image basedrendering
Original Image Source: Interactive Visual Media Groupof Microsoft Research
Kars
ten
Mül
ler
Slide 27
18.03.2010 Universität Klagenfurt
![Page 28: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/28.jpg)
Kars
ten
Mül
ler
Advanced 3D Video Solutions Required
• Color-only multi-view video too restricted– High number of views cannot be efficiently
compressed and transmitted– View synthesis of new views at the receiver
side from color-only data gives badreconstruction quality
• Additional Scene Geometry information required, e.g. per-pixel depth/disparity data
18.03.2010 Universität Klagenfurt
Slide 28
![Page 29: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/29.jpg)
18.03.2010© Fraunhofer HHI
Folie 29
Kars
ten
Mül
ler
Depth-enhanced 3D Video
• Stereo / multiview capturing, depth provision
• Production format specification (e.g. SMPTE)
• Rectification, color correction, format conversion
• 3D format specification
• 3D video coding (e.g. MPEG)
• High quality intermediate view synthesis (e.g. 2 view + 2 depth N view)
• High-resolution stereo/multiview display
Capture
Post-processing
Coding
View Synthesis
ProductionFormat
TransportFormat
Transport Format
Display
Display Format
![Page 30: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/30.jpg)
Kars
ten
Mül
ler
3D Video Formats
• Conventional stereo video (CSV)• Mixed resolution stereo (MRS)• Video plus depth (V+D)
• Multiview video (MVV)• Multiview video + depth (MVD)• Layered depth video (LDV)
• Depth-enhanced stereo (DES)
Slide 30
18.03.2010 Universität Klagenfurt
![Page 31: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/31.jpg)
assuming multiple views with color & depth… Ka
rste
n M
ülle
r
Slide 31
18.03.2010 Universität Klagenfurt
![Page 32: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/32.jpg)
Stereo video (CSV)
CSV
Kars
ten
Mül
ler
Slide 32
18.03.2010 Universität Klagenfurt
![Page 33: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/33.jpg)
Mixed resolution stereo (MRS)
MRS
Kars
ten
Mül
ler
Slide 33
18.03.2010 Universität Klagenfurt
![Page 34: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/34.jpg)
Video plus depth (V+D)
V+D
Kars
ten
Mül
ler
Slide 34
18.03.2010 Universität Klagenfurt
![Page 35: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/35.jpg)
MVV
Multiview video (MVV)Ka
rste
n M
ülle
r
Slide 35
18.03.2010 Universität Klagenfurt
![Page 36: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/36.jpg)
MVD
Multiview video plus depth (MVD)Ka
rste
n M
ülle
r
Slide 36
18.03.2010 Universität Klagenfurt
![Page 37: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/37.jpg)
LDV
Layered depth video (LDV)Ka
rste
n M
ülle
r
Slide 37
18.03.2010 Universität Klagenfurt
![Page 38: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/38.jpg)
DES
Depth-enhance Stereo (DES)Ka
rste
n M
ülle
r
Slide 38
18.03.2010 Universität Klagenfurt
![Page 39: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/39.jpg)
Kars
ten
Mül
ler
Towards a new 3D Video Coding Standard
• MPEG develops a new 3D video coding standard
• Motivation for new standard:– Decouple production from coding format– MPEG-4 AVC/H.264 only optimized for 2D color video, but not for
depth information
Slide 39
18.03.2010 Universität Klagenfurt
![Page 40: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/40.jpg)
Advanced 3DTV concept based on MVDKa
rste
n M
ülle
r
Slide 40
18.03.2010 Universität Klagenfurt
![Page 41: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/41.jpg)
3D components under consideration:– Transport format– 3D video coding methods– Intermediate view synthesis
18.03.2010© Fraunhofer HHI
Folie 41
Kars
ten
Mül
ler
Coverage of 3D Video Coding
Capture
Post-processing
Coding
View Synthesis
ProductionFormat
Transport Format
Transport Format
Display
Display Format
![Page 42: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/42.jpg)
Kars
ten
Mül
ler
Challenges for 3DVC
• Consider capturing technology, i.e. maximal 2-3 recorded views
• Break linear dependency of coding bit rate from number of target views (e.g. MVC)
• Provide scene geometry data in general form, i.e. pixel-wise depth data
• Consider statistical properties of depth (and supplementary) data
• Consider new quality evaluation methods for intermediate views
• Provide high-quality view synthesis for continuous viewing range
• Decrease depth and coding errors
Slide 42
18.03.2010 Universität Klagenfurt
![Page 43: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/43.jpg)
Kars
ten
Mül
ler
Coding Experiments MVD with MVC
• Joint color and depth coding for optimal intermediate view quality
• Evaluation of total bit rate (e.g. of 2 color and 2 depth maps) vs. quality of decoded synthesized views
• View synthesis is part of coding optimization
• Reduction of coding and interpolation artifacts through high-quality view synthesis
Slide 43
18.03.2010 Universität Klagenfurt
![Page 44: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/44.jpg)
Kars
ten
Mül
ler
Coding Results Intermediate View
• Ballet (JMVM 7.0.1, GOP16, κ=½)
Slide 44
18.03.2010 Universität Klagenfurt
![Page 45: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/45.jpg)
Kars
ten
Mül
ler
Coding Results Intermediate View
• Breakdancers (JMVM 7.0.1, GOP16, κ=½)
Slide 45
18.03.2010 Universität Klagenfurt
![Page 46: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/46.jpg)
Kars
ten
Mül
ler
Color Depth Bit Rate Distribution
• Objective and subjective examples for optimal and non-optimal bit rate distribution
• Variation of color and depth quality
• Tradeoff between synthesized views and views at original positions
Slide 46
18.03.2010 Universität Klagenfurt
![Page 47: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/47.jpg)
Kars
ten
Mül
ler
Coding Results Intermediate View
• Ballet (JMVM 7.0.1, GOP16, κ=½)
C30D30
C24D40
Slide 47
18.03.2010 Universität Klagenfurt
![Page 48: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/48.jpg)
Kars
ten
Mül
ler
Coding Results across Viewing Range
• Ballet (JMVM 7.0.1, GOP16)
Slide 48
18.03.2010 Universität Klagenfurt
![Page 49: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/49.jpg)
Kars
ten
Mül
ler
Coding Results Intermediate View
• Breakdancers (JMVM 7.0.1, GOP16, κ=½)
C30D36
C36D30
Slide 49
18.03.2010 Universität Klagenfurt
![Page 50: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/50.jpg)
Kars
ten
Mül
ler
Coding Results across Viewing Range
• Breakdancers (JMVM 7.0.1, GOP16)
Slide 50
18.03.2010 Universität Klagenfurt
![Page 51: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/51.jpg)
View Synthesis for MVD
– High quality view interpolationis essential for 3D video applications with MVD data.
– Processing steps:- Layer Extraction- Layer Projection- Hole Filling + Filtering
Kars
ten
Mül
ler
Slide 51
18.03.2010 Universität Klagenfurt
![Page 52: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/52.jpg)
Kars
ten
Mül
ler
Synthesis Improvements (Ballet)
compressed uncompressed
simple merging
layer-basedview synthesis
Slide 52
18.03.2010 Universität Klagenfurt
![Page 53: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/53.jpg)
International R&D in 3D Media
• ISO/IEC MPEG 3DVC (3D video coding)• SMPTE Task Force of 3D to the home (3D master format)• ITU-R SG6• DVB – TM - ES Stereoscopic (3D) TV• 3D@Home Consortium• Association in Korea, Japan,…• EU 3D, Immersive, Interactive Media Cluster
• International Conferences– 3DTV-CON– 3DPVT
Kars
ten
Mül
ler
Slide 53
18.03.2010 Universität Klagenfurt
![Page 54: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/54.jpg)
3D Video Coding Group in MPEG
• Nagoya Univ. • NTT• UPM• Philips• GIST• ETRI• JVC• Poznan Univ. of
Technology• Thomson• MERL• Peking Universtity• Tsinghua Univ.• Huawei• Xidian University• Telefonica
• Samsung• LG Electronics• Nokia• NICT• Kwangwoon Univ.• Sharp• Fraunhofer• NCTU/ITRI• Motorola• NXP• Ericsson• Logitech• Orange/France
Telecom • FUB• Sony
• Zhejiang Univ.• ASTRI• Qualcomm
• Coordination:• Karsten Müller (HHI)
Anthony Vetro (MERL)
Kars
ten
Mül
ler
Slide 54
18.03.2010 Universität Klagenfurt
![Page 55: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/55.jpg)
Kars
ten
Mül
ler
3D@Home Consortium
• Steering Team 1: Content Creation & ProductionObjective: To focus on projects which improve the quality of 3D content for the home audience.
• Steering Team 2: Content Storage, Transmission & DistributionObjective: To develop useful definitions and guidelines for the successful storage, transmission and distribution of 3D content.
• Steering Team 3: 3D PromotionObjective: To promote the adoption of 3D into the home by promoting 3D format and the 3D@Home Consortium.
• Steering Team 4: 3D DisplaysObjective: To identify issues around 3D Displays and associated hardware.
Slide 55
18.03.2010 Universität Klagenfurt
![Page 56: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/56.jpg)
Kars
ten
Mül
ler
3D@Home Consortium
Slide 56
18.03.2010 Universität Klagenfurt
![Page 57: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/57.jpg)
Kars
ten
Mül
ler
EU 3D, Immersive, Interactive Media Cluster• Currently, a number of EU projects develop parts and products of the 3D
media chain for certain application areas, e.g. 3D home entertainment or mobile services
• Enhanced success of these EC funded projects and subsequent associated commercialization efforts if simultaneous operations of individual projects can be further orchestrated towards a strong mutual presence
• Better highlight of Europe‘s good position in worldwide R&D
• 3D, Immersive, Interactive Media Cluster is an umbrella structure for the projects and provides international contact and information about project activities
• 3D Media, Immersive, Interactive Cluster is one of the main drivers towards "3D Media" in the context of the "Future Internet" activities supported by EU Commission (DG INFSO)
Slide 57
18.03.2010 Universität Klagenfurt
![Page 58: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/58.jpg)
Kars
ten
Mül
ler
3DII Media Cluster – Current Projects
• Leadership: Prof. Levent Onural and Dr. Karsten Müller
• Projects: Status: Framework Program• 3DTV ended 2008 FP6• 3DPHONE ongoing FP7• MOBILE3DTV ongoing FP7• 3D4YOU ongoing FP7• 2020 3D Media ongoing FP7• 3DPresence ongoing FP7• MUTED ended 2008 FP6• HELIUM3D ongoing FP7• Real3D ongoing FP7• i3DPost ongoing FP7• VICTORY ended 2009 FP7
3D for MobileSystems
3D for BroadcastSystems
3D DisplayTechnology
3D ContentDescription
Slide 58
3D Videoconference
18.03.2010 Universität Klagenfurt
![Page 59: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/59.jpg)
Kars
ten
Mül
ler
3DII Media Cluster – New Projects
• Projects: Status: Framework Program• DIOMEDES new 2010 FP7• MUSCADE new 2010 FP7• 3D VIVANT new 2010 FP7• BEAMING new 2010 FP7• FINE new 2010 FP7• SKYMEDIA new 2010 FP7• SALA3D new 2010 FP7• 3DLife new 2010 FP7• FascinatE new 2010 FP7
Slide 59
18.03.2010 Universität Klagenfurt
![Page 60: 3D Video: From Stereo to Multi-View](https://reader038.vdocuments.net/reader038/viewer/2022103016/554a0f86b4c9055c598b4a3a/html5/thumbnails/60.jpg)
Kars
ten
Mül
ler
Summary
• 3D technology is maturing due to world wide development from capturing to display
• First systems are in use, mainly using stereoscopic displays and MVC coding
• New challenging research topics in all areas of 3D video:– Reliable capturing devices with synchronized views– Improved time-consistent depth estimation– Geometry-enhanced multi-view coding (e.g. 3DVC)– Robust view synthesis– New high-resolution multi-view displays
(e.g. 50 views, each with HDTV resolution!)
Slide 60
18.03.2010 Universität Klagenfurt