The Future of TelepresenceAlex Eleftheriadis, alex@vidyo.com

2 IMTC 2010/06/15

What is Telepresence?

Wikipedia:Telepresence refers to a set of technologies which allow a person to feel as if they were present, to give the appearance that they were present, or to have an effect, at a location other than their true location.

Alternatively:Videoconferencing sucks! What can you do to make it better if you have all the money you need?

3 IMTC 2010/06/15

Lessons Learned Elsewhere

Computing

Networking

Communications

4 IMTC 2010/06/15

DEC VAX (1977, 0.5 MIPS)

VAX 11/780 (first two cabinet sections), shown with Unibus expansion cabinet (middle cabinet section), two tape drives, two RP05 or RP06 removable pack disk drives, a DECwriter printing terminal, and a VT52 CRT terminal

5 IMTC 2010/06/15

iPhone 3G (2008, 1250 MIPS)

6 IMTC 2010/06/15

Lessons Learned Elsewhere

ComputingFastPersonalPortable

NetworkingScalableUbiquitous

CommunicationsReliableUbiquitousMobile

7 IMTC 2010/06/15

Telepresence Today

Extremely expensive

Not scalable

Not portable

Not personal

Very high quality

=> Traditional videoconferencing: same, but sacrifices quality for low cost.

8 IMTC 2010/06/15

Quality of Experience

Cost

Type Desktop Executive Room Telepresence

$100’s $1000’s $10,000’s $100,000’s

Qua

lity

of E

xper

ienc

e

10 hours/month

100 hours/monthTelepresence quality

• At least SD quality per face

• Total delay under 250ms latency

Legacy video conferencing quality

9 IMTC 2010/06/15

Key Videoconferencing System Characteristics

Rate Matching Support for heterogeneous endpoints and access networks

Personalized Layout Each user selects the resolution and users he/she wants to see

Low Delay Interactivity requires <250msec end-to-end

Error Resilience Tolerance to packet loss rates >10%

Error Localization An error in one user should not affect other users

Complexity Lower complexity = Lower cost Low complexity allows to integrate with existing network services (e.g., in cheap routers)

=> How can we have all that, but with high quality and scalability?

10 IMTC 2010/06/15

H.264 SVC

Scalable Video Coding (SVC) Developed by JVT (=Joint Video Team of ITU and ISO)

SVC is Amendment 3 to H.264 AVC Most of SVC is in Annex G of H.264 AVC CFP April 2004, Consented in Nov 2007 RTP payload format nearly completed

Architecture centered on VidyoRouter™ (“VRU”)

11 IMTC 2010/06/15

Temporal Structure of Non-Scalable Codecs

12 IMTC 2010/06/15

Temporal Scalability

13 IMTC 2010/06/15

Spatial Scalability

MultiplexSVC

Bitstream

Prediction Coding

H.264/AVC-compatiblebase layer

Prediction Coding

PredictionCoding

Scale

Scale

14 IMTC 2010/06/15

Temporal + Spatial Scalability

15 IMTC 2010/06/15

Temporal + Spatial Scalability

QVGA30 fps

16 IMTC 2010/06/15

Temporal + Spatial Scalability

VGA15 fps

17 IMTC 2010/06/15

What does the VidyoRouter do?

1 Mbps

500 Kbps

150 Kbps

Vidy

oRou

ter™

High ResolutionHigh Frame Rate

Low ResolutionLow Frame Rate

2 Mbps

High ResolutionMedium Frame Rate

Medium ResolutionMedium Frame Rate

High ResolutionHigh Frame Rate

18 IMTC 2010/06/15

Traditional MCU Architecture

ENCODE

DECODE

ENCODE

DECODE

Quality Loss from Cascaded

Encodings

High CostHigh Delay

DECODE

ENCODE

COMPOSE

19 IMTC 2010/06/15

VidyoRouter™ vs. MCU

VidyoRouter is simply an application layer router Nearly zero-delay (~20 msec), allows interactive multi-point sessions Eliminates video quality loss due to transcoding Rate matching and personal layout are simple routing decisions Error localization and robustness up to 20% packet loss ratesComputing power that rides the Intel® curve

Endpoint processing of the video from different participants makes the VidyoRouter highly scalable Similar complexity to other network appliances

20 IMTC 2010/06/15

Enhancement Layer

Base Layer

Significant ImpactConventional Coding

Scalable Coding

Minor or No Impact

SVC+VR Eliminates Error Resilience Problem

21 IMTC 2010/06/15

Temporal Scalability

22 IMTC 2010/06/15

Concluding Remarks

Videoconferencing and telepresence will remain “small”, unless our technology enables scaling with high quality and low cost.Video coding scalability and the video router architecture allow:

Replacement of the MCU with a much simpler, scalable device – the Video Routing Unit (VRU)Very high error resilience (=high quality user experience)Co-existence of lower-end systems with telepresence systemsUse of standard (Intel) hardware components – riding the Intel curve

These concepts finally solve 30-year-old packet video problems, and are the key ingredients for making telepresence “personal”.

23 IMTC 2010/06/15

IMTC SuperOp! Room

24 IMTC 2010/06/15

IMTC SuperOp! Room

25 IMTC 2010/06/15

Concluding Remarks

Videoconferencing and telepresence will remain “small”, unless our technology enables scaling with high quality and low cost.Video coding scalability and the video router architecture allow:

Replacement of the MCU with a much simpler, scalable deviceVery high error resilience (=high quality user experience)Use of standard (Intel) hardware components – riding the Intel curve

These concepts finally solve 30-year-old packet video problems, and are the key ingredients for making telepresence “personal”.

Can I get the same benefit from simulcasting? Or without two spatial layers? Or ….?

Partially. If you use part of the recipe, you get only part of the flavor.

Wait … So, the future of telepresence is … Vidyo?

Thank You!