internet media delivery systems: current design, open issues, research problems
DESCRIPTION
Internet Media Delivery Systems: Current Design, Open Issues, Research Problems. G. S. Greenbaum, Ph.D RealNetworks – Codec Group Stanford University Feb 13, 2001. Outline. Introduction to Internet-based Media Distribution Systems Components - PowerPoint PPT PresentationTRANSCRIPT
Internet Media Delivery Systems: Current Design, Open Issues,
Research Problems.
G. S. Greenbaum, Ph.D
RealNetworks – Codec Group
Stanford University
Feb 13, 2001
Introduction to Internet-based Media Distribution Systems– Components
A Real Example of a Media System: RealSystem iQ
– RealProducer 8.5– RealSystem Sever 8 and Proxy 8 with Neuralcast
Technologies– RealPlayer 8
Open Problems– source/channel coding– server storage/complexity tradeoffs– network traffic optimization problems
Outline
Introduction:: Ambitions of a Media Delivery System
The Overall Goal is to Maximize the End-User Subjective Experience to Streaming Multimedia
A Successful Internet Media Delivery System Must Provide:– Tools for the easy re-purposing of new and extant
content distribution over digital networks– Robust, Distributed, Scalable, Secure, and Flexible
Architecture for content distribution– Ubiquitous client distribution supporting diverse
data types
Introduction:: Components
Capture Encoding Serving Internet distribution Playback
RealServer
RealProducer
RealPlayer
Plug-ins Plug-ins Plug-insRealProxy
Plug-ins
IP network
SDK SDK SDK
Content Creation: Digitization and Encoding– Most general purpose content starts life in analog form
which needs conversion to digital form– Rich multimedia data types: audio, video, image, VRML,
etc. • Lossy and lossless modes of compression• Download vs Broadcast (streamed) modes
– Encoding tools typically have little knowledge of the distribution network or computational resources of the client playback device
– Metadata for efficient search and retrieval– Rights of content provider must be insured
Introduction:: Components
Introduction:: Components
Transmission and Distribution– Live and pre-recorded Broadcast and VOD modes of
operation– Redundant, Intelligent, and Reliable architecture needed
• Redundant:: Zero points of failure at each critical node of the distribution chain
• Intelligent: Load balancing (dynamic capacity allocation) for efficient distribution
• Reliable: Stream distributions that traverse public and private network segments; fortified against loss and congested
– Architecture usually contains splitters/repeaters and caches for efficient usage of network/gateway capacity working in both a push and pull mode
Introduction:: Components
Client-side Player residing on different appliances and diverse networks– DSL, Cable Modem, Intranets/firewalls,
POTS, ISDN, etc.– PC’s, cellular devices, PDA’s, set-top
boxes, digital VCRs, etc.
State of the Art Narrowband and Broadband Audio and Video Codecs: RealVideo 8 and RealAudio 8
Pre-processing:– Noise removal– Resampling– Inverse-Telecine– Deinterlacing
Rate control:– Two-pass, VBR
Channel adaptation:– SureStream, forward unequal ECC, data interleaving
RealSystem iQ:: RealProducer 8.5
Broadcast
Remote Broadcast Library
Producer
Internet/Intranet
SLTA library
Live Text Live PixFile Format
File System “Storage”
Adaptive serving:– Bandwidth detection– Stream switching– Thinning– multicast
Designed for distributed delivery networks in mind:
– proxy plug-ins– edge-server plug-ins– distributed licensing
Features:– Ad Insertion, Authentication extensions, remote administration,
firewall proxy, etc. (see www.realnetworks.com for more details)
RealSystem iQ:: Server 8
RealSystem iQ:: Server 8
Broadcast Applications
Admin, monitoring Applications
Broadcast Plug-in
File Format Plug-in
File System Plug-in
Generic Plug-ins (logging, monitoring, etc.)
Server Core
“Storage”
Packet Sink Plug-in
Allowance Plug-in
Server RegistryNetwork Services
Internet/Intranet
Server Clustering
Originate & Deliver from Any WhereOriginate & Deliver from Any WhereAn architecture free from ‘origins’ and ‘edges’An architecture free from ‘origins’ and ‘edges’
RealServer
RealServer
RealServer
RealServer
RealServer
RealServer
Switch
RealPlayer
RealPlayerRealPlayer
RealPlayerRealPlayer
RealPlayer
RealPlayer
Distributed Architectures
Cache
RealServer
RealServer
Router
Storage Array
Storage Array
RealServer(Extra-Net)
Firew all RouterRealProxy
RouterPublic
Internet(Terestrial)
Router
Router
SWITCH
CLEC/TELCO
CABLE
RealProxy
Cache
SWITCH
RealServer
Storage Array
RealServerStorage
Router
RealServer
RealServer
Router
Storage Array
Storage Array
CLEC/TELCO
ENTERPRISE
BROADCASTER: NONSUBSCRIBER BASED
CONTENT DELIVERYNETWORK POP
NON SUBSCRIBER BASED
SUBSCRIBER ACCESS PROVIDER- ISP
BROADCAST NETWORKSUBSCRIBER BASED
RealSystem iQ:: Server 8 Neuralcast Technologies
NeuralCast Communications Protocol– The ability for a network of RealSystem Servers to become self-
aware by exchanging information and making decisions. Current abilities include capacity sharing and capacity fail over in the event of a network or equipment issue.
NeuralCast Live Distribution– The intelligent and reliable delivery of broadcasts through a network
or RealSystem Servers. Current abilities include multi-protocol transmission between servers, error correcting methods for streams, terrestrial and satellite multicast support.
NeuralCast Live Redundancy– From encoder to server or from server to server, the ability to send
redundant streams, providing a fail-over feed in the event of a network or equipment outage.
RealSystem iQ:: Server 8 Neuralcast Technologies
NeuralCast Live Distribution– Push and Pull models– Re-Transmission requests made at receiver not
originating transmitter– FEC configurable– Multiple Transport Support allows the same
broadcast to be transmitted via conventional unicast UPD or TCP across non-multicast enabled segments
RealServertransmitter
Unidirectional channel(UDP/unicast) RealServer
receiverEncoder/
SLTA live.rm
RealServerreceiver
RealServerreceiver
Unidirectional channel(UDP/multicast)
RealSystem iQ:: Server 8 Neuralcast Technologies NeuralCast Live Redundancy
– Stateless protocol decouples live broadcast packet contents from the transmitting network
– Live broadcast streams that are sent redundantly over different networks may be re-converged at the Server receiver
Network B
Network A
Router
RealServertransmitter
Encoder/SLTA live.rm
RealServerreceiver
192.168.110.23
204.24.101.34Router
Encoder/SLTA
RealSystem iQ:: Live Webcast Deployment
Server Splitter RelaysServer Splitter Relays
PublicInternet
RealServerrelay
RealServerrelay
RealServerchanges thetransport on thefly, enabling thetransmission tospan the unicastnetwork segement
RealServertransmitter
Encoder/SLTA live.rm
RealServertransmitter
Encoder/SLTA live.rm
RealServerreceiver
RealServerreceiver
RealServerreceiver
RealServerreceiver
Signal Acquistion Point
Point of Presence
RealSystem iQ:: Real Proxy 8 Access Provider Deployment Goals
– Optimize gateway bandwidth for streaming media• Eliminate redundant requests for media from subscribers
for content that is back hauled from the public Internet
– Improve playback quality experience by migrating content closer to the viewing subscriber
Deployed Technologies– RealSystem Proxy near Telco/Cable cross
connects or at data center
Public Internet orNetwork Access Point
CacheRouter
Router
Switch
CLEC/TELCO
CABLE
RealProxy
ACCESS PROVIDER - ISP
Router
RealSystem iQ:: Real Proxy 8 Deployment
Traffic ManagementTraffic Management
PublicInternet
Subscriber AccessNetwork
RealProxyswitch
RealServer
RealPlayer(s)
Access Provider Point of Presence
RTSP RTSP
Media Stream: RTD/RTP v UDP or TCP
Control Protocol: RTSP
Media Stream: RTD/RTP v UDPor TCP
Cache Acquisition (Cold)Cache Acquisition (Cold)
PublicInternet
RealProxy
RealServer
Access Provider Point of Presence
RTSP
Cache
TCP Cache Acquistion
RTSP://RealServer.com/Media.rm
Media Stream: RTD/RTP v UDP or TCPRealPlayer
RealSystem iQ:: Real Proxy 8 Deployment
Cache Playback (Hot)Cache Playback (Hot)
Live SplittingLive Splitting
PublicInternet
RealProxy
RealServer
Access Provider Point of Presence
RTSP://RealServer.com/Media.rm
Media Stream: RTD/RTP v UDP or TCP
RTSP
CacheRealPlayer
PublicInternet
RealProxy
RealServer
Access Provider Point of Presence
RTSP
Live Pull Split Stream: RTD/RTP v UDP or TCP
RTSP://RealServer.com/Live.rm
Media Stream: RTD/RTP v UDP or TCPRealPlayer
RealSystem iQ:: RealPlayer 8
Post-Filters– Sharpening– color correction– graphic equalizer– Rescaling– Bandwidth Simulator– Reverb
Network Aware– Monitors Network connection– Auto-update enabled to obtain latest player components– Auto-configures for best transport method
Features– RV8, RA8, Visualizations, Radio Tuner, Media Search,
localizations for different languages, etc. see www.real.com for a complete list.
Client Core
RealSystem iQ::Real Client Architecture
RealPlayer
File Format Plug-in
File System Plug-in
“Storage”
Generic Plug-ins
Client Registry
Network Services
Internet/Intranet
RealJukebox Netscape Plug-in ActiveX
Rendering Plug-in
Audio Services
Video Surface
Open Problems...
RealServer
RealProducer
RealPlayer
RealProxy
IP network
Source/Channel coding Internet traffic problems
Server storage vs.complexity tradeoff
Overall minimum-distortion optimization
Traffic optimization indedicated delivery network
Video: no adequate distortion measure
Human retina - an ultimate 120:1 lossy compressor
Source Coding Issues:The Challange
Human eye has around 6M cones + 120M rods;but only around 1M ganglion fibers in the optical nerve.
Some other facts from physiology:– visual recognition requires power of ~30% of
the cortex– hearing -- uses just 3% of the cortex
Ultimate video decoder = 1/3 of a human (adequately educated and trained)
Source Coding Issues:The Challange
Constraints:– Bounded delay (1-10 s and/or MB constraint)– Targeting many discrete, fixed bit rate connections– Random-access is necessary
• Seek/rewind on pre-recorded media• Ability to join live stream
– Packetization: Each packet must contain sufficient information to be decoded.
– Low complexity decoding– Real-time encoding– Need to assume inherently lossy network transport
Source Coding Issues
Source Coding Issues
Bounded Delay– Not as much an issue for Downloadable Media
– Leveraged to solve inhomogeneity in content bit allocation (VBR), network jitter, random access, re-transmission requests, etc.
– Imposes constraints on look-ahead dependencies
– Imposes constraints on interleaving techniques for error mitigation
– Cannot replicated the “remote-control” paradigm of broadcast television.
Source Coding Issues
Bandwidth Scalability– Layered Scalability, e.g. FGS, annex O.
• Introduces additional delay• Hit in coding efficiency
– Embedded techniques, e.g. FGS, wavelets• Introduces additional delay• Possible hit in coding efficiency
– Multi-Encoding e.g. SureStream• Increased file size and possible delivery costs• Optimal coding efficiency for targeted rate
– ?
How to facilitate random access decoding?– independent encoding – pre-encoded transitions – pre-encoded correlated fragments
combined with Slepian-Wolf encoding– use of multiple-description codes– something else???
Source Coding Issues
?
Source Coding Issues
Packetization– Not an issue for downloadable media– Imposes constraints on dependencies for streaming content
leading to a decrease in coding efficiency– Due to IP and packet reconstruction overhead, packet size
should be nearly as large as the packet fragmentation limit imposed by routers, 1500 bytes. => increased latency of stream
Complexity Scalability– How does the algorithm scale on limited resource devices for the
encoder and decoder?• Scalable Post-filters• Limited decode: B-frames dropped or I-frame only• Short-cuts: limited ME/MC
IP have been designed to relieve engineers from thinking about physical connection and channel coding problems
Ironically enough, with streaming media we now have to go back to basics…
UDP is mainly used=> packet erasures– ARQ and ECC– Client side mitigation techniques
Ideally, we want the server to do adaptive channel coding/ transmission. Cost is reduction in server load.
How much of this work can be done ahead of the time by encoder?
Is there a way to store all necessary information in a compact form? Is SureStream the ideal?
Channel Coding Issues
Separation Theorem may not hold:– packets have finite lengths– packets can be sent via multiuser channels
Should we use joint source/channel coding?– How to combine distortion due to lossy
compression and one due to channel errors– can we design a universal lossy joint
source/channel code?
Source/Channel Coding Issues
Another way to deal with IP packets is to use multiple-description coding:
Source/Channel coding problems
Ozarow (1980) The achievable set of quantuples (R1,R2 ,d1,d2,d3) is given by:
1 1
2 2
1 23 2
1 2 1 2 1 2
exp 2
exp 2
exp 2
1 1 1 exp 2
d R
d R
R Rd
d d d d R R
Several embodiments of Media Delivery Systems exist today, but the technology is still in its toddler years.
Many more issues, problems, and questions than answers.
Plenty of opportunities for grad students, post-docs, and professors! (and job security for those outside of academia)
Summary