Dynamic Reconfiguration of IP Domain Middleware Stacks to Support Multicast Multimedia

Distribution in a Heterogeneous Environment

….Basically attempting to give the optimum multimedia reception quality

to unequally yoked receivers on a Mobile IP network using features

implemented in the industry standard RTP protocol designed for streaming

networked multimedia. The result should be ideal for resource constrained mobile devices.

Stop waffling…….Explain it to me in simple terms before I call security….

Internet …. A gigantic leap for mankind …. A small step for those under the age of 5

All machines are equal…but some are more equal than others eg. Laptops, Sun Sparcs

Mbone, Multicast, layered encoding….My God, what are you talking about? All we want to do is broadcast a baywatch video

Existing framework for multimedia conferencing on Internet (This won’t change, I’ll just enhance it)

Conferencing applications use the Real-time Transport Protocol

Receivers express interest in receiving traffic by tuning into a multicast address & network forwards traffic only along links with down-steam recipients

No knowledge of group membership or routing is available at source or receivers

Differently ‘shaped’ computers on the

internet… A sort of ushering in the new socialist computer

society...

Back to fluffy clouds or is it a sheep with seven legs?

14.4 k 28.8 k 33.6 k

64 k

128 k 512 k

1.5 MB Internet

Backbone

Layered Encoding....Not bad...but not good enough

S

R1

R2

R3

1.5Mb/s1.5Mb/s

1.5Mb/s512 kb/s

128 kb/sLink capacity

Enhancement layer 1

Enhancement layer 2

Enhancement layer 3

Router

S1

Sender

R1, R2, R3 : Receiver 1, 2, 3

Simply, three different quality streams sent over the network (each one building on the previous)

The Kerryman’s Solution….replicated streams

S

R1

R4

R3

100K

64k

28K

42K 8K

HQ Video (16k)

Audio (4k)

Text (2k)

Router

S1

Sender

Receiver

Cross-traffic

Cross traffic

LQ Video (9k)

(15K) (20K)

R2

20K

R

Sender creates various quality streams and lets receivers pick and mix….. wasteful of bandwidth…. Each up-stream link carrys all streams

Let me see a conceptual diagram before I throw up.….

The framework is a pure Java implementation using the Java Media Framework (JMF) to transport multimedia

Chameleon Framework

Network Interface Card

MAC Drivers

IP / Mobile IP

RTP/RTCP

Codecs Renderers Effects QoS

Java Media Framework 2.0 API

Streaming Applications

Physical

Datalink

Network

Transport

Session

Presentation

Application

2

4

6

7

5

3

1

Layer

Mobile Host

Home Agent

Router x.xx.xx

Mobile Host

Foreign Agent

Router

LAN (Ethernet)

Wireless Antenna

Video Stream

Video Stream FA 1

Wireless Service Provider

Mobile device with no connectivity

(Out of range)

Internet

Correspondent Host

Mobile Host

FA 2

FA 3

Near Region

Distant Region

Far Region

Out of range Region

Nope.. Still doesn’t make sense to me…I’ll just sit here and nod intelligently

5. Multicast groups for heterogeneous Clients

Object adaptor

Object skeleton

Text

Video

Audio

Video, Audio and Text

Client Stub Middleware library

Desktop 10MB Client Wireless LAN Client

Audio Video Text

Memory Audio Video Text

Memory

Audio Video Text

Memory

Multimedia Sender

Video Media and Stack Repository

TCP Socket

Text Video

Audio

Object Request Broker

TCP Socket

Chameleon Multimedia Sender

Video

Text

Text

Group

Video

Video Specific Stack

Video Group

Audio Specific Stack

Audio Group

Audio

Chameleon Middleware

Media and Stack Repository

B & W Frames

Filtered

Text Group

Filtered Audio Group

Filtered Video Group

System Monitor

Stream Control & Updates

Stream Subscription

Statistics & Stack Component Requests

Text Specific Stack

Desktop 10MB Client

Audio Video Text

Text

Memory

GSM PDA Client

Text Memory

Wireless LAN Client

Audio Video

Memory

New QoS

modules

New QoS

modules

Session Manager

Mobility, Bandwidth,

Display, Location Events

1

Experimental Evaluation

Points

1,2. Media Specific Stacks & Dynamic Reconfiguration

1 + 2

3. Proxies to offload computation for mobile

devices

3

4. QoS negotiation components

4

5

Chameleon Architecture Components

GUI Events

Stream Manager

Server Manager

Session Manager

Adaptation Manager

Mobility Manager

Media Text Protocol Profiler

Server

Mobile Hosts (Clients)

Proxy

Audio

Stack Requests

System Monitor Control Channel

Data Channel

Location Updates

Session Announcements

Monitor Resource & Enforce QoS

Stack Manager

System Resource Manager

Policy Module

Monitor Throughput

Stack Repository

Video Security/QoS

Policies

4.1 4.2 4.3 4.4 4.5 4.6 4.7

x = Section Heading

Reflective Class Loader

4.13

4.12

4.11

4.10

4.9

4.8

What happens when you don’t eat your Weetabix

The failings of a research scientist

Stream Management

Display Interface

Transmission Application

Capture Application

Storage Facility

Multi-Stream Management

Server Interface

Monitor Interface

Media Recording

Compression

Capture Interface

Encoding Module

Streaming Server

Stream Monitoring

Media Database

File/Data Transfer

Security

Data Query Tool

Multi-Stream GUI

Monitor QoS

Location Monitor

Bandwidth Monitor

Display Monitor

Memory Monitor

System Monitor

Screen resolution Display capabilities…

Maximum bandwidth Available bandwidth Packet loss….

Location Velocity ….

Total Memory Available Memory ...

how to overload a diagram & confuse an audience

Receiver

Transcoder

Router

Max Bandwidth

Med Bandwidth

Min Bandwidth

University of Ulster, Magee

Remote Site A

Remote Site B

Laptop Machine X

Machine X

Machine Z

Machine Y

Machine Y

Machine Y

Machine Z

Machine Z

Video Server

Router x.xx.xx

Router x.xx.xx

Router x.xx.xx

Machine X

Machine Y

Transcoder

CongestionRegion

Transcoder

Basically - let these ‘transcoder’ thing-a-mi-jigs do all the hard work. Let them resend the missing pieces - let them act as big brother.

•Saves Bandwidth•Faster recovery•Optimum quality

1 – Media Specific Stack Configuration

Chameleon Multimedia Sender

Video

Text Video

Video Specific Stack

Audio Specific Stack

Audio

Media and Stack Repository

Text Specific Stack

Problem. Protocols stacks have traditionally been monolithic chunks of code where all data regardless of whether it is a continuous stream of bits with strict time dependencies between those bits, or the packets comprising an asynchronous message are sent through the same stack.

The nature of the data is not considered however and therefore there is no room for optimisation of the code to create a more efficient service. In addition protocol functionality can exist which is never invoked.

Solution. We propose a propose a paradigm whereby media are transported through an optimised stack constructed solely for that media/medium allowing improved multimedia Quality of Services to be achieved even at run-time as illustrated above. In our approach, for making applications adaptive, the applications are composed from micro stack objects and the composition is reconfigured when the operational environment is changed. Each stack object has a function such as filtering and caching and a protocol profile manager can dynamically reconfigure the object compositions for increased performance.

2 – Dynamic Runtime Stack Configuration

Cryptography

Fragmentation

Queuing

Acknowledge Network

CRYPT

FRAG

FIFO

NACK

Wired/Wireless

Protocol Graph Module Graph

Chameleon Multimedia Sender

Video

Text

Text

Group

Video

Video Specific Stack

Video Group

Audio Specific Stack

Audio Group

Audio

Chameleon Middleware

Media and Stack Repository

B & W Frames

Filtered

Text Group

Filtered Audio Group

Filtered Video Group

Text Specific Stack

Desktop 10MB

Client

Audio

Video

Text

Text

Memory

GSM PDA Client

Text Memory

Wireless LAN Client

Audio

Video

Memory

New QoS

modules

New QoS

modules

3

1

2

Example Protocol

Stack

IPMCAST

Problem. Mobile devices will be connected to a diverse range of network types offering vastly differing levels of bandwidth throughput.

In addition, fluctuations in throughput and delay may also be experienced due to congestion even while connected to a particular network, (e.g. as witnessed in the Internet) therefore it is important that systems have the means to adapt to the offered network QoS.

Solution. Dynamic protocol stacks can be invoked which provide a ‘best-fit’ for each and every situation.

For example, in the case of a mobile device moving from a LAN into a cellular network where the available bandwidth and the error rate change significantly, the middleware could adopt bandwidth-conserving mechanisms, such as compressing the requests, (i.e. trading off CPU for bandwidth in order to adapt itself to environmental changes) through dynamic protocol stacks.

3 – Computation Offload for Constrained Devices

Problem. Current PDA/wireless devices have limited processing power in comparison to their desktop counterparts. This seems likely to be the trend for the near future. Multimedia however has strict real-time requirements with regards reception; processing and display of media therefore large reservoirs of multimedia are likely to be inaccessible to these devices for the foreseeable future. The need also exists for facilitating mobile mobile users in handling intermittent connectivity, location tracking, link QOS constraint & session migration among others.

Solution. Proxies located at the home agent/base station of each mobile client can offload processing for memory-constrained devices. The proxy becomes responsible for activities such as message queuing and forwarding, compression, access to streaming sessions, message encryption and protocol translation among others.

..for smallest possible client footprint, the proxy relieves the client library of most of the work of maintaining state information about active sessions, filters and general system monitoring facilities….may also perform another function as a cache for recent clips

Chameleon Middleware

B & W Frames

Filtered

Text Group

Filtered Audio Group

Filtered Video Group

System Monitor

Desktop 10MB

Client

Audio

Video

Text

Text

Memory

GSM PDA Client

Text Memory

Wireless LAN Client

Audio

Video

Memory

New QoS

modules

New QoS

modules

Session Manager

Mobility, Bandwidth,

Display, Location Events

4 – Quality of Service Middleware Components

Problem. TCP/IP networks are essentially best-effort conduits in which packets are sent wholesale, without waiting in line for each other. Packets routinely collide and hit congestion, causing tiny delays and often requiring resends. Those delays, usually measured in milliseconds, are meaningless with data traffic but wreak havoc with time sensitive multimedia traffic.

Solution. A set of negotiation protocols allowing real-time negotiation on QoS through the application of dynamic transformation of media. Protocol profilers maintain a list of adaptation thresholds and decisions to transcode data are taken in accordance in accordance with media profiles mapped to bandwidth and end-host resource characteristics. In addition, a Priority Queuing technique can give specific media streams higher priority than less critical traffic so that during congestion periods, media streams are queued as high, normal, medium, or low. Therefore, using Priority Queuing, all high-priority traffic is serviced first, then normal etc. This allows additional resource reservation protocols and selected pricing schemes to be ‘slotted in’ at a later date.

Chameleon Middleware

System Monitor

New QoS

modules

New QoS

modules

Session Manager

Mobility, Bandwidth,

Display, Location Events

There is no clear picture of what it will take for better QoS to be deployed in enterprise networks, and demand is not forcing fast solutions however solutions to many QoS problems can be solved using application level QoS mechanisms and to date much of the enterprise middleware available ignores or implements poorly priority assignments to media.

5 – Heterogenity of Client Population

Problem. There are many types of network connected mobile devices today with capabilities ranging from powerful full specification laptop PC’s, to lower powered PDA’s. Some devices will be capable of displaying full colour 1600x1200 while others may only manage black and white 100x60 screen resolution.

Networked devices range from broadband others to GSM type service. Approaches to the problem have involved sending the lowest common denominator stream to all receivers, however this penalises the more powerful mobile clients that receive far below their true capabilities. Solution. Providing various qualities of media through separate multicast groups can cater for a heterogeneous mobile client population. Here a client can ‘pick and choose’ a Quality of Service (QoS) in accordance with its capabilities. This mechanism offers movement between multicast groups and transcoding of media within groups (no need to join a separate group). A Secondary transcoding mechanism can complement the above coarser grained technique by assuming responsibility for responding to quality fluctuations within each group. Both techniques can be optimised to work with priorities being assigned to differing streams within a link in order to allow different streams to be rate controlled according to application-implied importance.

Mobile Device

Transcoder

MPEG/Audio-1 Layer 3, 44 kHz, 16 bit, stereo 128 kbps

Server

DVI 8 kHz, 4 bit, mono, 32 kbps

MPEG/Audio-1 Layer 3 44kHz, 16 bit, mono 64 kbps

Mobile Device

Laptop, network 128 kbps PDA, network 28 kbps

Proxy GUI for snooping and manipulating

streams

Simple Client Player

Server GUI for streaming multiple QoS streams to Multiple Mulitcast Group Locations

Server, Proxy and Client GUIs

Chameleon allows the insertion of a retransmission policy object such as ACK or NAK to ensure reliability of data should the need arise. Using the NAK scheme, the sender tags a message with a sequence number and the receiver only requests retransmission if there is a gap between the expected and actually received message sequence number. However, in the ACK scheme the sender tags a message with a sequence number and the receiver acknowledges every message by sending an ACK back to the sender.

When no ACK has been received for a certain time, the sender retransmits the message. In general, the NAK scheme uses very few extra messages if transmission failures are rare, but latency can be very high if the transmission rate is low or if the failure rate is high. This is due in part to the buffering of larger groups of data and the repercussions upon other data should packets go astray.

Experiment Goal: To demonstrate that adapting the epoch message size of the NAK protocol in relation to network losses can lead to an optimal protocol. The first part of the experiment compares the performance of varying epoch sizes over a lossy network. The second part utilises the secondary quality transformation technique, which reconfigures the epoch policy to achieve an optimal throughout over a lossy network.

Experimental Verification of (1) Media Specific Stack Configuration

Experimental Verification of (1) Media Specific Stack Configuration

4

5

6

7

8

9

10

0 5 10 15 20 25 20 35 40Packet Loss (% )

Th

ro

ug

hp

ut

(M

Bp

s)

epoch size 100 epoch size 300epoch size 600 epoch size 900

4

5

6

7

8

9

10

0 5 10 15 20 25 20 35 40Packet Loss (%)

Th

ro

ug

hp

ut

(M

Bp

s)

epoch size 100

Here we achieve an optimal stack through dynamic reconfiguration. Examination of the raw data however reveals that there is 18% lower throughput than the theoretical maximum. We believe this is due to the actual overhead involved in the system reconfiguration. We have also to some degree weighted the experiment to suit our system as we have performed runs of sufficient length in time to allow adaptation to be detected and to benefit the system, once reconfigured.

We believe the other cause of poorer performance is simply the self-imposed delay on the system when responding to brief fluctuations in quality.

This is necessary to prevent incorrect responses to random spikes in throughput.

Future Work or (Dream Work)* By registering a domain name the package names used in the framework will be assigned a unique name that conforms to the naming conventions that Sun Microsystems have specified. This is important, as a unique name space for the framework will be essential to prevent it from conflicting with other packages. Therefore this should help to ease acceptance of the framework by developers and to ensure that the framework is industry standard; exception handling will have to be incorporated for every eventuality.

* Adoption of XML to allow system administrators to customise the multicast groups will allow the system to be flexible enough for non-programmers to alter its behaviour without the need for recompiling the system. It would also be useful to implement a group selector that uses XML configuration for each multicast group processor, which contains details about the bandwidth limitations the processor is designed to support. Therefore a Group Selector could be created to use this information when assigning clients to Multicast groups. Group processors could also be distributed to relieve the bandwidth requirement on any particular broadcaster.

Goodness, will this guy ever shut up?

* A layer broker similar to the over the air (OTA) mechanism that J2me uses to download application to a mobile phone could also be developed. This functionality will allow layers to be dynamically downloaded by a client as needed. Security should be tightened (e.g. a form of encryption can be introduced into the group processors). Another is that clients could be validated by their IP address, thus only authorized clients would be able to subscribe to a group processor.

* The files could be encoded using the Sorensen encoder. Files are currently encoded using the CINEPAK codec. Compared with Cinepak, Sorenson Video generally achieves higher image quality at a fraction of the data rate allowing for higher quality, and faster downloading. It is designed for high quality at data rates less than 200 kBps. This codec is capable of better picture quality and smaller files than Cinepak. It requires more compression time than Cinepak however but it supports temporal scalability, which lets a movie exported for a high-end computer play back smoothly on a low-end computer.