a multi-path routing service for immersive environments

A Multi-Path A Multi-Path Routing Service for Routing Service for

Immersive Immersive EnvironmentsEnvironmentsSherlia Shi, Lili Wang, Kenneth L. Calvert, Sherlia Shi, Lili Wang, Kenneth L. Calvert,

and and James GriffioenJames Griffioen

Laboratory for Advanced NetworkingLaboratory for Advanced NetworkingUniversity of KentuckyUniversity of Kentucky

Supported by NSF, DARPA, and Intel

April 22, 2004 CCGrid: Workshop on Grids and Advanced Networks

The MetaverseThe Metaverse Goal:Goal: design, implement, deploy, use and design, implement, deploy, use and

evaluate evaluate networked, collaborative, networked, collaborative, visualization environmentsvisualization environments that act as portals that act as portals into theinto the MetaverseMetaverse**..

Basic Problem:Basic Problem: Visualize and interact with 3D Visualize and interact with 3D models, simulated objects, time-dependent models, simulated objects, time-dependent data, image data, avatars, and video streamsdata, image data, avatars, and video streams in collaboration with remote users across the in collaboration with remote users across the InternetInternet

Research Areas:Research Areas: computer vision, graphics, computer vision, graphics, networking protocols and servicesnetworking protocols and services

* Neil Stephenson, Snowcrash, 1997.


An Example Metaverse An Example Metaverse AppApp

Example Fire Whirl Data

A CFD Example- Visualize a 3D animated simulation- Interact with the model- Collaborate with colleagues or students at remote metaverse portals


Metaverse Components Metaverse Components

Network

ModelServer

ModelServer

ModelServer


Metaverse PortalMetaverse Portal

Network



Immersive Display Environment

Based on inexpensive, commodity, self-configuring,

networked components



• rendering engines




• rendering engines• projectors• cameras


Driven by cheap,commodity, graphics

and video cards



• rendering engines• projectors• cameras• motion sensors




• rendering engines• projectors• cameras• motion sensors• local area network




• rendering engines• projectors• cameras• motion sensors• local area network • control processors



Immersive, Multiprojector Immersive, Multiprojector renderingrendering

(18 projectors)(18 projectors)

Walkthrough 1

Walkthrough 2

View 1

View 2


Metaverse DataMetaverse Data Data consists of multiple componentsData consists of multiple components

Texture mapsTexture maps Polygonal meshesPolygonal meshes Volumetric DataVolumetric Data User Control Data (position tracking, 3D mouse)User Control Data (position tracking, 3D mouse)

Same data may be represented in arbitrary Same data may be represented in arbitrary waysways Resolutions – potentially variable resolutionResolutions – potentially variable resolution EncodingsEncodings

Time Series Data (=> video-like)Time Series Data (=> video-like) Generated DataGenerated Data


Metaverse ApplicationsMetaverse Applications Goal:Goal: Offer the most compelling immersive Offer the most compelling immersive

experience possible!experience possible! Assumptions:Assumptions:

Massive amounts of complex data to be transmittedMassive amounts of complex data to be transmitted Cannot assume over-provisioning or QoS guaranteesCannot assume over-provisioning or QoS guarantees User’s definition of compelling depends on several factorsUser’s definition of compelling depends on several factors Application can design for these factorsApplication can design for these factors Application does not want to know details of transportApplication does not want to know details of transport

Implications:Implications: Not all data can/needs to be sentNot all data can/needs to be sent Data that is “the most important” to send depends on the Data that is “the most important” to send depends on the

useruser and the and the networknetwork.. Communication channel must support Communication channel must support cooperationcooperation between between

the network and the applications.the network and the applications. Need to maximize bandwidth subject to (good “net citizen”) Need to maximize bandwidth subject to (good “net citizen”)

constraintsconstraints


Transport API Transport API RequirementsRequirements

Feedback: Feedback: about transport capabilities.about transport capabilities. Single end-to-end communication Single end-to-end communication

channelchannel: need a simple “session” : need a simple “session” abstraction -- App is not interested in abstraction -- App is not interested in implementation details.implementation details.

Data “importance”:Data “importance”: App specifies what App specifies what data to transmit to achieve “compelling”data to transmit to achieve “compelling”

Sender / Receiver Cooperation: Sender / Receiver Cooperation: sender sender specifies data, receiver defines qualityspecifies data, receiver defines quality

Reliability:Reliability: data transmitted reliably data transmitted reliably


FeedbackFeedback Feedback Feedback could could describedescribe any “aspect” of the any “aspect” of the

serviceservice Feedback “aspects” Feedback “aspects” may bemay be network-dependent network-dependent Feedback Feedback could becould be at many levels of detail. at many levels of detail. Apps don’t want to deal with the above complexity!Apps don’t want to deal with the above complexity! Apps have no desire to control the transport Apps have no desire to control the transport

service (i.e., twist transport knobs). service (i.e., twist transport knobs). Understanding and mapping knobs to user Understanding and mapping knobs to user satisfaction is too hard.satisfaction is too hard.

Simply want to know “what if” answers! Let Simply want to know “what if” answers! Let transport service figure out the mapping!transport service figure out the mapping!


New Service AbstractionNew Service Abstraction

SenderSender1)1) Specify “important” dataSpecify “important” data2)2) Change definition of “important” as neededChange definition of “important” as needed

ReceiverReceiver1)1) Specify a prioritized delivery planSpecify a prioritized delivery plan2)2) Let service map the delivery plan onto the Let service map the delivery plan onto the

underlying networkunderlying network3)3) Provide predictions of how well the service will Provide predictions of how well the service will

performperform4)4) Goto 1Goto 1


Prioritized DataPrioritized Data Specified as a hierarchy of nodesSpecified as a hierarchy of nodes Leaf nodes represent a semantically Leaf nodes represent a semantically

meaningful data unit, called an “object”. meaningful data unit, called an “object”. Objects are the smallest logical unit and Objects are the smallest logical unit and are delivered as a wholeare delivered as a whole

Internal nodes represent groups of objects Internal nodes represent groups of objects and define the percentage of bandwidth to and define the percentage of bandwidth to give to each subtree.give to each subtree.

Each tree denotes a priority levelEach tree denotes a priority level Children nodes may require ordered Children nodes may require ordered

deliverydelivery


Example Transmission Example Transmission SpecificationSpecification

1

0.2

0.5

0.3

0.40.6

1

Ordered

Priority 0

Priority j

Unordered, No weights

0.2

0.5

0.3a

bc


Metaverse Transport Metaverse Transport Service APIService API

int int open_sessionopen_session(struct sockaddr* (struct sockaddr* svrAddr)svrAddr)

intint close_sessionclose_session(int sessId)(int sessId) int int set_recv_handleset_recv_handle(int sessId, int (int sessId, int

objId, void *buf, int len, objId, void *buf, int len, CallBackFunction* call_back)CallBackFunction* call_back)

MetaInfo*MetaInfo* schedule_objectsschedule_objects(int (int sessId, MetaInfo* objects, int sessId, MetaInfo* objects, int modified)modified)


A Multi-path Transport A Multi-path Transport ServiceService

Goal: Goal: Maximize throughputMaximize throughput

Find independent pathsFind independent paths Distribute load across pathsDistribute load across paths

Minimizing network loadMinimizing network load Limit the hops and delayLimit the hops and delay

Share bandwidth across “links” fairlyShare bandwidth across “links” fairly Need a sharing policyNeed a sharing policy

Implication:Implication: Need more control: Use Overlay/GridNeed more control: Use Overlay/Grid

Control routingControl routing It becomes a “Shared Service”It becomes a “Shared Service”


PlanetLab ExamplePlanetLab Example

714 Kbps

21 Mbps!

Improving Throughput (v1)

- use TCP unfriendly protocols

- use multiple TCP flows

Improving Throughput (v2)!

12 Mbps!

3 Mbps


Transport Service TasksTransport Service Tasks

1)1) DiscoverDiscover network characteristics to find network characteristics to find high-bandwidth communication high-bandwidth communication channel(s) between source and channel(s) between source and destinationdestination

2)2) SelectSelect a a set of routesset of routes subject to subject to constraints limiting the load imposed on constraints limiting the load imposed on the networkthe network

3)3) Dynamically Dynamically mapmap object hierarchy to object hierarchy to paths (i.e., distributed load across the paths (i.e., distributed load across the selected paths).selected paths).


Residual Bandwidth Residual Bandwidth DiscoveryDiscovery

Each overlay node Each overlay node passively monitorspassively monitors current current bandwidth usage to other overlay nodesbandwidth usage to other overlay nodes

Avoids active probing (ping, iperf, traceroute,…)Avoids active probing (ping, iperf, traceroute,…) Record two variables for an active “virtual link”Record two variables for an active “virtual link”

Peak bandwidthPeak bandwidth Average measured bandwidthAverage measured bandwidth

Difference is an estimate of residual bandwidthDifference is an estimate of residual bandwidth Information built up over time and shared among Information built up over time and shared among

nodesnodes


Path SelectionPath Selection

Selection algorithm must choose multiple Selection algorithm must choose multiple paths to optimize some objective paths to optimize some objective function, subject to various constraints.function, subject to various constraints.

How many paths should be selected?How many paths should be selected? What is the maximum length of a path?What is the maximum length of a path? Are some paths preferred over others?Are some paths preferred over others? Are the paths disjoint?Are the paths disjoint? We evaluated three path selection We evaluated three path selection

algorithmsalgorithms


Random Path SelectionRandom Path Selection

Randomly select paths from the Randomly select paths from the source to the destination.source to the destination.

Start from source and randomly Start from source and randomly select a link that satisfies the select a link that satisfies the constraints, otherwise backtrack and constraints, otherwise backtrack and try another linktry another link

Only guarantees that the virtual Only guarantees that the virtual paths are disjoint, not the physical paths are disjoint, not the physical pathspaths


Closest First Path Closest First Path SelectionSelection

Closest First Multipath (CFM) routing Closest First Multipath (CFM) routing uses a breadth-first approach to select uses a breadth-first approach to select paths with the shortest hop count.paths with the shortest hop count.

Start from the source and select all edgesStart from the source and select all edges For each selected node, select all edgesFor each selected node, select all edges When destination is reached, add path to When destination is reached, add path to

feasible setfeasible set If more feasible paths than allowed, select If more feasible paths than allowed, select

from feasible set randomly.from feasible set randomly.


Widest First Path Widest First Path SelectionSelection

Widest First Multipath (WFM) routing Widest First Multipath (WFM) routing selects the paths with the highest bandwidthselects the paths with the highest bandwidth

Start with the source, following the widest Start with the source, following the widest pathpath

If the destination is reached, the widest path If the destination is reached, the widest path edges are removed, and the algorithm edges are removed, and the algorithm restartsrestarts

If there are more feasible paths than If there are more feasible paths than allowed, paths are selected from the feasible allowed, paths are selected from the feasible set randomlyset randomly


Mapping Data to PathsMapping Data to Paths

A variety of mappings possibleA variety of mappings possible Our approach sends all fragments of Our approach sends all fragments of

an object over the same path to an object over the same path to minimize fragmentation effectsminimize fragmentation effects

Our goal is to minimize the “finish Our goal is to minimize the “finish time” (adhering to the data time” (adhering to the data prioritization plan), while satisfying prioritization plan), while satisfying the buffering constraints at the the buffering constraints at the receiver.receiver.


Experimental ResultsExperimental Results

Alpha = sharing policy (% of residual for local traffic)Alpha = sharing policy (% of residual for local traffic) When alpha is 0 (no protection) and network load is high, all When alpha is 0 (no protection) and network load is high, all

algorithms perform poorlyalgorithms perform poorly Medium values of alpha produce good throughput at low load and Medium values of alpha produce good throughput at low load and

converge to the shortest path at high loadconverge to the shortest path at high load Random tends to pick paths that are unnecessarily long and have Random tends to pick paths that are unnecessarily long and have

low bandwidth low bandwidth

Random Closest First

Widest First


55thth Percentile Results Percentile Results


Correlated PathsCorrelated Paths

Are hard to identify, because Are hard to identify, because topology info is not readily availabletopology info is not readily available Traceroute is not sufficientTraceroute is not sufficient

Not clear how important it is to Not clear how important it is to know if paths are correlated or notknow if paths are correlated or not

We developed two passive We developed two passive techniques to identify correlated techniques to identify correlated links – each has its own advantages.links – each has its own advantages.


SummarySummary Metaverse-like applications require a new Metaverse-like applications require a new

APIAPI We proposed an API that allows application We proposed an API that allows application

and network to work togetherand network to work together Proposed a passive monitoring approach to Proposed a passive monitoring approach to

estimate available bandwidthestimate available bandwidth Designed and evaluated three algorithms Designed and evaluated three algorithms

for path selectionfor path selection Developed an algorithm to load balance Developed an algorithm to load balance

across paths – based on the data priority across paths – based on the data priority spec.spec.

a multi-path routing service for immersive environments

Documents