university of illinois at chicago electronic visualization laboratory (evl) application-level...
TRANSCRIPT
University of Illinois at ChicagoElectronic Visualization Laboratory (EVL)
Application-Level Network Performance / Measurement Tools
Jason Leigh, Oliver Yu,
Alan Verlo, Linda Winkler
University of Illinois at ChicagoElectronic Visualization Laboratory (EVL)
Tele-Immersion is the synthesis of Virtual Reality, video conferencing, and advanced computation
University of Illinois at ChicagoElectronic Visualization Laboratory (EVL)
Example of a Data Intensive Tele-Immersion Application
• TIDE: Tele-Immersive Data Explorer
• Collaborative Large Scale Data Visualization
• In collaboration with National Center for Data Mining
• General framework for collaborative visualization of massive data-sets
• Current data-set is ozone data from NOAA
University of Illinois at ChicagoElectronic Visualization Laboratory (EVL)
Example of a Data Intensive Tele-Immersion Application
• CIBRView: Collaborative Image Based Rendering Viewer
• Cosmology Hydrodynamic code by Julian Borrill, LBNL/NERSC shows theoretical condensation of diffuse matter into string-like formations during early stages of universe evolution
• Accesses volume data:512x256x256x 256 frames ~ 40Gig data-sets
• Generates image slices that are distributed to collaborating clients
• Sent about 500, 1M slices/filesfrom Chicago to Japan using parallelTCP.
• It was also the application over whichwe tested DiffServ.
University of Illinois at ChicagoElectronic Visualization Laboratory (EVL)
BW Latency Sensitive
Jitter Sensitive
Reliability Reqrmnt
Burstiness Packet Size
Protocol
Avatar (virtual representation)
Low Y Y Low Constant Small RTP / UDP
Audio conference
Med Y Y Low Constant Med RTP / UDP
Video conference
High Y Y Low Constant Large RTP / UDP
Realtime state updates
Low Y Y Med Short bursts
Small FEC
Non-realtime state updates
Low N N High App dependent
Small TCP
Small bulk data
(image files)
Med N N High Medium bursts
Large TCP/CPTCP
Large bulk data(raw data sets)
High N N High Very long bursts
Large PTCP / RBUDP
Streaming bulk data
(high quality audio/video/bitmap/polygons)
High Y N High Long bursts
Large SRBUDP / AFEC
Characteristics of Tele-Immersion Flows
University of Illinois at ChicagoElectronic Visualization Laboratory (EVL)
Motivation for Application-level networking tools
• Bandwidth is becoming increasingly available. • Networking QoS is still under research and difficult to deploy and
use. It is not as easy as “flipping a switch.”• Network QoS is not only about bandwidth, it’s about latency and
jitter.• Applications today are not ready to use the extra bandwidth even
if available.• Application developers have to be increasingly network savvy in
order to be able to convert application requirements to networking services.
• Need to make advanced networking easier for the average application developer.
• Need to provide a higher level application framework to keep pace with network advances.
University of Illinois at ChicagoElectronic Visualization Laboratory (EVL)
Advanced Data Transport Techniques for Tele-Immersion
• Maybe we can compensate for latency:– Reliable Low-Latency Data Transfer for Tele-
Immersion
• Even if you had QoS could you really take advantage of it?– High Throughput Techniques for Tele-Immersion
University of Illinois at ChicagoElectronic Visualization Laboratory (EVL)
Reliable Low-Latency Data Transfer for Tele-Immersion
• In Tele-Immersion it is desirable to be able to transmit state information with minimum latency and jitter while preserving reliability
• Rather than use TCP which uses acknowledgments to obtain reliability, try UDP augmented with error correction codes:Forward Error Correction
University of Illinois at ChicagoElectronic Visualization Laboratory (EVL)
`Latency of transmitting 100 packets underUDP, TCP, FEC/UDP with 3:1 redundancy from EVL (Chicago) to
SARA (Amsterdam)
0
50
100
150
200
250
300
350
400
0 500 1000 1500 2000 2500
Packet size in bytes
1-w
ay la
ten
cy in
ms
UDP
TCP
FEC over UDP
FEC greatest benefit is in small packets.
Larger packets impose greater overhead.
As redundancy decreases FEC approaches UDP.
goal
University of Illinois at ChicagoElectronic Visualization Laboratory (EVL)
Jitter for UDP, TCP and FEC over UDPMoving average (over 20 successive data points) of deviations of Short Term Latency (also over 20
successive data points)
0
2
4
6
8
10
12
14
1 5 9
13
17
21
25
29
33
37
41
45
49
53
57
61
65
69
73
77
Jit
ter
UDP
TCP
FEC/UDP
G o a l
University of Illinois at ChicagoElectronic Visualization Laboratory (EVL)
Packet Loss over UDP vs FEC/UDP between Chicago & Amsterdam
50Mbps UDP or FEC
+50Mbps UDP congestion
Packet Loss
UDP 1.90%
FEC 0.05%
UDP with congestion 17.40%
FEC with congestion 4.15%
University of Illinois at ChicagoElectronic Visualization Laboratory (EVL)
High Throughput Techniques for Tele-Immersion
• In Tele-Immersion it is desirable to share data files as rapidly as possible
• Even if you had QoS, you couldn’t take advantage of it
• Long Fat Network problem: an ftp session will max out at 3.5Mbps over a 100Mbps link between Chicago and Amsterdam (and Switzerland)
• 2 Techniques:– Parallel TCP Socket Striping– Reliable Blast UDP
University of Illinois at ChicagoElectronic Visualization Laboratory (EVL)
Parallel Sockets : 64K Window SizeAmsterdam (SARA) to Chicago (EVL)
Plot of Average Achievable Bandwidth vs # of Paralle TCP Sockets Used to Deliver a 50M File from Amsterdam to Chicago over 100Mbps Link
0
10
20
30
40
50
60
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
# of Sockets
Band
wid
th (M
bps)
Found it difficult to achieve more than50Mbps on a 100Mbps link.Have been able to achieve 80Mbpson rare occasions.
University of Illinois at ChicagoElectronic Visualization Laboratory (EVL)
Reliable Blast UDP (RBUDP)
• RBUDP - An old idea that may be useful now that networking bandwidth is increasing
• Use UDP for bulk data transmission rather than TCP
• If bandwidth can be guaranteed by QoS –reliability will be high- chances of errors will be few
University of Illinois at ChicagoElectronic Visualization Laboratory (EVL)
RBUDP Performance(EVL to SARA)
Average throughput vs transmission unit size(size before an ACK is required) over 100Mbps link between EVL and SARA.
01020
30405060
708090
1 2 3 4 6 8 10 15 20 22 23 25 30 40 60 80 100 150 200 400
Unit Size (MBytes)
Aver
age
Tho
ught
put
(Mbp
s)
Realtime Throughput Maximum UDP throughput
University of Illinois at ChicagoElectronic Visualization Laboratory (EVL)
RBUDP Performance(EVL to CERN)
Streaming RBUDP from CERN to EVL over 45Mbps link
0
5
10
15
20
25
30
35
Unit Size (Mbytes)
Th
rou
gh
pu
t (M
bp
s)
RUDP sending at 33Mbps
Maximum throughput achieved by UDP(33Mbps)
University of Illinois at ChicagoElectronic Visualization Laboratory (EVL)
DiffServ results over a testbed between EVL and Argonne National Lab
25Mbps80Mbps
ANL
EVL
42Mbps 42Mbps
100Mbps 100Mbps
100Mbps 100Mbps
fore
back
x
x
x x
Cisco7507
DiffServ Bandwidth
0
5
10
15
20
25
1 10
19
28
37
46
55
64
73
82
91
100
109
118
127
136
145
154
163
Time (s)
Ban
dw
idth
(M
bp
s)
DiffServ Latency
0
50
100
150
200
250
300
1 10
19
28
37
46
55
64
73
82
91
100
109
118
127
136
145
154
163
Time (s)
1 w
ay L
ate
ncy (
ms
)
DiffServ Packet Loss
0200400600800
1000120014001600
1 9 17 25 33 41 49 57 65 73 81 89 97 105
113
121
129
137
145
153
161
Time (s)
Pac
ke
t L
os
s (
pac
ke
ts/s
)
Bandwidth recovery good
Latency recovery not good
Packet loss double
+ background + DiffServ
150ms 1-way latency
University of Illinois at ChicagoElectronic Visualization Laboratory (EVL)
Collaborative Coordination Experiments betweenChicago and Singapore
• Tightly coordinated collaborative interaction task between 2 remote users
• 200ms RTT is the threshold where performance begins to suffer• 200ms RTT with 0 jitter is same as 10ms RTT with 7ms jitter• So DiffServ is not suited for Tele-Immersion
University of Illinois at ChicagoElectronic Visualization Laboratory (EVL)
• G2 is C++ toolkit for building Tele-Immersive applications with special emphasis on networking
• G2 is the Grid’s main tele-immersion library• Networking:
– UDP, TCP, Multicast, HTTP.– UDP reflector and multicast bridge.– TCP reflector.– Remote procedure calls.– 32 and 64bit Remote file I/O.– Parallel 32 & 64 bit TCP socket striping for high throughput data
delivery.– FEC.– Client/Server distributed shared memory persistent database.– Threading, Mutual Exclusion.– Built-in Instrumentation of networking services.
University of Illinois at ChicagoElectronic Visualization Laboratory (EVL)
• Tools for higher level application development:– Audio streaming– Articulated Avatars– VR navigation– VR menus– Speech recognition with IBM ViaVoice– Collaborative application shell to jumpstart development– Network visualization tools
University of Illinois at ChicagoElectronic Visualization Laboratory (EVL)
QoS Internet Monitoring ToolQoSIMoto
• Provides real time viewing of CAVERNsoft data streams
• Visualizes bandwidth, latency, jitter of multiple network flows
• Accepts Netlogger compatible format
University of Illinois at ChicagoElectronic Visualization Laboratory (EVL)
These techniques are needed more than ever in STAR LIGHT
• Being able to dedicate lambdas will ease our latency and jitter problems but we still can’t beat the speed of light.
• Lambda switching requires more intelligence at the edges to perform traffic shaping, bandwidth management, error correction.
• Need to provide high level API for applications to select Lambdas and define which application flow will go over the lambda.
• Long Fat Network problems do not go away with GMPLS.• Need to provide a higher level application programmer’s
framework to keep pace with network advances. We need more than just sockets( ) API.
University of Illinois at ChicagoElectronic Visualization Laboratory (EVL)
What new things can we do withSTAR LIGHT?
• Streaming uncompressed high resolution stereoscopic 3D movies (1024x768x24bitsx30fps) ~ 1.05 Gbps
• Imagine magnifying this to multi-tiled displays for ultra high resolution displays such as CAVEs and Active Murals
University of Illinois at ChicagoElectronic Visualization Laboratory (EVL)
What new things can we do withSTAR LIGHT?
Distributed Tera-snap:• Perform a tera-mining correlation between
distributed databases and generate a visual overview
• 1 PC can absorb ~500Mbps => min 4.6 hours to perform a tera-snap
• 20 PCs can absorb data and produce image composites in a min of 13 mins using 10Gbps
University of Illinois at ChicagoElectronic Visualization Laboratory (EVL)
What new things can we do withSTAR LIGHT?
Digital Continuums: Distributed collaboratories with linked tiled, tele-immersive and mobile displays enhanced with cluster computing and data mining services
University of Illinois at ChicagoElectronic Visualization Laboratory (EVL)
Thanks
• Thanks SARA (Amsterdam), CERN (Switzerland), IHPC (Singapore) for graciously participating in these network experiments
• For more info:– www.evl.uic.edu/cavern– [email protected]