lola - terena · lola “design” concept •!lola has been conceived and developed for dedicated...
TRANSCRIPT
LOLA (LOw LAtency audio visual streaming system)
A LOW LATENCY, HIGH QUALITY AUDIO/VIDEO TRANSMISSION SYSTEM FOR NETWORK MUSICAL PERFORMANCES AND INTERACTION
Conservatorio G. Tartini Consortium GARR
The Dream...
...at the GARR Conference 2005, in Pisa...
To perform together, from distant locations... as if distance has vanished
in a click of a computer mouse.
LOLA (the team working to make the dream come true)
Production
Conservatorio di musica G. Tartini - Trieste
Implementation
Paolo Pachini: general coordination
Carlo Drioli: programming
Nicola Buso: testing and musical advice
Claudio Allocchio (Consortium GARR): testing and networking advice
Massimo Parovel: conception and supervision
What do we need to make the dream real?
•! A very fast CAPTURING and RENDERING device (audio & video)
•! A very fast ENCODING and DECODING (audio & video)
•! A network service which is •! Very FAST •! Very STABLE •! ERROR FREE
!!no commodity Internet can deliver these features
! it’s a big challenge ALSO for National Research and Education Networks
LOLA “design” concept
•! LOLA has been conceived and developed for dedicated end-to-end connections (circuits, optical lambdas) because there is no protection against network problems in it and in full mode it requires up to 500M for Standard Resolution (color).
•! LOLA has been also adapted to run in minimal configuration over very reliable NRENs… in the middle of the highway traffic at 92M (Standard Resolution, 30fps, B/W)
•! two violins…
•! Trieste-Barcelona (via Madrid): ~ 2700km
The experiment we are going to try now:
Beware of Traffic Jams!
•! and in the middle of the Highway!
•! where also security attacks can happen…
•! and also something else…
The experiment we are going to try now:
DANTE TICKET: 15368 Type: Dashboard Alarm Status: Open
Description: [mad - gen] GEN-MAD-IP (interface so-7/2/0) is down Location A: Madrid, ES Location B: Geneva, CH
Incident Start: 14/06/2011 10:30:34 (UTC) Incident Resolved: (UTC) Ticket Open: 14/06/2011 10:37:29 (UTC) Ticket Close: (UTC)
Translation: the fiber was CUT!
The re-routing went via Geneva-Paris-Madrid Another 1000km !! 3700 !!!
Well cross finger!!
Caterpillars Exists!
Audio/Video tools before LOLA
H.323/SIP
•! High Video Compression •! Low Quality Audio
Codecs •! Video over Audio Priority •! Significant roundtrip
latency •! Optimised for Low
Bandiwidth •! Propietary Polycom
“Music Mode”
Audio/Video tools before LOLA
DVTS
•! No Video Compression •! DV Audio Codecs •! Audio/Video in DV
frames •! High roundtrip latency
due to IEEE1394 •! High Bandiwidth
(30Mbps) •! Good Quality •! OK for music lessons
Audio/Video tools before LOLA
ConferenceXP
•! Indipendent Audio and Video Handling (compression, codecs)
•! Low to High Quality Audio/Video Codecs
•! Multiple Audio/Video streams
•! High roundtrip latency •! Variable Bandiwidth
(2Mbps to ... inifinity)
Musical Requirements to Play Together
•! Audio Latency below ~75ms (depends on music gender) •! Eyes Contact to synchronize •! Spatial Sound immersion for expression •! Room Reverbering •! Continuous Presence of all musicians •! Non distracting environment
Motivations and Objectives
•! Target applications:
•! interactive musical tasks •! Network musical performances •! Network music education and training
... but also...
•! interactive on stage performances •! Network Dance performances •! Network Drama performances •! Network performing arts education and training •! ...
Motivations and Objectives
•! Challanges
•! Management of synchronous audio video streams over packet networks
•! Minimal delay requirements for interactive task and music performances
•! Optimal balance between AV presentation delay and quality
•! The speed of computer/networking equipments •! ... and beyond: the speed of light...
Motivations and Objectives
•! Targets of the project •! A system suitable for musical performances relying on both audio and visual communication (but also other applications)
•! A Transparent and Natural end-user interaction
•! Providing lowest possible delay using available technology
•! Low cost and portable equipment
•! Oriented to dedicated high performance networks (LightNet Project, GARR, GÈANT, Internet2,...)
The LOLA system a very basic schema
(well, what’s the difference with DVTS, CXP?)
Research & Education Networks
Functions and networking Connection negotiation and management
•!Send/receive and manage connection requests
•!Negotiation of audio/video formats and compatibility check Video: 640*480, 30-60 fps, BW/Color Audio: 44100, 16/24 bit, mono/stereo, multichannel...
•!Transmission of raw audio and video streams, monitoring
•!Grabbing and Tx with very low latency devices Video: Grabber BitFlow, BW progressive scan camera Audio: RME HDSP9632 Network: 1GBps Ethernet Card / 1 GBps Network Connection (Circuit?)
•!CODEC latency: < 5 ms (audio and video)
•!Network latency: <1 ms (LAN); ~1 ms per 100Km (WAN); stay below 60ms RTT (~4000Km) ? a few jitter buffers are required (delay vs quality balance)
Functions and networking Audio/Video data grabbing and transmission
GUI functions
•!Controls for connection negotiation and management •!Audio/Video buffering controls •!Visualization options, audio mixer, etc •!Recording
Functions and networking Graphical user interface
First results: 110km, 15 switches, 60fps, B/W
June 4th 2009 - Trieste fiber optics metropolitan network Lightnet
•!Two Pianos, in two studio rooms at Tartini, linked over the loop with LOLA •! Round Trip Latency ~90ms
•! monitor on the music score, “as if the other pianist was in his canonical ‘duo position’, e.g. in front”
•! Tests performed:
•! One piano plays alone, with the return audio channel open; sound was coming back, but no echo cancelling needed;
•! Two pianos play together some scales and easy execises; •! Two pianos play together some canone by J.S. Bach; •! The latency is artificially increased to test interaction limits;
•! We can try with a full setup and a real Piano Duo to get feedback
Yes, we can try to play together!
First test with music: September 21st 2009
•!Two Pianos, in two Concert Hall at Tartini, linked over the loop with LOLA •! Round Trip Latency ~80ms, mostly due to CODECs
•! Sound Rendering, Room Environment, musician interaction with LOLA environment.
•! Tests performed:
•! Play Bach Branderburgh Concerts •! Roundtrip Latency tests •! Remote sound in (insulating) earphone vs audio monitors •! Adaptation techniques to delay
•! They can play together, but too much attention is payed to handle the delay. No confortable environment for artistic performance. •! We need to go further down with CODECs delays
Bach Brandeburgh Concerts
Trevisan-Zaccaria Piano Duo: November 5th 2009
•! Just a Triangle, to make accurate latency measurments •! New Round Trip Latency ~20ms !
•! Tests performed:
•! Various buffers configuration to identify optimal results
•! Sound Quality check (sending recorded audio tracks)
•! We can give Bach another chance to meet technology!
The Triangle Test
Experiments without WinSock: December 21st 2009
•!Two Pianos, in two Concert Hall at Tartini, linked over the loop with LOLA •! Round Trip Latency 20ms (artifically up to 70ms)
•! Tests performed:
•! Play Bach Branderburgh Concerts
•! They started to play... and played them all! •! Natural interaction
•! Many tests to refine the Sound Rendering of the remote piano.
Bach Brandeburgh Concerts (bis)
Trevisan-Zaccaria Piano Duo: February 4th 2010
•! Sound equalisation and spatialisation tests for the “remote” piano;
•! it seems to run safely in the middle of the highway:
Let’s Try a shared real production network
Trevisan-Zaccaria Piano Duo: October 16th 2010 Trieste-Milan-Trieste loop; ~40ms
•! color impact (on h/w and network bandwidth);
•! QoS implementation through DiffServ
•! further optimize graphic card rendering;
•! “simple” user interface;
•! to develop a users’ community and to build common projects;
•! to liaise with other potential users
•! spatial audio rendering (multichannels)
The Work In Progress...
We won the CODECs challenge ... the (academic) Network Challenge now!
•! 95 to 450 Mbps (GigaEthernet circuits) •! Dedicated Ligthpaths
•! jitter < 3ms
•! ~1ms per 100Km
•! when the Atlantic will be large 30ms... we will call Captain Kirk and Scotty:
UltraLight Speed, Scotty!
Yes Sir! Done!
Questions?
http://www.conservatorio.trieste.it [email protected] [email protected]
http://www.garr.it [email protected]