history of internet audio experiments at mcgill
TRANSCRIPT
![Page 1: History of Internet Audio Experiments at McGill](https://reader035.vdocuments.net/reader035/viewer/2022070604/54b484ab4a7959fd348b45a8/html5/thumbnails/1.jpg)
ANET Workshop
History of Internet Audio Experiments at McGill (and elsewhere)
Jeremy R. CooperstockCentre for Intelligent Machines
Centre for Interdisciplinary Research in Music Media and Technology
McGill University
![Page 2: History of Internet Audio Experiments at McGill](https://reader035.vdocuments.net/reader035/viewer/2022070604/54b484ab4a7959fd348b45a8/html5/thumbnails/2.jpg)
ANET Workshop
Software available from:http://ultravideo.mcgill.edu
Research supported by:
![Page 3: History of Internet Audio Experiments at McGill](https://reader035.vdocuments.net/reader035/viewer/2022070604/54b484ab4a7959fd348b45a8/html5/thumbnails/3.jpg)
ANET Workshop
Some Early (Pre-)History
Eve Schooler: Distributed Music: A Foray into Net Performance (Sept. 1993)
synchronized three real-time streams from different hosts; delays in the order of 200 ms difficult for performers to be listeners
Paul Hoffert: CyberSoiree (Feb. 1996) ATM-based technology for audio and video streaming of a four-way jazz performance delay >0.5s delay but musicians earned to compensate through extensive practice
Dimitri Konstantas: Distributed Musical Rehearsal Studio (May 1996) ATM based distributed rehearsal with conductor at different location from musicians 80 ms one-way delay for audio-video synch; echo resulted in "extreme confusion"
Seiji Ozawa: Opening Ceremony Nagano Winter Olympics (1998) conduct choruses on 5 continents: singers in Sydney, New York, Beijing, Berlin, False Bay time lag adjustor used to eliminate satellite delay
![Page 4: History of Internet Audio Experiments at McGill](https://reader035.vdocuments.net/reader035/viewer/2022070604/54b484ab4a7959fd348b45a8/html5/thumbnails/4.jpg)
ANET Workshop
World's first Internet AC-3 streamMontreal-New York for the AES 107th Convention, Sept. 26, 1999
![Page 5: History of Internet Audio Experiments at McGill](https://reader035.vdocuments.net/reader035/viewer/2022070604/54b484ab4a7959fd348b45a8/html5/thumbnails/5.jpg)
ANET Workshop
System Diagram
1.5 Mbps
1.5 MbpsSGI Indy
PC ClientCisco IP/TVPC Server
SGI IndyNetwork
Dolby encoder
AC-3 (640 Kbps) (5.1 channels,16 bits @ 48kHz)
coded in AES/EBU
McGill UniversityMontreal, Canada
NYU Cantor HallNew York, USA
![Page 6: History of Internet Audio Experiments at McGill](https://reader035.vdocuments.net/reader035/viewer/2022070604/54b484ab4a7959fd348b45a8/html5/thumbnails/6.jpg)
ANET Workshop
In action
Receiver
Transmit Receive,check, playback
Send AudioQueue
Sender
Receive AudioQueue
Read
![Page 7: History of Internet Audio Experiments at McGill](https://reader035.vdocuments.net/reader035/viewer/2022070604/54b484ab4a7959fd348b45a8/html5/thumbnails/7.jpg)
ANET Workshop
Recording Studio that Spanned a ContinentMontreal-Los Angeles for the AES 109th Convention, Sept. 23, 2000
McGill Redpath Hall, Montreal
USC Norris Theatre, LA
12 channels, 96kHz, 24
bits/channel
![Page 8: History of Internet Audio Experiments at McGill](https://reader035.vdocuments.net/reader035/viewer/2022070604/54b484ab4a7959fd348b45a8/html5/thumbnails/8.jpg)
ANET Workshop
Microphone Configuration
![Page 9: History of Internet Audio Experiments at McGill](https://reader035.vdocuments.net/reader035/viewer/2022070604/54b484ab4a7959fd348b45a8/html5/thumbnails/9.jpg)
ANET Workshop
3 Mbps
![Page 10: History of Internet Audio Experiments at McGill](https://reader035.vdocuments.net/reader035/viewer/2022070604/54b484ab4a7959fd348b45a8/html5/thumbnails/10.jpg)
ANET Workshop
27.6 Mbps
![Page 11: History of Internet Audio Experiments at McGill](https://reader035.vdocuments.net/reader035/viewer/2022070604/54b484ab4a7959fd348b45a8/html5/thumbnails/11.jpg)
ANET Workshop
Some Modern History
Internet2 Initiative: World's First Remote Barbershop Quartet (Nov. 2000) multi-location quartet; each of the 4 singers in different cities, conductor in 5th network delay variances prevented singers from hearing each other or conductor
Internet2 Initiative: Music Video Recording via Internet2 (Nov. 2000) multi-location music video recording session using real-time streaming video musicians simultaneously connected via timing tracks to a mixing board
Chris Chafe: QoS Enabled Audio Teleportation (Nov. 2000) CD quality sound (750 kbps) of 2 separated musicians in Dallas streamed to Stanford musicians played "together" in same space (Stanford) but delay was severe
John Wawrzynek: Network Musical Performance (May 2001) gestural coding (e.g. MIDI) used to manage data for distributed musical performance musicians at Berkeley and CalTech, playing on MIDI keyboards; local feedback only
![Page 12: History of Internet Audio Experiments at McGill](https://reader035.vdocuments.net/reader035/viewer/2022070604/54b484ab4a7959fd348b45a8/html5/thumbnails/12.jpg)
ANET Workshop
"Frères Jacques" experimentMontreal, RISQ 2000 Conference
"low-latency" MPEG-2 videoconferencing
![Page 13: History of Internet Audio Experiments at McGill](https://reader035.vdocuments.net/reader035/viewer/2022070604/54b484ab4a7959fd348b45a8/html5/thumbnails/13.jpg)
ANET Workshop
"Playing Together" experimentNew York-Ottawa, Remote Masters class, Dec. 8, 2000
used Litton MPEG-2 codec @ 10 Mbps
Pinchas Zuckerman, Columbia U, NYC
Wu Ji, Canarie ARDNOC, Ottawa
![Page 14: History of Internet Audio Experiments at McGill](https://reader035.vdocuments.net/reader035/viewer/2022070604/54b484ab4a7959fd348b45a8/html5/thumbnails/14.jpg)
ANET Workshop
Quality vs. Latency
near-equivalent quality of encoded video at greatly reduced bandwidth
but... compression takes time
0
50
100
150
200
250
300
uncompressedSDI
M-JPEG/DV MPEG-4
Bandwidth (Mbps)
![Page 15: History of Internet Audio Experiments at McGill](https://reader035.vdocuments.net/reader035/viewer/2022070604/54b484ab4a7959fd348b45a8/html5/thumbnails/15.jpg)
ANET Workshop
Tolerated Synchronicity
compress transmit
100 20 4030 50 7060 80msspeed of
light: Mtl-LAnet latency
Mtl-LAnetworks
smallensemble
large ensemble
lip synch
musicalactivity
* This figure is a theoretical “best-case” scenario. Actual performance may vary based on local hardware. The manufacturer makes no warranties, either express or implied, and will not be held liable for damage resulting from use of the product. Certain restrictions may apply. Offer void where prohibited by law. Batteries not included.
MPEG*acquire
![Page 16: History of Internet Audio Experiments at McGill](https://reader035.vdocuments.net/reader035/viewer/2022070604/54b484ab4a7959fd348b45a8/html5/thumbnails/16.jpg)
Latency:Latency: The Interaction-KillerThe Interaction-Killer
or put another way...
![Page 17: History of Internet Audio Experiments at McGill](https://reader035.vdocuments.net/reader035/viewer/2022070604/54b484ab4a7959fd348b45a8/html5/thumbnails/17.jpg)
ANET Workshop
Interactive Success!
Chris Chafe: SoundWIRE (August 2001) streaming pro audio from remote locations for musical collaboration UDP/IP streaming: good results between Armonk, NY and Stanford round trip delay 125ms, musicians able to "catch-up" during pauses
![Page 18: History of Internet Audio Experiments at McGill](https://reader035.vdocuments.net/reader035/viewer/2022070604/54b484ab4a7959fd348b45a8/html5/thumbnails/18.jpg)
ANET Workshop
The future according to Sympatico?
![Page 19: History of Internet Audio Experiments at McGill](https://reader035.vdocuments.net/reader035/viewer/2022070604/54b484ab4a7959fd348b45a8/html5/thumbnails/19.jpg)
ANET Workshop
For those with DSL at home...For those with DSL at home...
yeah, right!(tu n'es pas
sérieux!)
![Page 20: History of Internet Audio Experiments at McGill](https://reader035.vdocuments.net/reader035/viewer/2022070604/54b484ab4a7959fd348b45a8/html5/thumbnails/20.jpg)
ANET Workshop
So what does it take?
much bigger network pipes ("abusive bandwidth") need Fast Ether (100Mbps) for audio GigE (1000Mbps) for video more efficient network topologies – hops add delay
![Page 21: History of Internet Audio Experiments at McGill](https://reader035.vdocuments.net/reader035/viewer/2022070604/54b484ab4a7959fd348b45a8/html5/thumbnails/21.jpg)
ANET Workshop
So what does it take?
much bigger network pipes ("abusive bandwidth") need Fast Ether (100Mbps) for audio GigE (1000Mbps) for video more efficient network topologies – hops add delay
more powerful hardware CPU speed (fast process switching, especially for mixed media) pro-audio interfaces with small buffers
![Page 22: History of Internet Audio Experiments at McGill](https://reader035.vdocuments.net/reader035/viewer/2022070604/54b484ab4a7959fd348b45a8/html5/thumbnails/22.jpg)
ANET Workshop
So what does it take?
much bigger network pipes ("abusive bandwidth") need Fast Ether (100Mbps) for audio GigE (1000Mbps) for video more efficient network topologies – hops add delay
more powerful hardware CPU speed (fast process switching, especially for mixed media) pro-audio interfaces with small buffers
more masochistic researchers
![Page 23: History of Internet Audio Experiments at McGill](https://reader035.vdocuments.net/reader035/viewer/2022070604/54b484ab4a7959fd348b45a8/html5/thumbnails/23.jpg)
ANET Workshop
Sometimes things work...
![Page 24: History of Internet Audio Experiments at McGill](https://reader035.vdocuments.net/reader035/viewer/2022070604/54b484ab4a7959fd348b45a8/html5/thumbnails/24.jpg)
ANET Workshop
Sometimes things work...
and sometimes they don't
note: look of extreme anguish
![Page 25: History of Internet Audio Experiments at McGill](https://reader035.vdocuments.net/reader035/viewer/2022070604/54b484ab4a7959fd348b45a8/html5/thumbnails/25.jpg)
ANET Workshop
Distributed Violin DuetMcGill-Université de Montréal for RISQnet Conference, Nov. 21, 2001
connected two Montreal universities via IP over 1 Gbps fiber link
uncompressed audio and video
< 30ms latency
![Page 26: History of Internet Audio Experiments at McGill](https://reader035.vdocuments.net/reader035/viewer/2022070604/54b484ab4a7959fd348b45a8/html5/thumbnails/26.jpg)
ANET Workshop
"The medium is the message" Marshall McLuhan
Cooperstock's Corollary:
"The medium must be faster than the message."
![Page 27: History of Internet Audio Experiments at McGill](https://reader035.vdocuments.net/reader035/viewer/2022070604/54b484ab4a7959fd348b45a8/html5/thumbnails/27.jpg)
ANET Workshop
Size does matterMontreal-Ottawa (NRC), Distance Teaching Trial, Feb. 20, 2002
connected McGill to National Research Council (Ottawa) with SDI video and multichannel 96kHz/24bit audio
display on 50" plasma screen (near life-size)
![Page 28: History of Internet Audio Experiments at McGill](https://reader035.vdocuments.net/reader035/viewer/2022070604/54b484ab4a7959fd348b45a8/html5/thumbnails/28.jpg)
ANET Workshop
Distributed Jazz JamMontreal-Stanford (CCRMA), Research Demo, Oct. 24, 2002
uncompressed audio and video @ < 50ms latency
![Page 29: History of Internet Audio Experiments at McGill](https://reader035.vdocuments.net/reader035/viewer/2022070604/54b484ab4a7959fd348b45a8/html5/thumbnails/29.jpg)
ANET Workshop
Remote Video Interpretation (Sign Language)Montreal-Fredericton (UNB), research trials, 2003
DV (patient, physician) exchanged with remote sign-language interpreter
physician's setup
patient's setup
![Page 30: History of Internet Audio Experiments at McGill](https://reader035.vdocuments.net/reader035/viewer/2022070604/54b484ab4a7959fd348b45a8/html5/thumbnails/30.jpg)
ANET Workshop
Vibrosensory TransmissionVRQ demo, March 8, 2004
![Page 32: History of Internet Audio Experiments at McGill](https://reader035.vdocuments.net/reader035/viewer/2022070604/54b484ab4a7959fd348b45a8/html5/thumbnails/32.jpg)
ANET Workshop
Milestones Ahead
multicast traffic shaping region-of-interest client selection multichannel echo-supression/cancellation
![Page 33: History of Internet Audio Experiments at McGill](https://reader035.vdocuments.net/reader035/viewer/2022070604/54b484ab4a7959fd348b45a8/html5/thumbnails/33.jpg)
Self-Signal Isolation: Echo-suppression
the big problem in videoconferencing same issue for video and vibrosensory data
![Page 34: History of Internet Audio Experiments at McGill](https://reader035.vdocuments.net/reader035/viewer/2022070604/54b484ab4a7959fd348b45a8/html5/thumbnails/34.jpg)
achieving high-fidelity distributed interaction, at levels of presence and latency that support the
most demanding applications and doing so in spite of sensor and bandwidth limitations
![Page 36: History of Internet Audio Experiments at McGill](https://reader035.vdocuments.net/reader035/viewer/2022070604/54b484ab4a7959fd348b45a8/html5/thumbnails/36.jpg)
http://ultravideo.mcgill.edu