to the edge with china: explorations in network performance...to the edge with china: explorations...

To the Edge with China: Explorations in Network Performance

Juan-Pablo Caceres, Robert Hamilton, Deepak Iyer, Chris Chafe, and Ge Wang{jcaceres, rob, ideepak, cc, ge}@ccrma.stanford.edu

Center for Computer Research in Music and Acoustics (CCRMA), Stanford UniversityThe Knoll, 660 Lomita Dr., Stanford, CA 94305, U.S.A.

Abstract—The complex nature of distributed network-based musical performance served as the starting pointfor the Stanford University SoundWIRE group’s 2008collaboration with Peking University in Beijing, China.In planning and executing the multi-ensemble networkedconcert entitled “Pacific Rim of Wire” at Stanford onApril 29, 2008, musicians and engineers from Stanford andBeijing undertook issues—technical and musical—rangingfrom the use of incompatible networking address protocolsto the synchronization of performers, human and computer,across a 6000 mile span of network. This paper outlinesthe technical and musical strategies employed to supportthe production’s demands, as well as specific methodologiesemployed for the realization of Terry Riley’s In C.

I. INTRODUCTION

Musical and technical strategies in local performancesneed to be re-factored when musicians are separatedby long physical and acoustical distances. We addresssome of these issues in a large scale concert betweentwo venues separated by 6000 miles: Stanford, Californiaand Beijing, China. The SoundWIRE research groupat Stanford [1] has been organizing and implementingtechnologies for real-time distributed performance for thelast several years. The “Pacific Rim of Wire” concerthighlights a number of key musical and technical chal-lenges that still loom large above current attempts toperform using communication technologies.

The technical and musical demands of the “PacificRim of Wire” collaboration required the initiation ofnew types of network connectivity, the development ofsoftware to deal with next-generation Internet backbones,the implementation of musical strategies to deal withnetwork-induced acoustical delays, and the organizationof network-based metronomic systems with which alaptop orchestra synchronizes its performance with anensemble of acoustic instruments.

Performing with ensembles based in other countriesaffords musicians and researchers the opportunity toexplore edges of musical and technological strategies.The “Pacific Rim of Wire” performances included anensemble of traditional Chinese instruments perform-ing with the Stanford New Ensemble, using traditionalWestern instruments, all mixed together with the newlyminted Stanford Laptop Orchestra (SLOrk) [2]. Such acollaboration also offers a unique opportunity to explorethe manner in which traditional pieces can be performedin this medium. As part of this effort, a performance ofTerry Rilley’s In C is showcased.

II. COLLABORATION WITH CHINA

In April of 2008, the annual Stanford Pan-Asian MusicFestival [3] turned its focus towards China, featuringits music and musicians from that country. Seeking aforward-looking production that would bridge the geo-graphical distance between American and Chinese cul-tures, maestro Jindong Cai and CCRMA faculty memberand SLOrk director Ge Wang conceived the “Pacific Rimof Wire”, a networked musical collaboration betweenmusicians at Stanford and in Beijing, performing togetheracross fast Internet connections. Based on the ongoingresearch of CCRMA’s SoundWIRE group, directed byChris Chafe, the “Pacific Rim of Wire” concert wouldmake use of SoundWIRE’s JackTrip software [4] runningon Linux, allowing the use of low-latency uncompressedbi-directional multi-channel audio streams.

Working with Kenneth Fields of the Peking Univer-sity and China’s Central Conservatory of Music, net-work testing between CCRMA and the Computer Sci-ence/Networking department of Peking University ini-tially showed encouraging results, but network traffic pat-terns and insufficient stability left a number of problemswith no immediate resolution. With the goal of betterunderstanding the specific issues at hand, members ofthe SoundWIRE group traveled to Beijing to meet withtheir team and to run additional tests.

Although Peking University serves as a key hub inChina’s CERNET2 next-generation education and re-search network, testing conducted from a multimediaconference room within the University grounds showedthat the Stanford-Beijing connection was in fact notutilizing the CERNET2 connection, running instead onChina’s standard first-generation Internet. As CERNET2required the use of the IPv6 protocol, and as JackTriponly supported IPv4, some software changes were re-quired.

A. Reaching China: a path to CERNET2 and IPv6

Connecting to Beijing with the requirements for a high-quality low-latency musical experience requires the useof the fastest and most stable high-bandwidth networksand backbones available. CERNET2 [5] is the most well-provisioned Internet backbone in China, with speeds of2.5∼10 Gbps (Giga bit per second). CERNET2 is a nativeIPv6 [6] backbone; to connect to it, hosts are required touse IPv6 protocols.

ARTECH 2008, 4th International Conference on Digital Arts, 7- 8 November, Portuguese Catholic University, Porto

Page 61

!"#$%&'()*+,+$-

./0/

Fig. 1. Simplified network path between Stanford and Peking University during the “Pacific Rim of Wire” concert

The development of IPv6 started with the need toaddress the scaling problems caused by Internet growth,and hence the need for more IP numbers than the onespresently provided by IPv4, the most widespread protocolcurrently in use. IPv4 uses a 32 bit address space, whileIPv6 was designed with 128 bits, with a potential (as-suming 100% efficiency) of addressing 3.4 · 1038 nodes.Based on even the most pessimistic estimates, IPv6 mayprovide over 1500 addresses per square foot of the earth’ssurface [6].

To communicate with CERNET2, Stanford Universitypeers via Internet2 [7], the U.S. research and educationnetwork. IPv6 static routing was set up, allowing directcommunication with the IPv6 Internet from Internet2’sAbilene backbone.1

Stanford University’s network was still exclusivelyIPv4 and required that Stanford hosts use tunneling toconnect to the IPv6 router. The portion of the connectionthat runs inside the university, i.e. from the CCRMAcomputer to the IPv6 router, runs within a tunnel: overthe piece of network that only understands IPv4, the IPv6packets “travel inside” IPv4 ones. Packets are encapsu-lated and decapsulated at each end of the tunnel—the hostand the router. Across the rest of the network packetstravel as normal IPv6 ones. In testing, the overhead ofthe encapsulation/decapsulation was found to be insignifi-cant, without any important processing spikes on the IPv6Stanford router, translating into no additional latency.

Figure 1 shows a simplified version of the network path

1The static router was specifically connected with the CENIC HPR(high-performance research network) 10Gbps backbone [8].

between Stanford Campus and Peking University duringthe concert. The packets first travel from Stanford to LosAngeles, then cross the Pacific Ocean, passing throughKorea before finally reaching China’s CERNET2. Theconnection was symmetric—packets in both directionsfollow the same network path—and highly stable. Theround trip time (RTT), measured with ping6, was ∼220milliseconds.

The “Pacific Rim of Wire” performance made useof full-duplex, uncompressed audio (thus avoiding ad-ditional latency and audio artifacts of perceptual-audiocompression), at 16bits and a 44.1KHz sampling rate,equivalent to Compact Disc quality. Two channels ofaudio were sent to the concert hall stereo PA speakerswith one extra channel used for synchronization in theperformance of Terry Rilley’s In C (see Sec. III). Thesoftware used was an IPv6 version of JackTrip2 [10],[4], a system for multi-channel uncompressed audiostreaming.

Video streaming was done using the free open-sourceand cross platform software VLC [11], that supportsvarious video codecs and streaming protocols. The in-put stream from the digital video camera was set tohigh-definition quality (720x480 pixel resolution). Thedeinterlaced video was transcoded using the MPEG4codec and streamed as UDP packets, both operationsprovided by VLC. As there is an inherent trade-offbetween compression and bandwidth, the encoder settingswere selected to minimize processing and thereby avoid

2The machine used in the performance was running Fedora distribu-tion with Planet CCRMA [9].


Page 62

Fig. 2. “Pacific Rim of Wire” concert: performance of Terry Riley’s In C. Onstage: Stanford Laptop Orchestra (SLOrk) and Stanford NewEnsemble. Onscreen: real-time video of musicians at Peking University.

latency, making use of available bandwidth as needed.Camera movements in concerts are typically not very

aggressive (low-motion). Video encoding exploits thisfact effectively with a peak bandwidth utilization that didnot exceed 8 Mbps (Mega bit per second) with cameramovement and less than 1 Mbps without.

The total video latency was on the order of one second,composed of image capture delay in the camera, networkdelay and encoding/decoding time. The audio and videowere not synchronized as both were streamed separatelyand had significantly different latencies. Although thismight seem disruptive for the performers, our previousexperience in network performance shows that musiciansusually don’t look at the video when they perform;it serves primarily the purpose of providing an expe-rience for the audience—while also adding additionalreassurance and comfort to the musicians during setup,discussion and other communication needs. Until videocan match audio in terms of latency, the trade-offs forsynchronizing video and audio are a significantly higherbandwidth utilization (for uncompressed video) vs. acorrespondingly longer latency for audio (to match videocodec lags).

With audio, any small dropouts or artifacts can be verynoticeable and potentially annoying, focusing attention onglitches and sound quality [12]. In comparison, dropouts

and latency in video delivery seem more tolerable. Whileuncompressed video is preferable, its enormous band-width demands and the difficulty of obtaining videocameras with fast capture motivated us to employ videoencoding this time around. Uncompressed should providea much better solution for the future.

B. Nested Rims of Wire and Laptop Orchestra

Within our wide-area, ocean-spanning network con-necting Stanford and Beijing, an onstage local areanetwork at Stanford University kept the computers inthe laptop orchestra tightly synchronized. Each of the20 hosts was connected via wireless Ethernet to an802.11n switch, and used Open Sound Control [13],[14] to transmit low-latency control messages across theensemble. Equipped with a custom hemispherical speakerarray and paired with a human performer, each laptopstation represented a single, localized meta-instrumentwith its own sonic presence and identity. With 20 suchstations, the Stanford Laptop Orchestra leveraged itscapability to project an ocean of sound, while fusing itwith that of acoustic instruments playing on the samestage (Fig. 2). In the “Pacific Rim of Wire”, these twoaspects of synchronization and sound projection werefully explored in our networked performance of TerryRiley’s In C. Here the laptop orchestra contributed a point


Page 63

of local synchronization as well as a centralized, dynamicsonic “anchor” for musicians at Stanford and Beijing. Inthe next section, we present and discuss our computer-mediated, wide-area, and yet rather traditional realizationof In C, combining SoundWIRE, laptop orchestra, andacoustic musicians.

III. A BI-LOCATED TERRY RILEY’S In CTo best showcase the trans-continental collaboration

between Stanford and Peking University, the decision wasmade to perform an ensemble musical work featuring per-formers located in both locations. A performance of TerryRiley’s In C, led by Michael Bussiere, was performed byparticipants of the 2008 ANET II (High Quality Audioover Networks) Summit at the Banff Centre for the Arts[15]. This experience suggested that Riley’s work mightprove a good choice for the Beijing collaboration.

For the performance between Stanford and Beijing,instrumental performers at Stanford playing a varietyof traditionally Western instruments joined instrumentalperformers at Beijing, playing a variety of traditionalChinese instruments, and the Stanford Laptop Orchestra.The choice to perform In C was in hindsight evenmore fitting than previously intended as Terry Rileyhimself brought the work to Beijing in 1989 where heperformed and recorded the work with Chinese musiciansof the Shanghai Film Orchestra performing on traditionalChinese instruments [16].

A. Performance DetailsComposed and premiered in 1964, Riley’s In C con-

sists of 53 melodic patterns—or cells—each composedwith a loose tonal center based around the pitch-class C.Instrumentation for In C is not set by the composer andcan be performed by virtually any instrument capable ofproducing diatonic pitches. The instructions for the scorerequire that beginning with the first musical cell, eachpattern must be played in sequence by each performer,moving through the sequence of cells at their own discre-tion. Performers may choose to repeat cells as many timesas they wish and may also pause between performance ofdifferent cells. The work ends after all performers havearrived at the final cell of the composition.

Riley’s instructions in the written score [17] includethe following:

Each pattern can be played in unison orcanonically in any alignment with itself or withits neighboring patterns. One of the joys of In Cis the interaction of the players in polyrhythmiccombinations that spontaneously arise betweenpatterns. Some quite fantastic shapes will ariseand disintegrate as the group moves throughthe piece when it is properly played. [. . . ] Theensemble can be aided by the means of aneighth note pulse played on the high c’s ofthe piano or on a mallet instrument. [. . . ] Allperformers must play strictly in rhythm and

it is essential that everyone play each patterncarefully. It is advised to rehearse patterns inunison before attempting to play the piece, todetermine that everyone is playing correctly.

In C presents several challenges for a distributednetwork performance context, one of the most impor-tant being that it requires tight synchronization betweenmusicians. It is a well known phenomenon that rhythmicsynchronization is problematic when the acoustic delaybetween musicians becomes too long, with significantproblems occurring at delay thresholds of just 20 mil-liseconds [18]. Faced with a single-direction base delaypath between Beijing and Stanford of approximately110 milliseconds—a delay already significantly greaterthan this 20 millisecond threshold—it was clear that thegoal of a rhythmically-synchronized distributed ensembleperforming with a signal path greater than 6000 mileswould require a different solution.

B. Distributing the PulseAs Riley’s instructions indicate, one of In C’s most

striking features is the work’s ability to create com-plex polyrhythms through the repetition and alignmentof each musical phrase into patterns of tight rhythmicsynchronization. The use of an audible metronomic pulseallows performers to concentrate on the phrasing andalignment of tonal and rhythmic patterns between theirown individual performance and the performances ofeach member of the ensemble, safe in the knowledgethat each musician is locked in step with the samepulse. However, while a metronomic pulse will clearlyaid performers sitting in the same performance space,the introduction of a significant signal path latency andpotentially a dynamic latency effectively renders a staticmetronomic pulse useless: this leaves the two ensemblesout of step.

The solution put forth to provide a stable pulse forthe entire distributed ensemble for this performance ofIn C—the technique of feedback locking [19]—relies ona metronomic pulse transmitted to both performance lo-cations, with its rate based on the current dynamic signalpath delay between Stanford and Beijing. The tempo ofthe audible eighth note pulse, set, performed and adjustedby a musician listening to a hidden audio channel (i.e.not broadcast to the ensemble and audience) is basedon the round-trip network feedback. In this manner, atight rhythmic alignment between both locations canbe maintained. The RTT in the “Pacific Rim of Wire”concert was ∼220 milliseconds. A simple calculationserves to obtain the tempo for the performance in beatsper minute (BPM):

Tempo =60 (seconds in a minute)

0.220 (seconds)= 272.73 (BPM)

This result is used for the tempo of the eighth note(!) pulse. The piece was performed at approximately270 BPM.


Page 64

Stanford

Stanford

Stanford

Beijing

Beijing

Beijing

&3 ‰ jœ œ œ ‰ jœ œ œ

4‰ jœ œ œ5

œ œ jœ ‰

&3‰ jœ œ œ ‰ jœ œ œ

4‰ jœ œ œ5

œ œ jœ ‰&

3‰ jœ œ œ ‰ jœ œ œ ‰ jœ œ œ ‰ jœ œ œ

&3 ‰ jœ œ œ ‰ jœ œ œ ‰ jœ œ œ ‰ jœ œ œ

Fig. 3. Feedback locking In C. The center of the figure shows in blue squares the pulse as heard in both locations. A performer at Stanford lockswith its own feedback. The top part shows the musical cells as performed at Stanford and heard from Beijing. The bottom shows the musicalcells performed in Beijing and the ones heard from Stanford.

Figure 3 shows the feedback locking approach, with ametronomic pulse originating at Stanford sent to perform-ers in Beijing. The blue square at the center of the figurerepresents the pulse. The horizontal time axis illustratesthe arrival of the pulse to each location. Three musicalcells performed by one performer at each location areshown as well as an example of an interesting extensionof Riley’s desire for a variable interlocking composi-tion: at any given time, the performance will soundsignificantly different in each location. This extensionof the composer’s intent is happily furthered by the actof network distribution, simultaneously introducing twounique variations on the piece into the world duringany given performance. Figure 4 shows the compoundmusical figure heard at one point in performance, wherea performer in Beijing loops musical Cell 3 four timesagainst a performer in Stanford performing two loops ofCell 3, immediately followed by Cell 4 and Cell 5.

Stanford

Beijing

Composite phrases as heard in Beijing

Stanford

Beijing

Composite phrases as heard in Stanford

&

&

6 Œ œ œ

‰ jœ œ œ

jœ ‰ œ œ

‰ jœ œ œ

jœ ‰ œ œ

‰ jœ œ œ

jœ ‰ œ œ

œ œ jœ ‰

jœ ‰ Œ

∑

&

&

11 ‰ jœ œ œ

‰ jœ œ œ

‰ jœ œ œ

‰ jœ œ œ

‰ jœ œ œ

‰ jœ œ œ

‰ jœ œ œ

œ œ jœ ‰

Fig. 4. Composite phrases resulting at Stanford (top staff) and Beijing(bottom staff)

C. ChucK Implementation

To create both a sonified metronomic pulse whichcould not only be easily tuned by the network engineerbut could also regulate timing for the 20 laptop per-formers, as well as a performable version of In C forLaptop Orchestra, a custom client-server implementationwas written in the ChucK language [20]. A slider onthe screen of a laptop in front of the network engineersimultaneously regulated both the audible metronomicpulse and an inaudible Open Sound Control data pulse,projected to each of the 20 laptop performers over awireless 802.11n network. Performers on each laptopwould select an instrument at the start of the performancefrom a selection of digital physical models in the Syn-thesis ToolKit (STK) [21]. As seen in Figure 5, eachlaptop performer was presented a small GUI windowwith controls to start and stop musical cells numbering1-53, as well as a toggle switch to loop the selected cell.The timing for each individual note was clocked to theOpen Sound Control pulse and subsequently was alwaysperfectly in sync with the network-synchronized audiblemetronome. In this way, the laptop instruments served asan audible reinforcement to the metronomic pulse.

IV. CONCLUSIONS

The application of recent low-latency audio transmis-sion technologies to a real musical scenario served asa good example of the challenges facing musicians andengineers alike in the realization of real-time networked-based musical performance. The showcasing of thesetechniques resulting in a concert between Stanford Uni-versity and China’s Peking University brought forth aseries of network-based obstacles which required novel


Page 65

Fig. 5. In C’s ChucK client interface for laptop performer. The pictureshows the interface as used by one of the 20 SLOrk performers.

solutions to produce satisfying engineering and musicalresults.

The successful “Pacific Rim of Wire” concert hasshown the SoundWIRE group that with the addition ofnetworking and distributed performance practice, it ispossible to “enhance” the experience of existing musicalrepertoire. The presentation of Terry Riley’s In C, ex-panded Riley’s own concept of a loose-but-synchronousensemble to include dual related but significantly differ-ent performances in each distributed location. The appli-cation of musical strategies that use network-based time-delay to synchronize and to distribute musical patternswas successfully applied in this performance, paving theway for future distributed performances of rhythmically-strict works. The successes outlined above have shownus that through the use of these techniques, we cansuccessfully synchronize musicians through a consistentdistribution of the musical pulse.

Network performance has proven to be also a good op-portunity to experiment with non-traditional instrumentalcombinations. A laptop orchestra performing in real-timewith a traditional Chinese erhu, combined with westernorchestral instruments may not be the most standardinstrumental ensemble, but through the use of distributedperformance practices, such a grouping, even performedin a small performance space, is made possible. In thismanner, the use of powerful networking technologieshas shown itself as an effective paradigm for musicalperformance, well worthy of future technological andmusical efforts.

ACKNOWLEDGMENTS

Ma Hao, Kenneth Fields and Haku Wang from PekingUniversity, Lea Roberts of the Stanford’s Network-ing Systems group, Carr Wilkerson, Fernando Lopez-Lezcano and Chryssie Nanou from CCRMA, ScottGresham-Lancaster and maestro Jindong Cai.

REFERENCES

[1] (2008) SoundWIRE research group at CCRMA, StanfordUniversity. [Online]. Available: http://ccrma.stanford.edu/groups/soundwire/

[2] G. Wang. (2008) Stanford Laptop Orchestra (SLOrk). [Online].Available: http://slork.stanford.edu/

[3] (2008) Stanford Pan-Asian Music Festival. [Online]. Available:http://panasianmusicfestival.stanford.edu/

[4] J.-P. Caceres. (2008) Jacktrip: Multimachine jam sessions over theInternet2. [Online]. Available: http://ccrma.stanford.edu/groups/soundwire/software/jacktrip/

[5] (2008) CERNET2. [Online]. Available: http://www.edu.cn/cernet%202 1382/

[6] L. L. Peterson and B. S. Davie, Computer Networks: A SystemsApproach, 3rd Edition, 3rd ed. Morgan Kaufmann, May 2003.

[7] (2008) Internet2. [Online]. Available: http://www.internet2.edu/[8] (2008) Corporation for Education Network Initiatives in

California, CENIC. [Online]. Available: http://www.cenic.org/[9] F. Lopez-Lezcano. (2008) Planet CCRMA. [Online]. Available:

http://ccrma.stanford.edu/planetccrma/software/[10] C. Chafe, S. Wilson, R. Leistikow, D. Chisholm, and G. Scavone,

“A simplified approach to high quality music and sound over IP,”in Proceedings of the COST G-6 Conference on Digital AudioEffects (DAFX-00), Dec. 2000.

[11] (2008) VideoLAN (VLC). [Online]. Available: http://www.videolan.org/

[12] S. Gulliver and G. Ghinea, “The perceptual and attentive impactof delay and jitter in multimedia delivery,” Broadcasting, IEEETransactions on, vol. 53, pp. 449–458, 2007.

[13] M. Wright, A. Freed, and A. Momeni, “OpenSound Control: Stateof the art 2003,” in NIME ’03: Proceedings of the 3th internationalconference on New Interfaces for Musical Expression, Montreal,Canada, 2003, pp. 153–159. [Online]. Available: http://cnmat.berkeley.edu/publications/open sound control state art 2003

[14] M. Wright. (2002) Open sound control 1.0 specification. [Online].Available: http://opensoundcontrol.org/spec-1 0

[15] (2008) The Banff Centre Programs—ANET II: High QualityAudio over Networks Summit. [Online]. Available: http://www.banffcentre.ca/programs/program.aspx?id=721

[16] D. M. Liang, T. Riley, and Shanghai Film Orchestra, “In C,” AudioCD, Nov. 1992.

[17] T. Riley, “In C,” Musical score, 1964.[18] C. Chafe and M. Gurevich, “Network time delay and ensemble

accuracy: Effects of latency, asymmetry,” in Proceedings of theAES 117th Convention, 2004.

[19] J.-P. Caceres and A. B. Renaud, “Playing the network: the use oftime delays as musical devices,” in Proceedings of InternationalComputer Music Conference, Belfast, Northern Ireland, 2008, pp.244–250.

[20] G. Wang and P. R. Cook, “ChucK: A concurrent, on-the-fly,audio programming language,” in Proceedings of InternationalComputer Music Conference, Singapore, 2003.

[21] P. R. Cook and G. P. Scavone. (2007) The Synthesis Toolkitin C++ (STK). [Online]. Available: http://ccrma.stanford.edu/software/stk/


6

to the edge with china: explorations in network performance...to the edge with china: explorations...

Documents