8 2 I E E E S O F T W A R E P u b l i s h e d b y t h e I E E E C o m p u t e r S o c i e t y 0 7 4 0 - 7 4 5 9 / 0 7 / $ 2 0 . 0 0 © 2 0 0 7 I E E E

open sourceE d i t o r : C h r i s t o f E b e r t ■ V e c t o r C o n s u l t i n g ■ c h r i s t o f . e b e r t @ v e c t o r - c o n s u l t i n g . d e

The importance of visual and graphical ap-plication frameworks such as MVC(model-view-controller) and Interviews tothe development of software frameworksis well known. But the audio and musiccomputing field also has a long, rich his-

tory of frameworks for application builders.And it’s here we find CLAM, winner of the 2006

ACM Open Source MultimediaSoftware Competition.

CLAM stands for C++ Li-brary for Audio and Music, aname that doesn’t reflect its cur-rent status as a full-fledgedframework. CLAM originated inan effort to organize a repositoryof audio-processing algorithms,and the early development goalwas simply to offer a class li-

brary. But it soon became obvious that the projectwas of interest to a much broader audience.CLAM adopted the GNU General Public License(GPL) and became public in the context of theAGNULA (A GNU/Linux Audio) distribution.

What is CLAM?CLAM is a C++ framework that offers a com-

plete R&D platform for the audio and music do-

main. It includes an abstract model for audio sys-tems, a repository of processing algorithms anddata types, and a number of tools such as audioand MIDI (musical instrument digital interface)I/O. Developers can exploit all these features tobuild cross-platform applications or rapid proto-types for testing signal- and media-processing al-gorithms and systems. Everything in CLAM—from the low-level, operating-system accessprimitives to the GUI—is cross-platform and reg-ularly compiled under several Linux distributionsas well as Mac OS X and Windows.

The framework also offers several stand-alone applications, including

■ SMSTools, a graphical tool for audio analy-sis, synthesis, and transformation;

■ Annotator, an application for semiautomaticmusic analysis that includes goodies such asautomatic chord recognition;

■ Salto, a real-time brass-instrument synthe-sizer; and

■ Network Editor, a visual-building tool.

The Network Editor has become CLAM’s flag-ship application. It lets developers dynamicallypatch process graphs by inserting ready-to-useblackbox components in an application through

CLAM: A Framework for Audio and Music Application Development

Xavier Amatriain, University of California, Santa Barbara

Music and open source? Are there any connections between these two? After several years ofdescribing tools, middleware, services, and applications up the open software systems stack, let’slook now at frameworks. CLAM is an OSS framework for processing audio and music. It letsdevelopers create and manage complex audio effects and applications on top of them, whetherfor experiments, research, or business. And it’s built on OSS libraries. Xavier Amatriain high-lights the major features of this award-winning framework and points to a variety of links forfurther information. Let there be music! —Christof Ebert

J a n u a r y / F e b r u a r y 2 0 0 7 I E E E S O F T W A R E 8 3

OPEN SOURCE

a GUI (see figure 1). Furthermore, youcan use the resulting XML file to createstand-alone applications without writinga single line of code.

CLAM isn’t the only open source en-vironment for audio and music develop-ment. Table 1 compares it to several

other environments. Each environmentfocuses on a particular aspect of the au-dio domain and how to present it to theuser. In many senses, CLAM is a super-set of the other environments because itoffers a metamodel and tools for all pos-sible applications, ranging from audio

analysis to synthesis and real-time pro-cessing. You can also use it as a completeapplication development framework, aclass library, or a rapid-prototyping en-vironment with a GUI.

Audio and music open source libraries

Figure 2 shows the components of atypical audio application. To offer sucha wide range of services, CLAM relieson third-party open source audio tools.CLAM and its metamodel act as a glu-ing point and offer a common serviceinterface.

Streaming audio to and from thesound card is one of the most demand-ing audio-application services. If youneed to offer it both efficiently andacross different platforms, the task canbe daunting. Fortunately, several opensource solutions exist. The most well-known and widely used is PortAudio, acomplete toolkit for cross-platform au-dio input and output. RtAudio is a sim-ilar library but more modest in its scope.

Handling sound files is a similarlycomplex service. Supporting the manyavailable formats (each with its own bit

Figure 1. The CLAM Network Editor. Developers select processing objects,which are CLAM’s atomic units, from the left menu categories. Then theyplace and connect the objects in the canvas to build a processing graph or network. Brown connections represent data flows; blue connections represent event-driven control flows.

Table 1Open source audio and music environments

Open Sound STK (SynthesisFeatures CLAM Jsyn Marsyas World Pure Data ToolKit in C++)

Language C++ Java C++ C++ C C++

Can be used Yes Yes Yes Yes No Yesfor stand-aloneapplication development

Visual-building Yes Yes No Yes Yes Noapplication

Description Complete Audio synthesis Conceptually Extensible Graphical Software toolkit software API for Java similar to CLAM programming programming for audio framework application but with fewer environment environment synthesis, for developing developers tools and a focus for interactive used mainly especially audio and music dealing with on music- audio processing for artistic physical modeling. applications. computer music. information and synthesis. purposes. It Doesn’t depend Offers a You can use retrieval and It allows for has a large user on external conceptual Jsyn to create audio analysis. visual patching base and is libraries.metamodel as applets for and high-level similar to the well as tools Web pages. C++ expressions. commercialand a visual Max/MSP builder. package.

Web page www.clam.iua. www.softsynth. http://opihi.cs. http://osw. http://puredata. http://ccrma.upf.edu com/jsyn/ uvic.ca/marsyas sourceforge.net info stanford.edu/

software/stk

8 4 I E E E S O F T W A R E w w w. c o m p u t e r. o r g / s o f t w a r e

OPEN SOURCE

rates, encodings, and so on) would be al-most unthinkable without relying on anexisting library. Fortunately, libsndfilesupports virtually any sound file format.But to support compressed formats,you’ll need to look elsewhere. ForMPEG1 (aka MP3) and MPEG2 decod-ing, you can use either Underbit’s MPEGAudio Decoder (MAD) or the mpg123 li-brary; for encoding, refer to Lame. Ifyou’d rather support a patent-free formatsuch as ogg, the libvorbis library will giveyou all the needed support for that com-pression technology, which many listen-ers consider superior to MP3.

Other typical audio-related servicesthat you might need are support forID3 metadata tags (offered by ID3libor MAD), for MIDI I/O (see PortMidi),and for the Open Sound Control net-working protocol, which is enabled byseveral implementations such as Osc-pack in C++.

Open source tools and libraries to build CLAM

We’ve developed CLAM entirely us-ing open source tools and third-party li-braries that relate not only to our partic-ular domain but also to general-purposedomains. Although the framework iscompletely multiplatform, we prefer todevelop on Linux using open sourcetools. It’s beyond this column’s scope todescribe all the tools and decisions in-volved, but figure 3 should give an over-

all picture of the process and how it’ssupported.

We had to overcome three nonaudio-related open source challenges. The firstone was storage. Apart from the audio-specific storage formats, applicationsneed to store different data such as con-figurations, metadata, or process out-puts. CLAM supports this requirementby basing all data storage on the popularXML format. In CLAM, every object inmemory has automatically built-in XMLpersistency. If your framework or appli-cation needs cross-platform open sourceXML support, you should look atlibxml and XercesC. We started usingXercesC because it was more developedand well supported in all platforms atthe time. However, we’ve since switchedto libxml for its greater efficiency andstability. If your application requiresonly event-based XML processing, takea look at Arabica, a toolkit that providesuniform access to a number of librariesincluding libxml and XercesC.

Second, we needed to build efficientcross-platform GUIs, and we needed torely on an existing toolkit. We first choseFltk, a light toolkit that can yield goodresults. But after the latest release of Qtfrom Trolltech, we’ve made it our pre-ferred toolkit. Once Trolltech solved theproblem of a GPL-compatible license forMS Windows, Qt stood out as a supe-rior choice.

Finally, we also needed an open

source solution to fast-Fourier-transformcomputing. Performing FFTs efficientlyis a major issue in many signal-process-ing-intensive software packages. Ar-guably, the most reliable open sourcepackage for the task is the Massachu-setts Institute of Technology’s FFTW. It’soffered on a dual-licensing scheme. Youcan use FFTW under the GPL conditionsor link it into proprietary software bypaying a licensing fee. If the license fee isan issue, you might want to check outthe Ooura FFT implementation, whichcan be almost as efficient in most cases.CLAM supports both.

A s you can see, the audio and musicfield is very active in the open sourcearena, where application frame-

works also flourish. In both these con-texts, CLAM is a long-term, relativelysuccessful OSS project that’s used opensource tools exclusively in its develop-ment. Even before its first stable 1.0 re-lease, CLAM has already proved its po-tential and quality. A framework’ssuccess tends to depend more on theusers than on the quality of the develop-ment itself. Nevertheless, if offeringgood quality software is the first step inattracting open source users, the almost200 registered users of the frameworkshould grow exponentially in the nearfuture.

Network protocol (OSC)

Libsndfile

Compressed soundfileI/O

PortAudio

MIDI I/O

LameMAD

Vorbis

SoundfileI/O

Soundcard input Soundcard output

PortAudioPlaylists

Browse

Extras

Settings

About

Audioapplication

core

PortMidi Oscpack

Figure 2. A typical audio application. Generic services are in red font; opensource libraries that can provide the service are in black font.

Automatictest

management

Automaticprofiling

management

Testing Profiling

Cross-platformbuild Scons

DebuggingGdbBug

tracking Mantis

ValgrindCppunit

Coding Text editorGcc

EfficiencyGuardianTestFarm

Figure 3. CLAM process activities (inblack) and tools to support them (inred).

J a n u a r y / F e b r u a r y 2 0 0 7 I E E E S O F T W A R E 8 5

OPEN SOURCE

AcknowledgmentsCLAM is a team effort that throughout the

years has included around 20 different contribu-tors with a stable core development team of threeor four developers. At this point, Pau Arumí andDavid García anchor the team in Barcelona withme in Santa Barbara. The project has receivedseveral grants, and the Catalan government issupporting its current development.

I thank Christof Ebert for his support in writ-ing this column.

Xavier Amatriain is research director of the Center forResearch in Electronic Art Technology at the University of Califor-nia, Santa Barbara. Contact him at xavier@create.ucsb.edu;www.create.ucsb.edu/xavier.

Related Web Sites

CLAM: www.clam.iua.upf.edu

Distributions and open source audio repositoriesAGNULA: www.agnula.orgLinux Sound: http://linuxsound.orgPlanet CCRMA: http://ccrma.stanford.edu/planetccrma/software

CLAM-related environmentsCREATE Signal Library: www.create.ucsb.edu/CSLJSyn: www.softsynth.com/jsyn/Marsyas: http://opihi.cs.uvic.ca/marsyasOpen Sound World: http://osw.sourceforge.netPure Data: http://puredata.infoSound Object Library: http://music.nuim.ie/musictec/SndObjSynthesis Toolkit in C++: http://ccrma.stanford.edu/software/stk

Audio librariesID3lib: http://id3lib.sourceforge.netLame: http://lame.sourceforge.netLibsndfile: www.mega-nerd.com/libsndfileLibvorbis: http://xiph.org/vorbisMPEG Audio Decoder: www.underbit.com/products/madMpg123: www.mpg123.deOscpack: www.audiomulch.com/~rossb/code/oscpackPortAudio: www.portaudio.comPortMIDI: www.cs.cmu.edu/~music/portmusicRtAudio: www.music.mcgill.ca/~gary/rtaudio

General interest libraries and toolsArabica: www.jezuk.co.uk/cgibin/view/arabicaBugzilla: www.bugzilla.orgDoxygen: www.stack.nl/~dimitri/doxygenEfficiencyGuardian: http://sourceforge.net/projects/efficiencyguardFFTW: www.fftw.orgFltk: www.fltk.orgGforge: http://gforge.orgLibxml: http://xmlsoft.orgMantis: www.mantisbt.orgOoura FFT: http://momonga.t.utokyo.ac.jp/~ooura/fft.htmlScons: www.scons.orgTestfarm: www.iua.upf.es/~parumi/testfarmTrolltech’s Qt: www.trolltech.com/products/qtXercesC: http://xml.apache.org/xercesc

Tomorrow'sPCs,handhelds, andInternet will usetechnology that exploitscurrent research in artificialintelligence. Breakthroughs inareas such as intelligentagents, the Semantic Web,data mining, and naturallanguage processing willrevolutionize your work andleisure activities. Read aboutthis research as it happens in IEEE Intelligent Systems.

ww

w.c

om

pu

ter.

org

/in

tell

igen

t/su

bsc

rib

e.h

tm

SEE THEFUTURE OFCOMPUTING

NOWin IEEEIntelligent Systems