Software Visualization with Audio Supported Cognitive Glyphs

Sandro Boccuzzo and Harald C. GallDepartment of Informatics, University of Zurich, Switzerland

{boccuzzo, gall}@ifi.uzh.ch

Abstract

There exist numerous software visualization techniques thataim to facilitate program comprehension. One of the mainconcerns in every such software visualization is to identifyrelevant aspects fast and provide information in an effectiveway. In previous work, we developed a cognitive visualiza-tion technique and tool called CocoViz that uses commonplace metaphors for an intuitive understanding of softwarestructures and evolution. In this paper, we address soft-ware comprehension by a combination of visualization andaudio. Evolution and structural aspects are annotated withdifferent audio to represent concepts such as design erosion,code smells or evolution metrics. We use audio conceptssuch as loudness, sharpness, tone pitch, roughness or oscil-lation and map those to properties of classes and packages.As such we provide an audio annotation of software entitiesalong their version history for software analysis and soft-ware browsing. Our first results with the prototype and asmall user study show that with this combination of visualand aural means we can facilitate program comprehensionand provide additional information that usually is not pro-vided by current visualization approaches.

1 Introduction

With the increasing complexity of software systems, pro-gram comprehension is a major concern in maintenance andevolution. The amount of data, the relationships betweenthe entities, and missing or out of date documentation makeit almost impossible for engineers to maintain an accurateunderstanding of an evolving system without effective toolsupport. A variety of project stakeholders are interested indifferent aspects of a system. A project manager for exam-ple might not be interested in the entire system, but onlyin a reflection of the state of a project. Other stakehold-ers such as auditors or customers might want to have deeperinsights into the project, while not even being allowed to ac-cess the source code. There is an opportunity in providingthem with visualizations that support their work and offer aquick and comprehensive status on a software project. Such

a software visualization needs to aggregate all the gatheredinformation about a project in an effective visual represen-tation.

With the CocoViz project we aim to enhance existingmaintenance and evolution analysis methods to present asoftware system in an intuitively understandable visualiza-tion. The benefit of such a cognitive visualization lies inrepresenting the context of interest with perceivable glyphs,abstracting from the real complexity of a system. The dif-ferent stakeholders can analyze a system within their partic-ular context.

Perceivable glyphs could be any object known from ourday to day lives. In our recent work we focus on a housemetaphor in which a well-shaped house represents a well-designed software entity and a miss-shaped house showsevolutionary decay. To further improve the perception ofour software visualization we looked at multimedia contentwe use in our daily lives. The perception of multimediacontent nowadays is often supported with audio: From amovie, in which the dramaturgy is enhanced with music,to games that produce a virtual reality, or to the interactionwith a computer operating system. All these support theirvisual content with appropriate audio. Therefore, we startedto investigate in what way software visualization can benefitfrom the addition of audio.

In this paper, we describe our approach of annotatingvisual cognitive glyphs with diverse audio concepts, andpresent our first results on how software comprehensiontasks are improved by using software visualization enrichedwith audio.

The main contribution is an extended CocoViz [7] vi-sualization approach, where we implemented the describedaudio concepts to show the benefits of an audio-supportedsoftware visualization. The tool architecture was extendedto include information gathered with various other softwareanalysis techniques into a single cognitive software visual-ization.

The remainder of this paper is organized as follows. Sec-tion 2 covers the key visualization and navigational con-cepts, used in cognitive software visualization to map soft-ware metrics to cognitive glyphs, as far as needed to under-

978-1-4244-2614-0/08/$25.00 © 2008 IEEE ICSM 2008366

stand their application in the extended audio context. InSection 3 we describe our approach to support softwarevisualization with audio, with regard to specific tasks ofprogram comprehension and navigation. In Section 4 wediscuss concepts for interacting with an audio supportedsoftware visualization. Section 5 discusses how we imple-mented audio in our cognitive software visualization ap-proach. In Section 6 we present example scenarios basedon a commercial web application, and in Section 7 we de-scribe the results of a small user study. We address relatedwork in Section 8 and summarize with our conclusions andfuture work in Section 9.

2 Cognitive Software Visualization (CSV)

In cognitive software visualization structural and evolu-tionary metrics are mapped to graphical elements in 2D and3D as cognitive glyphs. The concept of mapping metricshas already been described in [30] and was introduced asPolymetric Views by Lanza et al. [19]. With the CocoVizapproach we investigate the usefulness of the third dimen-sion and other improvements with respect to the compre-hension of a visualized software project.

In the following we discuss the key concepts in a cog-nitive software visualization that are later used in the audiocontext. 1) Metrics Clusters; 2) metrics configuration; and3) Glyphs.

2.1 Metric Clusters

We define a Metrics Clusters to be a set of specific met-rics that in combination enable analysis of particular soft-ware entities in terms of their structure (i.e., size or com-plexity) and evolution (i.e., change coupling or bug den-sity). A similar concept is used in [29] where Pinzger etal. offer a solution to build characteristic views on sourcecode evolution. According to [29], a combination of mean-ingfully clustered metrics can facilitate the comprehensi-bility of a software visualization. In a Hot-Spot-View, forexample, the metrics cluster consisting of number of func-tions, lines of code, Cyclomatic Complexity [23] and Hal-stead Programm Difficulty [15] accentuates complex soft-ware components that exhibit a variety of functionality.

In CocoVizwe implemented a list of Metric Clusters aspreset mappings for our metric configurator. With that weare capable to define clusters independently from the differ-ent visualizations. The power of Metric Clusters becomesclear if we use our capabilities to import analysis data. Wecan easily combine results from completely different anal-ysis methods to new metric clusters and visualize the newcontext according to our needs. Currently CocoViz sup-ports importing data from the Eclipse1 metrics plugin2 and

1http://www.eclipse.org/2http://metrics.sourceforge.net/

Together’s3 metric analysis functionality. However the ar-chitecture is extensible for other data sources.

In the audio context this concept becomes handy as it al-lows one to combine metrics suitable for a specific audioalgorithm to metric clusters that can easily be applied after-wards.

2.2 Metrics Configuration and SV-Mixer

The next key concept we address in this section is theSoftware Visualization Mixer (SV-Mixer). The SV-Mixeradopts the concept of an audio mixer for software visualiza-tion. An audio mixer processes audio signals before send-ing the result to an amplifier. The same way the SV-Mixermaps the particular software metrics to the visual represen-tations of a cognitive glyph. The metric values are filtered,normalized or transformed according to the SV-Mixer con-figuration before composing a visualization. Similar to theaudio mixer channels, every visualization has a specified setof visual representations. The idea is to quickly adjust thevisual mappings according to our focus while looking at theview.

In our extended audio context, the SV-Mixer in additionto the set of visual representations has a set of audio rep-resentations. Depending on the specific amount of metricsused from an applied audio algorithm, the correspondingamount of audio representation channels becomes availablein the SV-mixer.

2.3 Cognitive Glyphs

Glyphs are represented as a set of visual representationsmapped to software metrics (e.g., the hight of a house roofis mapped to lines of code). The mapped metric values ofa specific entity specify the glyphs representation. Beyondthat, cognitive glyphs visualize software in a comprehensi-ble way, leading to a faster comprehension of the relevantaspects compared to glyphs that are not based on a metaphor(e.g., Starglyphs [13]).

To build a house glyph, for example, four parameters to-gether with their metric mappings are used. Two metricmappings represent the width and height of the roof, whereas other metrics are mapped to the width and height of thebody of the house. In the context of a Hot-Spot-View Met-ric Cluster the example in Figure 1a) represents a complexclass, visualized with a large house body, and a comparablesmall house roof width (number of functions) and a mediumto large roof height (lines of code) the glyph would repre-sent a software component that condense reasonably-sizedcomplex code on few functions like a class implementing ancomplex algorithm. These components might be consideredproblematic candidates to maintain and evolve.

3http://www.borland.com/us/products/together/index.html

367

In our audio extension, every cognitive glyph represent-ing a software entity has its audio context. Based on thecomprehension or analysis task, the audio context offersfurther details on the particular software entity.

3 Why Audio in Software Visualization

One of the main concerns in a software visualization re-mains to find relevant aspects in a complex system as fast aspossible. Within a Cognitive Software Visualization (CSV)as explained in [7] the glyphs representing the various enti-ties and their aspects are distinguable and perceivable. TheSV-Mixer allows one to customize the view and interactwith it. Still there are cases in CSV as well as general insoftware visualizations, where after filtering out irrelevantentities, one finds himself with hundreds of potential enti-ties. In many layouts these entities even overlap each other.

To overcome these shortcomings and further improveour CSV approach, we were looking for components thatare fundamental in program understanding and navigation.We want to mention some of the work we found impor-tant for our purpose. According to Pennington’s [27, 28]bottom-up theory of program comprehension e.g., a pro-grammer focuses first on the basic structural entities. A fun-damental component therefore is an adequate highlightingof basic text structure units. Mosemann and Wiedenbeck’sresults presented in [24] stated that reading a program byfollowing the control flow offers a high performance wayof navigation even for novices.

In [25] Pacione proposed ways to increase the utility ofvisualization for software comprehension. He classified vi-sualization to five levels of abstraction for software com-prehension. Pacione suggested that software comprehen-sion can be facilitated in adequately using multiple of thoselevels of abstraction combined with multiple facets and theintegration of static and dynamic information. Pacione etal. did a case study in a realistic software comprehensionscenario [26]. According to them visualizating an object-or class-level representation of the system and providingan architectural-level view were optimal in terms of an-swering most of the scenario questions. With our cognitivesoftware visualization approach we place ourself already inbetween the object-/class- representation and architectural-level. Nevertheless Pacione stated that adequately usingmultiple of those levels can further facilitate software com-prehension. Therefore we were looking for concepts to en-hance our approach to address his suggestion.

Popups or tooltips are concepts that are used in similarsituations, where an observer needs additional informationon entities. Implementing these concepts for our purposeshowed some problems. Whenever we have more than 10entities to further investigate we found popups / tooltips tobecome suboptimal. Specially when the extra informationshowed exceeded a simple sentence, the time used to read

the tooltip of the entities that were not interesting at all,ended abstracting the observer from the main task.

Not satisfied with the tooltips solution, we thought aboutthe fundamental components for program understandingand the suggestion for facilitating software comprehension.We investigated supporting our visualization and interactionin our visual CSV extending it to an audio visual CSV ap-proach. In particular we found that by using audio to sup-port our cognitive software visualization we solved the pre-vious mentioned shortcomings. Beyond that we were ableto improve the navigation and program comprehension ca-pabilities of our CocoViz approach in general.

4 Audio Concepts

Audio has been used for software analysis in previouswork e.g., in [10] to monitor control flow or in [1] to provideprogrammers with debugging feedback. In the context ofsoftware visualization we use audio for tasks, in which wecan effectively support interaction and navigation. In thissection we discuss our extended visualization with audiosupport.

4.1 Audio in Data Analysis

Even after filtering out irrelevant data the remaining vi-sualized data often contains a variety of entities. It becomesimportant that one can get extended information about aparticular entity. To do this in a traditional visualizationone would represent the entity in a different view, loosingthe focus from the original visualization. In supporting thevisualization with audio we can provide the user with addi-tional information, while preserving the current visualiza-tion state.

4.1.1 Detect threshold exceeding with audio

A simple way to provide a user with extended information isto check a mapped metric for exceeding thresholds. When-ever an entity exhibits a metric value beyond a specifiedvalue, a sound (e.g., in the simplest case an acoustic signal)would provide the information to the user. This approachis convenient in cases where we are looking for entities thatare outliers. For instance we visualized a system with a Hot-Spot-View (by applying the size-complexity-metrics clusterpreset) and were interested in entities for which many criti-cal bugs were reported. With threshold exceeding audio an-notation we can map Nr of critical Bugs to the entities andget notified which entities in the Hot-Spot-View are critical.

With a slightly modified version we are able to track notonly threshold exceeding but also intervals. That allows usto simply classify the entities based on a mapped metric.The advantage is that we can give an audio feedback towhich subgroup an entity would belong in an extended or

368

Figure 1. Cognitive Glyphs showing two miss-shaped a) + c) and a well-shaped House b). Besides the shapeevery glyph has its audio context offering further detail or different aspects of the represented software entity.

completely different context from the one within the currentvisualization. In a Hot-Spot-View, for example, we can mapJUnit test Coverage to entities. Using one of our SV-Mixerconfiguration presets we can classify how well particularcritical entities are tested.

4.1.2 Detect code smells with audio

Another way to provide a user with extended informationis to check for code smells in entities of interest. Differ-ent approaches exist to find code smells or anti-patterns insource code. In our work, we use the approach developedby Lanza and Marinescu in [20]. In a traditional approachthe entities with detected code smells would be visualizedseparately. For example, we would color the detected enti-ties according to a color concept (Figure 2). The bottleneckin coloring the entities based on their detected code smellsis that we cannot apply another color concept to the dataset, and even worse that whenever an entity includes twoor more different code smells, we would need to color theentities with a default color. Unfortunately, a default colorleaves us with just little, and imprecise new information.We still would need to further dig into the entity to see whatcode smells were detected.

In an audio-supported visualization, an entity can giveus a more precise audio feedback. The feedback could be,for example, spoken or non-spoken, depending on the need.For instance clicking on an entity would trigger the audiofeedback. The entity would then speak its extended infor-mation, like ’the current entity is a potential god class and apotential shot gun surgery class’. Of course, spoken text isonly a rather simple example of audio support. Harmonies,disharmonies, twang or shrillness are other possibilities toenrich an entity’s audio-supported visualization. Comparedto the visual only approach we can get a very informativesummary of the entity’s extended context including caseswhere entities incorporate two or more code smells.

4.1.3 Entity description with synthesized audio

An even more sophisticated approach to use audio in dataanalysis is to synthesize an audio feedback based on the re-lated entities values. The important part in such an approachis that the audio feedback needs to maintain its distinctionfrom other similar entities. A lot of work has been donein the field of psycho-acoustics to address the issue of dis-tinguishability, especially in the context of audio compres-sion. Particularly worth mentioning in our context are the socalled Zwicker parameters [36]. According to Zwicker et al.the parameters loudness, sharpness, tone pitch, roughnessand oscillation do preserve the distinguishability of synthe-sized audio.

With synthesized audio we can generate a highly com-plex feedback that still remains compact. In an implemen-tation of a synthesized audio algorithm we map metric clus-ters to the various Zwicker parameters.

Example 1: A large and long lived entity with few crit-ical bugs and low coupling. As an example, we map thevalue of the entities for FanIn to loudness, the Number ofcritical Bugs to roughness, the Historical growth rate of theentity to tone pitch and the Lines of Code to the length ofthe tone. With that a large entity with few critical bugs, thatis not used often by other classes and has been in the systemfor quite some time, produces an audio feedback of a long,clear, but rather low tone.

Example 2: A large but young entity with many criti-cal bugs and high coupling. Such an entity that is widelyused by other classes but was introduced only in recent re-leases and incorporates several critical bugs would result ina long, rough, shrill and loud tone.

The advantage of our audio to metric cluster mapping isthat we can provide the user with quite a bit of extendedinformation including historical information as well as cur-rent health and importance of an entity with effective acous-tical feedback.

369

Figure 2. House glyphs showing a part of a commercial web application framework in a system hot-spot-view;the brighter the color the higher the god classes potential of a class

Table 1 shows two possible mappings of metric clustersto the Zwicker parameters that we found useful as describedin the examples above.

Zwicker Structural EvolutionParameter Metric Cluster Metric ClusterLoudness Complexity Fan inTone pitch Growth rate Growth rateRoughness - # critical BugsOscillation - Change rateTone length Lines of Code Lines of Code

Table 1. Example of two metric clusters mapped tosynthesized audio concepts

4.2 Audio in Historical Analysis

Besides using historical information to synthesize an au-dio feedback, audio offers support for other historical analy-sis tasks. Whenever we change the focus of a data set fromone version to another in traditional visualizations we en-counter a variety of changes happening at the same time.Depending on the layout applied to a view this results inobjects changing position, objects disappearing or shrink-ing substantially in size. It can be hard to keep track ofa particular set of interesting entities during such changes.Traditional visualizations animate the entities to representtheir change in position and size from one version to thenext. Still, with a variety of changes happening at the sametime, it remains hard to perceive whether an entity of inter-est changed substantially from one version to another.

With the use of audio we can support this visualizationin simply notifying whenever an entity of interest changes

more than e.g., 10% from its previous version. The audiofeedback could again be spoken or non-spoken, dependingon our preference. For example, the evolutionary change ofentities from one version to another could give a simple spo-ken annotation such as ’from version 1.0 to 2.0 the taggedentities ListProducer and PersistentModel changed by morethan 10 percent’.

4.3 Audio in Trace Analysis

Within our EvoSpaces project4 we found visual tracingto be an informative way to analyze use cases [11]. Thebottleneck of visual tracing is that with the endless amountof actions one can get lost in a wonderful animation. Es-pecially in cases when we aim to understand how often andfrom where specific entities are called.

To overcome this we suggest to use audio in the visualtracing context similar to the one used by Baecker et al. in[1] to provide programmers with debugging feedback. Be-fore visualizing the trace we tag the entities of interest. Withthat we can keep track of and get notified on the interactionsthat the particular set of entities is engaged in, while the vi-sualization shows us the system interaction during our usecase as a whole.

5 Audio Supported Cognitive Glyphs

In our CocoViz approach we implemented audio sup-port on the level of supporting navigation to achieve get-ting fast end effective familiarity with program entities andcollecting information about a program as referred in [24].This is done by binding audio feedback to the results of al-gorithms that gather extended information on specific pro-gram entities in an actual visualization.

4http://www.inf.unisi.ch/projects/evospaces/

370

We found such audio feedback particularly useful incases where even after adequately filtering visual entitiesin the SV-Mixer we still ended up with hundreds of poten-tial entities relevant in the context of interest or wheneverwe need to fine-tune the visualized entities. In such casesan adequate context-oriented audio feedback can improveinteraction with a cognitive software visualization substan-tially, without the need of changing the visualization andloosing subsequently focus.

Beyond that we can use audio for finding and highlight-ing basic text structure units as Pennington suggested in hisbottom-up theory of program comprehension [27, 28], andsupport a programmer in finding and understanding basicstructural entities in a large program. With audio feedbackwe are also able to extend our level of abstraction as definedin [25], to include information from other abstraction levelssuch as object population, memory usage, load distributionand deployment informations from a microscopic level orinformations on business behaviour and use cases from amacroscopic level.

With audio feedback we found a way to improve inter-action in cognitive software visualizations, extend the vi-sualization approach to adequately address further softwarecomprehension tasks, while still preserving all the advan-tages of a non audio-supported cognitive software visual-ization.

6 Example Scenarios

In this section we present example scenarios with ananalysis of a commercial web application. We show situ-ations where audio feedback is particularly useful and com-pare it to the effort that would be needed to achieve the sameresult with a non audio supported visualization. The usedevolutionary data set consists of the basic framework of acommercial web application used in healthcare. We ana-lyzed six releases over the period of 3 years. The metricswere calculated per release. The framework has more than950 classes and approximately 90’000 lines of code. In thefollowing, we analyze the framework from a program com-prehension and a software analysis point of view.

6.1 Program comprehension task

In an example program comprehension scenario aquality-assurance team could ask for critical classes of thesystem that where changed since the last release and need toundergo extensive testing. A set of such critical classes thatshould be included in the extensive testing can be found bylooking at potential big and complex classes that are widelyused by other classes and have been changed during the lastrelease.

For this we select all the class entities of the applicationand visualize them using our house glyphs. The visual rep-

resentations of the house glyphs are mapped to a Hot-Spot-View (size-complexity-metrics) as described in Section 2.1.With that we show complex software components that con-dense a variety of functionality. We arrange the entities onthe visualization axes using their lines of code and weightedmethods per class values. The colors in Figure 3a) repre-sent fanOut, a metric showing how extensively the classesuse other classes. We mapped and configured the differentcolors in the SV-Mixer to cluster the entities in a sensibleway. Classes using methods from less than 4 other classesare colored in blue. Classes that use up to 8 other classes arecolored in orange and classes with more than 8 are shownin red.

Figure 3a) shows the smaller classes of our visualiza-tion setup. In the upper right corner of the visualization wesee houses that are obviously complex (body size) and large(roof size). In a context of finding the crucial components ofour system, we need to take those into consideration for anin-depth analysis. However from a crucial components per-spective, we need to pay attention on some of the smallerclasses, too. Because they might only be small based ontheir inheritance.

To gain familiarity with the small but relevant classes,with current visualization approaches we end up having touse a new view. In our approach, for example, we do this byeither changing the layout mapping to rearrange the entitiesbased on their inheritance, with the disadvantage of losingorientation and finding all the entities on a new position, oras shown in Figure 3b), changing the mapping of the colorsand at least preserve a comparability between the two views.In any case we lose focus on our main context. Furthermorewe end up having a variety of less important extra viewsmaking the comprehension of a project as a whole moredifficult.

With audio feedback, a context like the small crucialclasses could be achieved without losing focus on the pri-mary view. For instance we can simply map the inheritancelevel of the entities to give us an audio feedback based ona threshold exceeding audio algorithm. We then hover withthe mouse over the entities of our interest. Whenever thereis an entity that we need to take into consideration we getan audio feedback and can select that entity right away.

Finally we get information on which of this classes werechanged during the last release and need extensive testingwith a historical audio algorithm. With that, classes notifyus their change from the last release with two musical notesplaying a interval representing the amount of the change.

To emphasize the usability of audio-supported softwarevisualization for program comprehension, we extend ourcurrent example to another common use case. Lets considerthat we are trying to find a bug that was introduced duringthe latest releases. We change focus from our entities of in-terest to the ones changed during the last releases. In a tra-

371

Figure 3. House glyphs showing a system hot-spot-view of a commercial web application framework in the latestrelease; in a) colors represent how often the entities use methods from other entities, in b) colors represent theinheritance level of the entities

ditional visualization approach we change the visualizationagain by mapping the color to the entities changed duringthe last releases. Even though we do not know whether anyof our entities under investigation has been changed duringthe latest releases, we change focus in our view to gain littleor no information.

With audio support we can run our entities through anaudio algorithm that notifies us whether the entities werechanged during the latest release. With that we again hoverover the entities under investigation. We then simply selectthe ones changed during the latest release that are poten-tially involved with the bug for an in-depth analysis withoutlosing focus.

6.2 Software analysis task

A common software analysis scenario is to detect codesmells. In a traditional CocoViz approach we analyzethe entities color by a god classes detection algorithm. Asshown in Figure 2 the brighter the color the higher the godclass potential of a class.

If we are further interested in which of the potential godclasses are affected by another code smell, we can map thecolor of the entities to another code smell algorithm. With-out losing focus we can run the entities through a code smellaudio algorithm and hover over the entities under investiga-tion. Immediately, we then can hear with what other codesmells the classes are potentially affected.

7 User Study

In this section we present the first results of a user studywe have done with 10 individuals not involved within thisproject. For the user study we used an extended version ofour CocoViz-Implementation [6], which implements allthe previously mentioned audio concepts.

Characteristic of Participants The subjects inquired inthe experiment where categorized based on their musicaleducation and level of expertise in program comprehen-sion. For musical education, we distinguished individualsthat were able to read music sheets, play a music instrumentor sing, from the ones that did not have musical experienceand further divided them into candidates with and withoutsoftware engineering knowledge. We did so to check pre-liminary concerns on whether our audio approach would beharder to understand for individuals with no musical edu-cation. Beside that we wanted to know how much traininga new user would need to get familiar with the audio ap-proach and to distinguish the entity properties through au-dio. We took non software engineers into consideration be-cause one reason of the cognitive glyph abstraction, is thatimportant individuals like CFO in smaller companies with-out an own information system department, understand thesituation even though they often are not familiar with sourcecode.

Design In this user study we asked the participants to ad-dress the situation, in which a quality-assurance team wants

372

to know which critical classes of the system where changedsince the last release and need to undergo extensive test-ing. Like described in Section 6 we want them to find suchcritical classes that where changed. During the user studyall participants used the same data-set of a commercial webapplication. In the end we asked the Individuals questionslike whether the audio supported visualization was usefulfor them or what benefit they encountered, compared to avisual only approach.

Preliminary results of our user study showed that in gen-eral the usefulness of the idea was clearly received. In ourstudy all the participants, no matter on their musical educa-tion, did not encounter any problems in instantaneous un-derstanding the audio feedback. Our concerns on whethertraining would be needed was pretty much disproved. Fornon engineers with the cognitive glyph metaphor, only lit-tle explanation of the general cognitive glyph representa-tion was needed, to understand the task, not mentioning thesource code at all.

No general preference for speech versus non-speech au-dio feedback was found. Depending on the task one waspreferred over the other and vice versa.

Concerns arise in audio algorithms using the Zwickerparameters[36]. Tone length and loudness were harder toperceive by the individuals in a noisy environment or with-out headphones. Especially loudness showed itself as beinghard to map linear differences correctly, as it seams that ev-ery individual does percept loudness different and not lin-ear.

Some individual stated that for certain tasks a popupwould be as useful as audio feedback. Still the same onefound the idea to support the visualization with an extra di-mension that prevents further overloading the screen verypromising and even suggested future work to areas thatpopup can not keep up.

8 Related Work

The goal of Software visualization is to represent thecomplex context of today’s software projects. To visual-ize software is essential, due to the abstract nature of thecontext and the amount of information that needs to be un-derstood. Most visualization methods use a graphical rep-resentation of data rendered either in a two-dimensional orthree-dimensional view. In the past few years a variety ofapproaches dedicated to software visualization and softwarereengineering emerged.

Hierarchical visualization approaches aim to displaylarge hierarchies in a comprehensible form. With Treemaps[17], Johnson and Shneidermann proposed to map treestructures to rectangular regions. Very large hierarchieswith thousands of leaves, can be displayed space efficient,

while still being comprehensible. However the readabilitydecreases very large hierarchies and the attraction in the vi-sualization is often centered on relatively unimportant enti-ties that are represented as a large rectangular reagion.

In contrast with Cone Trees, Robertson et al. [31] sug-gested to laid out the hierarchy in a three-dimensional way,where the children of a node are placed evenly spaced alonga cone base. Through rotation of the cone base a viewerbrings different parts of the tree into focus. But, as stated in[18] Cone Trees with more than 1000 nodes are difficult tomanipulate. Therefore, Cone Trees might be considered formedium-sized trees only.

An other technique for interaction with medium-sizedtrees Dachselt and Ebert recommend in [9]. The CollapsibleCylindrical Trees (CCT) map the child nodes on a rotatingcylinder. This offers a fast and intuitive interaction and al-lows one to dynamically hide or show further details. Theinteresting part of this work is that beside most other workin the field of hierarchical views CCT do not concentrate onhow to display large hierarchies in a comprehensible formbut concentrate on the interaction with the data itself.

With our CocoViz approach we aim to avoid the shortcomings of the mentioned hierarchical visualizations, incombining their key concepts. Among other things we usea 3D view to avoiding space limitations, appropriate layoutalgorithms to prevent dispensable overlapping and an ad-vanced dynamic approach that allows intuitive interaction.

Metrics visualization in contrast to hierarchical visual-izations, describe a software state or situation. Metrics de-scribe a specific software entity and are not part of a hier-archy. The goal of these approaches is to show aspects of asoftware by visualizing the representing metrics.

In Seesoft [12] Eick represents the lines of code of ev-ery software entity as a thin row. The rows are then coloredbased on a statistics of interest, e.g., most recently, least re-cently changed, or locations of characters. With that, onecan quickly overview the fragmentation of a software andhighlight parts of interest. Marcus et al.’s sv3D[21] extendsthe Seesoft approach to the third dimension, and adds dif-ferent manipulation techniques. They use cylinders wherethe height, depth, color, and position would represent themetrics.

Lanza and Ducasse’s Polymetric Views [19] attempt todetect problems as early as possible in the initial phases ofa reverse engineering process and aim to help understand-ing the structure of a software system. In their concept theydisplay the software entities based on their metric values asa rectangular shape. Whereas the position, the height, thewidth and the color of one rectangle each represents a met-ric value of the same software entity. This approach offers aquick overview of the softwares subdivision. The Polymet-ric Views in addition to Seesoft include a representation of

373

the relations within the software entities.Inselberg and and Dimsdale presented a way to visual-

ize multi-dimentional analytic and synthetic geometry [16].In the parallel coordinates, they arrange the various met-ric scales vertically one after the other. For every softwareentity the metric values are marked on the correspondingmetric scale. A line connecting all the marks of one entitythen represents that software entity.

In [3] Benedix et al. explain how the layout of paral-lel coordinates can be used to visualize categorical data.In their adoption the data points are substituted with afrequency-based representation offering auxiliary efficientwork with meta-data.

In [13] Fanea et al. combined parallel coordinates andstar glyphs to provide a more efficient analysis compared tothe original parallel coordinates.

Pinzger et al. proposed to use star glyphs to visualizecondensed graphical views on source code and relation his-tory data [30]. In their Kiviat diagram, metric values ofdifferent releases are reflected like annual rings on a tree-stump. The diagrams can be used to show one metric inmultiple modules or multiple metrics in one module. Fur-thermore relation of modules are characterized with con-nections between those modules.

CocoViz distinguishes itself from the other metrics visu-alization approaches through its metaphor glyphs an the re-sulting improved software comprehension compared to ab-stract graphical representation used in other approaches. Aninteractive approach where a viewer analyses the softwarein walking through the views and tagging elements. Andlast but not least a dynamic approach that allows to quicklyfilter temporary non relevant elements out.

Audio supported visualization To our best knowledgewe found only little work done with audio in software vi-sualization. However there is work done in the context ofsoftware analysis and auditory display.

Vickers in [33] gives a good introduction in summarizingvarious approaches present in the field auditory representa-tion of programs.

Brown and Hershberger in [8] use audio to enhance al-gorithm animations. In their work they enhance the Zeusalgorithm animation systems using a MIDI synthesizer andgive a introduction to the use of colour and sound in algo-rithm animations.

In [10] DiGiano et al. explored with a sound-enhancedprogramming environment they call LogoMedia. LogoMe-dia allows programmers to associate non-speech audio withprogram events while the code is being developed. The in-terface is designed for specifying visualization events withsound and monitoring variables or control flow.

Baecker et al. in [1] suggest to use audio to provide pro-grammers with debugging and profiling feedback without

disturbing the integrity of the graphical interface. Accord-ing to them audio may be a more salient representation forcertain types of program information like repetitious pat-terns in control flow and nonlinear sequences of variablevalues.

In [14] Finlayson and Mellish compared speech, non-speech sound and a combination of them. They recommendthat a combination where non-speech sound be used as asupplement to speech as it shows slightly better results com-pared to pure speech or non-speech.

Berman and Gallagher in [5] present techniques to listento program slices that help software developer in undertak-ing program comprehension activities.

Recent work with audio in software analysis has beendone by Stefik et al. in [32]. In their work they use auralfeedback to sonify computer code as an aid to non-sightedprogrammers.CocoViz distinguishes itself from the other approaches

in that they focus mainly on tracking the value of state vari-ables and control flow during debugging or visualizing al-gorithms. Meanwhile our focus is more in supporting theinteraction within a visualization.

9 Conclusions & Future Work

In this paper we discussed improvements to the per-ception of relevant aspects in evolving software projects.We proposed an audio extension to our cognitive soft-ware visualization approach CocoViz [7], where metricbased analysis of entities are visualized in form of cogni-tive glyphs. With our extended cognitive software visual-ization approach structural and evolutionary aspects of en-tities not only are distinguished faster, but we are also capa-ble to combine existing program comprehension techniqueswith audio-supported visualizations. The audio parametersof Zwicker (loudness, tone pitch, roughness, oscillation andtone length) were mapped to metric clusters. These met-rics clusters that are presets in our SV-Mixer of CocoVizcan therefore be used to set to sound distinguished entities.As a results, conventional visualization techniques are en-riched with particular audio sounds to enable a user a moreeffective program comprehension scenario without losingcontext in multiple views.

Based on previous work in visualization, we introducedconcepts where our cognitive software visualization sub-stantially benefits from audio support over non audio sup-ported visualizations. Audio feedback is used to supportthe dynamic interaction with a system’s visualization. Epe-cially in visualizations of large systems with many cogni-tive perceivable glyphs audio feedback facilitates the under-standing of relevant aspects. It does so in allowing to hoverover potential entities of interest an get instant feedback ornotify a user over relevant changes during an animation ortrace view.

374

The approach is currently evaluated with a large set ofsoftware projects and against other known visualization ap-proaches to document in which situations an audio feed-back offers substantial advantages over more traditional ap-proaches. We are furthermore working on a user study toget substantial data on the benefits of our audio-supportedapproach CocoViz.

Future work aims to improve the audio feedbacks withuser configuration and will consider more sophisticated au-dio algorithms and to extend the use cases where traditionalvisualization tasks benefits from an audio support. With oneparticular audio algorithm we try to synthesize an ambientsound that would get a feedback based on the code smells orproperties presented in the proximate entities. Besides thatwe are currently experimenting with clustering algorithmsand synthesized audio algorithms.

Acknowledgments

We are grateful to Emanuel Giger, Patrick Knab,Michael Wursch and Martin Pinzger for their valuable in-put. This work was partially supported by the HaslerStiftung Switzerland within the Project “EvoSpaces II”.

References

[1] R. Baecker, C. DiGiano, and M. Aaron. Software visualization fordebugging. Commun. ACM, 40(4):44–54, 1997.

[2] S. Barrass. Sonification design patterns. In Proc. Int’l Conf. onAuditory Display, 2003.

[3] F. Bendix, R. Kosara, and H. Hauser. Parallel sets: visual analysis ofcategorical data. IEEE Symp. on Info. Visualization, pages 133–140,2005.

[4] L. Berman, S. Danicic, K. Gallagher, and N. Gold. The sound ofsoftware: Using sonification to aid comprehension. In Proc. IEEEInt’l Conf. on Program Comprehension, pages 225–229, 2006.

[5] L. I. Berman and K. B. Gallagher. Listening to program slices. InProc. Int’l Conf. on Auditory Display, 2006.

[6] S. Boccuzzo and H. C. Gall. Cocoviz: Supported cognitive softwarevisualization. In Proc. Working Conf. on Reverse Eng., 2007.

[7] S. Boccuzzo and H. C. Gall. Cocoviz: Towards cognitive softwarevisualization. In Proc. IEEE Int’l Workshop on Visualizing Softw.for Understanding and Analysis, 2007.

[8] M. Brown and J. Hershberger. Colour and sound in algorithm ani-mation. In Proc. IEEE Workshop on Visual Languages, page 5263,1991.

[9] R. Dachselt and J. Ebert. Collapsible cylindrical trees: A fast hier-archical navigation technique. IEEE Symp. on Info. Visualization,pages 79–86, 2001.

[10] C. J. DiGiano, R. M. Baecker, and R. N. Owen. Logomedia: asound-enhanced programming environment for monitoring programbehavior. In Proc. Conf. on Human factors in computing systems,pages 301–302, 1993.

[11] P. Dugerdil and S. Alam. Execution trace visualization in a 3d space.In Proc. Int’l Conf. on Information Technology, 2008.

[12] S. G. Eick, J. L. Steffen, and E. E. Sumner, Jr. Seesoft - a toolfor visualizing line oriented software statistics. IEEE Trans. Softw.Eng., 18(11):957–968, 1992.

[13] E. Fanea, S. Carpendale, and T. Isenberg. An interactive 3d inte-gration of parallel coordinates and star glyphs. IEEE Symp. on Info.Visualization, pages 149–156, 2005.

[14] L. J. Finlayson and C. Mellish. The audioview - providing a glanceat java source code. In Proc. Int’l Conf. on Auditory Display, 2005.

[15] M. H. Halstead. Elements of software science, operating and pro-gramming system series. Elsevier, 7, 1977.

[16] A. Inselberg and B. Dimsdale. Parallel coordinates: a tool for visu-alizing multi-dimensional geometry. In Proc. IEEE Conf. on Visu-alization, pages 361–378, 1990.

[17] B. Johnson and B. Shneiderman. Tree-maps: a space-filling ap-proach to the visualization of hierarchical information structures. InProc. IEEE Conf. on Visualization, pages 284–291, 1991.

[18] J. Lamping, R. Rao, and P. Pirolli. A focus+context technique basedon hyperbolic geometry for visualizing large hierarchies. In Proc.SIGCHI Conf. on Human factors in computing systems, pages 401–408, 1995.

[19] M. Lanza and S. Ducasse. Polymetric views — a lightweight vi-sual approach to reverse engineering. IEEE Trans. on Softw. Eng.,29(9):782–795, 2003.

[20] M. Lanza and R. Marinescu. Object-Oriented Metrics in Prctice.Springer, 2006.

[21] A. Marcus, L. Feng, and J. I. Maletic. 3d representations for soft-ware visualization. In Proc. ACM Symp. on Softw. Visualization,pages 27–36, 2003.

[22] R. Marinescu. Detection strategies: Metrics-based rules for detect-ing design flaws. In Proc. Int’l Conf. on Software Maintenance,2004.

[23] T. J. McCabe. A complexity measure. IEEE Trans. on Softw. Eng.,2(4), 1976.

[24] R. Mosemann and S. Wiedenbeck. Navigation and comprehensionof programs by novice programmers. In Proc. Int’l Workshop onProgram Comprehension, page 79, 2001.

[25] M. J. Pacione. Software visualisation for object-oriented programcomprehension. In Proc. Int’l Conf. on Softw. Eng., pages 63–65,2004.

[26] M. J. Pacione, M. Roper, and M. Wood. A comparative evaluationof dynamic visualisation tools. In Proc. Working Conf. on ReverseEng., pages 80–89, 2003.

[27] N. Pennington. Comprehension strategies in programming. In InG. M. Olson, S. Sheppard & E. Soloway, Eds. Empirical Studies ofProgrammers: Second Workshop, pages 100– 113, 1987.

[28] N. Pennington. Stimulus structures and mental representations inexpert comprehension of computer programs. In Cognitive Psy-chology, pages 295–341, 1987.

[29] M. Pinzger. ArchView - Analyzing Evolutionary Aspects of ComplexSoftware Systems. Vienna University of Technology, 2005.

[30] M. Pinzger, H. Gall, M. Fischer, and M. Lanza. Visualizing multi-ple evolution metrics. In Proc. ACM Symp. on Softw. Visualization,pages 67–75, 2005.

[31] G. G. Robertson, J. D. Mackinlay, and S. K. Card. Cone trees:animated 3d visualizations of hierarchical information. In Proc.SIGCHI Conf. on Human factors in computing systems, pages 189–194, 1991.

[32] A. Stefik, R. Alexander, R. Patterson, and J. Brown. Wad: A fea-sibility study using the wicked audio debugger. In Proc. IEEE Int’lConf. on Program Comprehension, pages 69–80, 2007.

[33] P. Vickers. External auditory representations of programs: Past,present, and futurean aesthetic perspective. In Proc. Int’l Conf. onAuditory Display, 2004.

[34] P. Vickers and J. L. Alty. Caitlin: A musical program auralizationtool to assist novice programmers with debugging. In Proc. Int’lConf. on Auditory Display, 1996.

[35] P. Vickers and J. L. Alty. Siren songs and swan songs: Debuggingwith music. In Commun. ACM 46, 7, page 8692, 2003.

[36] E. Zwicker, H. Fastl, and W. M. Hartmann. ”psychoacoustics: Factsand models”. Physics Today, 54:64–65, 2001.

375