knowledge-based software representation, querying and...

Knowledge-based Software Representation, Queryingand Visualization

Peter Kapec∗

Institute of Applied InformaticsFaculty of Informatics and Information Technologies

Slovak University of Technology in BratislavaIlkovicova 3, 842 16 Bratislava, Slovakia

[email protected]

AbstractAmong various data types that can be visualized, recentlysoftware became one of the most important candidate forvisualization. With the increasing complexity of devel-oped software systems the need for better means to un-derstand software are needed. Software visualization aimsto help with these problems by providing graphical pre-sentations and thus allows discussion between developersin the domain of these graphical representations ratherthan mental concepts. In this work we presents an al-ternative software visualization method based on hyper-graphs. We have developed a visualization system thatutilize hypergraph-based representation of software arti-facts and relations between them. We present how hy-pergraphs can be used in all stages of the visualizationprocess and how they are used as the unifying data rep-resentation. Interactive 3D visualization displays directlythese hypergraph representations of software. To allowfiltering we have developed a query mechanism in whichqueries are hypergraph patterns and results are also hy-pergraphs. This way the visualization system is buildaround only one data structure.

Categories and Subject DescriptorsD.2.6 [ Software engineering]: Programming Environ-ments—Graphical environments; G.2.2 [Discrete math-ematics]: Graph Theory—Hypergraphs; H.3.3 [Infor-mation storage and retrieval]: Information Searchand Retrieval—Query formulation; H.5.2 [Informationinterfaces and presentation]: User Interfaces—Graph-ical user interfaces; I.3.7 [Computer graphics]: Three-Dimensional Graphics and Realism—Virtual reality

∗Recommended by thesis supervisor: Assoc. Prof. Mar-tin Sperka. Defended at Faculty of Informatics and Infor-mation Technologies, Slovak University of Technology inBratislava on April 7, 2011.

c© Copyright 2011. All rights reserved. Permission to make digitalor hard copies of part or all of this work for personal or classroom useis granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies show this notice onthe first page or initial screen of a display along with the full citation.Copyrights for components of this work owned by others than ACMmust be honored. Abstracting with credit is permitted. To copy other-wise, to republish, to post on servers, to redistribute to lists, or to useany component of this work in other works requires prior specific per-mission and/or a fee. Permissions may be requested from STU Press,Vazovova 5, 811 07 Bratislava, Slovakia.Kapec, P. Knowledge-based Software Representation, Querying andVisualization. Information Sciences and Technologies Bulletin of theACM Slovakia, Vol. 3, No. 2 (2011) 1-11

Keywordssoftware visualization, hypergraphs, query language,visual programming environment

1. IntroductionSoftware development has become very important in pastyears. As more and more difficult problems should besolved using software systems, the software programs havebecome also more complex. With increased software com-plexity and several developers working on the same project,the design and maintenance of complex software systemsis more difficult than ever. Techniques and tools helpingto overcome these problems, especially covering tasks likeprogramming, program comprehension or program mod-ification, become very necessary. Software visualizationaspires to help in the development process. Software vi-sualization, an interesting and large research area, aimsto help us with the intangible software making it easierto understand. Software, opposed to other engineeringproducts, is untouchable. Although in past years manysoftware visualization systems and visual programminglanguages were developed, developers still use in practicestandard integrated development environments and tex-tual languages.

Software development is not only about writing sourcecode, but also involves the management of the softwaredevelopment process and creating / modifying varioussoftware artifacts during development. Currently existingintegrated development environments rarely utilize visu-alizations (except popular UML diagrams). With the in-creasing size and complexity of software products it wouldbe beneficial for software developers to look at software asa knowledge repository. This way better understandingof software could be achieved by visualizations of well-formed queries about software.

2. Software visualizationMany information visualization techniques, which dealwith non-perceptible abstract data, have been developedin past years and they can be classified according to fol-lowing three orthogonal criteria [5]: data to be visual-ized, the visualization technique, interaction and distor-tion technique used, When looking at data types thatcan by visualized, we can identify following types: multi-dimensional data, text and hypertext, temporal data, (geo)-spatial data, hierarchies and graphs, algorithms and soft-ware.

2 Kapec, P.: Knowledge-based Software Representation, Querying and Visualization

Among the mentioned data types, software has becomerecently target to many visualization approaches. Soft-ware is very suitable for visualization due to software’sintangibility, which is an additional complication for soft-ware engineers compared to other engineering fields.

It seems that software comprehension is easier when thesoftware is graphically represented. For example, control-flow diagrams are popular aids for program understanding[16]. Graphical representations allow using, except two-dimensional positioning of text, also attributes like form,color, size, position, orientation etc.

Software visualization aims at helping with software com-prehension using graphically representations. Accordingto a discussion mentioned by Petre et al. [17], graphicalrepresentations may offer following advantages:

• better information content and information density

• higher level of abstraction

• providing overview (structures may become easierrecognizable)

• easier remembering

In the work by Larkin and Simon [12], which comparesdifferences of textual and graphical notations, the authorsstate that recognition of information using graphical rep-resentation is not basically more efficient than using tex-tual. However, e.g. in searching problems the graphicalrepresentation may outperform textual. Larkin and Si-mon note that most benefits of using graphical represen-tations can be seen in deriving additional information notdirectly visible.

In the past twenty years more than hundred software vi-sualization prototypes and systems were developed. How-ever, only few managed it to move from research projectsto systems usable in praxis. Respondents in recent stud-ies stated the following challenges in the field of softwaremaintenance, reverse engineering and re-engineering [11]:

• visualization of large datasets and dynamic aspects

• navigation between multiple views

• automatic selection of visualization techniques

• ”semantic” visualization

• integration with other tools

One of the mentioned challenges is semantic visualization.Software visualization should allow tho explore seman-tic relations between software artifacts so that previouslyhidden or hard to obtain information about software canbe obtained.

Although most software visualization approaches usegraph visualizations, they often do not put high empha-sis on the used graph structures. However, when look-ing at possibilities of knowledge representations, see nextSection, that are also often based on various graph struc-tures, introducing concepts and approaches from knowl-edge representation into the software visualization fieldcould bring new benefits. Therefore in the next sectionwe discus the problematics of knowledge representation.

3. Representing knowledgeIn the long history of information management systemsmany data models have been developed. Before the ad-vent of the relational model, the research also focused ongraph-based data models. Currently, graph-based datamodels gain again focus of researchers, because they canbetter cope with current trends and requirements in theera of Internet. The problem the relational model facestoday is the fact that most information found on Internetis heterogeneous and highly interconnected. Databasesbased on the relational model are well suited for caseswhen the problem domain can be thoroughly analyzedand a static database schema can be created. However,to store knowledge, that is information and their inter-connections, from a domain that we can not analyze inadvance and may dynamically change in time, the rela-tional model is not capable to handle such requirements.

Graph data models are well suited for such situation inwhich interconnections between data are at least as im-portant as data itself. Many graph data models have beendeveloped – a detailed overview can be found in [2]. Insummary, graph data models use various graph types andtwo data models (The Hypergraph Data Model [18] andGROOVY [14]) also use generalized graph concepts: hy-pergraphs or hypernodes to represent data and/or schema.

Under knowledge we often understand information placedinto a meaningful context, combined with experience, in-terpreted etc. Ontologies allow representing and storingknowledge about a certain domain in a declarative way.One of benefits of ontologies is that they can be inter-preted by humans and also computers. Of course pre-senting ontologies in a navigable form is important andoften visualizations based on graphs are used.

The term ontology originates in the philosophy field andis used to describe set of concepts and their interrela-tions in a specific domain [1]. Concepts, often termedclasses, represent real-world objects and often form a hi-erarchical structure. Concepts may be related to otherconcepts, may play different roles in relations and canbe described by various attributes. Furthermore conceptsmay point to existing instances that can be stored in com-puters. For modeling ontologies various approaches havebeen developed [13]. Among the existing approaches thegraph-based or graph-oriented models are popular andoffer possibilities to present ontologies in graphical formfor users. Recently is has been shown that two popularontology storage formats (Topic maps and RDF) can beformally defined trough hypergraphs [3, 8].

The mentioned hypergraph-based data models and on-tological representations can be considered as knowledgestorages. For information retrieval often query languagesare used. Several query languages for graphs and hyper-graph-based representation have been developed, e.g.GraphQL [9] for common graphs, HQL defined for theHypergraph Data Model [19], or tolog [7] and TMQL [22]query languages for ontology storages.

The need to store graphs in computers led to the develop-ment of many file-based formats. Among many existinggraph formats currently the XML-based formats GXL [21]and GraphML [4] are very popular. Both allow to storecommon graph types, but they also support hypergraphsand hierarchic graphs with subgraphs within graphs.

Information Sciences and Technologies Bulletin of the ACM Slovakia, Vol. 3, No. 2 (2011) 1-11 3

4. GoalsThe aim of this thesis is to introduce approaches fromknowledge representation field into software visualization.Our work focuses on the idea of uniform hypergraph rep-resentation of software artifacts that would allow to rep-resent and visualize semantic relations between softwareartifacts that occur at various stages of software develop-ment. Combining hypergraph representations and query-ing techniques with already existing information visual-ization techniques, we could achieve interesting results.

To achieve knowledge-based software visualization, thefollowing partial goals of this thesis can be specified:

• Analyze current state-of-the-art in software visual-izationResearch in software visualization is producing manyinteresting approaches often utilizing graph visual-izations. However, very few works focus on thegraph representations itself, probably missing keyfeatures: the possibility to use graph data modelsdeveloped in the field of knowledge representationand semantic web. Therefor a thorough analysis ofthe software visualization field and important ap-proaches of knowledge representation is to be pro-vided.

• Propose a model for storing and querying hypergraphrepresentations of software artifactsThe aim is to develop, based on existing knowledgerepresentation approaches, a suitable hypergraph-based data model to represent software artifacts andtheir interrelations. However, we should look at theproposed hypergraph-based data model as a unify-ing data storage for software artifacts that can bequeried. Based on the hypergraph model a querymechanism should allow to filter the visualized soft-ware artifacts and relations, thus reducing the visualload the user experiences when dealing with visual-izations of very large graphs.

• Utilize 3D information visualization techniquesTo verify possible benefits and/or bottlenecks of hy-pergraph-based software representations a visualiza-tion prototype is needed. 3D graph visualizationsbecame very popular and may provide interestingviews, but effective browsing and interaction is stillan open challenge, especially accessing the data a (hy-per)graph represents.

• Develop a visualization and programming environ-ment on top of hypergraph representationsPart of our goals is to allow to access and modifythe software artifacts that are represented by hy-pergraphs and not to develop a visualization systemonly for gaining insight into software. Therefore thedesign of our visualization method should be ori-ented towards a visual programming environmentthat allows programming tasks.

5. Knowledge-based software visualizationIn this section we discuss our approach for knowledge-based software visualization. First we formally introducehypergraphs that where used to develop a hypergraph-based data model. We then discuss how source code arti-facts and relations for a concrete programming languageand present how they can be represented in the proposed

data model. Afterwards we demonstrate a hypergraph-based query language and finish this section with detailsabout hypergraph visualization.

5.1 HypergraphsCommon graphs, people are familiar with, usually connecttwo nodes with an edge displayed as a line. Hypergraphsare generalized graphs in which an edge can connect morethan two nodes. A graph with edges connecting node-pairs is just a special case of a hypergraph. Definition 1formally defines a hypergraph.

Definition 1. Let V = {v1, . . . , vn} be a finite set, whosemembers are called nodes. A hypergraph on V is pairH = (V, ε), where ε is a family (Ei)i∈I of subsets of V .The members of ε are called hyperedges.

The following Definitions 2, 3 illustrate the mathematicaltransformation of hypergraph into a bipartite incidencegraph, using hypergraph’s incidence matrix.

Definition 2. Let H = (V, ε) be a hypergraph with m =|ε| edges and n = |V | nodes. The edge-node incidencematrix of H is:

MH ∈Mm×n({0, 1})

and defined as:

mi,j =

{1 if vj ∈ Ei0 else

Definition 3. For a hypergraph H = (V, ε) with an in-cidence matrix MH the bipartite incidence graph

BH = (NV ∪Nε, E)

is defined as follows:

E = {{mi, nj} : mi ∈ Nε, nj ∈ Nv, and mi,j = 1}Nε = {mi : Ei ∈ ε}NV = {nj : vj ∈ V }

The incidence matrix representation can be used for ac-tual implementation, however for effective memory stor-age an implementation using sparse matrices is needed.Using Definition 3 it is possible to utilize well known graphlayout algorithm to visualize hypergraphs as described inSection 5.6.

5.2 Hypergraph-based data modelInspired by the existing approaches for representing knowl-edge, mentioned in Section 3, we have defined a suitabledata model based on hypergraph representations to storeknowledge about software artifacts and their interrela-tions.

The hypergraph-based data model is based on the fol-lowing Definition of a modified, possibly oriented and or-dered, hypergraph that utilizes incidences to specify howhyperedges connect nodes. This model is an enhancementof the hypergraph-based representation of Topic Maps in-troduced in [3].

Definition 4. A labeled, possibly oriented, hypergraphis H = (V,E, I, L, λV , λE , λI , λL) where V,E, I, L are dis-joint finite sets and we call V the node set of H, E the


hyperedge set of H, I the incidence set of H and L theset of labels of H. The mappings λV : V → P (I) andλE : E → P (I) satisfy the following conditions:

∀v 6= v′ λV (v) ∩ λV (v′) = 0 ∪v∈V λV (v) = I

∀e 6= e′ λE(e) ∩ λE(e′) = 0 ∪e∈E λE(e) = I

The map λI : I → {IN,OUT,NONE} specifies orienta-tion of hyperedges and map λL : N ∪ E ∪ I → L assignslabels to hypergraph elements.

Figure 1 illustrates the model from above Definition. Thishypergraph definition allows to model complex relationssimilar to Topic Maps. Based on this hypergraph defi-nition we can define a simplified hypergraph-based datamodel similar to GXL mentioned in Section 3.

Hypergraph

NodeIncidence+Direction+Order

Hyperedge

GraphElement+Label

1

0..n

Attribute+Name+Value

n 1

1 0..n

Figure 1: Hypergraph-based data model.

This hypergraph data model does not limit incidences toalways connect one hyperedge and one node, but alsoallows that a hyperedge is connected through an inci-dence to another hyperedge or another incidence. Thisapproach, as proposed in works described in Section 3,actually allows to represent knowledge.

Allowing incidences to connect not only hyperedeges andnodes is the key aspect in modeling heterogeneous dataand their relations. This approach allows reification ofconcepts that represent objects of interest. A (hyper)graphdata model, opposed to e.g. a relation data model, doesnot necessarily need to known the application area in ad-vance, thus the structure of stored information does notneed to be know in advance and can be heterogeneous.

5.3 Identification of software artifacts and relationsThere are many software artifacts that can be found onvarious levels of software and software development pro-cesses. To verify our approach we have focused on soft-ware artifacts at the core of each software – artifacts foundin source code and related source code documentations.

We have focused on the programming language Lua [10],which has academic origin, and software systems imple-mented in it. Lua is a simple embeddable scripting lan-guage with very clean syntax which retains high expres-siveness. Lua uses only simple programming conceptswhich are common among many other languages, how-ever it provides mechanisms to extend its behavior that

may become complicated and difficult to understand forbeginners. For example Lua does not support object ori-ented programming but it is capable to simulate manyobject oriented features found in other languages withpowerful meta-table mechanisms. Visualization of theselanguage specific features would certainly help with un-derstanding. Before we can visualize software artifactsthat can be found in projects implemented in Lua, it isnecessary to identify them and to determine possible ar-tifact relations.

Identification of software artifacts is not a straightforwardprocess and we have to incorporate users of Lua languageand their point of view. It is important to summarize andcategorize software artifacts that developers and sourcecode analysts may be interested in. Of course identifyingrelations should undergo a similar process. Identifyingartifacts and relations is important for searching tasks,because repository queering is often done by using a querylanguage that utilizes relations.

For defining Lua hypergraph representations we decidedto start with its documentation extracting relevant topicsand their relations. We have identified several categoriesof software artifacts and their relations. The first cate-gory is the syntax level and lexical conventions that canbe derived from language syntax. Relevant artifacts inthis category are: identifiers, keywords, symbols, com-ments and finally Lua syntax rules. Each artifact can berepresented by a hypergraph node and therefore all arti-facts are equal. Differences between them are stored asspecific relations. For example we can use is-of-type rela-tion between type-nodes and any other nodes. There aremany other relations between artifacts and other artifacttypes such as variables, values and data types.

As our system is intended for software analysis we focusmainly on relations which emerge from syntax rules butare not directly observable from source code. For exam-ple during functional analysis we would like to observefunction relations as mentioned in Table 1.

Table 1: Relations a function can participate in.Relation Descriptionis documented Is documented with short

and/or long descriptionis nested in Is placed in a tableis method of Function acts as a method in

table (defines implicit self pa-rameter)

has parameter Function has parametertail calls Function calls another func-

tion via tail callreturns Returns some variable or ex-

pressiondefines Defines local variables or local

functionscalls Calls functionaccess Function accesses/uses global

variables or upvalueshas environment Function has defines environ-

menthas body Executes statements in func-

tion’s body


Similarly we can identify other software artifacts and theirrelations, for example to understand architectural design,we need to look at the program decomposition into Luamodules and files. A module written in Lua is usuallydefined by it’s name, consists of several functions and mayuse other modules.

5.4 Representing software artifacts and relationsTo illustrate the hypergraph-based representation, we canuse the software artifacts and relations extracted froma software system in our study in Section 6

From the system’s source code we extracted the downloadfunction software artifact that is involved in two relations:calls and tail calls related to function calls. We can for-mulate following statements describing relations betweenfunctions:

• ”Function download calls functions sys.isDir,sys.copy, pcall”

• ”Function download tail-calls function curlDown-load and socketDownload”

Illustration of these relations is shown in Figure 2.1

download

caller

tail_calls

called

curlDownload

SocketDownload

called

download

caller

called

called

called

sys.isDir

sys.copy

pcall

calls

Figure 2: Relations related to the download func-tion.

As can be seen in Figure 2, we used incidences to clarifyroles of the functions, thus it is easily recognizable whichfunction calls and which functions are called. Although astandard function call and a tail call may seem the same,in Lua the later is internally optimized and is used in spe-cial programming cases, thus differentiating them may behelpful. Similarly we can approach other possible soft-ware artifacts and relations. Important to note is thatnodes may play different roles in different relations. Al-though the functions displayed in Figure 2 play only rolescaller or called, we could easily find out by looking intosource code, that e.g. the function curlDownload callsother functions not mentioned here. Thus the role of thecurlDownload function is called as shown in Figure 2, butmay be also caller for other functions. Differentiatingroles a node plays in different relations can be of advancewhen we search for other nodes connected to a node byhyperedges.

1We use following graphical notation: blue spheres repre-sent nodes, green incidences and red hyperedges.

The proposed hypergraph-based data model also offersthe possibility to use oriented and/or ordered incidences,making the hyperedge with these incidences also oriented/ ordered2. In the above example we used only non-oriented and non-ordered incidences, thus the relationswhere not oriented nor ordered. To clarify the ”orien-tation” of hyperedges we used only labels of incidences.Oriented or ordered incidences can be used to enhancethe information stored in incidences (node’s role(s)). Us-ing ordered incidences could be used to order parametersof a function. Oriented incidences could be used when afunction returns values by function’s parameters. Suchsituations are common in C programs. For example thefunction from OpenGL’s API

void glGetShaderSource(GLuint shader, GL-sizei bufSize, GLsizei *length, GLchar *source);

takes four parameters, but the length, source are usedas return parameters, and of course the parameters areordered. Figure 3 illustrates the has parameters relations

glGetShaderSource

functionparameter

shader

bufSize

length

has_parameters

source

parameter

parameter

parameter

1

2

3

4

Figure 3: Hypergraph-based data model.

with oriented (displayed as lines with arrows) and orderedincidences (numbers shown inside circles)

5.5 Hypergraph-based query languageDisplaying large hypergraphs might be difficult to com-prehend. Therefore some filtering or querying is needed.Inspired by the graph query languages mentioned in Sec-tion 3, we can also develop a query language for our hy-pergraph data model.

The proposed hypergraph query language is based on hy-pergraph patterns that can be represented in graphicaland textual form and we define a transformation betweenthese two representations. The querying itself is done byan algorithm that receives as parameters the hypergraphquery and the queried hypergraphs and returns a hyper-graph with matched hypergraph elements.

The syntax of the textual query language is based on sim-ple hyperedge queries in form

E(I1 : N1, . . . , In : Nn)

where E is a constrain for hyperedge label and I1 : N1, . . . ,In : Nn is a sequence of I : N pairs which denote con-

2Oriented incidences make the hyperedge oriented. Or-dering incidences orders hyperedge’s ”tentacles”.


strains for labels of incidence-node pairs. These con-strains are actually patterns that are searched in the la-bels of hypregraph elements. We used the ∗ symbol forconvenience as a short pattern that matches any label.

The proposed query language is based on hypergraphs,which can be also visualized. The Figure 4 illustratesthe relation between the textual notation of the followingsimple query,

calls(caller:* ; called:*)

which searches for all calls relations and receives all nodesconnected trough these relations and play roles caller orcalled and its graphical hypergraph representation.

calls*

called

*

caller

calls(caller : *, called : *)

E(I : N , I : N )11 2 2

Figure 4: Mapping between textual and graphicalrepresentation of a query.

The discussed simple hyperedge queries can be chainedtogether by an and operator to form complex hypergraphpatterns. The following snippet shows an example ofa complex textual query:

has-parameters(*:installPackage, parameter:*) and

is-documented(function:installPackage,

short-description:*) and

is-documented(of_function:installPackage,

parameter:*,

description:*)

This query defines a hypergraph pattern that searchesfor a function labeled installPackage. Using the has-parameters relation it receives all parameters of this func-tion and using the is-documented relations also function’sshort-description and description of function’s parame-ters. Visualization of this query is shown in Figure 5 andthe result of this query is shown in Figure 6. Detailsabout the results of this query and from what they wereobtained are described in Section 6.

Figures 5 and 6 show visualizations of a hypergraph queryand the corresponding result hypergraph. The Figureswere taken from an initial hypergraph query prototypethat used on 2D hypergraph visualizations. In these Fig-ures hyperedges are displayed as red spheres, incidencesas green and nodes as yellow spheres. For the hypergraphlayout an early version of the layout Algorithm 1 was used.

Formally, the syntax of the textual representation of thehypergraph queries can be defined by the following Pars-ing Expression Grammar (PEG), where terminals are dis-

Figure 5: Example of complex query

Figure 6: Results of the query shown in Figure 5

played in bold, nonterminals in italic and S is the top-level grammar rule:

S ← query (and query) ∗query ← name (body (, body) ∗ )body ← name : name

name ← * | patternpattern ← regexp

The hypergraph matching algorithm iterates over hyper-edges of the query tries to match hyperedges of the queriedhypergraph by iterating over them. For every matchedhyperedge the algorithm then tries to match all incidence-node pairs of the query hyperedge with incidence-nodepairs of the queried hyperedge. If at least one incidence-node pairs is matched then the matched hyperedge towhich they belong is added to a temporal hyperedge list.If at least one hyperedge is found then it is added to theresulting hypergraph also with matched incidences andnodes.


5.6 Hypergraph visualizationGraph visualization faces several problems and variousgraph visualization techniques have been developed. Whenconsidering graph-like visualization, hypergraph visual-ization can be based on two approaches: (a) develop aspecialized layout algorithm specially for hypergraphs (b)transform hypergraphs into bipartite graph. The first ap-proach has been proposed in [15] and uses a modifiedforce-based layout algorithm.

We selected the transformation into bipartite graph in ourapproach, because it allows us to utilize existing layoutalgorithms developed for common graphs. However, ourhypergaph model uses incidences which along nodes andhyperedges also must be considered by the hypergraphlayout algorithm. The simplest solution is to representhypergraph’s nodes, hyperedges and incidences as nodesof the bipartite graph and for each hyperedge-incidence-node triple two edges (hyperedge-incidence and incidence-node) in the bipartite graph are created. This approachhowever unnecessarily introduces for each incidence a nodein the bipartite graph thus increasing the number of bi-partite graph nodes the layout algorithm has to process.This can be easily solved by ignoring incidences by the lay-out algorithm. Position of incidences can be then easilycalculated from node’s and hyperedge’s position as centerbetween them. However as our hypergraph data modelallows to connect hyperedges also to incidences, so in-cidences can not be displayed only as lines. Also theirposition might be affected by forces created by hyper-edges connected to them. Therefore incidences should beconsidered by the layout algorithm as nodes.

The modified Fruchterman-Reingold layout algorithm weused in our prototype visualization system is shown inAlgorithm 1. It introduces the concept of graph centerwhich is calculated to easily position the layouted graphto origin of world coordinates. The algorithm assigns ran-dom positions to nodes at start and then iteratively movesthem to new positions based on attracting and repellingforces these nodes produce.

During visualization hypergraph elements are displayedas colored spheres connected with lines representing hy-peredges with following color mapping:

• red spheres – hyperedges

• yellow spheres – incidences

• green spheres – nodes

We can look at red spheres as centers of hyperedges. Vi-sualization prototype can also hide incidences either todecrease visual clutter or to weaken the number of nodesthe layout algorithm has to process.

The visualization takes place in 3D space in which the usercan interactively navigate and explore presented hyper-graph representation of analyzed software projects. Themodifications we made to the algorithm allow to pausethe layout algorithm globally or for selected nodes. Thisallows the user to interactively interact with the layoutalgorithm and to manipulate nodes or hyperedges of in-terest into preferred positions, possibly making the visu-alization more comprehensible. The implemented graph

layout algorithm is capable to handle thousands of hyper-graph nodes at feasible comfort for the user, however forlarger hypergraphs more effective layout algorithms mustbe considered.

The bipartite graph representation of a hypergraph how-ever opens possible research questions for optimizing hy-pergraph layout. When considering hypergraph nodesand hyperedges represented as two disjoint node sets ofthe representing bipartite graph, we could tweak the lay-out algorithm to consider this fact. Recently multi-levelgraph layout algorithms showed their potential in visual-izing large graphs [6]. They construct a set of sub-graphswith various graph coarsening and layout the final graphby layouting the hierarchy of subgraphs. The two sets ofthe bipartite graph could be the initial clustering for thesemethod and further sub-graph cluster could be created byconsidering e.g. the node degree. However these are ideasthat require further research as graph layout algorithmsare not in main focus of this thesis.

Algorithm 1 Modified Fruchterman-Reingold layout al-gorithm [20]

Require: graph G = (V,E), world origin pSEnsure: position pu for ∀u ∈ V

for all u ∈ V dopu ← random()V (u)← 0 . reset total speed

end forloop

pb ← 0 . reset center positionfor all u ∈ V do

F (u)← 0 . reset total forcepb ← pb + pu . add to centerF (u)← F (u) +

∑fr(duv) . add repeling forces

end forfor all (u, v) ∈ E do

F (u)← F (u) + fa(duv)F (v)← F (v) + fa(duv) . add attractive forces

end forpb ← pb/|V | . set center positionfor all u ∈ V do

F (u)← F (u) + fc(dbS). add force between center and world origin

if |F (u)| < MIN F then~p← F (u)/|F (u)| ×min{MAX F, |F (u)|} .

force vector~p← α~ppu ← pu + ~p+ V (u) . move according force

and actual speedV (u)← γ(V (u) + vecp)

. set new speed, γ < 1 stiffnesselse

V (u)← 0 . reset total speedend if

end forend loop

6. Visualization of an existing software systemThe software in our study is a an open-source projectcalled LuaDist3, which is written in the Lua program-ming language and well documented. It is a small smallproject containing 80kB of source code, but the project’s

3Available at: www.luadist.org


documentation make nearly 30% of source code. The pur-pose of the analyzed software is management, automaticbuilding and deployment of Lua modules and general li-braries in an multi-platform development environment. Itconsist of various modules ranging from network commu-nication, trough dependency resolving to compilation andinstallation modules.

We focused on obtaining not directly visible relations andartifacts. Together we were searching for just eight nodetypes and seven different hyperedge types. For the com-pleteness of the extracted hypergraph the above identifiedartifacts types (module, function etc.) where automati-cally inserted into the hypergraph as nodes. Additionallythe is_of_type relation was used to connect the extractedsoftware artifacts to the corresponding nodes representingthe type of these nodes.

We were able to extract 1233 nodes and 459 hyperedges,what are for such a small project relatively high numbers,considering the small amount of different node and hyper-edge types. Searching for other node or hyperedge typeswould certainly dramatically increase the number of ex-tracted artifacts and relations. For large-scale projects wecan expect very high numbers of artifacts and relations.

Nearly the whole extracted hypergraph is displayed inFigure 7. The user can explore this visualization by ro-tating the scene and freely fly into the scene using a vir-tual camera, thus focusing the view towards hypergraphparts the user is interested in. In Figure 7 we can seethree highly interconnected hyperedges in the right mid-dle part, which where manually positioned by user furtherfrom hypergraph center. Moving the virtual camera closeto these hyperedges reveals their labels and that they rep-resent is_of_type relations, thus their connection to mostnodes is obvious.

Figure 7: Visualization of the whole extracted hy-pergraph.

The extracted hypergraph contains many nodes and rela-tions, which make comprehending relations difficult. The-refore we can use the query language proposed in Sec-tion 5.5 to filter the hypergprah to contain only parts wewould like to observe. For example to visualize the projectdecomposition into modules we can use following simplequery:

defines(module:*; function:*)

This query receives all modules together with their func-tions, thus obtaining information on architecture as wellinformation about modules’ interfaces. Figure 8 illus-trates the hypergraph obtained by this query, from whichthe number of modules is easily observable. From theFigure 8 we can also visually identify modules containinga higher number of functions (hypergraph cluster in Fig-ure’s middle and the cluster bottom-right), thus revealingthat these two modules provide a rich API.

Figure 8: Visualization of modules and their func-tions.

To obtain this information from source code the user wouldhave to read through the actual source code files search-ing for module definitions and modules’ functions – whichis difficult due to the fact, that Lua is similar to Javain the way that Lua does not uses header files like e.g.C++. Although a good IDE might provide a tree-view ofmodules and their functions, still identifying the size ofmodules (number of functions) is difficult and currentlysource code metrics tools are used for this purpose.

Identifying modules and their functions can be quite easycompared to understanding all function call relations. Forthis purpose, the user can use the following query:

calls(caller:*; called:*)

This query receives all calls relations between functionsand the results of this query are shown in Figure 9. TheFigure displays a cluster of call relations between func-tions, however the resulting hypergraph is not a full call-graph.

As functions are one of the most important language con-struct of Lua language (separation into modules is doneby splitting code into files and additionally defined by call-ing module function), we can filter the whole hypergraphwith the following query to obtain all functions and theirparameters:

is_of_type(object:*, type:function) and

has_parameters(function:*, parameter:*)


Figure 9: Visualization of the function calls in Lu-aDist.

The query searches for all nodes that play role objectand are connected to a node labeled function playingrole type in the relation is_of_type. This ensures thatthe right is_of_type relation, the one connected to anode function, is selected among other relations with samename. The second part of this query receives all nodesplaying role function and parameter in the relation has-parameters, thus effectively retrieving all parameters offound functions.

The results of this query are shown in Figure 10. In figurecenter the is of type relation is placed to which individualnodes representing system’s functions are connected. Thefunction parameters are positioned at most outer rim ofthis cluster, e.g. the top and bottom right figure part showtwo functions with several parameters. This cluster canbe considered as semantically closed as it contains nodesrepresenting functions and their parameters connected byrelations.

Of course, the functions are not the only artifact that wasextracted. To show crossing semantic boundaries we canuse this query:

is_of_type(object:*, type:function) and

has_parameters(*:installPackage, parameter:*) and

is_documented(function:installPackage,

short_description:*) and

is_documented(of_function:installPackage,

parameter:*,

description:*)

This query receives all functions and for one function,installPackage, receives function’s parameters and relateddocumentation strings of the function and parameters.Visualization of the result hypergraphs is shown in Fig-ure 11. We can identify two clusters: the top right clus-ter represents all the functions of the system centeredaround the is-instance-of relation, and the left bottomcluster contains documentation nodes and parameters re-

Figure 10: All functions of the software projectobtained by a query.

lated to the installPackage nodes. The line that connectsthese two cluster is actually one of hypergraph arcs of theis-instance-of relation; the node connected to this hyper-graph arc in the left bottom cluster is the installPackagenode.

7. Towards a Visual Programming EnvironmentThe presented visualizations in the previous section aresuitable for interactive exploring and searching tasks. How-ever our goal is to make also the step towards a visualprogramming environment in which programming taskscan be performed. Therefor we need access to the under-lying source code. For programming tasks the interfacecan change individual nodes into floating billboards con-taining a 2D textual editor. These billboards are alwaysparallel to the projection plane, thus are not distorted byperspective projection making the text they display easilyreadable. The user can dynamically zoom into these bill-boards to modify the source code these nodes represent.

Billboards are just another way of visualizing hypergraphnodes and are also affected by the force directed layout al-gorithm, but can be attracted into specified positions. Forthis purpose the user uses meta-nodes and meta-edges,which are not part of the visualized hypergraph, to selectand move nodes of interest. This allows to force positionof e.g. documentation nodes to upper part of a window,billboards containing development cycle information e.g.into right window part and source code and software ar-tifact revisions into left window part.

From software artifacts displayed in these billboards linksto other nodes or billboards show different relations. Oneof our goals is to implement a simple method for dis-playing contextual information around focused fragmentswith dynamic weighting of spring forces, based on rela-tion relevancy in hypergraph representation. Hyperedgerelevancy below certain threshold will not be displayed inorder to conserve screen space and reduce the complexityof the visualized graph.


Figure 11: All functions cluster with details abouta function.

Figure 12: Nodes containing text editors withsyntax-highlighting.

The Figure 12 shows a situation, where the user displayedseveral nodes as billboards containing text editor. Thefirst figure displays a billboard with source code of theinstallPackage function and in the background three func-tions that this function calls are shown. The call-graphrelations are displayed as a hyperedge.

8. ConclusionsIn this thesis we presented a method for software visual-ization that is based on hypergraph representations thatare used during the whole visualization process: from soft-ware representation, trough visualization to query mech-anisms and visual programming. Our approach combinesknowledge-based software representation with graph visu-alization techniques to provide an interesting visual pro-gramming environment that features visual browsing andquerying.

The hypergraph data model was inspired by the exist-ing approaches in knowledge representation field. Hyper-graph representations of software artifacts allow to storesemantic relations, which are currently often lost in var-ious file formats of common integrated development en-

vironments. Hypergraph representations also are capa-ble to store heterogeneous data which make them moreflexible than common databases that need to be designedwith exact application domain knowledge. We utilized thehypergraph data model that uses incidences that allowto model more complex relations than common graphs.We implemented an effective hypergraph implementationthat uses nested hash-tables to represent hypergraph’s in-cidence matrix. For filtering hypergraphs we introduced aquery language in which queries can be defined in textualand also graphical form. The queries are hypergraph pat-terns that are searched in the queried hypergraph. Theresults of queries are again hypergraphs, thus making thisapproach transparent for visualization methods as it op-erates on one data structure.

Visualization of hypergraph representation of software ar-tifacts is based on the well known force-based graph layoutalgorithms. We utilized the mathematical transformationof hypergraphs into bipartite graphs, thus allowing thepossible use of state-of-art graph layout algorithms. Theimplemented layout algorithm is suitable only for smallgraphs, however it was sufficient for experimental visual-izations of an existing software system.

The proposed visual programming environment on topof hypergraph visualizations can be considered as impor-tant as the hypergraph representations itself. Althoughplacing 2D windows in 3D space has been previously re-searched, applications in software visualization can beconsidered as a new approach. Also the dynamic bill-board placement caused by the connection to force-basedhypergraph layouting can be the base to future human-computer interaction methods in 3D space.

The experimental visualizations of an existing system sho-wed, that even in small software projects the number ofsoftware artifacts and their relations can be high. Andwe only focused on source code artifacts. Applying ourhypergraph-based approach to large software projectswould be certainly challenging, especially for visualiza-tion and interaction.

Acknowledgements. This work was partially supportedby the Scientific Grant Agency of Slovak Republic, grantVEGA No. 1/3103/06: Information infrastructure for in-formation processing in distributed environments; and theCultural and Educational Grant Agency of the Slovak Re-public, grant KEGA No. 244-022STU-4/2010: Supportfor Parallel and Distributed Computing Education.

References[1] B. Amann and I. Fundulaki. Integrating ontologies and thesauri to

build RDF schemas. In Proceedings of the Third EuropeanConference on Research and Advanced Technology for DigitalLibraries, page 234U253, 1999.

[2] R. Angles and C. Gutierrez. Survey of graph database models.ACM Computing Surveys (CSUR), 40(1):1–39, 2008.

[3] P. Auillans, P. O. de Mendez, P. Rosenstiehl, and B. Vatant. Aformal model for topic maps. In ISWC ’02: Proceedings of theFirst International Semantic Web Conference on The SemanticWeb, pages 69–83, London, UK, 2002. Springer-Verlag.

[4] U. Brandes, J. Lerner, and C. Pich. GXL to GraphML and ViceVersa with XSLT1. Electronic Notes in Theoretical ComputerScience, 127(1):113–125, 2005.

[5] K. D. Visual exploration of large databases. In Communications ofthe ACM, volume 44, pages 38–44, 2001.


[6] Y. Frishman and A. Tal. Multi-level graph layout on the GPU.IEEE Transactions on Visualization and Computer Graphics,13(6):1310–1319, 2007.

[7] L. Garshol. tolog–a topic maps query language. Charting theTopic Maps Research and Applications Landscape, pages183–196, 2006.

[8] J. Hayes. A graph model for RDF. Master’s thesis, TechnischeUniversität Darmstadt, Universidad de Chile, August 2004.

[9] H. He and A. Singh. Graphs-at-a-time: query language and accessmethods for graph databases. In Proc. of SIGMOD, pages405–418. Citeseer, 2008.

[10] R. Ierusalimschy, L. H. de Figueiredo, and W. C. Filho. Lua—anextensible extension language. Softw. Pract. Exper.,26(6):635–652, 1996.

[11] R. Koschke. Software visualization in software maintenance,reverse engineering, and re-engineering: a research survey.Journal of Software Maintenance, 15(2):87–109, 2003.

[12] J. H. Larkin and H. A. Simon. Why a diagram is (sometimes)worth ten thousand words. Cognitive Science, 11(1):65–100,1987.

[13] M. Lenzerini, D. Milano, A. Poggi, and I. Roma. OntologyRepresentation & Reasoning. Universit di Roma La Sapienza,Roma, Italy, Tech. Rep. NoE InterOp (IST-508011), WP8, subtask,8, 2004.

[14] M. Levene and A. Poulovassilis. An object-oriented data modelformalised through hypergraphs. Data & Knowledge Engineering,6(3):205–224, 1991.

[15] E. Makinen. How to draw a hypergraph. 34:177–185, 1990.[16] B. Myers. Taxonomies of visual programming and program

visualization. Journal of Visual Languages and Computing,1(1):97–123, 1990.

[17] M. Petre, A. Blackwell, and T. Green. Cognitive questions insoftware visualisation. In J. Stasko, J. Domingue, M. H. Brown,and B. A. Price, editors, Software Visualization: Programming asa Multimedia Experience, chapter 30, pages 453–480. MIT Press,1997.

[18] A. Poulovassilis and P. McBrien. A general formal framework forschema transformation. DATA AND KNOWLEDGEENGINEERING, 28:47–71, 1998.

[19] D. Theodoratos. Semantic integration and querying ofheterogeneous data sources using a hypergraph data model. In InProc. BNCOD’02, LNCS 2405, pages 166–182, 2002.

[20] J. Ukrop. Constraints in force-directed graph visualization,bachelor thesis, 2009. supervisor: P. Kapec.

[21] A. Winter. Exchanging graphs with GXL. In Graph Drawing,pages 54–58. Springer, 2002.

[22] A. Wrightson. TMQL Draft (Topic Map Query Language).Ontopia, BSI. Online: http://www. y12. doe.gov/sgml/sc34/document/0186. doc, 2000.

Selected Papers by the AuthorP. Kapec, M. Šperka. Interactive Visualization of Abstract Data.

Science & Military, Vol. 5, No. 1, 84–90, 2010.

P. Kapec. Visual Programming Environment Based on HypergraphRepresentations. ICCVG 2010 – LNCS 6375: InternationalConference on Computer Vision and Graphics Proceedings, PartII, pages 9–16, 2010.

L’. Ukrop, M. Jakubéci, P. Kapec. Metaphorical Visualizations ofGraph Structures. GraVisMa 2010: 2nd International Workshopon Computer Graphics, Computer Vision and Mathematics,pages 115–122, 2010.

M. Šperka, P. Kapec, I. Ruttkay-Nedecký. Exploring andUnderstanding Software Behaviour Using Interactive 3DVisualization. In: ICETA 2010: 8th International Conference onEmerging eLearning Technologies and Applications,pages281–287, 2010.

P. Kapec, M. Šperka. Visual Information Discovery in Large DataSets. In: INTELS 2010 – Intelligent Systems: Proceedings of theNinth International Symposium, pages 123–128, 2010.

P. Kapec. Visualizing software artifacts using hypergraphs. In: SCCG2010 Spring Conference on Computer Graphics, pages 33–38,2010.

P. Kapec. Hypergraph-based software visualization. In:GraVisMa2009 Workshop Proceedings : International Workshop onComputer Graphics, Computer Vision and Mathematics, pages149–153, 2009.

M. Kottman, P. Kapec. Villie – a hypergraph-based visualprogramming language. In: ITAT 2010: Applications andTheory: Conference proceedings, pages 69–74, 2010.

P. Kapec, P. Drahoš. Hypergraph based software representation,mining and visualization. In: INFORMATICS 2009 :Proceedings of the Tenth International Conference onInformatics, pages 253–258, 2009

P. Kapec, P. Drahoš. Knowledge-based programming environments.In: ITAT 2008: Information Technologies – Applications andTheory: Conference proceedings, pages 27–30, 2008.

M. Šperka, P. Kapec. Visual Information Discovery in SoftwareSystems. In: Znalosti 2008, pages 339–342, 2008.

knowledge-based software representation, querying and...

Documents