Bi-Annual Report 2009-2010

Advanced School for Computing and Imaging

2012 Advanced School for Computing and Imaging

p/a DUT / EWIP.O. Box 50312600 GA DELFTThe Netherlands

Phone: + 31 15 27 88032Fax: +31 15 27 86632asci@ewi.tudelft.nl http://www.asci.tudelft.nl

ASCI is a Dutch graduate school established in 1993 and accredited by the Royal Netherlands Academy of Arts and Sciences. Research groups of Delft University of Technology, Vrije Universiteit, University of Amsterdam, Leiden University, University Utrecht, University of Twente, University of Groningen, Eindhoven University of Technology, Erasmus University Rotterdam and Radboud University Nijmegen participate in ASCI.

Contents

Preface...................................................................................................................................................................................... 51 ASCI and its Research.................................................................................................................................................... 7

1.1 About ASCI.......................................................................................................................................................... 71.2 Participating Groups............................................................................................................................................ 71.3 ASCI Research Themes.................................................................................................................................... 10

2 Cooperation within ASCI.............................................................................................................................................. 123 Scientific Output along Research Themes................................................................................................................. 13

3.1 Contribution of VU-EW-CS..........................................................................................................….13 3.2 Contribution of UvA-FNWI-SNE......................................................................................................................... 153.3 Contribution of UvA-FNWI-IAS.......................................................................................................................... .183.4 Contribution of UvA FNWI-ISIS.......................................................................................................................... 203.5 Contribution of UvA-FNWI-CSA......................................................................................................................... 213.6 Contribution of TUD-EWI-ST-PGS..................................................................................................................... 233.7 Contribution of TUD-EWI-MM-CGCC................................................................................................................. 243.8 Contribution of TUD-TNW-QI............................................................................................................................. 253.9 Contribution of UL-LIACS................................................................................................................................... 273.10 Contribution of UU-ICS-GMT............................................................................................................................. 293.11 Contribution of TUD-L&R-FRS........................................................................................................................... 303.12 Contribution of RUG-CS-SVCG......................................................................................................................... 313.13 Contribution of RUG-CS-IS................................................................................................................................ 333.14 Contribution of TUE-ET...................................................................................................................................... 353.15 Contribution of TUE-WI...................................................................................................................................... 363.16 Contribution of TUE-BMT................................................................................................................................... 383.17 Contribution of UL-LUMC................................................................................................................................... 393.18 Contribution of UT-EWI-DACS........................................................................................................................... 413.19 Contribution of UT-EWI-CAES........................................................................................................................... 433.20 Contribution of EUR-UMCR............................................................................................................................... 453.21 Contribution of RUN-UMCR............................................................................................................................... 46

Preface

Computer and network technology have a still rapidly increasing impact on society, business, and science. Digital media are quickly replacing classic media, and due to the availability of huge data collections Big Data has become the major challenge of Computer Science. The research of ASCI matches very well with these developments, as for instance fast networks, parallel processing, multi-media analysis and visualization are vital ingredients to deal with the data flood and to make optimal use of all opportunities.

In this report an overview is provided of the research of ASCI in the period 2009-2010. We have continued to produce a bi-annual report instead of annual ones. There ceased to be a need for presenting all results of individual groups, as these are readily available through the web. However, this presumes that it is known who is doing what, which requires an overview of a wide variety of groups and researchers. With this report we aim to provide such an overview, both for participants of ASCI and for others that are interested in our research. Instead of a focus on exhaustive enumeration of all results, we collected for each research group in ASCI a view on current and past research, future plans and key publications, and we hope this will lead to increasing awareness, new ideas and new collaborations.

The period 2009-2010 marked an important change in the way of organization of our major event. In 2009 the 15 th ASCI-conference was held in Zeewolde, as a three-day event at a Center Parcs location, following tradition. The audience consisted of some 75 ASCI PhD’s and 25 staff members. Keynote speakers were Marc Dacier, Symantec Research Labs; David Hogg, University of Leeds and David Stork, Ricoh InnovationsIn 2010 the ASCI-conference took place in Veldhoven as part of SIREN. SIREN stands for the Scientific ICT Research Event Netherlands, and was held in 2010 for the first time, organized by IPN, the ICT-research Platform Netherlands. SIREN is one example of the increasing awareness in the Computer Science research community that joining forces and exchanging information across borders of research schools is important. The program included events of all three research schools in Computer Science: the ASCI-conference, the IPA Fall days, and the Annual SIKS-day. As a result, a broad overview of ICT research in the Netherlands was provided and ample opportunities to network. Keynote speakers were Jasmin Fisher of Microsoft Research Cambridge and Daniel Rueckert of Imperial College London.

In the years 2009 and 2010 two ASCI GNARP workshops have been held on Parallel Systems, and one ASCI Winterschool on Embedded Systems.

We hope you enjoy reading this report.

Henri BalScientific Director

ASCI bi-annual Report 2009-2010 5

ASCI bi-annual Report 2009-2010 6

1 ASCI and its Research

1.1 About ASCI

ASCI is a national research school on advanced computer and imaging systems. The school was founded in December 1993, and it was approved by the KNAW (Dutch Royal Academy of Sciences) in May 1995. In 2005 the school got its new accreditation for the coming six years.Participants in ASCI are groups from Delft University of Technology, the University of Amsterdam, the Vrije Universiteit, Leiden University and the University of Utrecht; the University of Twente, the University of Groningen, Eindhoven University of Technology and Radboud University Nijmegen have joined ASCI by association agreements.

ASCI performs research in two main fields: computing and imaging. In the course of time ‘imaging’ broadened to ‘multimedia data processing’. The activities within these fields are further classified based on their main target, either Methods and Algorithms (development of models and tools for scientific and industrial applications) or Systems and Architecture (large-scale integration in areas like telematics, embedded systems, communication and networks). In both categories fundamental and applied research is done within ASCI. Much of the ASCI research is interdisciplinary, involving multiple groups and areas from computer science, electrical engineering, physics, and other departments.

The school organizes a graduate program and a research program covering all major subjects concerning parallel, distributed, embedded, and real-time systems, performance analysis, image processing, image analysis, image synthesis, sensor interpretation, pattern recognition and computer vision. Every year ASCI organises the Annual ASCI Conference, the scientific meeting place for all participants in ASCI. Another annual activity is the GNARP workshop (GNARP Graduate Network of Applied Research in Parallel systems) which is organized by PhD students and which is a platform for presenting work in progress. Every two years ASCI organizes a winter/springschool, in 2008 on Embedded Systems.

1.2 Participating Groups

The following reseach groups participate in ASCI. They are represented together with their abbreviations. For each group the members are listed (situation January 2009).

VU-EW-CS Vrije Universiteit, Faculty of Sciences, Division of Mathematics and Computer Science, Dept. of Computer Sciencehttp://www.cs.vu.nl Prof.dr. A.S. Tanenbaum, Prof.dr.ir. H.E. Bal, Prof.dr.ir. M.R. van Steen, Dr.ing. T. Kielmann, Dr. G.E.O. Pierre, Dr. R. van Nieuwpoort, Dr.ir. H.J. Bos, Dr.B. Crispo, Dr.ir. C. van Reeuwijk, Dr. F.J. Seinstra

UvA-FNWI-SNE University of Amsterdam, Department of Computer Science,System and Network Engineering

http://www.science.uva.nl/research/sne Prof.dr.ir. C.T.A.M. de Laat, Dr. P. Grosso

UvA-FNWI-IAS University of Amsterdam, Faculty of Science, Informatics Institute, Intelligent Autonomous Systemshttp://isla.science.uva.nlProf.dr.ir. F.C.A. Groen, Prof.dr. D.M. Gavrila, Dr.ir. B.J.A. Kröse, Dr.ir. L. Dorst, Dr. G. Pavlin

UvA- FNWI-ISIS University of Amsterdam, Faculty of Sciences, Informatics Institute, Intelligent Sensory Information Systems Grouphttp://isla.science.uva.nlProf.dr.ir. A.W.M. Smeulders, Dr.ir. R. van den Boomgaard, Dr. M. Worring, Dr.Ing. J.M. Geusebroek, Dr. Th. Gevers, Dr. C.J. Veenman, R.F. Aldershoff, Drs. C.G.M. Snoek,Dr. N. Sebe

UvA-FNWI-CSA University of Amsterdam, Faculty of Sicence, Informatics Institute, Computer Systems Architecture Grouphttp://www.science.uva.nl/research/csaProf.dr. C. Jesshope, Dr. A.D. Pimentel

TUD-EWI-ST-PGS Delft University of Technology, Faculty of Electrical Engineering, Mathematics & Computer Science, Parallel and Distributed Systems Grouphttp://www.pds.ewi.tudelft.nlProf.dr.ir. H.J. Sips, Dr.ir. D.H.J. Epema, Prof.dr.ir. A.J.C. van Gemund, Prof.dr. C. Witteveen, Dr. K.G. Langendoen

ASCI bi-annual Report 2009-2010 7

TUD-EWI-MM-CGCC Delft University of Technology, Faculty of Electrical Engineering, Mathematics & Computer Science, Computer Graphics and CAD/CAM Grouphttp://graphics.tudelft.nl Prof.dr.ir. F.W. Jansen, Dr. W.F. Bronsvoort, Ir. F.H. Post, Dr. C.P. Botha

TUD-TNW-QI Delft University of Technology, Faculty of Applied Physics, Imaging Science & Technology, Quantitative Imaging Grouphttp://www.ist.tudelft.nl/qiProf.dr. L.J. van Vliet, Dr. B. Rieger

UL-LIACS Leiden University, Faculty of Mathematics and Natural Sciences, Leiden Institute of Advanced Computer Science (LIACS)http://www.liacs.nl/researchProf.dr. H.A.G. Wijshoff, Dr. M.S. Lew, Dr. A.A. Wolters, Dr. D.P. Huijsmans, Dr. E.M. Bakker, Dr.ir. T.P. Stefanov, Dr.ir. B. Kienhuis, Prof.dr.ir. E.F. Deprettere

UU-ICS-GMT Utrecht University, Faculty of Science, Department of Information and Computing Sciences,

Center for Geometry, Imaging and Virtual Environmentshttp://www.cs.uu.nl/centers/give/give-center.html and http://www.cs.uu.nl/groups/MG Prof.dr. M.H. Overmars, Dr. M. van Kreveld, Dr.ir. F. van der Stappen, Dr. R. Veltkamp, Prof.dr P.J. Werkhoven

TUD-L&R-FRS Delft University of Technology, Faculty of Aerospace EngineeringDepartment of Earth Observation and Space systems (DEOS), Optical and Laser Remote Sensing Grouphttp://www.lr.tudelft.nl/olrs Dr.ir. B.G.H. Gorte

RUG-CS-SVCG University of Groningen, Faculty of Mathematics and Natural Sciences, Johann Bernoulli Institute of Mathematics and Computer Science, Scientific Visualization and Computer Graphics http://www.cs.rug.nl/svcg Prof.dr. J.B.T.M. Roerdink, Dr. H. Bekker

RUG-CS-IS University of Groningen, Faculty of Mathematics and Natural Sciences, Johann Bernoulli Institute of Mathematics and Computer Science, Intelligent Systemshttp://www.cs.rug.nl/is Prof.dr. N. Petkov, Dr. M.H.F. Wilkinson, Prof.dr. M. Biehl

TUE-ET Technische Universiteit Eindhoven, Faculty of Electrical Engineering, Design Methodology for Electronic Systemshttp://www.es.ele.tue.nl Prof.dr.ir. R.H.J.M.Otten, Dr.ir. T. Basten, Prof.dr. H. Corporaal, Dr.ir. M.C.W. Geilen, Prof.dr.ir. G. de Haan, Dr.ir. J.P.M. Voeten

TUE-WI Technische Universiteit Eindhoven, Department of Mathematics and Computer Science, Visualization Grouphttp://www.win.tue.nl/vis Prof.dr.ir. J.J. van Wijk, Prof.dr.ir. R. van Liere, Dr.ir. H.M.M. van de Wetering, Dr. M.A. Westenberg, Dr. A.C. Jalba

TUE-BMT Technische Universiteit Eindhoven, Department of Biomedical EngineeringBiomedical Image Analysishttp://bmia.bmt.tue.nl Prof.dr.ir. B.M. ter Haar Romeny, Prof.dr. F.A. Gerritsen, Prof.dr. L.M.J. Florack, Dr. A. Vilanova Bartroli, Dr.ir. H.C. van Assen, Dr.ir. R. Duits, Dr.ir. B. Platel

UL-LUMC Leiden University Medical Center, division of Image Processing, laboratorium voor klinische en Experimentele Beeldverwerkinghttp://www.lumc.nl/con/1010/83058/87360/87377Prof.dr.ir. Johan H.C. Reiber, Dr.ir. B.P.F. Lelieveldt, Prof.dr. R. Nelissen

UT-EWI-DACS University of Twente, Faculty of Electrical Engineering, Mathematics and Computer Science, Design and Analysis of Communication Systemshttp://utwente.nl/ewi/dacs/Prof.dr. B. Haverkort, Dr.ir. G.J. Heijenk, Dr.ir. A. Pras, Dr.ir. P.T. de Boer, Dr.ir. G. Karagiannis

ASCI bi-annual Report 2009-2010 8

UT-EWI-CAES University of Twente, Faculty of Electrical Engineering, Mathematics an Computer Science,Computer Architecture, Design & Test for Embedded Systemshttp://caes.ewi.utwente.nlProf.dr.ir. G.J.M. Smit, Dr.ir. A.B.J. Kokkeler, Ir. E. Molenkamp

EUR-UMCR Erasmus MC, Departments of Radiology and Medical Informatics, Biomedical Imaging Group Rotterdamhttp://www.bigr.nl Prof.dr. W.J. Niessen, Dr. J. Veenland, Dr.Ir. H. Vrooman

RUN-UMCR Radboud University Nijmegen Medical Centre, Department of Radiology, Diagnostic Image Analysis Group (DIAG) Nijmegenhttp://www.diagnijmegen.nlDr.ir. N. Karssemeijer, Dr.ir. H.J. Huisman, Dr. M. Velikova

ASCI bi-annual Report 2009-2010 9

1.3 ASCI Research Themes

ASCI research comprises historically two main interlocked themes. The ‘C’ in ASCI stands for Computing and the ‘I’ for Imaging. One half of ASCI is still best represented by the ‘C’ for Computation, but the ‘I’ is gradually developing into Sensory Information, which still justifies the ‘I’ if the emphasis is shifted from the word Imaging to Information. Within the two ASCI themes, the scientific activities can be divided in Methods & Algorithms on the one hand and Systems & Architectures on the other. Methods & Algorithms deal with the development of models and tools as such. They are eventually directed towards particular applications in science or industry and non-profit organizations. Systems & Architectures deal with the large-scale design and integration of tools, and the evaluation thereof at the system level. They are eventually directed to exemplary systems such as embedded systems, communication networks, information analysis systems, search engines, and visualization systems. Both types of activity are targets for fundamental and applied research.

These themes and target areas are combined in the following matrix, in which the different computer science research disciplines covered by ASCI can be placed.

Methods & Algorithms Systems & Architectures

ComputingA1. High Performance Computing2. Computational Science

B1. Large-Scale Information Systems2. Distributed Systems3. Embedded Systems4. Sensor networks

(Sensory) Information

C1. Image and Multimedia Sensing2. Processing3. Interpretation and Visualization

D1. MM Analysis & Search Systems2. Sensing and Learning Systems3. Acting and Visualization Systems

ASCI bi-annual Report 2009-2010 10

Within the area of Computing we have seen several important developments. The field of high performance computing (HPC) and computational science (theme A) has shifted focus from exclusively computation-intensive computing to also include data-intensive computing, which is required by many e-Science applications. Apart from traditional HPC, ASCI now also studies data-centric aspects, including networking, high-performance communication, I/O, and security. The result is a more balanced treatment of computation-intensive and data-intensive applications.

The field large-scale distributed information systems and embedded systems (theme B) has been extended considerably. The work on operating systems is now focusing on the highly successful Minix-3. The distributed systems subtheme has grown with many new activities on grids, peer-to-peer systems, and sensor networks, especially resource management, network management, programming, and workflow systems. Much of this work is related to the national BSIK projects VL-e and Freeband, and to the NWO program I-Science. ASCI´s research on embedded systems has expanded significantly. Many ASCI groups are studying Systems-on-a-chip (SoC) designs, for example multiprocessor SoCs. They collaborate extensively in ASCI and in large STW Progress projects. The field of sensor networks is gaining a lot of momentum within the ASCI community. Many groups deal with aspects of sensor networks, including distributed communication protocol such as gossiping, low-power sensor nodes, distributed information processing, and security aspects.

In the field of Sensory Information and Systems (theme D), large developments have taken place. Where sensory information has been an academic topic of study for twenty years, it recently became part of mainstream information and communication technology for two reasons. Firstly, massive digitization of all sensory data is taking place, for robots, science or popular use alike. Secondly, very large-scale archives are disclosed through digital media networks, again in science and society. Multimedia systems are no longer academic playgrounds but real platforms with many applications in science and the arts, cultural heritage, safety services, medical imaging, industry and the population at large.

In the field of Sensory Data Methods and Algorithms (theme C), steady developments take place to unravel the structure of multimedia data of many different sources. Examples are the understanding of the (deep) structure of images in for instance medicine, the structure of space observed through sensor networks or mobile robots, the learning of facts from multimedia information, and the understanding and exploitation of multimedia exchange, consumption, alteration and annotations in on-line social communities.

At the onset, ASCI anticipated the massive computation needs generated by the digitization of massive sensory data. This expectation has come true as can be seen from the processing of popular resources such as Hyves and Flickr, or professional archives like broadcast archives, or scientific resources with large archives in ecology, astronomy or geo-sciences. The research program and the educated PhD students of ASCI have and will remain to contribute to this confluence.

Furthermore, for large repositories of data and knowledge, also the structuring and computation of heterogeneous and multimedia sources of knowledge in ontologies and databases is increasingly important. This has lead to more co-operation with members of the Research School SIKS.

ASCI bi-annual Report 2009-2010 11

2 Cooperation within ASCI

ASCI researchers collaborate through the DAS infrastructure and throught numerous externally funded collaborative projects. Below, we describe DAS and some of the largest collaborative projects.

The DAS projects

DAS, Distributed ASCI Supercomputer, is the experimental infrastructure shared among all ASCI researchers. The first DAS system was set up in 1997, while DAS-3 was operational in 2006. The successive systems were funded largely by three equipment grants from NWO. Each generation consists of four to five clusters at different locations, connected by a wide-area network and integrated into a single, shared, distributed system. DAS-3 pioneers a novel flexible dedicated optical interconnect Starplane provided by SURFnet.

DAS is unique in that it is designed specifically for experimental computer science and (unlike grids) it is designed and managed by a single organization (ASCI). The DAS systems have been highly successful and clearly demonstrated that computer scientists need such a dedicated distributed infrastructure. Over a 100 researchers currently use DAS, including dozens of PhD students. In the period 2005-2008, 36 PhD-students used DAS for their research project. Only few other countries have such a facility; the most prominent similar system is the French Grid’5000, with which we collaborate intensively.

Large-scale collaborative projects

Ever since the start of ASCI, its researchers collaborated in numerous joint projects. ASCI researchers also participated in (or initiated) several very large collaborative programs, each involving many dozens of scientists, often from different areas. ASCI played a leading role in most of these programs, and we feel that the efforts we invested during the preceding decade in building a coherent research community paid off in these programs. The programs are described below.

MultimediaN is a 30 MEuro BSIK program that runs from 2004 to 2009 exploring (large-scale) digital sensory data, their storage and their interaction. Sensory mostly focused on image and video data, with small excursions to audio, speech and text processing. The program is well connected to industry and non-profit organizations. The operational model is based on the work-table model where intensive co-operation at the lowest level of the organization takes place and immediate transfer of knowledge and know-how is achieved.

VL-e (Virtual Laboratory for e-Science) is a 40 MEuro BSIK program running from 2004 to 2009 that studies virtual laboratories for e-Science. About one third of this program consists of Computer Scientists (mostly from ASCI) who design generic methods and tools for scientific applications from high-energy physics, medicine, bioinformatics, biodiversity, and other areas. This program boosts our research on grid programming environments, workflow systems, problem solving environments, resource management, and networking.

ICIS (Interactive Collaborative Information Systems) is a 14 MEuro BSIK project running from 2004 to 2009, to develop techniques that support humans in complex information environments and to facilitate distributed decision-making capabilities. ICIS emphasizes the importance of building actor-agent communities: close collaborations between human and artificial actors that highlight their complementary capabilities, and in which task distribution is flexible and adaptive. It moreover studies the interaction of humans with their artificial counterparts in such settings and how to meet the mutual information needs.

STW Progress program. ASCI has set up a research program together with Philips Corporation on future consumer electronics (CE) devices, resulting in three related projects that were granted from STW (the Dutch national technology foundation) in the Progress program on embedded systems. They study different aspects of System-on-a-Chip (SoC) architectures, including software engineering methods (SCALP), mapping domain specific (video) applications onto a domain specific Network-on-a-Chip platform (Artemisia) and the design of NoC-based real-time systems (PreMaDoNa). The projects collaborate intensively in regular tri-partite meetings.

NWO i-Science program. NWO has set up a cluster of three programs (GLANCE, VIEW, STARE) to advance the research in e-Science in the Netherlands. Numerous collaborations within ASCI and between ASCI and other partners were funded by these programs, including large-scale distributed systems projects in GLANCE (GUARD-MM, StarPlane, GUARD-G, MicroGrids), visualization projects in VIEW (EIO, IMOVIS, MFMV, Multi-Vis), and research on astronomy applications in STARE (Astrostream, SCARI, ASTROVIS). Almost two third of the i-Science projects contain ASCI researchers.

ASCI bi-annual Report 2009-2010 12

3 Scientific Output along Research Themes

This chapter contains the contributions of the various research groups in ASCI. The following table lists the enrollment of the groups with respect to the themes.

Methods & Algorithms Systems & Architectures

Computing

A1. High Performance Computing &2. Computational Science Wijshoff (UL)

B1. Large-Scale Information Systems Tanenbaum (VU)2. Distributed Systems Bal (VU), De Laat (UvA), Sips (TUD), Pras (UT)3. Embedded Systems Corporaal (TUE), Smit (UT), Jesshope (UvA), Haverkort (UT), Deprettere (UL)4. Sensor networks Langendoen (TUD), Van Steen (VU)

(Sensory) Information

C1. Image and Multimedia Sensing Van Vliet (TUD), Ter Haar Romeny (TUE)2. Processing Petkov (RUG)3. Interpretation and Visualization Roerdink (RUG), Jansen (TUD), Van Wijk (TUE), Overmars (UU)

D1. MM Analysis & Search Systems Veltkamp (UU), Smeulders (UvA), Lagendijk (TUD), Lew (UL)2. Sensing and Learning Systems Niessen (EUR), Lelieveldt (LUMC), Karssemeijer (RUN), Reinders (TUD), Groen (UvA)3. Acting and Visualization Systems Gorte (TUD)

3.1 Contribution of VU-EW-CS

Vrije Universiteit, Faculty of Sciences, Faculty of Sciences, Dept. of Computer ScienceProf.dr. A.S. Tanenbaum, Prof.dr.ir. H.E. Bal, Prof.dr.ir. M.R. van Steen, Dr.ing. T. Kielmann, Dr. G.E.O. Pierre, Dr. R.van Nieuwpoort, Dr.ir. H.J. Bos, Dr.B. Crispo, Dr. F.J. Seinstra

Research 2009-2010

The Section Computer Systems of the VU comprises three programs: Secure and Reliable Computer Systems(Tanenbaum), High Performance Distributed Computing (Bal), and Large-Scale Distributed Computer Systems (Van Steen).Secure and Reliable Computer SystemsOur focus has been on doing research which we hope will lead to more dependable and secure computer systems. While there are many aspects to the problem, much of our focus has been on operating systems. In particular, current operating systems typically consist of millions of lines of code running together in kernel mode. A consequence of this design is that a single bug anywhere in this code can bring down the system. We are trying to design a system that does not have this unfortunate characteristic.In our design, only a very small amount of code runs in kernel mode. The rest of the operating system runs as a collection of user-mode processes. These include the file system, memory manager, process manager, and al the device drivers. We have been able to make a design in which failures of most components are no longer fatal. In many cases a failed component can be restarted without affecting running user processes.In addition, we are working on a way to replace most of the operating system on the fly, while the system is running, and also without affecting running user processes. This means that when a new release of the operating system comes out, it is not necessary to stop and reboot the computer with the new version. For servers and embedded systems that must run 24 hours a day, 7 days a week, this feature is quite important.While this modular design improves security, we are also addressing security directly. This research includes novel techniques for malware detection with particular interest in breaching-privacy software (e.g. keyloggers and spyware). Different than other classes of malware, privacy-breaching are difficult to detect and current approaches based on attack signatures or fingerprinting system calls of the attach do not work. We propose a class of new techniques based on memory writes profiling that is both effective and efficient against this class of malware.

ASCI bi-annual Report 2009-2010 13

Another thread of research is related to smartphone security and privacy. We are working on extending the security model of existing smartphone operating systems, with particular focus on Android. We designed new solution capable of enforcing dynamically and seamless context-based policies and to provide full software isolation to protect selected application and/or related data.We are also investigating in new authentication methods suitable for modern platforms based on touch screens and typically without keyboard (e.g. smartphone, tablets, ipad, etc.). We aim at transparent authentication methods, where users are not required to perform any specific action for the sole purpose of authentication. The new methods are based on new behavioural biometrics.High Performance Distributed ComputingModern real-world distributed systems consist of clusters, grids, clouds, desktop grids, and mobile devices, often extended with accelerators like GPUs. Together with other researchers in ASCI, we have set up the DAS-4 system as a new distributed testbed on heterogeneous distributed computing and Green IT. Writing applications for such systems has become increasingly difficult. We try to drastically simplify the programming of such systems by studying the underlying fundamental problems of distributed computing hand-in-hand with major applications. These efforts are integrated into a single system called Ibis, consisting of:

- The Ibis High-Performance Application Programming System, containing a communication library designed for dynamically changing distributed environments (IPL), higher-level programming models, and a library (SmartSockets) that automatically solves connectivity problems.

- The Ibis Distributed Deployment System, containing the JavaGAT programming interface, a high-level GUI (IbisDeploy) for running and managing grid applications, and the Zorilla peer-to-peer layer.

Together with other research groups, we use Ibis for research on multimedia content analysis, medical image processing, distributed reasoning, graph algorithms, model checking, bioinformatics, and astronomy. Our work has won several awards at international venues (e.g., CCGrid SCALE 2010).

Many scientific problems are of such complexity that solutions are obtained only by using the wide variety of computing hardware all at once. Driven by this need for the concurrent use of multiple types of resources, we have laid the foundations of a new computing paradigm: Jungle Computing. We have extended Ibis to enable easy implementation and deployment of large-scale Jungle Computing applications consisting of any variety of tasks implemented using any popular language or model (e.g. C, Java, MPI, Python, CUDA). It also can select one of several alternative implementations that calculate the same result using different hardware. In the near future, we plan to work on real-life e-Science applications for Jungle computing, especially Computational Astrophysics, Climate Modeling, and Remote Sensing, and to work on energy-aware programming systems and applications.We also study the usage of GPUs to speed up applications, especially from astronomy (LOFAR) and multimedia. In addition, we use Ibis for distributed applications that involve mobile devices, in particular smart phones. We have built a framework that enables context-aware applications, provides distributed decision support and advanced data-management features (including versioning). It also supports offloading of computation and communication tasks to Clouds, to save energy and improve performance. Clouds have become an interesting platform for both commercial, Web-centric systems, as well as scientific computing. We are studying cloud infrastructures and platforms for use with scientific workloads, especially the trade-offs among execution speed and monetary costs. We have built a framework that schedules large, parameter-sweep applications within user-defined budget limitations. We continue our work in the area of system security. Specifically, we increased our efforts in information flow tracking, initially to chart out its usefulness and limitations. Based on this work, we then initiated a major new project on reverse engineering, now supported by an ERC Starting Grant and a Microsoft Academic Research grant. These projects, collectively referred to as Rosetta, aim to reverse complex binaries by extracting the data structures first. Unlike most other projects, we use dynamic analysis and extensive information flow tracking. Once we have the data structures, we expect the reversing of the code to be much simpler, but more importantly, we expect to be able to protect the binaries against common memory corruption attacks like buffer overflows. In addition to these projects, we keep working on detecting attacks in fast networks using Streamline (in the European i-Code and SysSec projects) and in mobile phones (Paranoid Android).

Large-Scale Distributed Computer SystemsLARGE-SCALE CLOUD INFRASTRUCTURESGlobule is now a subprogram lead by Guillaume Pierre. The research is dedicated to designing techniques for scalable Web hosting. Initially, it started with the development of the Globule open-source collaborative content distribution network. It now expands to other related topics such as scalable Web application hosting, Cloud data stores, resource provisioning, etc. During the report period, major effort has been put into dynamically reserving resources, along with the design and evaluation of a collaborative Wikipedia system. Future research concentrates on enhancing primitive cloud services (such as for storage and resource allocation) with better and easier support.EXTREME DISTRIBUTED SYSTEMSIn a separate program, we have been concentrating on very large wireless ad hoc systems. Starting with sensor networks, this work has been extended to scenarios encompassing large crowds of people equipped with electronic badges or similar devices. Research has concentrated on epidemic-based information dissemination and developing ultra-low duty-cycled networks. Future work involves automatically deriving and analyzing social graphs, so that feedback to users can be provided.

ASCI bi-annual Report 2009-2010 14

Next to wireless systems, we have started collaborations with the University of Oslo to develop secure, decentralized online social networks.

Key Publications 2009-2010

Appuswamy, R., Moolenbroek, D.C., and Tanenbaum, A.S.: Loris - A Dependable, Modular File-Based Storage Stack, Proc. 16th Pacific Rim Int'l Symp. on Dependable Computing, pp. 165-174, 2010.Herder, J.N., Moolenbroek, D.C. van, Appuswamy, R., Wu, B., Gras, B, and Tanenbaum, A.S.: Dealing with Driver Failures in the Storage Stack, Proc. Fourth Latin American Symposium on Dependable Computing, pp. 119-126, 2009 (Best Paper award).Mazzoleni, P., Crispo. B., Sivasubramanian, S., Bertino, E. "Efficient integration of fine-grained access control and resource brokering in grid", J. Supercomputing, Elsevier, vol.49 (1): pp. 108-126, 2009.Henri E. Bal, Jason Maassen, Rob van Nieuwpoort, Niels Drost, Roelof Kemp, Timo van Kessel, Nick Palmer, Gosia Wrzesinska, Thilo Kielmann, Kees van Reeuwijk, Frank J. Seinstra, Ceriel Jacobs, and Kees Verstoep Real-World Distributed Computing with Ibis, IEEE Computer magazine, Aug. 2010Rob V. van Nieuwpoort, Gosia Wrzesinska, Ceriel J.H. Jacobs and Henri E.Bal: Satin: a High-Level and Efficient Grid Programming Model, ACM Transactions on Programming Languages and Systems (TOPLAS), Volume 32, Issue 3, ACM Press New York, NY, USA, 2010F.J. Seinstra, J. Maassen, R.V. van Nieuwpoort, N. Drost, T. van Kessel, B. van Werkhoven, J. Urbani, C. Jacobs, T. Kielmann, and H.E. Bal. Jungle Computing: Distributed Supercomputing beyond Clusters, Grids, and Clouds In: M. Cafaro and G. Aloisio, editors, Grids, Clouds and Virtualization, pp. 167-197, Springer-Verlag, September 2010.Ana-Maria Oprescu, Thilo Kielmann, Bag-of-Tasks Scheduling under Budget Constraints, Proc. 2nd IEEE International Conference on cloud Computing Technology and Science (CloudCom 2010), Indianapolis, USA, Nov/Dec 2010.Asia Slowinska and Herbert Bos, Pointless tainting? Evaluating the practicality of pointer tainting, Proceedings of ACM SIGOPS EUROSYS, 2009, Nuremberg, Germany.P. Costa, J. Napper, G. Pierre and M. van Steen. Autonomous Resource Selection for Decentralized Utility Computing. Proc. 29th Int'l Conference on Distributed Computing Systems (ICDCS), Montreal, Canada, June 2009.K. Iwanicki, M. van Steen. On Hierarchical Routing in Wireless Sensor Networks Proc. 8th ACM/IEEE Int'l Conf. on Information Processing in Sensor Networks (IPSN), San Francisco, California, April 2009G. Urdaneta, G. Pierre, M. van Steen. Wikipedia Workload Analysis for Decentralized Hosting. Computer Networks, vol. 53(11), July 2009.P. Garbacki, D. Epema, M. van Steen. The Design and Evaluation of a Self-Organizing Super-Peer Network. IEEE Transactions on Computers, vol. 59(3):317-331, March 2010Jiang Dejun, Guillaume Pierre and Chi-Hung Chi. Autonomous Resource Provisioning for Multi-Service Web Applications. Proc. 19th Int'l World-Wide Web conference (WWW), April 2010.

3.2 Contribution of UvA-FNWI-SNE

System and Network Engineering research group, University of Amsterdam.prof.dr.ir. C.T.A.M. de Laat, prof.dr. R. Meijer, dr. P.Grosso, dr. Y. Demchencko, drs. G. van 't Noordende.

The System and Network Engineering research group (SNE) focuses its research on the complexity of emerging hybrid System and Network architectures and the associated models, protocols and system approaches for data processing in science. The group is building tools and proof of concept applications that promote optimal use of high- speed networks. The group develops middleware to empower applications to optimally allocate and use complex infrastructures. Security of the required mechanisms, infrastructure, middleware, applications and the privacy of data in distributed processing environments is an essential aspect of the research. The SNE research group is closely working together with the SNE Master (www.os3.nl), Software Engineering Master (SE) and the Computer (Grid) Science Master programs to disseminate knowledge through education. For more information see: System and Network Engineering (SNE)

Network Modeling

In 2009 and 2010 we have further developed the Network Description Language with support from the EU in the projects GEYSERS, GEANT3 and NOVI, within our participation in the GigaPort3 project, and in the CineGrid project . We extended our models to describe connected end system resources and the data they contain.

NDL is now a multi-schema language that can be used by network control planes to provision (i.e. configure) dedicated optical connections in a hybrid network. To facilitate path finding in such environments we have developed technology-independent schemas that wrap the multi-layer components of the network in standard interfaces. We have also further developed NDL to make it suitable for network emulations and simulations, so that new protocols can be tested in realistic network topologies. The group has also successfully enlarged the community of NDL users and to cooperate internationally with other researchers toward a standardized language for network descriptions. This effort has been carried out within the Open Grid Forum NML-WG (Network Markup Language Working Group). There have been demonstrations of the multi-layer

ASCI bi-annual Report 2009-2010 15

path finding algorithms based on NDL at several international venues as Terena, GLIF, GENI and SuperComputing conferences. Two Ph.D thesis on the topic were awarded in 2009 and 2010.

In 2009 and 2010 we worked at the integration of two NOW-funded projects: SCARIe and StarPlane. The SCARIe project was a collaboration between the JIVE, the UvA and SARA. StarPlane was a collaboration between UvA and VU.

SCARIe is focused on providing a software correlator for Very Long Base Interferometry, VLBI. In our implementation the correlation task follows a hierarchical master-worker model. In StarPlane we implemented this model on top of MPI and of the SATIN programming model, and we tested the high performances aspects of StarPlane. In our experiments Starplane is delivering 400MB/s of throughput between the cluster sites and thus permits us to distribute medium correlation jobs to the DAS-3 grid. This was demonstrated at the Terena and SC conferences in 2009.

CineGrid

The SNE group is a founding member of CineGrid, a collaboration in the field of high-quality media content delivery. CineGrid recognizes the need of new collaborative tools to enable the transport of this content. Data need to move seamlessly from the location where it is stored to the end user requesting it in a fast and automatic way. For this we need IT infrastructures that offer enhanced Quality-of-Service suitable for real-time data distribution. In 2007 we built the first European CineGrid distribution center. We now focus on the creation of a portal to the distributed resources that help the Cinema community to accomplish its creative work using elastic virtualized cloud resources. This work was demonstrated in the 2009 and in 2010 at the SuperComputing conferences.

Complex Resource Provisioning and Cloud Architecture Research

Complex Resource Provisioning (CRP) research focuses on models and architectures for on-demand CRP systems that may include both computing resources and dedicated network. The proposed Composable Services Architecture (CSA) is used as an architectural basis for developing GEANT Multidomain Service Bus (GEMBus). Currently this area of research is evolving into using Clouds as a universal platform for CRP systems and general infrastructure services. Cloud architecture research include definition and development of architectures and supporting services and mechanisms for on-demand provisioning of Cloud based infrastructure services, including security infrastructure, considering multi-domain and multi-stakeholder environment.

Generic AAA architecture and policy based access control

The Generic AAA Authorization Framework architecture and its toolkit functionality was enhanced in order to support Complex Resource Provisioning (CRP) in Cloud environment, including both network and ICT resources provisioned on-demand.

The management of the security context and authorization sessions in multi-domain dynamically provisioned services had specific attention. The proposed solution uses the access and pilot tokens for authorization and provisioning sessions management that allows communicating security context between domains and binding application related security context to the provisioning session and trusted platform. Proposed solutions are being implemented in the framework of the GEYSERS project, by extending the AAA Toolkit to work in the Enterprise Service Bus (ESB) and OSGi environments.

Actively contributed to OGF standardisation activity in the framework of the Infrastructure Services On-Demand provisioning Research Group (ISOD-RG) and NIST collaboration on Clouds.

This work has been resulted in several publications and contribution to the OGF ISOD-RG. This work was done in the context of the GigaPort research on networks, GEYSERS and the GEANT3 projects.

Programmable Networks and Clouds

We developed a framework that enables application developers to create complex and application specific network services. The essence of our approach is to utilize programmable network elements to create a software representation of network elements in the application. We show that the typical pattern of an application specific network service is a control loop in which topology, paths, and services are continuously monitored and adjusted to match application specific qualities. We present a platform in which network control applications can be developed and illustrate possible use cases. Based on these use cases, new research questions are identified.

2. Korte beschrijving van verwachtingen voor de komende verslagperiode

ASCI bi-annual Report 2009-2010 16

The System and Network Engineering group is looking beyond the horizon to bring together applications and new communication and computing technologies. We have introduced hybrid networking as a novel paradigm that provides additional scaling capabilities to Internet communications for large sciences when using 10’s of gigabits of network links. The technology has now further improved and it is now capable of delivering Terabit/s capacities. The coupling of this kind of networks with computational, data and visualisation resources will require innovation in the ways applications can express their needs to the infrastructure. Upcoming trends in e-Infrastructures for science include new capabilities in virtualisation, Cloud based infrastructure services provisioning, programmable infrastructures, and sustainability. Planning in these kinds of e-Infrastructures with constraints on all these capabilities gives an increase in flexibility, but also complexity. Harnessing the complexity in planning and executing these workflows will require innovation on a fundamental level. Security, privacy, policy and trust in data processing infrastructures are also major challenge that is under research in the group. [PG1] The handling of medical data in distributed environments requires mechanisms that ensure privacy as required by law.

The research work is developed in the context of a number of national and international collaborations such as the EU NOVI, GEYSERS, GEANT3 and ENVRI projects, the nationals COMMIT, Lifewatch, GigaPort3 and CineGrid-Amsterdam projects and in communities as GLIF and CineGrid.org. We are addressing in the GEYSERS and NOVI projects, some of the key technical challenges that enable on-demand service composition across multiple domains. We demonstrate solutions and functionalities across test-beds involving European NRENs, GÈANT3, Cross Border Dark Fibre Initiatives, Internet2, ESNET and GLIF connectivity infrastructure.

3. De 5-10 belangrijkste publicaties over de verslagperiode 2009-20010

Derek Groen, Steven Rieder, Paola Grosso, Cees de Laat, Simon Portegies Zwart, "A light-weight communication library for distributed computing", IOP journal Computational Science & Discovery 3 (2010) 015002 (14pp) doi:10.1088/1749-4699/3/1/015002.

Simon Portegies Zwart, Tomoaki Ishiyama, Derek Groen, Keigo Nitadori, Junichiro Makino, Cees de Laat, Stephen McMillan, Kei Hiraki, Stefan Harfst, Paola Grosso, "Simulating the Universe on an Intercontinental Grid", IEEE Computer, Volume 43, Issue 8, Aug. 2010, page 63-70.

Guido J. van 't Noordende, Silvia D. Olabarriaga, Matthijs R. Koot, Cees Th.A.M. de Laat, "A Trusted Data Storage Infrastructure for Grid-Based Medical Applications", International Journal of Grid and High Performance Computing, Vol. 1, Issue 2, April-June 2009, pages 1-14.

Leon Gommans, Li Xu, Fred Wan, Yuri Demchenko, Mihai Cristea, Robert Meijer, Cees de Laat , “Multi-Domain Lightpath Authorization using Tokens”, FGCS, Vol 25, issue 2, feb 2009, pages 153-160.

Freek Dijkstra, Jeroen J van der Ham, Paola Grosso, Cees de Laat, "A Path Finding Implementation for Multi-Layer Networks", Future Generation Computer Systems, Volume 25, Issue 2, February 2009, Pages 142-146.

Paola Grosso, Damien Marchal, Jason Maassen, Eric Bernier, Li Xu, Cees de Laat, "Dynamic Photonic Lightpaths in the StarPlane Network", FGCS, Vol 25, issue 2, feb 2009, pages 132-136.

Yuri Demchenko, Cees de Laat, Thierry Denys, Christian Toinard, "Authorisation session management in on-demand resource provisioning in collaborative applications", Proceedings of the 2009 International Symposium on Collaborative Technologies and Systems, ISBN:978-1-4244-4584-4, 2009, pages 201-208

Matthijs R. Koot, Guido van 't Noordende, Cees de Laat, "A Study on the Re-Identifiability of Dutch Citizens", 3rd Hot Topics in Privacy Enhancing Technologies (HotPETs 2010), PETS workshop, Berlin, Germany, July 21 - 23, 2010, http://petsymposium.org/2010/.

Rudolf Strijkers, Mihai Cristea, Cees de Laat, Robert Meijer, "Application Framework for Programmable Network Control," in 1st International Workshop on Network Embedded Management & Applications Niagara Falls, Canada: Springer, 2010.

Zhiming Zhao, Paola Grosso, Ralph Koning, Jeroen van der Ham, Cees de Laat, "Network resource selection for data transfer processes in scientific workflow", in: Workflows in Support of Large-Scale Science (WORKS10), New Orleans in conjunction with Supercomputing 10, 14th November.

Non-scientific book chapter publication:Gordon Cook, "Building a National Knowledge Infrastructure; How Dutch Pragmatism nurtures a 21st Century Economy", ISBN 978-90-814102-2-9, February 2010.

3.3 Contribution of UvA-FNWI-IAS

ASCI bi-annual Report 2009-2010 17

University of Amsterdam, Faculty of Science, Informatics Institute, Intelligent Autonomous SystemsProf.dr.ir. F.C.A. Groen, Prof.dr. D.M. Gavrila, Dr.ir. B.J.A. Kröse, Dr.ir. L. Dorst, Dr. G. Pavlin

Research 2009-2010We focus on intelligent autonomous systems operating in a world inhabited by humans, explored within two themes: ‘Perception and Modeling’ of the world, and `Decision Making’ to use these models for goal-directed behavior.

Perception and Modeling This theme develops cognitive sensors that enable machines to interact intelligently and naturally with a human-inhabited environment. We want to localize humans in their environment, track them over time, and recognize their activities. Also, we are interested in modeling the environment the humans are living in, typically from distributed or moving sensor systems. We have a number of projects addressing these issues.

In the project ‘Zorgen voor Morgen’ we equipped 6 assisted living apartments with sensor systems in order to monitor Activities of Daily Living (ADL) of elderly. This project is a collaboration with the Hogeschool van Amsterdam and nursing home Naarderheem, part of Vivium and is funded in the Pieken in de Delta program.

Another collaboration with the Hogeschool van Amsterdam is the ‘Mens voor de Lens’ project, aimed at the valorization of knowledge in the field of computer vision. In collaboration with a company ‘Eagle Vision’ we develop an study algorithms for people counting and pose estimation.

In the project AnaSiD we Analyse Social Interactions from a Distance using cameras and microphones. Social signal processing is becoming extremely important and we use advanced datafusion methods for detecting dominance in a group. The project is funded by EU Intra-European Fellowships (IEF) Marie Curie

In the NW0 CASSANDRA project we enriched the visual features used in our system for automatic aggression estimation by tracking multiple persons with overlapping cameras as “3D blobs”: the visual features now relate to 3D human body motion rather than to 2D scene motion. We furthermore developed a method for the adaptation of a generic 3D human model to a particular individual in a multi-view image sequence; we showed that the use of the resulting personalized model improves 3D articulated pose recovery.

In the DIADEM project, we investigated the use of image associations in mobile phone apps to report suspicious smells more accurately, and did large scale experiments with suitable feedback on text messages to increase the effectiveness of this application by increasing willingness to participate. We also studied the social acceptance of robots in situations of care and crisis management.

For ego-motion estimation of moving robots in collaboration with TNO, we developed a novel fast and robust outlier-filtering method called EM-SE(3) that utilizes Expectation Maximization (EM) on a local linearization of the rigid body motions group i.e. SE(3). With TNO, we study 3D reconstructioin based on handheld cameras. For the 3D reconstruction of environments and (articulated) structures, we are extending the data processing aspects of the new field of geometric algebra, partly within the NWO project ‘Discovery of Articulated Structures in Image Sequences’

Decision Making research theme This theme focuses on the action side of the complete perception-action loop of an intelligent autonomous system, which requires robust handling of uncertainty due to incomplete information. We are interested in real-world applications with direct relevance to society. Examples include crisis management, wherein intelligent systems must analyze crises and help human managers make decisions in real time; rescue robot teams, which help people in disaster areas; and traffic management, which can be optimized via intelligent agents. Funding is obtained via an NWO Free Competition grant.

We developed methods using neuroevolution for the autonomous control of helicopters under unknown conditions. We also developed new reinforcement learning algorithms that can trade off the sample efficiency of model-based methods with the space efficiency of model-free methods without compromising convergence guarantees. Finally, we developed methods for applying reinforcement learning to automatically optimize online learning-to-rank systems for information retrieval, such as search engines.

Our intelligent Decision Making methods are also being applied in the RoboCup international robot competitions, notably the Virtual Rescue competition aims to develop robot control systems in crisis environments. In this competition Amsterdam has a shared team with Oxford University. The heterogeneous team of robots (both ground and air robots) has to coordinate their actions with limited communication. Our effective methods have earned us several prices in the past few years.

In the problem of decentralized planning under uncertainty for teams of collaborating agents, we have focused on the case where the agents are fully cooperative. Finding a solution for the corresponding stochastic game model (known as DECPOMDP model) is NEXP-complete. We have provided theory concerning the existence of an optimal value function for DECPOMDPs, as well as tighter upper bounds to this optimal value function. We also proposed algorithms that use these tighter bounds to either obtain more accurate approximate results or to find exact solutions faster. We collaborated with

ASCI bi-annual Report 2009-2010 18

international partners to develop a multi-agent decision library, which provides functionality for planning in multi-agent domains, as well as a set of benchmark problems.

Future research plansPerception and modelingIn the “Looking at People” area, we will develop 3D human pose estimation algorithms that are robust to occlusion and work with several people in the scene. We aim to integrate the richer 3D pose human pose features (MultimediaN – Professional Dashboard) into our aggression detection system (CASSANDRA).

We will further focus on health monitoring systems. A proposal was granted to start a ‘living lab’ on health, in which we monitor a number of elderly with simple sensor systems. This is a collaboration with VU, HvA and SIGRA (Amsterdam Care organizations). The work on distributed cameras that was carried out two years ago will continue in the project ‘SCAN’, a PointOne project with Philips and Eagle Vision, on the calibration of cameras in intelligent street lighting posts.

Decision MakingWe will continue our work developing efficient, scalable methods for decision-theoretic planning and learning. In particular, we will undertake new directions concerning the use of reinforcement learning to improve human-computer interaction and the development of multi-objective methods for multi-agent planning and learning, with applications in traffic control.

Key Publications 2009-2010

Babuška, R. & Groen, F.C.A. (Eds.). (2010). Interactive collaborative information systems. (Studies in computational intelligence, 281). Berlin: Springer. Balaguer, B., Carpin, S., Balakirsky, S. & Visser, A. (2009). Evaluation of RoboCup maps . In Proceedings of the 9th Performance Metrics for Intelligent Systems workshop (PerMIS'09). Association for Computing Machinery (ACM). Dorst, L. (2010). Tutorial: Structure-preserving representation of Euclidean motions through conformal geometric algebra. In E. Bayro-Corrochano & G. Scheuermann (Eds.), Geometric algebra computing in engineering and computer science (pp. 35-52). London: Springer. Heerink, M., Kröse, B., Evers, V. & Wielinga, B. (2010). Relating conversational expressiveness to social presence and acceptance of an assistive social robot. Virtual Reality, 14(1), 77-84· Hofmann, M. & Gavrila, D.M. (2009). Multi-view 3D human pose estimation combining single-frame recovery, temporal integration and model adaptation. In 2009 IEEE Conference on Computer Vision and Pattern Recognition: CVPR 2009 ; Miami, Florida, USA, 20 - 25 June 2009 (pp. 2214-2221). Piscataway, NJ: IEEE.

3.4 Contribution of UvA FNWI-ISIS

University of Amsterdam, Faculty of Sciences, Informatics Institute, Intelligent Sensory Information Systems GroupProf.dr.ir. A.W.M. Smeulders, Dr.ir. R. van den Boomgaard, Dr. M. Worring, Dr.Ing. J.M. Geusebroek, Dr. Th. Gevers,Dr. C.J. Veenman, R.F. Aldershoff, Drs. C.G.M. Snoek, Dr. N. Sebe

Research 2007-2008

We consider the research areas of content-based access of visual data, computer vision, and systems for retrieval of video, more specifically:

Visual search enginesFor the retrieval of videos the aim is to make multimedia archives as accessible as their textual counterpart. To that end, our research efforts concentrate on automatic semantic indexing and interactive retrieval of multimedia sources. We have developed the MediaMill semantic video search engine, which uses a lexicon of detectable concepts in combination with several advanced user interfaces. To value the merit of our efforts on the highest international standards, all research is evaluated within the international TRECVID benchmark for multimedia retrieval, organized by NIST. In 2008, we obtained top rank performance in both the concept detection task and the supervised search task, and we secured the second rank in the automatic search task. Furthermore, we participated in the international PASCAL-VOC Visual Object Categorization challenge for image categorization, organized by the EU funded PASCAL network. We obtained top rank performance in the VOC challenge. These results, in such strong international benchmarks, confirm our research efforts over the past few years have given us a leading position in the field of image and video retrieval.

Computer vision and visual cognitionAn import asset in our endeavor for semantic access is indexing of semantic concepts. One approach is scene categorization by modeling ambiguity in the popular codebook approach. There are two drawbacks to the traditional codebook mode, which stem from the hard assignment of visual features to a single codeword. We have studied the learning of soft relations between visual words, and the effect on categorization of many popular datasets.

ASCI bi-annual Report 2009-2010 19

Another line of research is on emotion recognition and categorization. We considered the question: can a machine learn to perceive emotions as evoked by an artwork? Therefore, we proposed an emotion categorization system, trained by ground truth from psychology studies. The training data contains emotional valences scored by human subjects on the International Affective Picture System (IAPS), a standard emotion evoking image set in psychology. Our approach is based on the assessment of local image statistics which are learned per emotional category using support vector machines.

Colour in computer visionOur research on colour has led to an evaluation of edge-based colour constancy algorithms. The goal of color constancy is to measure image colors despite differences in the color of the light source. Traditionally, the computational method of obtaining this ability is by using pixel values only. Recently, methods using edges instead of pixel values have been proposed. However, different edge types exist, such as material, shadow and specular edges. Therefore, we analyzed the influence of different edge types on the performance of edge-based color constancy. We have shown that, on generated data without color clipping, specular edges deliver near-perfect color constancy and that shadow edges are more valuable than material edges. However, with color clipping, the performance using the specular edges decreases significantly, while the performance using the material or shadow edges is less affected.

Computer vision for humansThe main direction of research here addresses the problem of sensing and understanding users’ interactive actions and intentions for achieving multimodal human-computer interaction in natural settings. To that end, we developed and evaluated and algorithm for accurate eye center location and tracking. The ubiquitous application of eye tracking is precluded by the requirement of dedicated and expensive hardware, such as infrared high definition cameras. Our aim is to perform very accurate eye center location and tracking using a simple web cam. The proposed method makes use of isophote properties to gain invariance to linear lighting changes (contrast and brightness), to achieve rotational invariance and to keep low computational costs. We have tested our approach for accuracy and robustness using the BioID and the Yale Face B databases. Our system can achieve a considerable improvement in accuracy over state of the art techniques.

Spatial and extensible databasesThe activities are realized in close co-operation with the CWI-database group. Amongst the results obtained the Monet database kernel and its modules for image and geo-spatial reasoning stands out. In the area of database kernels, an innovative experimental analysis uncovered the lack of performance improvement in database technology over the last decade. The underlying reason is the relative progress in CPU- and RAM-technology, which shows an increasing performance bottleneck. This observation has led to novel techniques to measure the resource waste and new database algorithms to avoid resource stales. The open-source version of the MonetDB system has been downloaded more than 30.000 times.

Future Research Plans

Colour in computer vision will be structurally positioned within the Faculty by the recent VICI award for Dr. Gevers.

For video search engines, we are starting a close collaboration with the Faculty of Humanities, aiming at combined research in a Centre for Content Culture and Technology. Furthermore, we intent to collaborate with the Cognition Centre of the university on various aspects of vision. For computer vision and learning, we intent to concentrate on recognition and similarity-learning of scenes, objects, and actions.

We will continue the line of research as initiated for computer vision for humans and for visual data mining for safety. Recent EU funding has strengthened this line of research.

Key Publications 2007-2008

A. Diplaros, N. Vlassis and Th. Gevers. A Spatially Constrained GenerativeModel and an EM Algorithm for Image Segmentation. IEEE Trans. on Neural Networks, 18(3), 798-808, 2007.

J.C. van Gemert, J.-M. Geusebroek, C.J. Veenman, A.W.M. Smeulders, Kernel Codebooks for Scene Categorization, ECCV, Volume 3, page 696-709, 2008.

H.T. Nguyen, Q. Ji, and , A.W.M. Smeulders, Spatio-Temporal Context for Robust Multitarget Tracking. IEEE Transactions on Pattern Analysis andMachine Intelligence, 29, 52-64, 2007.

K. van de Sande, Th. Gevers, C. G. M. Snoek, Evaluation of Color Descriptors for Object and Scene Recognition, IEEE Conference on Computer Vision and Pattern Recognition, 2008.

Cees G. M. Snoek, M. Worring, O. de Rooij, K. E. A. van de Sande, R. Yan, A. G. Hauptmann, VideOlympics: Real-Time Evaluation of Multimedia Retrieval Systems, IEEE Multimedia, Volume 15 (1), page 86-91, 2008.

H.M.G. Stokman and Th. Gevers, Selection and Fusion of Colour Models for Image Feature Detection. IEEE Trans. on

ASCI bi-annual Report 2009-2010 20

Pattern Analysis and Machine Intelligence, 29, 371-381, 2007.

R. Valenti and Th. Gevers, Accurate Eye Center Location and Tracking Using Isophote Curvature, IEEE Conference on Computer Vision and Pattern Recognition, 2008.

A. Gijsenij, Th. Gevers, M. Lucassen, A Perceptual Comparison of Distance Measures for Color Constancy Algorithms, European Conference on Computer Vision, Volume 1, page 208-221, 2008.

J. van de Weijer, Th. Gevers and A. Gijsenij, Edge-Based Colour Constancy. IEEE Trans. on Image Processing, 16, 2207-2214, 2007.

M. Worring and G. Schreiber, Semantic, image and video indexing in broad domains, IEEE Transactions on Multimedia, 9, 909-911, 2007.

3.5 Contribution of UvA-FNWI-CSA

UvA-FNWI-csaComputer Systems Architecture GroupInformatics Institute, Faculty of Sciences, University of Amsterdam

Dr. Clemens Grelck, Prof. Dr. Chris Jesshope, Dr. Andy Pimentel

Research 2009-2010

The predictable but quite unprepared for shift from frequency scaling to concurrency in computer systems architecture has played into the strengths in the computer systems architecture group. We have extensive expertise both in concurrency management for systems-on-chip (SoC) design and in the design and use of high-performance computing (HPC) systems. This shift now puts concurrency firmly into mainstream computing. The major motivation for this seismic shift in computer architecture is power dissipation and we see power management as a key motivator for our research over the coming five-year period. It will span a broad range of computing platforms, from SoC to HPC, although we anticipate a convergence of techniques and methodology used to optimise design across this range. In all areas we see a shift in the provisioning of computing away from a focus on achieving theoretical peak performance and more towards achieving optimal efficiency while a resource is being used. This in turn leads to a requirement for dynamic provisioning. Our strategy will be to match the specific requirements of the heterogeneous components of a computation to processing resources in order to minimize energy consumption. This in turn requires the characterisation of both computations and targets for non-functional capabilities and requirements. A major emphasis here will be on maintaining spatial locality. We will therefore adopt appropriate concurrency models (with formal contacts) that expose these requirements and our approach will embrace both static and dynamic optimizations.

In the embedded system domain, one of the most important challenges is the development of design tools that allow for fully automated system-level synthesis and programming of multi-processor SoC architectures. Our joint work with Leiden University in developing the Daedalus system-level MP-SoC synthesis framework offers a fully integrated tool-flow in which the various steps of system-level MP-SoC synthesis, such as parallelisation, design space exploration, hardware-software partitioning, application mapping and system prototyping, are highly automated.

Our work on fine-grain threaded architectures with data-driven scheduling using the SVP concurrency model will continue but we are also exploring software implementations of SVP on other emerging multi-core architectures such as Niagara and Intel's SCC. This allows us to explore multi-grain architecture and develop an infrastructure to support such an approach. One of the major directions in this work is the development of a coherent set of operating system services that support space sharing in these heterogeneous environments and yet provide a secure operating environment that can be scaled from chip-level micro-grids to globally distributed Grids.

Future research plans

One of the major challenges, especially in mainstream computing, will be in making these systems programmable without specialised concurrency knowledge and we have designed programming language support to express parallel computations and systems at a very high level of abstraction and developed compilation technologies that effectively map the abstract descriptions to concurrent computing environments. Our long-term vision is in the direction of a compilation infrastructure that automatically adapts running programs derived from high-level specifications to a heterogeneous and dynamically varying execution environment based on continuous reflection of execution parameters. For our work in the embedded systems domain, future research extends in two main directions: 1) introducing technology awareness at the system level to e.g. improve the performance and energy-efficiency of the resulting designs, and 2) introducing adaptivity to handle changing workloads by dynamically (re-)mapping applications or even reconfiguring system components.

ASCI bi-annual Report 2009-2010 21

Key publications 2009 – 2010

M. W. van Tol, C. R. Jesshope, M. Lankamp and S. Polstra, “An implementation of the SANE Virtual Processor using POSIX threads”, Journal of Systems Architecture, Volume 55, Issue 3, March 2009, pp 162-169.

K. Bousias, L. Guang, C.R. Jesshope, M. Lankamp, “Implementation and Evaluation of a Microthread Architecture”, Journal of Systems Architecture, Vol. 55, Issue 3, March 2009, pp 149-161.

Thomas A. M. Bernard , Clemens Grelck, Chris R. Jesshope, “On the Compilation of a Language for General Concurrent Target Architectures”, Parallel Processing Letters 20(1): 51-69, 2010.

Chris R. Jesshope, Mike Lankamp, , “The implementation of an SVP many-core processor and the evaluation of its memory architecture”, SIGARCH Computer Architecture News 37(2): 38-45, 2009.

Clemens Grelck, Sven-Bodo Scholz, Alexander V. Shafarenko, “Asynchronous Stream Processing with S-Net”, International Journal of Parallel Programming 38(1): 38-67, 2010.

P. van Stralen and A. D. Pimentel, "A High-level Microprocessor Power Modeling Technique based on Event Signatures", in the Journal of Signal Processing Systems, pp. 239-250, Vol. 60 (No. 2), Aug. 2010, Springer.

S. Jaddoe, M. Thompson, and A. D. Pimentel, "Signature-based Calibration of Analytical Performance Models for System-level Design Space Exploration", in Transactions on High-Performance Embedded Architectures and Compilers (Trans. on HiPEAC), Vol. 4 (No. 4), 2009.

Gerstlauer, C. Haubelt, A.D. Pimentel, T. Stefanov, D.D. Gajski, and J. Teich, "Electronic System-Level Synthesis Methodologies", in IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, pp. 1517-1530, Vol. 28 (No. 10), Oct. 2009.

P. van Stralen and A.D. Pimentel, "Scenario-Based Design Space Exploration of MPSoCs", in the Proc. of the IEEE Int. Conference on Computer Design (ICCD '10), Amsterdam, Netherlands, Oct. 2010.

3.6 Contribution of TUD-EWI-ST-PGS

Delft University of Technology, Faculty of Electrical Engineering, Mathematics & Computer Science, Parallel and Distributed Systems GroupProf.dr.ir. H.J. Sips, Dr.ir. D.H.J. Epema, Prof.dr.ir. A.J.C. van Gemund, Prof.dr. C. Witteveen, Dr. K.G. Langendoen

Research Report

The mission of the PDS group of TU Delft is to contribute to the scientific advancement in selected fields of parallel and distributed systems both in fundamental as well as application driven aspects; fundamental in the sense that we aim at the development and evaluation of generic methods and techniques, application-driven in the sense that also actual implementations of the concepts are investigated. The PDS group performs research in the following three areas.

In the area of grid and cloud computing, we focus on many aspects of resource management and scheduling, with as our research vehicle the KOALA multicluster scheduler, which has been deployed on different generations of the DAS. We have added support to KOALA for Bags-of-Tasks (BoTs) and workflows, and scheduling policies for these application types. We have also completed our research into processor co-allocation for parallel applications. In addition, we have done a thorough investigation into failures and their correlations in large-scale distributed systems of many different types. Finally, we have started working on the performance analysis of clouds.

In the area of multi-core programming, we have worked on case-studies, programming models, and application characterization. We have started with hardware-centric programming of several case-studies: image content analysis, radio-astronomy, medical imaging, basic and advanced signal processing. We have done several studies to compare the available multi- and many-core platforms in terms of performance and we have thoroughly investigated the programming models used for these architectures, with a special focus on portability and programmability. Finally, we have started to work on application characterization, using quantitative metrics to determine good matches between applications and platforms.

In the area of peer-to-peer and online social networks, we focus on many aspects of video distribution, with as our research vehicle the BitTorrent-based P2P system Tribler, of which we have made available multiple releases. In particular, we have included support for video-on-demand and live streaming into Tribler, and we have improved and investigated the

ASCI bi-annual Report 2009-2010 22

BarterCast reputation mechanism. In addition, we have completed our research into super-peer networks, we have performed a large-scale measurement of private and public BitTorrent communities, and we have performed multiple performance analysis studies of swarm-based P2P systems.

Research Plans

In the area of grid and cloud computing, we will incorporate support and scheduling policies for MapReduce applications into KOALA, we will investigate cloud performance, and we will perform a thorough application-oriented investigation of the current KOALA mechanisms and policies. In the area of multicore programming, we will continue working on many-core programming, tackling three issues: (1) novel optimizations and auto-tuning techniques for many-core architectures, (2) application characterization and its applicability to application-centric programming of many-cores, and (3) we will start a new direction of research towards large-scale graph-processing algorithms for heterogeneous systems built with many-core architectures. In the area of peer-to-peer and online social networks, we will continue our investigations of reputation mechanisms, our performance modeling studies of swarm-based P2P systems, and our design and implementation efforts in video-on-demand. Furthermore, we will investigate wiki-style P2P systems, a new Internet transport protocol for P2P video distribution, and online social networks for games.

Key publications

A.S. van Amesfoort, A.L. Varbanescu, H.J. Sips, and R.V. van Nieuwpoort, “Evaluating Multi-Core Platforms for HPC Data-Intensive Kernels,” In Proceedings ACM Computing Frontiers, pp. 207-216, 2009.

O.O. Sonmez, B. Grundeken, H.H. Mohamed, A. Iosup, and D. Epema, “Scheduling Strategies for Cycle Scavenging in Multicluster Grid Systems,” 9th IEEE/ACM Int'l Symp. on Cluster Computing and the Grid (CCGrid), pp. 12-19, 2009.

O.O. Sonmez, N. Yigitbasi, A. Iosup, and D.H.J. Epema, “Trace-based Evaluation of Job Runtime and Queue Wait Time Predictions in Grids,” Proceedings of the 18th ACM International Symposium on High Performance Distributed Computing (HPDC), pp. 111-120, 2009.

A.L. Varbanescu, H.J. Sips, K.A. Ross, Q. Liu, A. Natsev, J.R. Smith, and L.-K. Liu, “Evaluating Application Scenarios on the Cell/B.E.,” Concurrency and Computation: Practice and Experience, Vol. 21, pp. 85-100, 2009.

P. Garbacki, D.H.J. Epema, and M. van Steen, “The Design and Evaluation of a Self-Organizing Super-Peer Network,” IEEE Transactions on Computers Vol. 59, pp. 317-331, 2010.

D. Kondo, B. Javadi, A. Iosup, and D.H.J. Epema, “The Failure Trace Archive: Enabling Comparative Analysis of Failures in Diverse Distributed Systems,” 10th IEEE/ACM Int'l Symp. on Cluster, Cloud, and Grid Computing (CCGrid), pp. 398-407, 2010 (Best Paper Award).

M. Meulpolder, L. D'Acunto, M. Capota, M. Wojciechowski, J.A. Pouwelse, D.H.J. Epema, H.J. Sips, “Public and private BitTorrent communities: A measurement study,” In IPTPS 2010.

O.O. Sonmez, H.H. Mohamed, and D.H.J. Epema, “On the Benefit of Processor Co-Allocation in Multicluster Grid Systems,” IEEE Trans. on Parallel and Distributed Systems, Vol. 21, pp. 778-789, 2010.

3.7 Contribution of TUD-EWI-MM-CGCC

Delft University of Technology, Faculty of Electrical Engineering, Mathematics & Computer Science, Computer Graphics and CAD/CAM GroupProf.dr.ir. F.W. Jansen, Dr. W.F. Bronsvoort, Ir. F.H. Post, Dr. C.P. Botha

ASCI report 2009 – 2010

Delft University of TechnologyFaculty of Electrical Engineering, Mathematics and Computer ScienceComputer Graphics http://graphics.tudelft.nl

Dr. Ir. A.R. Bidarra, Dr. Ir. C.P. Botha, Dr. W.F. Bronsvoort, Prof. Dr. Ir. F.W. Jansen, Ir. F.H. Post

Computer graphics

ASCI bi-annual Report 2009-2010 23

Research in this programme is concerned with modelling of 3D objects for virtual worlds (gaming), and with visualisation for scientific and medical applications.

Visualisation is concerned with analysis and display of large data sets. The purpose is to provide insight by extracting important information from data sets, and supporting interactive exploration of the data. The focus of our research is on improved 3D interaction tools, on data reduction and data exploration techniques. The developed techniques are integrated within a virtual envrionment. A typical application is the visualization of environmental data (point clouds) and climate data (e.g. for extreme weather predictions). Real-time simulation and integrated visualisation and computational steering is now feasible on desktop computers with GPU support.

Another important application area is the processing and visualization of various medical imaging modalities, such as CT and MRI. Medical visualization in the group runs along two major lines: Visual Analysis for Medical Applications and Surgical Planning. In the first line, we work on the visualisation of multi-field data and higher order data such as diffusion tensor imaging. We have recently started projects on molecular imaging in order to visualize biological processes at cellular and molecular level over multiple time points and across multiple subjects, and also on the visual analysis of medical retrospective cohort studies, containing multi-modal multi-timepoint datasets of multiple patients. In the second line, we develop surgical simulation tools for the pre-operative planning of shoulder replacement surgery and minimally-invasive hip refixation.

Game technology is concerned with advanced modelling for the next generation of entertainment and serious games, in particular the dynamic generation and consistency maintenance of virtual worlds. Ultimate goal is to effectively assist game level designers in expressing and consistently maintaining in a model of the virtual world all intent specified throughout the various iterations of the design process.

Research Plan

In the coming years, we will focus our research efforts on the following goals: Game adaptivity of content and gameplay: enable runtime content generation in order to materialize player-

matching game worlds from personal gameplay requirements. Large-scale modeling of semantics for (serious) games: in this area, we are investigating better and easier ways of

integrating semantics into the game development process. Extension and evaluation of visual analytics in population imaging (PI) research. In PI research, mixed-modality

(multiple image volumes, blood measurements, genetics) are made of thousands of persons over multiple irregularly spaced timepoints, the research goal being to find retrospectively data features that predict pathology.

Hyper-realism in interactive visualization: We have recently started investigating the role of raytraced lighting in interactive volume rendering, resulting in new levels of realism.

Model-based anatomy and surgical planning: The development of detailed models of human anatomy, based on multiple data sources such as high-resolution crysections, immunohistochemistry and traditional 3D imaging modalities, could lead to significantly improved anatomical visualization and surgical planning.

Integrated Simulation and Visualization of Flow Phenomena: we initiated work on hardware-accelerated simulations with integrated 3D visualization to allow real-time interactive exploration of flow phenomena for the first time (collaboration with researchers on atmospheric processes, flooding scenarios).

Interactive exploration of (3D) sensor data: While 3D sensing techniques (e.g. LIDAR, depth ranging) become more prevalent, we aim to address the real-time aspects of processing and and visualization of this data work flow, enabling touch-based, 3D and augmented reality interaction.

For video surveillance tasks, we are exploring the integration of video streams with synthetic and reconstructed 3D models.

Five Key Publications

1. Bronsvoort WF, Bidarra R, van der Meiden H and Tutenel T (2010) The Increasing Role of Semantics in Object Modeling. Computer-Aided Design and Applications 7(3): 431-440.

2. de Carpentier GJP and Bidarra R (2009) Interactive GPU-based procedural heightfield brushes. In: Proceedings of Fourth International Conference on the Foundations of Digital Games, 26-30 April, Port Canaveral, FL, pp. 55-62.

3. P. Kok, M. Baiker, E. A. Hendriks, F. H. Post, J. Dijkstra, C. W. G. M. Löwik, B. P. F. Lelieveldt, and C. P. Botha, "Articulated Planar Reformation for Change Visualization in Small Animal Imaging," IEEE Transactions on Visualization and Computer Graphics, vol. 16, no. 6, pp. 1396–1404, 2010.

4. P. R. Krekel, E. R. Valstar, F. H. Post, P. M. Rozing, and C. P. Botha, “Combined Surface and Volume Processing for Fused Joint Segmentation,” The International Journal for Computer Assisted Radiology and Surgery, vol. 5, no. 3, pp. 263–273, 2010.

5. Gerwin de Haan, Huib Piguillet, Frits Post, "Spatial Navigation for Context-Aware Video Surveillance," IEEE Computer Graphics and Applications, pp. 20-31, September/October, 2010.

ASCI bi-annual Report 2009-2010 24

3.8 Contribution of TUD-TNW-QI

Delft University of Technology, Faculty of Applied Physics, Imaging Science & Technology, Quantitative Imaging GroupProf.dr. L.J. van Vliet, Dr. B. Rieger

Quantitative Imaging Group (TUD-TNW-IST-QI) 2009-2010

The Quantitative Imaging Group invents new image-based measurement principles through a combination of imaging physics and digital imaging leading to novel algorithms for image processing, image analysis, image reconstruction, and image recognition. We perform fundamental research with a focus on applications in Life Sciences and Health in a multi-disciplinary environment of technical and (pre-)clinical scientists and in close collaboration with leading industrial parties. The core competence of the group is on quantitative imaging, i.e. extracting quantitative information from multi-dimensional image data sets. Important contributions herein have been made to the field of computational microscopy, super-resolution reconstruction, an medical imaging. Computational microscopy offers tools to break the diffraction barrier in single molecule fluorescence microscopy and to achieve nanometer resolution in 3D electron-tomographic image volumes of frozen hydrated biological specimen. These techniques open new avenues for studying the molecular machinery inside the cell. We have developed an iterative algorithm that converges to the maximum likelihood estimate of the position and intensity of a single fluorophore. Our technique efficiently computes and achieves the Cramér-Rao lower bound, an essential tool for parameter estimation. An implementation of the algorithm on graphics processing unit hardware achieved more than 105 combined fits and Cramér-Rao lower bound calculations per second, enabling real-time data analysis for super-resolution imaging and other applicationsSuper-resolution reconstruction. Combining breakthroughs in filtering, estimation and inverse problems have led to methods capable of achieving super-resolution reconstruction in undersampled image sequences. We have obtained super-resolution for small moving targets against a highly cluttered background, i.e. objects containing entirely of boundary pixels that mix a space varying fraction of the object with a varying structured background.

In medical image analysis we have worked on virtual colonoscopy, dual tensor atlas building in DTI, and multi-object registration of wrist bones subject to global constraints. We realized Computer Aided Detection of polyps in CT colonography, a diagnostic tool for minimally invasive early detection of colon cancer, based on protrusion detection in surface rendered and volumetric data. In the coming years we further improve our methodology by permitting limited patient preparations and low-dose CT recordings. In neuro image analysis we try to differentiate in an early stage between physiologically (aging) and pathologically (e.g. Alzheimer's disease) in the brain, we search for spatiotemporal biomarkers of neurological disorders using sophisticated image processing and pattern recognition methods in Diffusion Weighted MRI data.

Expectations for 2011-2012In computational microscopy we are developing novel algorithms for super-resolution. In optical microscopy we work on exploiting the blinking characteristics of fluorophors to disentangle overlapping point-spread-functions and in electron tomography we are working on better algorithms for modelling the imaging process. For phase contrast tomography with electrons and x-rays we are developing quantitative forward models for designing new acquisition and reconstruction strategies, sparsity promoting image reconstruction techniques for solving underdetermined problems, and image processing algorithms capable of handling very noise data. In medical image analysis for CT colonography we expect to reach several milestones: to develop novel algorithms for computer cleansing of mild patient preparations and of very low dose. In abdominal imaging we are developing tools for non-invasive assessment of bowel inflammation in Crohn’s disease. In diffusion weighted MRI we aim to develop advanced algorithms permitting the quantification of diffusivity in both constituents of crossing fibre bundles in the human brain.

Key-publications 20091. M. van de Giessen, G.J. Streekstra, S.D. Strackee, M. Maas, K.A. Grimbergen, L.J. van Vliet, F.M. Vos, Constrained

registration of the wrist joint, IEEE Transactions on Medical Imaging, 28(12), 2009, 1861-18692. F.G.A. Faas, B. Rieger, L.J. van Vliet, D.I. Cherny, DNA deformations near charged surfaces: electron and atomic force

microscopy views, Biophysical Journal, Vol 97, 2009, 1148-1157.3. S. Brinkers, H.R.C. Dietrich, F.H. de Groote, I.T. Young, B. Rieger, The Persistence Length of Double Stranded DNA

Determined Using Dark Field Tethered Particle Motion, Journal of Chemical Physics, 130(21):215105, 2009.4. E.J. Aukema, M.W.A. Caan, N. Oudhuis, C.B.L.M. Majoie, F.M. Vos, L. Reneman, B.F. Last, M.A. Grootenhuis, A.Y.N.

Schouten-van Meeteren, White matter fractional anisotropy correlates with speed of processing and motor speed in young childhood cancer survivors, International Journal of Radiation Oncology Biology Physics, Vol. 74, Issue 3, 2009, 837-843.

5. M.W.A. Caan, C.A. Sage, M.M. van der Graaf, K.A. Grimbergen, S.G. Sunaert, L.J. van Vliet, and F.M. Vos, Dual tensor atlas generation based on a cohort of coregistered non-HARDI datasets, in: G.-Z. Yang et al. (Eds.), Medical Image Computing and Computer-Assisted Intervention - MICCAI 2009 (12th International Conference, London, UK, September 20-24, 2009, Proceedings, Part I), Lecture Notes in Computer Science, vol. 5761, Springer-Verlag Berlin Heidelberg, 2009, 869-876.

ASCI bi-annual Report 2009-2010 25

Key-publications 20101. V.F. van Ravesteijn, C. van Wijk, F.M. Vos, R. Truyen, J.F. Peters, J. Stoker, L.J. van Vliet, Computer aided detection of

polyps in CT colonography using logistic regression, IEEE Transactions on Medical Imaging, Vol. 29, No. 1, 2010, 120-31, art. no. 5196824.

2. C. van Wijk, V.F. van Ravesteijn, F.M. Vos, L.J. van Vliet, Detection and segmentation of colonic polyps on implicit isosurfaces by second principal curvature flow, IEEE Transactions on Medical Imaging, Vol 29, No. 3, 688-698, 2010, art.no. 5423301.

3. M. Vulovic, B. Rieger, L.J. van Vliet, A.J. Koster, R.B.G. Ravelli, A toolkit for the characterization of CCD cameras for transmission electron microscopy, Acta Crystallographica Section D- Biological Crystallography, D66, 2010, 97-109.

4. C.S. Smith, N. Joseph, B. Rieger, K.A. Lidke, Fast, single-molecule localization that achieves theoretically minimum uncertainty, Nature Methods, 2010, 7(5):373-375.

5. A.W.M. van Eekeren, K. Schutte, L.J. van Vliet, Multi-frame super-resolution reconstruction of small moving objects, IEEE Transactions on Image Processing, Vol. 19, No. 11, 2010, 2901-2912.

3.9 Contribution of UL-LIACS

Leiden University, Faculty of Mathematics and Natural Sciences, Leiden Institute of Advanced Computer Science (LIACS)Prof.dr. H.A.G. Wijshoff, Dr. M.S. Lew, Dr. A.A. Wolters, Dr. D.P. Huijsmans, Dr. E.M. Bakker, Dr.ir. T.P. Stefanov,Dr.ir. B. Kienhuis, Prof.dr.ir. E.F. Deprettere

Research 2009-2010

The mission of LIACS research program ‘Computer Systems and Imagery & Media’ is to improve the state of the art in systems and media analysis, and to guide new research areas which are of clear importance to society. In our research, we rigorously demonstrate the effectiveness of the resulting novel techniques and contribute to the leading evaluation and benchmarking projects. This programme covers a wide variety of research topics, including embedded systems and software, parallel and distributed computing, and media research and technology, including imaging and its applications to bio informatics. The research effort is clustered around three themes.

Embedded Systems and Software The research in embedded systems and software deals with modelling of applications and multi-processor architectures, and mapping methods in these domains, at various levels of abstraction, for exploration and design, theoretically and practically, down to real platforms. The working methodologies relies on advocating and applying modern state–of–the–art Software Engineering Practice both in the way the group’s projects are integrated, documented, and assessed, and in the way software that implements research results is written, tested and assessed. There are several themes that are researched in several projects and are jointly contributing to the knowledge, expertise, and skills on local, national, and international competitive settings. The research focuses on strategically well-chosen core activities that are briefly described below:

• Research on novel methods and tools for automated parallelization and porting of dynamic/adaptive applications on an MPSoC architecture. These tools will simplify the manual effort that is currently needed by the application developers to transfer sequential software in a concurrent programming model.• Research on novel methods, techniques, and tools for designing Embedded Multi-processor Systems-on-Chip with adaptivity and reliability support. The static approach to embedded MPSoC design will be extended to address adaptivity requirements by setting up a methodology and the corresponding tool support for adaptive (i.e., run-time) mapping of application tasks to the underlying architecture resources to cope with QoS and/or dependability demands.

Research on novel techniques and tools to tightly couple system-level power/ energy modeling with system-level synthesis techniques in order to significantly improve the trustworthiness of system-level power/energy models and to quickly explore and design energy-efficient MPSoCs. More specifically, MPSoC synthesis and measurement techniques are investigated that are needed for dynamically validating and calibrating system-level power/energy and performance models.

Parallel and Distributed Computing Systems

Research in parallel and distributed computing concentrates on: (optimizing) compilers, grid computing, application drivers for large-scale applications, and large-scale database systems. As within the embedded system research activities the approach taken evolves around the mapping problem of (large scale) applications onto (existing) computing platforms. The research focuses on strategically well–chosen core activities that are briefly described below:

• Performance Predictions and Resource Management in a Grid To design an efficient scheduler is one of the most challenging problem for the efficient utilization of parallel computers. This leads to a need for representative workload models that allow the generation of specific workloads for the evaluation of

ASCI bi-annual Report 2009-2010 26

different scheduling techniques. During 2009 and 2010 we have developed a new workload model toolkit. In addition an efficient runtime predictor has been developed for designing smart scheduling decisions.• HIRLAM on a Grid Environment The performance of the operational numerical weather forecast system HIRLAM on the new DAS-3 grid environment was evaluated. DAS-3 consists of five clusters located at five different locations. Furthermore, we investigated the possibilities in overlapping computations and communications for HIRLAM to decrease the communication overhead.• Data Structure Independent Programming Traditional analyses break down when they encounter pointer based codes. Therefore, as the potential performance of processors increases, new means must be explored to extract parallelism. We have developed new pointer optimizations by using split pool techniques combined with aggressive sublimation techniques to eliminate pointer occurrences in scientific computing based codes• Reshaping Memory Access Patterns Using a combination of compile-time and run-time techniques, the access patterns of pointer based codes are restructured such that the resulting code is analyzable by existing techniques for regular codes. These steps result in an intermediate code, which is amenable to data dependency analysis and as such can be compiled into a highly optimized executable.

Media SystemsMedia systems evolves around two approaches. The first approach focuses on the scientific investigation of novel directions and paradigms in the field of multimedia retrieval with emphasis on content-based methods in images, video, audio, and scientific data. As a special application, the integration bio-imaging information and image information with other bio-molecular information resources is studied. The second approach is the embedding of human beings in a computerized world, which is studied by creating such embeddings.Media systems evolve around the scientific investigation of novel directions and paradigms in the field of multimedia retrieval with emphasis on content-based methods in images, video, audio, and scientific data. As a special application, the integration bio-imaging information and image information with other bio-molecular information resources is studied.

In multimedia retrieval, we completed the second stage of research and experiments on the new paradigm of Artificial Imagination in multimedia information retrieval. We also introduced the largest open-access and legally redistributable community benchmark for image retrieval (over a million images). We also developed a new salient point paradigm which outperforms the leading descriptors from the research community such as SIFT and SURF.

In the area of bioinformatics, we developed new algorithms for genotype/phenotype data mining. The development of an Hidden Markov Model based classifier for biological imagery was completed and tested under a wide variety of different tissue samples. A new method for viewing the development of micro-tubules was completed.

In the next phase, we will continue investigating a general artificial imagination model. The artificial imagination will be used for improving both machine learning and information retrieval.

Future Research Plans

The research activities changed over the last couple of years from a strong focus on high performance computer systems towards more and more data-oriented computations. Specific examples of these shifts can be found in the themes “Content based image retrieval”, in which the main challenge lies in indexing and classifying tremendous amounts of visual images, “HIRLAM on the Grid”, in which the main challenge will be to integrate and reformat tremendous amounts of meteorological data distributed over a number of different networks (the Internet, the world-wide meteorological World Weather Watch network, etc.), “Data compilation”, in which the direct target is the integration and coupling of large legacy database applications and “Hardware dependant Software (HdS) solutions to improve IP integration in the SoC design process (quality / productivity)”, in which the operating system and the application software are separated from the underlying hardware.As our programme has grown, our research activities have also evolved to address important new research areas and critical challenges in transferring technology from research labs to society. In particular, it is well known that research systems, which work well in a laboratory environment, may not work well in the real world. In some cases, this is because there was a significant lack of communication between the researchers and the practitioners/users. In other cases the problem has to do with developing high quality evaluation and benchmarking procedures, which reflect real world usage.The changes as described above became firmly incorporated into the research activities in this programme. In fact our research activities became more focused on data representation, processing data representation, and the reduction of data representation by employing (semantic) transformation and interpretation of media data.

Key publications

A. Gerstlauer, C. Haubelt, A.D. Pimentel, Todor Stefanov, D.D. Gajski, and J. Teich, "Electronic System-Level Synthesis Methodologies", IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems (TCAD), vol. 28, No. 10, pp. 1517-1530, Oct 2009.

Ozana Silvia Dragomir, Todor Stefanov, and Koen Bertels, "Optimal Loop Unrolling and Shifting for Reconfigurable Architectures", ACM Transactions on Reconfigurable Technology and Systems (TRETS), vol. 2, No. 4, pp. 1-24, Sept 2009.

ASCI bi-annual Report 2009-2010 27

Harmen L.A. van der Spek, C.W. Mattias Holm, Harry A.G. Wijshoff, “Automatic Restructuring of Linked Data Structures”, LCPC 2009, pp. 263-277, 2009.

Harmen L.A. van der Spek, C.W. Mattias Holm, Harry A.G. Wijshoff, “How to unleash array optimizations on codes using recursive data structures”, Proceedings of the ACM International Conference on Supercomputing (ICS), pp. 275-284, 2010.

Bart Thomee, Mark J. Huiskens, Erwin M. Bakker, Michael S. Lew: Deep exploration for experiential image retrieval. ACM Multimedia 2009: 673-676

Mark J. Huiskess, Bart Thomee, Michael S. Lew: New trends and ideas in visual concept detection: the MIR flickr retrieval evaluation initiative. Multimedia Information Retrieval 2010: 527-536

3.10 Contribution of UU-ICS-GMT

Contribution of UU-ICS-GMTUtrecht University, Faculty of Science, Department of Information and Computing Sciences, Multimedia and Geometry; Games and Virtual WorldsProf.dr. M.H. Overmars, Dr. M. van Kreveld, Dr.ir. F. van der Stappen, Prof.dr. R. C. Veltkamp, Prof.dr P.J. Werkhoven

Research 2009-2010Research in Games and Virtual World has focused on path planning and crowd simulation, and on manipulation planning. One of the fundamental tasks that virtual characters have to perform is to plan their paths between different locations in the virtual world. Modern virtual environment applications require a path planner that is fast to ensure real-time planning of large crowds of characters and flexible so that the characters can avoid local hazards, such as small and dynamic obstacles. In addition, the computed paths need to be smooth to ensure natural looking motions. To address these issues, we recently introduced the Indicative Route Method as new path planning approach in interactive worlds and games.

Besides exhibiting believable path planning behavior, the virtual characters should also be able to move towards their desired locations in a life-like manner avoiding collisions with each other. Therefore, we proposed a physically-based model for solving interactions between virtual pedestrians that have converging trajectories]. The method is fast, simple to implement and exhibits important emergent phenomena that have been noted in the pedestrian literature. We also addressed the issue of realistic collision avoidance among virtual humans by exploiting experimental interaction data between real pedestrians. In the resulting method, the characters predict collisions and take early action to avoid them by adapting their direction and/or speed. Recently, we extended this technique to simulate small groups of virtual humans.

Research in manipulation planning has considered different forms of grasping. The efforts have focused on caging grasps and grasps of flexible structures. Caging grasps are loose grasps that do not exclude all motions of the grasped object but only prevent it from escaping, which is sufficient in many applications. We have studied algorithms for computing caging grasps of an object, and have established a remarkable upper bound on the complexity of the set of all three-finger caging grasps of a convex object, which stays several orders of magnitude below the best known bound for arbitrary objects. In addition we have continued our work on the immobilization of hinged chains of rigid objects and obtained various new insights. Finally we have developed algorithms for grasping and retracting layers of human tissue, in order to minimize stress, strain, and tissue damage during surgery.

In media technology, research was performed in the following areas: image retrieval, shape retrieval, 3D modeling, and music retrieval. We have developed a number of different methods for shape recognition, shape indexing with vantage objects, and layout indexing with Laplacian and Hermitian graph spectra. Apart from provable properties, the behavior in practice has been rigorously evaluated on the basis of ground truth test sets, and compared with many other content-based image retrieval methods.

In computational geometry our research in data imprecision has continued, but will receive less attention since the relevant project has ended. We have finished and published the obtained results regarding convex hulls and triangulations of imprecisie points. We have also shown how to compute the similarity of two polyhedral terrains under vertical scaling and translation.

Of relevance to GIS is our research on trajectories. We have developed the concept of a median trajectory and algorithms that show how to compute it. Furthermore, we have developed an algorithmic framework for segmenting trajectories of moving objects such that the pieces have similar characteristics within each piece. Also, we have given a new, efficient algorithm for detecting the most similar subtrajectories for two given trajectories. In graph drawing, we have performed research on drawing graphs with thick vertices and edges, and on simultaneous embeddings of two graphs. In 3D reconstruction, we have performed research on LiDAR point sets from areal and ground-based scans. The resulting point clouds were analyzed algorithmically, and more meaningful shapes like rectangles and composite objects were computed.

ASCI bi-annual Report 2009-2010 28

We have organized the 3D Shape Retrieval Contest (SHREC), which has a large impact on the community. We have performed research on the mapping of music notation and audio into a symbolic representation. Then the symbolic representation is indexed and matched to a query. One specific application is to find folk songs that are similar to a query, yet not exactly the same, due to oral transmission. Our retrieval methods are integrated with the Nederlandse Liederenbank, the Dutch folk songs database at the Meertens Institute for ethnological research.

Future Research PlansThe focus of our path planning and crowd simulation for the forthcoming years will be on crowd management in evacuation scenarios. The goal of the work is a serious game for managing panicking crowds. The work takes place at three interrelated levels. Path planning will play a role at the top level where the purpose of the training is to intervene in order to distribute and steer groups of people. At the intermediate level, the actions of individuals as a result of external triggers is modeled and executed. At the lower level, realistic emotions and gestures of individuals need to be animated. In our manipulation planning research, we will concentrate on dealing with uncertainty. We will study imprecision in the shape of the manipulated objects as well as inaccuracy of the manipulator. The aim is to develop algorithms that determine manipulation plans that work despite the presence of uncertainty. We will also study grasps that do not immobilize an object, but rather provide support in a limited set of directions. While we plan to extend results in the above areas, new research directions will be animation, multi-modal interaction, computer vision, game content generation, and mobile interaction. The animation research will mainly focus on generating realistic virtual character motion. In the animation community, character motion is generally divided into facial and body animation, which require different approaches. In collaboration with TNO, we will start a new line of research in multi-modal interaction. The goal of this research is to develop new concepts and techniques with which users can steer the behavior of virtual characters. In particular we are interested in techniques with which users can control the navigation and manipulation performed by such characters. We will start working on detecting, tracking, modeling, and recognition of persons, their pose, and gestures, from multiple video cameras. Our approach is to exploit 3D information that we derive from coherence in views. Research on mobile interaction will be directed towards new, better ways for accessing, handling, and manipulating different kind of media on handheld devices. With respect to spatial data mining we will study concepts like flocking, meeting, convergence, etc., for set of trajectories, and developed algorithms to detect these patterns.

Key Publications 2009-2010Maike Buchin, Anne Driemel, Marc J. van Kreveld, Vera Sacristan: An algorithmic framework for segmenting trajectories based on spatio-temporal criteria. Proceedings of ACM SIGSPATIAL 2010: 202-211.Marc J. van Kreveld, Maarten Löffler, Joseph S. B. Mitchell: Preprocessing Imprecise Points and Splitting Triangulations. SIAM J. Comput. 39(7): 2990-3000 (2010).M. Vahedi, A.F. van der Stappen, On the complexity of the set of three-finger caging grasps of convex polygons, Proceedings of Robotics: Science and Systems (2009), pp. 57-64.I. Karamouzas, R. Geraerts, and M. Overmars. Indicative Routes for Path Planning and Crowd Simulation. In: FDG '09: Proc. of the 4th International Conference on Foundations of Digital Games, 2009, pp. 113-120.I. Karamouzas and M. Overmars. Simulating the Local Behaviour of Small Pedestrian Groups. In: VRST '10: Proc. of the 17th ACM Symposium on Virtual Reality Software and Technology, 2010, pp. 183-190. Frank B. ter Haar, Remco C. Veltkamp. Expression modeling for expression-invariant face recognition. Computers & Graphics, vol. 34, 2010, 231–241. Sergio Cabello, Mark de Berg, Panos Giannopoulos, Christian Knauer, René van Oostrum, Remco C. Veltkamp. Maximizing the Area of Overlap of two Unions of Disks under Rigid Motion. International Journal of Computational Geometry and Applications, 19(6), 2009, 533-556. Frank B. ter Haar, Remco C. Veltkamp. A 3D face matching framework for facial curves. Graphical Models 71(2), 2009, 77-91, DOI 10.1016/j.gmod.2008.12.003.

3.11 Contribution of TUD-L&R-FRS

Staff: Dr.ir. BGH GortePhD Students: Alexander Bucksch, Fatemeh Karimi Nejadasl

Just before the start of the reporting period (December 2008) the chair of OLRS was re-occupied and subsequently two new assistant professors were hired. Also the section saw an increase in the number of PhD candidates, which however did not yet materialize into new ASCI memberships, as a consequence of a shift of focus in the section's research portfolio.

One project executed within the ASCI context concerned analysis of helicopter-recorded image sequences to measure and monitor driver behaviour in traffic congestion at motorways. This was a collaboration with the faculty of Civil Engineering and Geosciences at TU Delft with NWO funding.The resulting thesis was defended in February 2010.

A second PhD project of an ASCI PhD student member concerned using terrestrial LiDAR to measure the structure of trees,

ASCI bi-annual Report 2009-2010 29

as well other forestry-parameters like wood volume and biomass. The thesis was defended shortly after the reporting period (21 April 2011). The project was executed in cooperation with the University of Goettingen and the German LiDAR manufacturer Z&F.

Earth observation from satellites operating in the visible, near infra-red and thermal infra-red ranges of the electromagnetic spectrum is gaining significance with the increasing needs for high-frequency uniform monitoring of processes related to vegetation and the water and carbon cycles, against the background of increasing population pressure and climate change. In addition to imagery originating from satellite sensors also airborne and 'close range' image data are analysed at OLRS. Interesting technology is provided by laser range measurement devices that nowadays can be operated 'close range', airborne and even from satellite (ICESat). These three forms are studied with similarly large interest for very diverse applications in hydrology, forestry, flood modeling, (urban) climate modeling and land use/land cover mapping.

As mentioned above, the range of subjects studied at the chair is being extended. The computer vision/automatic interpretation based analysis of image and range data, as described above, will remain one of the the fields of focus. A new activity (within the chair) is physics based retrieval of parameters of geo- and biodynamical processes on the basis of multi-temporal, multi-angular and multi-spectral EM-radiation measurements, from both active and passive sensors. Naturally these two fields will not be seen in isolation. Their integration into a comprehensive modeling and monitoring strategy will be the focus of the chair during the coming years.

Key publications by ASCI members of OLRS:

Bucksch, A.K., Lindenbergh, R.C. and Menenti, M. (2010), "SkelTre - Robust skeleton extraction from imperfect point clouds", The Visual Computer: International journal of computer graphics, Vol. 26(10), pp. 1283-1284.

Bucksch, A.K., Lindenbergh, R.C., Menenti, M. and Abd Rahman, M.Z. (2009), "Skeleton-based botanic tree diameter estimation from dense LiDAR data". In: Proceedings Optics and Photonics 2009; LiDAR Remote Sensing for environment X, Washington, USA, Vol. 7460, pp. 1-11. SPIE.

Abd Rahman, M.Z., Gorte, B.G.H. and Bucksch, A.K. (2009), "A new method for individual tree delineation and undergrowth removal from high resolution airborne LiDAR". In: Proceedings ISPRS workshop laserscanning 2009, Paris, Vol. XXXVIII, pp. 283-288. ISPRS.

Bucksch, A.K. and Fleck, S. (2011), "Automated detection of branch dimensions in woody skeletons of fruit tree canopies", Photogrammetric Engineering & Remote Sensing, Vol. 77(3), pp. 229-240. [article]

Karimi Nejadasl, F. and Lindenbergh, R.C. (2011), "Vehicle detection from an image sequence collected by a hovering helicopter". In: Photogrammetric Image Analysis, ISPRS Conference, PIA 2011, Munchen, Vol. 38, pp. 209-214. Institute of Photogrammetry and Cartography.

3.12 Contribution of RUG-CS-SVCG

1. Global research descriptionThe research group Scientific Visualization and Computer Graphics carries out research in the area of scientific visualization, information and software visualization, computer graphics and innovative interfaces using large, touch-sensitive displays. With respect to applications, the research concentrates on fundamental and applied problems from the life sciences, in particular functional brain imaging and bioinformatics, and astronomy.

In interactive data visualization, we worked on surface reconstruction from noisy and sparse data sets. Special attention is devoted to acceleration of algorithms through special (multiscale) data structures and/or by mapping the involved computations to programmable Graphics Processing Units (GPUs). Examples are accelerated wavelet lifting or sparse level-set methods on graphics hardware using CUDA. Previous research in skeletonization was extended by saliency-dependent simplification. We have developed image-based edge bundles, which depict a bundled graph layout at increasingly coarse levels of detail. A neurophysiologically plausible population code model for feature integration was developed which explains visual crowding. In software visualization, we studied techniques for visualizing combinations of software architecture diagrams and quality metrics. Several new methods were developed for interactive static analysis and understanding of large software systems. The approaches were validated in collaboration with industrial partners and the open-source community. In non-photorealistic rendering (NPR) we worked on hand-posture based interaction that supports the exploration and visualization of flow datasets, and on illustrative visualization of line data. In addition, we explored interaction concepts for digital concept sketching and extended it to support the interaction with 3D visualization spaces on large touch surfaces.

Applications. We applied our methodological work in several application domains. Anatomical and functional brain visualization techniques were developed, based on a variety of data: EEG, functional MRI, or diffusion tensor imaging (DTI). As an example, we developed the depth-dependent halo technique for illustrative visualization of fiber tracts that were

ASCI bi-annual Report 2009-2010 30

extracted from DTI data of the human brain. A method was proposed for quantifying differences between multichannel EEG coherence networks. In bioinformatics, we worked on the visualization of gene expression time series in both a gene regulatory network and metabolic pathway context. In astronomy, we used visual analytics methods to study the relations between the spatial arrangement of galaxies and the distribution of various attributes in very high-dimensional parameter spaces based on ranking subspaces for clustering. With partners from astronomy we are investigating collaborative interaction using large touch-sensitive displays.

2. Expectations for the coming periodThere are two new international initiatives in which the group is extending its efforts, i.e., Visual Analytics and Neuroinformatics. Visual Analytics concerns the integration of visualization with other analytical methodologies, such as statistics, data-mining, and cognition. The group already collaborated within the European project Vismaster (2008-2010), which developed a European roadmap for Visual Analytics. The group is active in establishing new research networks in this area on the international level. Interesting connections are being explored with artificial intelligence and cognitive science. The second area is Neuroinformatics, which supports discovery and innovation in neuroscience by information technology, for example in the study of neurodegenerative diseases. The group is actively participating in local, national and international initiatives in this area.

3. Selected publications from the period 2009-2010R. van den Berg, J.B.T.M. Roerdink, and F.W. Cornelissen (2010) A Neurophysiologically Plausible Population Code Model for Feature Integration Explains Visual Crowding. PLoS Comput Biol, 6(1):e1000646, January 2010.

W.J. van der Laan, A.C. Jalba, and J.B.T.M. Roerdink (2010) A Memory and Computation Efficient Sparse Level-Set Method. Journal of Scientific Computing, 2010.

A. Telea and O. Ersoy (2010) Image-Based Edge Bundles: Simplified Visualization of LargeGraphs. Computer Graphics Forum, 29(3):843–852, June 2010. Second Best Paper Award at EuroVis 2010.

M.A. Westenberg, J.B.T.M. Roerdink and O.P. Kuipers, and S.A.F.T. van Hijum (2010) SpotXplore: a Cytoscape plugin for visual exploration of hotspot expression in gene regulatory networks. Bioinformatics, 26(22):2922–2923, 2010.

L. Yu, P. Svetachov, P. Isenberg, M.H. Everts, and T. Isenberg (2010) FI3D: Direct-Touch Interaction for the Exploration of 3D Scientific Visualization Spaces. IEEE Transactions on Visualization and Computer Graphics, 16(6):1613–1622, November/December 2010

H. Bekker, A.A. Brink, and J.B.T.M. Roerdink (2009) Reducing the time complexity and identifying ill-posed problem instances of Minkowski sum based similarity calculations. International Journal of Computational Geometry and Applications, 19(5):441–456, October 2009.

M.H. Everts, H. Bekker, J.B.T.M. Roerdink, and T. Isenberg (2009) Depth-Dependent Halos: Illustrative Rendering of Dense Line Data. IEEE Transactions on Visualization and Computer Graphics, 15(6):1299–1306, November/December 2009. Best Paper Award at IEEE Visualization 2009

A.C. Jalba and J.B.T.M. Roerdink (2009) Efficient Surface Reconstruction from Noisy Data using Regularized Membrane Potentials. IEEE Trans. Image Processing, 18(5):1119–1134, May 2009

M.M. Lorist, Eniko Bezdan, Michael ten Caat , M.M. Span, J.B.T.M. Roerdink, and N.M. Maurits (2009) The influence of mental fatigue and motivation on task switching; an EEG coherence study. Brain Research, 1270:95–106, 2009.

L. Voinea and A. Telea (2009) Visual Querying and Analysis of Large Software Repositories. Empirical Software Engineering, 14(3):316–340, 2009

3.13 Contribution of RUG-CS-IS

University of Groningen, Faculty of Mathematics and Natural Sciences, Johann Bernoulli Institute of Mathematics and Computer Science, Intelligent SystemsProf.dr. N. Petkov, Dr. M.H.F. Wilkinson

QUESTION 1

What’s the name of your group?University of GroningenFaculty of Mathematics and Natural SciencesJohann Bernoulli Institute of Mathematics and Computing ScienceIntelligent Systems

ASCI bi-annual Report 2009-2010 31

http://www.cs.rug.nl

QUESTION 2Image processing, computer vision, pattern recognition, brain-inspired computing,Applications in life sciences and health care, Connected filters and morphological operators (C.2)Machine learning and neural networks (C.3)

Question 3

Brain-inspired computer visionModels of information processing in visual cortex are developed and used in com- puter algorithms. This research is relevant for the areas of image processing, com- puter vision, pattern recognition, visual perception, and computational neuroscience. Our goal is to understand how people see and to deploy principles of natural vision in computer algorithms for artificial vision. Using facts from neuroscience and vi- sual perception, we build models of visual information processing in the brain and use them in computer simulations to obtain insights and derive practical computer vision algorithms.One example is a model of a grating cell that we developed [Petkov, Kruizinga: 1997 Biological Cybernetics 76: 83-96] and used in a texture operator [Kruizinga, Petkov: 1999 IEEE Trans. on Image Processing 8: 1395-1407], [Grigorescu, Petkov, Kruizinga: 2002 IEEE Trans. on Image Processing 11: 1160-1167]. By means of computer simulations we demonstrated that grating cells may play an important role in the disambiguation of edge information in early vision (texture vs. contours).Another example is our model of non-classical receptive filed inhibition, also called surround suppression, in orientation selective neurons [Petkov, Westenberg: 2003 Bi- ological Cybernetics 88: 236-246]. We demonstrated that the biological role of this inhibitory mechanism is quick pre-attentive detection of object contours and region boundaries in natural images that are rich in texture. We proposed various contour de-tection algorithms that deploy this mechanism and showed that they are more effective in detecting object contours and region boundaries than traditional computer vision al- gorithms for edge detection [Grigorescu, Petkov, Westenberg: 2003 IEEE Trans. on Image Processing 12: 729-739], [Grigorescu, Petkov, Westenberg: 2004 Image and Vision Computing 22: 609–622], [Papari, Campisi, Petkov, Neri: 2007 EURASIP J. on Advances in Signal Processing, Article ID 71828]. This work has been extended by applying gestalt principles to edge grouping [Papari, Petkov: 2008 IEEE Trans. Im- age Processing 17: 1950-1962], [Papari, Petkov: Proc. SPIE 2008, vol. 6812, art. no. 68121B]. Meanwhile, the article ”Contour detection based on nonclassical recep- tive field inhibition” [Grigorescu, Petkov, Westenberg: 2003 IEEE Trans. on Image Processing 12: 729-739] received a large number of citations.We also studied the orientation and speed tuning properties of spatio-temporal 3D Gabor and motion energy filters with surround suppression as models of time- dependent receptive fields of simple and complex cells in primary visual cortex (V1) [Petkov, Subramanian: 2007 Biological Cybernetics, 97: 423-439]. We demonstrated how these filters are related to motion detection, noise reduction, texture suppression and contour enhancement.In the same line of research we model the detection of contour segments and shape representation in areas V4 and TEO of visual cortex.Another result of our research that is inspired by psychological research on the human visual system is a method for the evaluation of the robustness of shape recog- nition algorithms to incompleteness of contours [Ghosh, Petkov: 2005 IEEE Trans. on Pattern Analysis and Machine Intelligence 27: 1793-1804].Image processing and computer visionIn shape analysis we study geometrical approaches in which a feature point is char- acterized by the spatial arrangement of other feature points around it. The collection of local geometrical descriptors for the different feature points of an object is used as a shape characteristics of that object [Grigorescu, Petkov: 2003 IEEE Trans. on Image Processing 12: 1274-1286].Another direction in our work is the development of image processing operators that add artistic effects to photographic images [Papari, Petkov, Campisi: 2007 IEEE Trans. on Image Processing, 16: 2449-2662], [Papari, Petkov: 2009 IEEE Trans. on Image Processing 18: 652-664].On the applications side, we collaborate with researchers from the University of Leon, Spain, in the area of automatic classification of boar spermatozoa [Sanchez, Petkov, Alegre: 2006 Cellular and Molecular Biology, 52: 38-43], [Petkov, Alegre, Biehl, Sanchez: 2008 Comp. in Biol. and Medicine 38: 461-468]. We also collabo- rate with the Department of Dermatology of the University Medical Center Groningen [Bosman, Petkov, Jonkman: 2010 Skin Research and Technology 16: 109-113] on the application of content based image retrieval and expert systems to dermatological prob- lems and with the Department of Ophtamology on the automatic detection of diabetic retinopathy.Connected filters, Connectivity Theory and SegmentationConnected filters are a comparatively new field of research within mathematical morphology. They are edge preserving operators which have found use in noise re- moval, texture analysis, image compression and description, and feature extraction. Research on connected operators in our group entails algorithm development (includ- ing parallelization), development of new classes of filters, applications to 2-D and 3-D medical images, and the development of new connectivity measures for these filters for increased robustness. Recently, content-based image retrieval (CBIR) has been added to the list of application areas. One line of this research links to visual cortex modelling: developing morphological analogues of texture operators based on mod- els of certain visual cortical cells. It is hoped these morphological counterparts will be an order of magnitude faster, whilst retaining the useful properties of the cortical cell models. Finally, fast visualization based on connected attribute filters is being ex- plored. Recently, work has begun expanding this line into hyperconnected filters and attribute-space-connected filters, which increase the flexibility of perceptual groupings available, and allow dealing with overlap explicitly.

ASCI bi-annual Report 2009-2010 32

Segmentation is a core problem in image analysis, and methods based on both simple thresholding methods and more advanced methods such as watersheds and de- formable models are being explored. Application areas are many, but the focus lies on biomedical imaging, both macroscopic (MRI, CT) and microscopic. New application domains in astronomy are also being explored.Machine learning and neural networksThe term machine learning refers to an area of computer science in which given example data is analysed. The aim could be, for instance, the identification of a classi- fication scheme based on labeled examples or interpolation/extrapolation in regression problems. Methods of unsupervised learning address problems like clustering, dimen- sion reduction, or correlation analysis in, potentially, high-dimensional data.In our research group, learning algorithms are developed and employed for data driven parameter adaptation in, for instance, prototype-based classifiers or other adap- tive systems.In the context of supervised learning, the so-called Learning Vector Quantization (LVQ) constitutes a family of algorithms which identify typical representatives of the classes. Together with a measure of distance or similarity, these prototypes are used to parameterize the classification scheme. Similarly, in unsupervised Vector Quantization (VQ), prototypes are used to represent large amounts of data by means of clustering or dimension reduction.Obviously, a key question in the above mentioned and many other similarity based methods is the choice of an appropriate distance measure. In recent years we have put forward methods of Relevance Learning, in which a generalized Euclidean distance measure is parameterized and adapted in the training phase. Hence, the classifier and a discriminative similarity measure are identified in the same data driven process.As a particular successful approach we have established Matrix Relevance Learning and applied it in various practical contexts, including content based image retrieval and tumor classification. Most recently we have applied matrix relevance learning in the low-dimensional representation and visualization of complex, labeled data sets.In a complementary line of research we investigate the use of novel, alternative distance measures which are not related to conventional Euclidean metrics. Statistical divergences, for instance, can be used as meaningful distance measures whenever the data consists of non-negative, potentially normalized features. Divergences are partic- ularly promising if feature vectors comprise statistical or functional information, e.g. in the form of histograms or spectra.In our theoretical investigations we aim at a better understanding of algorithms, learning processes and their performance. In the framework of model situations, we study the dynamics of on-line learning and systematically compare different algo- rithms. Similar investigations, which borrow basic concepts from statistical physics, concern the typical properties of large learning systems in the context of batch- or of- fline training. The ultimate goal of these studies is to design and optimize practical training algorithms.Consequently, we aim at testing the developed methods in real world applications. Recently, we have addressed several classification problems in the life sciences and image processing. Important examples are the classification of adrenal tumors based on metabolomics data and content based image retrieval in dermatology. Another project deals with the detection of Cassava Mosaic Disease based on color histograms obtained from leaf images.

QUESTION 4Please indicate in which external projects (2nd and 3rd funding projects) your group participated in in 2009-2010 – new and old projects.

The position of Kerstin Bunte is financed by a grant to Biehl and Petkov in the Open Competition 2006/07 of NWO. New funding for the project Connected Mor- phological Operators for Tensor Images (COMOTI) was awared to Jos Roerdink and Michael Wilkinson in the Open Competition 2010 of NWO. A PhD student has now been appointed. Wilkinson is also co-applicant in the ASTROVIS project, which funds two PhD students (Hugo Budelmeijer at the Kapteyn Astronomical Institute, and Bilkis Ferdosi at the Johann Bernoulli Institute). The positions of Tushabe and Kiwanuka are funded through a grant by NUFFIC. The position of Emerencia is financed by a grant to Aiello and Petkov from NWO Medical Sciences, 2009.QUESTION 5Doctoral Degrees: Which promotions took place in 2009-2010?a) G.K. Ouzounis. Generalized Connected Morphologcial Operators for Robust Shape Extraction. Promotor: N. Petkov, co-promotor: M.H.F. Wilkinson, Uni- versity of Groningen, 2009, ISBN 978-90-367-3698-5, 140 pages. b) G. Papari. Texture vs. Cotours: Explorations in the fields of contour detection and artistic imaging. Promotor: N. Petkov, University of Groningen, 2009, ISBN 978-90-367-3837-8, 186 pages. c) P. Schneider, Advanced methods for prototype-based classification. Promotors: M. Biehl, N. Petkov, Faculty of Mathematics and Natural Sciences, University of Groningen, 2010, ISBN 978-90-367-4405-8, 95 pages. d) F. Tushabe, Extending Attribut Filters to Color Processing and Multi-Media Ap- plications. Promotor: N. Petkov, co-promotor: M.H.F. Wilkinson, Faculty of Mathematics and Natural Sciences, University of Groningen, 2010, ISBN 978- 90-367-4630-4, 126 pages. e) A. Witoelar, Statistical Physics of Learning Vector Quantization. Promotor: M. Biehl, N. Petkov, Faculty of Mathematics and Natural Sciences, University of Groningen, 2010, ISBN 978-90-367-4409-6, 114 pages.

3.14 Contribution of TUE-ET

Technische Universiteit Eindhoven, Faculty of Electrical Engineering, Design Methodology for Electronic SystemsProf.dr.ir. R.H.J.M.Otten, Dr.ir. T. Basten, Prof.dr. H. Corporaal, Dr.ir. M.C.W. Geilen, Prof.dr.ir. G. de Haan,

ASCI bi-annual Report 2009-2010 33

Dr.ir. J.P.M. Voeten

Research 2009-2010

Over decades the chair electronic systems has focused on iteration free design. This utopian goal has always provided up-to-date challenges and resulted in a wealth of contributions to the field of design automation: "how to specify a function to be implemented on a chip, feed it to an eda tool, and get, without further interaction, a design that meets all requirements concerning functionality, speed, size, power, yield and other costs". The chair decided to embark on a more general approach where the set of all possible performance characteristics and, in principle, all combinations of realization values that can be optimal under the usual cost functions, are considered. Apart from the generalization from pairwise to multiple trade-offs, the approach lends itself well to dynamic reconfiguration of modern systems, and multimedia systems in particular, and offers a flexible transition between pre-silicon and post-silicon decision-making, without imposing any compromise with respect to optimality. Such an approach reveals fundamental problems that are turned into the following focus areas and results.Composability. A composable real-time operating systems (RTOS) using a two-level arbitration scheme. Composable (and predictable) Aethereal Network on Chip using pipelined virtual circuits based on TDM arbitration. Composable (and predictable) memory controller supporting both SRAM and SDRAM and a wide range of arbiters. Composable power management that enables independent dynamic voltage and frequency scaling per application. Predictability. Cyclo-Static Dataflow (CSDF) based throughput and latency analysis techniques. CSDF buffer sizing techniques to explore trade-offs between buffer size and throughput. SDF-based resource allocation and scheduling for shared resources. Predictable RTOS for CSDF applications. Modeling and analysis techniques to predict performance trade-offs in highly dynamic and distributed sensor networks. Predictable automatic synthesis for real-time control-dominated systems from formal real-time concurrent specifications such that qualitative properties are preserved and all real-time properties are preserved up to a small timing deviation.Compositionality. Theoretic model of timed interfaces to support compositional step-wise refinement and abstraction of worst-case performance models. Compositional (per-connection) communication models in RTOS and Network on Chip.Scenario-aware design. New model-of-computation (MoC): Scenario-Aware Dataflow (SADF), combining the strengths of determinate dataflow analysis and limited scenario indeterminacy. Analysis techniques leveraging the theory of Max-Plus algebra. Automated design flow to map applications modeled with a SADF graph onto a multi-processor platform. Structured, automated approach to identify and detect scenarios in application source code. Low-power architectures and power-aware design. Technology-enabled design methodologies towards minimum-area maximum-performance designs in sub-/near-threshold operation. Concurrent adaptive control of voltage scaling and forward body biasing to minimize power consumption under fixed performance. Sub-threshold JPEG encoder (down-to 0.75 pJ/operation; in 65 nm CMOS) has been successfully realized and tested. Proposal for a scalable SIMD processor, consuming down-to 1 pJ/operation (the first fully programmable processor beating the 1 pJ/op boundary),partly prototyped in VHDL and CMOS 65 nm.Complete mapping flow. The open-source software tool SDF3 implementing all developed dataflow analysis and mapping techniques. A light-weight, open-source multi-processor platform, using SDF3, that can be used for educational purposes. An industrial relevant multi-processor platform, using SDF3, offering composability, predictability and low power.Video processing for multimedia systems. A versatile platform for high quality image enhancement, suitable for real-time implementation of noise reduction, coding artifact reduction, sharpness enhancement, contrast enhancement, and resolution up-conversion with very limited time for parameter optimization. A theoretical framework for describing the influence of picture registration, picture reconstruction and perception on picture quality, extensively applied for industrial lcd displays. A HW-platform for real-time 2Dto3D-conversion with low power dissipation (<20 Watt), small form factor (1 card in a 19 inch rack), excellent video quality (1080 lines progressive with 60 frames per second), reliable and robust (24/7).Smart camera. We have built an experimental set-up for heartbeat detection from camera signals in a gym-setting. Advanced face detection where we can discriminate between actual people and pictures or other artifacts. Experimental set-up with infrared cameras to monitor a sleeping person to replace accelerometers physically attached to the wrists and ankles of the sleeper with motion analysis of video. Machine learning for detection of malaria infection in blood smears, the detection of TB, and the detection of breast cancer.

Future Research Plans

There will be in the coming years, focus on the following fundamental topics. Composability refers to the methodology of separating applications on a shared platform, in such a way that independent development, verification, and execution is possible. Verifying (real-time) requirements becomes much easier, and a mix of real-time and non-real-time applications can run on the same shared hardware. Predictability in real-time computing is a necessity for guaranteeing performance properties and constraints. Timing is a performance characteristic that has been studied extensively in order to analyze resource requirements for providing guarantees, but others deserve attention. Compositionality is a scalable approach to predictability. It allows for the decomposition in smaller parts or views that can be re-assembled without re-analysis. Scenario-aware design makes use of the fact that, when run-time conditions are known or can be predicted, applications can often be mapped and executed more efficiently than when simply designed for the statically 'worst case'. Low-power architectures and power-aware design are both essential in staying within realistic power budgets in the future. This requires extremely computationally-efficient processing and the incorporation of adaptive techniques at all levels. A complete mapping flow will be, faithful to the mission stated above, maintained and tools will be incorporated, whether in-house

ASCI bi-annual Report 2009-2010 34

developed or shared with partners. Our contributions will be in the design trajectory from data-flow specifications to multi-processor systems realized/prototyped for field programmable gate arrays.

Key Publications 2009-2010

• Marc Geilen, Stavros Tripakis, Maarten Wiggers: The earlier the better: a theory of timed actor interfaces. HSCC 2011: 23-32• Sander Stuijk, Marc Geilen, Twan Basten: A Predictable Multiprocessor Design Flow for Streaming Applications with Dynamic Behaviour. DSD 2010: 548-555• Akash Kumar, Bart Mesman, Henk Corporaal, Yajun Ha: Iterative Probabilistic Performance Prediction for Multi-Application Multiprocessor Systems. IEEE Trans. on CAD of Integrated Circuits and Systems 29(4): 538-551 (2010)• Yu Pu, J. Pineda de Gyvez, H. Corporaal, Yahun Ya: “An ultra-low-energy multi-standard JPEG Coprocessor in 65 nm CMOS with sub/near threshold supply voltage”, IEEE Journal of Solid State Circuits, 2010, vol 45, n. 3, pp 668-680a.• Jeroen Voeten, Oana Florescu, Jinfeng Huang, Henk Corporaal: Error computation for predictable real-time software synthesis. Simulation 87(4): 334-350 (2011).

3.15 Contribution of TUE-WI

Eindhoven University of Technology, Department of Mathematics and Computer Science, Visualization GroupProf.dr.ir. J.J. van Wijk, Prof.dr.ir. R. van Liere, Dr.ir. H.M.M. van de Wetering, Dr. M.A. Westenberg, Dr. A.C. Jalba

Research 2009-2010

The aim of visualization is to develop methods and techniques using interactive computer graphics such that most insight in large data sets can be obtained. The TU/e Visualization group is active in the following areas:

Information Visualization. We study how large amounts of abstract data, such as tables, trees, networks and combinations of these, can be visualized. Our research has led a number of PhD theses, focusing on compound graphs (PhD Holten, 2009, awarded as best PhD thesis TU/e 2009), support for navigation (PhD Shrinivasan, 2010), perceptual issues (PhD Li, 2011), and vessel traffic (PhD Willems, 2011). Research included the development of new methods and techniques for presentation and interaction, and evaluation based on user experiments; with application domains such as software analysis, bio-informatics, and spatial movement analysis.

Scientific Visualization. Scientific visualization concerns data from simulations and measurements, defined over geometric spaces. Within this area we study tensor field analysis and visualization, mathematical visualization, and, in cooperation with the TU/e Department of Biomedical Engineering, medical visualization.

3D interaction. In cooperation with CWI we study how affordable desktop Virtual Reality systems (hard- and software) can be designed to simplify interaction with 3D data and objects for interrogation and navigation. This has led to new methods and techniques for interactive measurements of 3D objects (PhD Kruszynski, 2010), and on new insights on interactive 3D manipulation (PhD Liu, 2011).

Future research plans

In the next period we aim to continue and expand our research. Visualization is almost by definition an applied discipline, close cooperation with end-users is stimulating and rewarding. We currently work on visualization of bio-informatics, and aim to expand our cooperation with Wageningen, Nijmegen and increase our role in NBIC. Our spin-off companies MagnaView and SynerScope provide interesting use cases from practice, including visualization of multivariate data for professionals and fraud detection for banking. Our research on moving object visualization will continue in METIS, a follow-up project of POSEIDON, in cooperation with Thales, with a focus on uncertainty visualization. Furthermore, we aim to expand cooperation here with our colleagues from the algoritmics group. Other challenges come from for instance CBS (demographic data), Kempenhaeghe (medicine use), and ASML (log data).

Also, we aim to expand our work in Visual Analytics: The science of analytical reasoning, supported by interactive visual interfaces. Integration of other data analysis methodologies (statistics, data mining); heterogenous data; and consideration of the complete data analysis process, from collection to presentation, are key aspects. We have participated in the EU VisMaster project, which has set up a European research agenda for Visual Analytics; a proposal for a EU research project is in preparation. Furthermore, we aim to contribute to FuturICT, an EU flagship project in preparation. We aim at extension of cooperation with data-mining experts at TU/e. Also, we cooperate with dr. M. Worring, University of Amsterdam, on interactive multimedia analysis, and aim to increase this; we continue cooperation with prof. W. van der Aalst on interactive process mining.

Furthermore, our cooperation with the groups of prof. L. Florack (TU/e Mathematics) and dr. A. Vilanova (Department of Biomedical Engineering) on medical visualization will be continued.

ASCI bi-annual Report 2009-2010 35

Key Publications 2009-2010

Willems, N., H. van de Wetering, J.J. van Wijk. Visualization of Vessel Movements. Computer Graphics Forum, 28(3), 959-966, 2009.

L. Liu, R. van Liere, K. Nieuwenhuizen, Jean-Bernard Martens (2009). Comparing Aimed Movements in the Real World and in Virtual Reality. In Proceedings IEEE Virtual Reality Conference 2009 (VR 2009), 14-18 March 2009, Lafayette, Louisiana, USA, 219-222.

J.J. van Wijk (2009). Symmetric Tiling of Closed Surfaces: Visualization of Regular Maps. ACM Transactions on Graphics 28(3), Article 49, 12 pages. Proceedings ACM SIGGRAPH 2009, New Orleans.

R. Otten, A. Vilanova, H. van de Wetering (2010). Illustrative White Matter Fiber Bundles. Computer Graphics Forum, 29(3), 1013-1022.

M.A. Westenberg, J.B.T.M. Roerdink, O.P. Kuipers, S.A.F.T. van Hijum (2010). SpotXplore: a Cytoscape plugin for visual exploration of hotspot expression in gene regulatory networks. Bioinformatics 26(22), 2922-2923.

J. Li, J.-B. Martens, J.J. van Wijk (2010). A model of symbol size discrimination in scatterplots. Proceedings of the 28th International Conference on Human Factors in Computing Systems, ACM CHI 2010, Atlanta, Georgia, USA, April 10-15, 2553-2562.

D. Holten, J.J. van Wijk (2010). Evaluation of Cluster Identification Performance for Different PCP Variants. Computer Graphics Forum 29(3), 793-802.

Y.B. Shrinivasan, J.J. van Wijk (2009). Supporting Exploration Awareness in Information Visualization. IEEE Computer Graphics and Applications, 29(5), p. 24-33.

de Rooij, J.J. van Wijk, M. Worring (2010). MediaTable: Interactive Categorization of Multimedia Collections. IEEE Computer Graphics and Applications 30(5), 42-51.

L. Astola, A. Jalba, E. Balmashnova, L. Florack (2011). Finsler Streamline Tracking with Single Tensor Orientation Distribution Function for High Angular Resolution Diffusion Imaging. Journal of Mathematical Imaging and Vision 41(3), 170-181.

3.16 Contribution of TUE-BMT

ASCI Annual Report 2009-2010 – Biomedical Image Analysis – Eindhoven University of Technology

Staff: Prof.dr.ir. B.M. ter Haar Romeny (project leader), dr. A. Vilanova, dr.ir. H.C. van Assen, dr. ir. B.Platel, prof.dr. L.M.J. Florack, dr.ir. R. Duits, prof.dr.ir. M. Breeuwer (Philips Healthcare).

Background:The group Biomedical Image Analysis focuses on generic mathematical approaches to solve image analysis problems in cardiovascular and neurological applications, and advanced visualization. The group collaborates with the TU/e Magnetic Resonance Lab, Philips Healthcare, the FC Donders Institute, the University of Zürich, the Maastricht, Utrecht, Nijmegen and Leiden University Hospitals and the Epilepsy Center Kempenhaeghe. Education is an important aspect, being housed in the largest BME Dept. in the Netherlands, with 500 students. A full range of courses is given, from 1st – 5th year, and at PhD level (ASCI course a8).

Research:In collaboration with Philips Healthcare (Best) work on computer aided diagnosis focuses on dynamic contrast enhanced MRI of breast tumors, low-dose catheter tracking, and cardiovascular dynamics from MRI tagging data to study local dense ventricular optic flow and deformation for non-invasive ventricular infarct size estimation. The challenge is taken to segment the thin atrial wall, to optimize cardiac ablation procedures.Much effort has been given to the GPU-based visualization and analysis of tensor fields of DTI (Diffusion Tensor Imaging) data, and High Angular Resolution Diffusion Imaging (HARDI). The interactive visualization of 4D flow data of turbulent aorta blood flow is also GPU-based. The clinical branch at the Maastricht University Hospital focuses on precise navigation and targeting for Deep Brain Stimulation, in close collaboration with the neurosurgery dept.A multi-scale framework has been established over the years for doing high-order differential geometry on high-dimensional images, with applications as adaptive ‘geometry-driven’ edge preserving enhancement, multi-scale optic flow extraction, and deep structure analysis for content-based image retrieval. This visual perception-inspired framework is expanded to multi-orientation analysis in 2D and 3D, giving rise to powerful contextual operators.

ASCI bi-annual Report 2009-2010 36

The mathematical analysis of (higher order) tensor fields, either for DTI, HARDI or strain tensor fields in cardiac deformation, includes methods from Finsler geometry, numerical methods for geodesic ray tracing, and a new Lie-group based theory on 3D orientation ‘scores’. A new start-up company (InViso) is etablished to implement these methods in massively parallel FPGA-based hardware. This project is rewarded a 200 K€ STW valorization grant.To inspect and interact with these complex tensor fields a sophisticated and flexible GPU-based visualization tool is developed, ‘DTItool’, which enables the interactive manipulation of all relevant parameters, 3D orientation glyphs, and tractography. The tool is used in many collaborating labs.

The current and near-future focus of research is on: cardiovascular applications, primarily with X-Ray and MRI, exploiting multi-valued images for local strain analysis of

ventricular deformation from tagged MRI sequences, and 4D flow visualization; neuro applications, primarily focusing on DTI and HARDI exploited in tractography for brain connectivity and muscle

fibre orientation analysis. GPU-based visualization of multi-valued information. The hardly investigated visualization of uncertainties is an

important research topic. Applications include the surgical preparation for safe epilepsy surgery with fiber visualization to spare the optic radiation, and brain tractography in relation to other brain imaging modalities (fMRI, EEG). The many parameters of cardiac function are integrated into a clinically more effective comprehensive visualization.

generic and advanced mathematical methods for multi-valued image analysis, for segmentation, enhancement, morphological and contextual operations, and tractography of crossing and splitting fibers..

In 2010 the research collaboration IST/e (Image Science & Technology / Eindhoven) has been established as a TU/e High Potential program, with a grant of 1 M€ of the TU/e board, with partners the Depts. of Mathematics & Computing Science (Florack, van Wijk) and Biomedical Engineering (ter Haar Romeny, Nicolay).

Publications 2009-2010 (selection):2010 Duits, R. & Franken, E.M. (2010). Left-invariant parabolic evolutions on SE(2) and contour enhancement via invertible

orientation scores. Part I: Linear left-invariant diffusion equations on SE(2). Quarterly of Applied Mathematics, 68(2), 255-292. Part II: Non-linear left-invariant diffusions on invertible orientation scores. Quarterly of Applied Mathematics, 68(2), 293-331.

Florack, L.M.J., Balmachnov - Sizykh, E.G., Astola, L.J. & Brunenberg, E.J.L. (2010). A new tensorial framework for single-shell high angular resolution diffusion imaging. Journal of Mathematical Imaging and Vision, 38(3), 171-181.

Pelt, R.F.P. van, Olivan Bescos, J., Breeuwer, M., Clough, R.E., Gröller, E., Haar Romenij, B.M. ter & Vilanova, A. (2010). Exploration of 4D MRI blood-flow using stylistic visualization. IEEE Transactions on Visualization and Computer Graphics, 16(6), 1339-1347.

Heisen, M., Fan, X., Buurman, J., Riel, N.A.W. van, Karczmar, G.S. & Haar Romenij, B.M. ter (2010). The influence of temporal resolution in determining pharmacokinetic parameters from DCE-MRI data. Magnetic Resonance in Medicine, 63(3), 811-816.

Neubauer, A.M., Garcia, J.A., Messenger, J.C., Hansis, E., Kim, M.S., Klein, A.J.P., Schoonenberg, G.A.F., Grass, M. & Carroll, J.D. (2010). Clinical feasibility of a fully automated 3D reconstruction of rotational coronary X-ray angiograms. Circulation: Cardiovascular Interventions, 3(2), 71-79.

2009 Brecheisen, R., Vilanova, A., Platel, B. & Haar Romenij, B.M. ter (2009). Parameter sensitivity visualization for DTI fiber

tracking. IEEE Transactions on Visualization and Computer Graphics, 15(6), 1441-1448. Bouma, H., Sonnemans, J.J., Vilanova, A. & Gerritsen, F.A. (2009). Automatic detection of pulmonary embolism in CTA.

IEEE Transactions on Medical Imaging, 28(8), 1223-1230. Franken, E.M. & Duits, R. (2009). Crossing-preserving coherence-enhancing diffusion on invertible orientation scores.

International Journal of Computer Vision, 85(3), 253-278. Peeters, T.H.J.M., Vilanova, A. & Haar Romenij, B.M. ter (2009). Interactive fiber structure visualization of the heart.

Computer Graphics Forum, 28(8), 2140-2150. Ruijters, D., Haar Romenij, B.M. ter & Suetens, P. (2009). Vesselness-based 2D-3D registration of the coronary

arteries. International Journal of Computer Assisted Radiology and Surgery, 4(4), 391-397. Schoonenberg, G.A.F., Florent, R., Lelong, P., Wink, O., Ruijters, D., Carroll, J.D. & Haar Romenij, B.M. ter (2009).

Projection-based motion compensation and reconstruction of coronary segments and cardiac implantable devices using rotational X-ray angiography. Medical Image Analysis, 13(5), 785-792.

3.17 Contribution of UL-LUMC

Leiden University Medical Center, division of Image Processing, laboratorium voor klinische en Experimentele BeeldverwerkingProf.dr.ir. Johan H.C. Reiber, Dr.ir. B.P.F. Lelieveldt, Prof.dr. R. Nelissen

ASCI bi-annual Report 2009-2010 37

ASCI enquete 2009-2010

Question 1: Name of the groupDivision of Image Processing (Dutch abbreviation LKEB)Dept of Radiology,Leiden University Medical Center,P.O. Box 96002300 RC Leidenthe Netherlandswww.lkeb.nl

Head in 2009-2010: Prof fr ir J.H.C. Reiber

Question 2: ASCI research themeProcessing, Interpretation and Visualization of Medical Images (Theme C)

Question 3: Short description of research

Contribution of LUMC-lkebThe main goal of the Division of Image Processing is the research, implementation and validation of image processing approaches, which allow the objective and reproducible assessment of objects in medical images. LKEB activities belong to one of the seven main research fields of the LUMC under the headings “Vascular Medicine”, “Neuro-science” and “Biomedical Imaging”. Part of the research involves computer vision research and algorithm development, whereas clinical applications also play an important role. Applications focus on Neuro-imaging, Pulmonology, Orthopaedics, Cardiology and molecular and cellular imaging. In 2009-2010, important research directions were:

Statistical shape modeling fitting to sparse image dataStatistical shape models are widely used to integrate a-priori knowledge about shape and image appearance into segmentation algorithms. Research at LKEB is directed towards fitting statistical shape models to sparse image observations. A 3D Active Shape Model has been developed, along with a number of techniques to fit these models to bi-plane X-ray data, and radially scanned echocardiographic imaging; Apart from segmentation, we have developed statistical shape models for computer-aided diagnosis to detect cardiac shape- and motion abnormalities in MR images for patients with a cardiac infarction, and for quantifying local shape changes caused by brain diseases.

Molecular image integrationIn this project, we are investigating novel algorithms to combine complementary information in molecular, structural and functional imaging. We address novel image processing challenges brought on by new molecular imaging modalities such as bioluminenscence imaging and fluorescence imaging. We focus on whole body registration between optical and structural data in follow-up studies, detection of changes and abnormalities in follow-up studies and on integrating information sources over the scale range from molecule to organism.

Elastic image registrationWe are also developing core technologies for non-rigid image registration. In collaboration with UMCU and Erasmus MC, we are maintaining and extending the well-known registration software elastiX, where we focus on parallel optimization algorithms to achieve near-real-time registration performance. The elastiX software has been downloaded > 6000 times, and is regarded as a worldwide reference for image registration

Clinical Image Analysis ApplicationsMuch of the research at LKEB is driven by questions from clinical partners. To this end, we are developing algorithms and software for:

Detection and quantification of pulmonary emphysema in CT Images Early detection of micro motion of prosthetic implants in bi-plane X-ray images Automatic analysis of coronary vessels in CT and intravascular ultrasound images Automatic analysis of coronary and left-ventricular angiograms Automatic analysis of cardiac function in MR and CT patient studies Automatic analysis of changes in brain structure with ageing and disease Automatic analysis of vascular MR data Automatic white matter lesion detection in MR images of the brain

ASCI bi-annual Report 2009-2010 38

Question 6: key publications

Peer-reviewed journal papers M. Baiker, J. Milles, J. Dijkstra, T. Henning, A.W. Weber, I. Que, E.L. Kaijzel, C.W.G.M. Lowik, J.H.C. Reiber,

B.P.F. Lelieveldt, "Atlas-based whole-body segmentation of mice from low-contrast µCT data", Medical Image Analysis, vol. 14(6), pp 723-737, 2010

M.Ma, M. van Stralen, J.H.C. Reiber, J.G. Bosch, B.P.F. Lelieveldt, "Model Driven Quantification of Left Ventricular Function from Sparse Single-beat 3D Echocardiography", Medical Image Analysis, vol 14(4), pp 582-593, 2010.

P.Kok, M. Baiker, E.A. Hendriks, F.H. Post, J. Dijkstra, C.W.G.M. Löwik, B.P.F. Lelieveldt, C.P. Botha, "Articulated Planar Reformatting for change visualization in small animal imaging", IEEE Transactions on Visualization and Computer Graphics, vol. 16(6), 1396-1404, 2010

Suinesiaputra, A.F. Frangi, A.M. Kaandorp, H.J. Lamb, J.J. Bax, J.H.C. Reiber, B.P.F. Lelieveldt, “Automatic Detection of Regional Wall Motion Abnormalities Based on a Statistical Model Applied to Multi-Slice Short-Axis Cardiac MR Images”, IEEE Transactions on Medical Imaging, vol. 28(4), pp 595-607, 2009.

S Klein, M Staring, K Murphy, MA Viergever, J Pluim , Elastix: a toolbox for intensity-based medical image registration, IEEE Transactions on Medical Imaging, 29 (1), 196-205 , 2010

Bovenkamp EG, Dijkstra J, Bosch JG, Reiber JH. “User-agent cooperation in multiagent IVUS image segmentation”. IEEE Trans Med Imaging. 2009 Jan;28(1):94-105.

L. Ferrarini, I.M. Veer, E. Baerends, M.J. van Tol, L.R. Demanescu, N.J.A. van der Wee, D. Veltman, A. Aleman, M.A. van Buchem, J.H.C. Reiber, S.A.R.B. Rombouts and J. Milles, “Hierarchical functional modularity in resting-state human brain”, Human Brain Mapping, 30(7):2220-2231, 2009.

Questions for five year research planQuestion 10: description of 5 year research planThe next five years the research at LKEB will continue to focus on development of generic methodologies for quantification, visualization and analysis techniques for biomedical images. Concrete clinical applications will involve:

Pulmonology: analysis of pulmonary disease from follow up CT scans, Orthopaedics: pre-operative surgery simulation and post-operative evaluation of prosthesis fixation Vascular image analysis: quantification and risk stratification of vascular disease from several imaging modalities

Cardiac image analysis: quantification and information fusion to support cardiac diagnostics from large, heterogeneous image data sets.

Neurological image analysis: development of shape comparison methods to detect pathologies compared to an atlas of normal subjects, and development of algorithms to monitor the progression of white matter lesions and ageing effects.

Molecular image analysis: fusion of and tracking in complementary imaging data from the whole-body scale to the molecular scale, with an emphasis on translational biomedical research.

3.18 Contribution of UT-EWI-DACS

1. DACS MissionDACS focuses on the design and analysis of dependable networked systems. A system is called dependable, whenever reliance can justifiably be placed on the services it delivers. Tailored to communication systems, which can be wired, wireless, or embedded in other systems, this means that we aim:

“to contribute to the design and implementation of dependable networked systems, as well as to methods and techniques to support the design and dimensioning of such systems, such that they are dependable, in all phases of their lifecycle.”

We thereby interpret the term dependability as encompassing availability, reliability, performance (quality of service) and security.

2. DACS StrategyThree phases in system design. We distinguish three phases in the design of dependable networked systems. In the first (exploratory) phase the emphasis lies on the development of system models and the model-based evaluation of key system characteristics. In the second, more concrete phase the emphasis lies on designing and/or standardizing systems architectures, protocols, and algorithms, as well as on implementing prototype systems. In the third, operational phase the systems have been implemented and the emphasis lies on operationally managing them. As an example of these three phases, consider the Internet backbone. During 1965-1985, research concentrated on a variety of networking models

ASCI bi-annual Report 2009-2010 39

(circuit-switching vs. packet-switching, various local area access mechanisms). In the period 1980-1995, research focus shifted toward designing, implementing and standardizing protocols (TCP/IP, Ethernet). Since the beginning of the 1990s, an ever more important challenge is to keep the Internet up and running (dependability), and solve problems related to (performance) management, security and scalability. A similar transition in focus can be seen for wireless communication system. The figure below illustrates, through a horizontal structuring, the above three phases in system design.

Three technologies for dependable networked systems. Research within DACS covers the whole spectrum of network technologies: from well-established technologies (like the wired Internet), via technologies that are under development (such as wireless networks) to emerging technologies (like embedded network systems).In the case of well-established technologies, research concentrates on operational aspects, here, in particular, of the wired Internet. Specific topics include bandwidth allocation, accounting, self-management of lambda switches and protection against scans, denial-of-service attacks and phishing. Taking and interpreting measurements plays an important role in this research.For technologies under development, research focuses on the design, evaluation, and prototype implementation of new protocols and algorithms for wireless and ad-hoc networks. Topics include algorithms for context- and power-aware routing in ad-hoc networks, and, lately, more and more on car-to-car communications and wireless sensor networks. These types of networking systems are exponents of embedded networking technologies as well.The remaining research on embedded networking technologies focuses on system specification and evaluation techniques to describe such systems, and the resource constraints (performance, dependability, energy usage) they have to operate under. This includes the development of new stochastic model checking techniques and the application thereof to predict dependability and performance properties.The figure above also shows these three technology domains (where a clear cut separation can, of course, never be made). When technologies mature over time, a shift in phase and type of activity is foreseen. Inherent to university research, DACS moves on the increasing side of the wave (to the right). Note that the presented structure is based on technical content, not an organizational sub-structuring of the group. There is strong collaboration between the various group members, and projects generally cover multiple issues.

3. Key journal publications for 2009-2010van Brandenburg, R. and van Deventer, M.O. and Karagiannis, G. and Schenk, M.

Towards personalized TV for concurrent use; challenges and opportunities for IMS-based IPTV In: Journal of Internet Engineering December 2010

van Wanrooij, W. and Pras, A. Filtering spam from bad neighborhoods In: International Journal of Network Management 15 November 2010

Klink, D. and Remke, A.K.I. and Haverkort, B.R.H.M. and Katoen, J.P. Time-bounded reachability in tree-structured QBDs by abstraction In: Performance evaluation 31 May 2010

ASCI bi-annual Report 2009-2010 40

Jongerden, M.R. and Mereacre, A. and Bohnenkamp, H.C. and Haverkort, B.R.H.M. and Katoen, J.P. Computing Optimal Schedules of Battery Usage in Embedded Systems In: IEEE Transactions on Industrial Informatics 2010

Menth, M. and Lehrieder, F. and Briscoe, B. and Eardley, P. and Moncaster, T. and Babiarz, J. and Charny, A. and Zhang, Sinyang and Taylor, T. and Chan, Kwok-Ho and Satoh, Daisuke and Geib, R. and Karagiannis, G. A Survey of PCN-Based Admission Control and Flow Termination In: IEEE Communications Surveys & Tutorials 2010

Sperotto, A. and Schaffrath, G. and Sadre, R. and Morariu, C. and Pras, A. and Stiller, B. An Overview of IP Flow-Based Intrusion Detection In: IEEE Communications Surveys & Tutorials 2010

Andrey, L. and Festor, O. and Lahmadi, A. and Pras, A. and Schoenwaelder, J. Survey of SNMP performance analysis studies In: International Journal of Network Management November 2009

Liu, Fei and Heijenk, G.J.

3.19 Contribution of UT-EWI-CAES

University of Twente, Faculty of Electrical Engineering, Mathematics an Computer Science, Computer Architecture, Design & Test for Embedded SystemsProf.dr.ir. G.J.M. Smit, Dr.ir. A.B.J. Kokkeler, Ir. E. Molenkamp

Program Design

The main emphasis of the group is on efficient architectures for dependable networked embedded systems. Within this theme the chair performs research on three related key areas:1) Efficient architectures for streaming applications,2) Architectures for efficient energy management,3) Dependability issues of embedded systems.Energy-efficiency and dependability are the main drivers of our research. Energy-efficiency is important for streaming applications found in battery powered mobile devices, and for high-performance systems. The effort on energy efficient architectures focuses on reconfigurable processors for streaming applications, e.g. found in battery powered mobile devices (e.g. portable multimedia players) or sensor networks. However, within high performance embedded computing (e.g. medical imaging, radar processing), extrapolating the current trend of using general purpose processors for future systems predicts excessive power consumption. MPSoC (Multi Processor Systems-on-Chip) devices for streaming applications are prime candidates for use in this application domain as well. Dependability plays an important role in sensor networks (nodes may fail or run out of energy unexpectedly). In MPSoC (Multi Processor Systems-on-Chip) systems for streaming applications dependability techniques also play an important role.

Results in 2009

Efficient Architectures for Streaming Applications (Dr. A.B.J. Kokkeler, Dr. J. Kuper, Prof. G.J.M. Smit, Prof. M. Bekooij)Research on techniques for analysis and synthesis of predictable multi-processor systems continued in cooperation with NXP. This resulted e.g. in new techniques for modeling run-time arbitration and fast calculation of buffer capacities for multi-core architectures that was published in ACM TECS (2010 issue). The predicatable NoC developed by Wolkotte was published in IEEE Transactions on VLSI. The results of run-time spatial mapping (Hölzenspies and ter Braak) resulted in a publication in the Journal of Parallel Programming and a paper on DATE2010.This theme resulted in four Ph.D. theses in 2009 of Maarten Wiggers “Aperiodic Multiprocessor Scheduling for Real-Time Stream Processing Applications” , Pascal Wolkotte “Exploration within the Network-on-Chip Paradigm”, Qiwei Zhang “Cognitive Radio on a Reconfigurable MPSoC Platform”, and Mohammed Khatib “MEMS-Based Storage Devices - Integration in Energy-Constrained Mobile Systems”.

Architectures for efficient energy management (Prof. G.J.M. Smit)Besides the development of efficient architectures, the use of these architectures to increase energy efficiency in a more general sense, is a subject of research. Distributed micro-generation techniques (micro-CHP, solar cells, micro windmills) play an important role in this research. Because of the reduced peak load a power plant can generate energy more efficiently. The research topics resulted in a number of conference papers and a journal publication is in preparation. Smit was one of the initiators of the STW/NWO/ICTRegie programme SES (Smart Energy Systems), a programme with a budget of 6 M Euro.

Efficient dependable networked embedded systems (Dr. H.G. Kerkhoff)

ASCI bi-annual Report 2009-2010 41

The research in dependable systems, including mixed-signal IPs and sensors, is centred on the design and implementation of architectures, IPs and circuits for dependable networked embedded systems. The key research challenge is to foster dependability (i.e. availability, reliability, integrity and maintainability) as embedded systems are business or safety critical in many applications. The group made the dependability managerfor the RFD chip in the FP7 the CRISP project.This theme resulted in a number of conference publications and a publication in the Microelectronics journal.

Results in 2010

Efficient Architectures for Streaming Applications (Dr. A.B.J. Kokkeler, Dr. J. Kuper, Prof. G.J.M. Smit, Prof. M. Bekooij)Research on techniques for analysis and synthesis of predictable multi-processor systems continued in cooperation with NXP. This resulted e.g. in new techniques for modeling run-time arbitration and fast calculation of buffer capacities for multi-core architectures that was published in ACM TECS. The results of run-time spatial mapping (Hölzenspies and ter Braak) resulted in a publication in the Journal of Parallel Programming and a paper on DATE2010. Smit and ter Braak gave a tutorial at the SoC conference in Tampere. Wiggers and Bekooij gave a tutorial at the Embedded Systems week. One of the post-docs Maarten Wiggers has set up a cooperation with UC Berkeley with the group of prof. Edward Lee. This has resulted in two papers in 2010 / 2011. In 2010 the group also contributed to three “kenniswerkers” projects: AIPRO together with Océ, and RFTT together with Nedap. Projects that started in 2010 are STARS (BSIK FES2008 on reconfigurable sensor systems), STW project SeaSTAR (on underwater communication), Pieken in de Delta project DTFC (on test platforms), EFRO project TSR (reading RFID tags from a long distance), FP7 projects So(o)S and ADVANCE (on programming multi-core architectures) This theme resulted in three Ph.D. theses in 2010 of Philip Hölzenspies “On run-time exploitation of concurrency”, Marcel van de Burgwal “Interfacing Networks-on-Chip: Hardware meeting Software” and Leon Evers “Concise and flexible programming of wireless sensor networks”.

Architectures for efficient energy management (Prof. G.J.M. Smit)Besides the development of efficient architectures, the use of these architectures to increase energy efficiency in a more general sense is a subject of research. Distributed micro-generation techniques (micro-CHP, solar cells, micro windmills) play an important role in this research. Because of the reduced peak load a power plant can generate energy more efficiently. The research topics resulted in a number of conference papers and a journal publication is IEEE Transactions on Smart Grid. Smit was one of the initiators of the STW/NWO/ICTRegie programme SES (Smart Energy Systems). The group contributed to four project proposals for SES; all four proposals were accepted.

Efficient dependable networked embedded systems (Dr. H.G. Kerkhoff)The research in dependable systems, including mixed-signal IPs and sensors, is centred on the design and implementation of architectures, IPs and circuits for dependable networked embedded systems. The key research challenge is to foster dependability (i.e. availability, reliability, integrity and maintainability) as embedded systems are business or safety critical in many applications. There was cooperation with NXP and Recore systems in the Catrene TOETS project with 3 PhD students, all related to dependability in automotive applications. The group designed the dependability manager for the RFD chip in the FP7 the CRISP project. In November 2010 we received samples of the RFD chip and the dependability manager is working as expected. In the “Kenniswerkingsregeling”, there was the cooperation of postdoc Dr. Kerzerho with NXP automotive in Nijmegen resulting in a redesign of an existing product to improve the resilience of a CAN transceiver. In the Strategic Research Orientation DSN (Dependability Sensor Networks) with shared postdoc Dr. Senouci, a wireless hardware platform is being developed for real-time dependability tests. In the cluster project EMPATIE, new algorithms were developed for NXP Eindhoven for evaluating embedded data converters. In total 10 IEEE contributions in dependability were accepted for European and international conferences.

Five key publications

[1] Banerjee, A. and Wolkotte, P.T. and Mullins, R.D. and Moore, S.W. and Smit, G.J.M., “An Energy and Performance Exploration of Network-on-Chip Architectures” IEEE Transactions on Very Large Scale Integration (VLSI) Systems, 17 (3), 2009, pp. 319-329. ISSN 1063-8210[2] van de Burgwal, M.D. and Wolkotte, P.T. and Smit, G.J.M. “Non-power-of-Two FFTs: Exploring the Flexibility of the Montium TP” International Journal on Reconfigurable Computing, 2009. 678045. ISSN 1687-7195[3] Hölzenspies, P.K.F. and ter Braak, T.D. and Kuper, J. and Smit, G.J.M. and Hurink, J.L. “Run-time Spatial Mapping of Streaming Applications to Heterogeneous Multi-Processor Systems” International Journal of Parallel Programming, 38 (1), 2009, pp. 68-83. ISSN 1573-7640[4] Oude Alink, M.S. and Kokkeler, A.B.J. and Klumperink, E.A.M. and Rovers, K.C. and Smit, G.J.M. and Nauta, B. “Spurious-Free Dynamic Range of a Uniform Quantizer” IEEE transactions on circuits and systems II: Express Briefs, Volume 56 (6), 2009, pp. 434-438. ISSN 1549-7747[5] Kerkhoff, H.G. and Zhang, Xiao “Fault co-simulation for test evaluation of heterogeneous integrated biological systems” Microelectronics journal, Volume 40, Issue 7, 2009, ISSN 0026-2692[6] Wiggers, M.H. and Bekooij, M.J.G. and Smit, G.J.M. “Buffer Capacity Computation for Throughput-Constrained Modal Task Graphs”, ACM Transactions on Embedded Computing Systems, 10 (2). Article 17. ISSN 1539-9087[7] Molderink, A. and Bakker, V. and Bosman, M.G.C. and Hurink, J.L. and Smit, G.J.M. “Management and Control of Domestic Smart Grid Technology”, IEEE transaction on Smart Grid, 1 (2). pp. 109-119. ISSN 1949-3053

ASCI bi-annual Report 2009-2010 42

[8] ter Braak, T.D. and Hölzenspies, P.K.F. and Kuper, J. and Hurink, J.L. and Smit, G.J.M. “Run-time Spatial Resource Management for Real-Time Applications on Heterogeneous MPSoCs”, In: Proceedings of the Conference on Design, Automation and Test in Europe (DATE 2010), 8-12 Mar 2010, Dresden. pp. 357-362. ISBN 978-3-9810801-6-2 [9] Oude Alink, M.S. and Klumperink, E.A.M. and Soer, M.C.M. and Kokkeler, A.B.J. and Nauta, B., “A 50MHz-to-1.5GHz Cross-Correlation CMOS Spectrum Analyzer for Cognitive Radio with 89dB SFDR in 1MHz RBW”, In: IEEE Symposium on New Frontiers in Dynamic Spectrum Access Networks (DySPAN 2010), 6-9-April-2010, Signapore. IEEE Press. ISBN 978-1-4244-5188-[10] Kerkhoff, H.G. and Zhang, Xiao “Design of an Infrastructural IP Dependability Manager for a Dependable Reconfigurable Many-Core Processor”, In: Fifth IEEE International Symposium on Electronic Design, Test & Applications, DELTA 2010, 13-15 Jan 2010, Ho Chi Minh City, Vietnam. pp. 270-275. IEEE Computer Society Press. ISBN 978-0-7695-3978-2

3.20 Contribution of EUR-UMCR

The Biomedical Imaging Group Rotterdam develops and validates advanced image analysis algorithms for a number of applications. The research is organized in 6 research lines, cardiovascular image analysis, neuro image analysis, image analysis in oncology, model-based image analysis, cellular and molecular image analysis, and image guided interventions.

Within the cardiovascular image analysis research line, the focus has been on (i) development and evaluation of plaque detection and characterization in carotid arteries from CT and MRI data, to support improved diagnosis and prognosis in stroke (ii) development of methods for functional and anatomical assessment of the heart chambers and coronaries from CT, and (iii) development of motion estimation techniques from 4D imaging data, a.o. to study morphodynamics of cerebral aneurysms, distensibility of carotid arteries, and coronary motion. Within the neuro group, the focus has been on the development and application of new software tools to extract imaging biomarkers from MR brain data. Although volumetric features are still a point of attention, we extended our work towards the measurement of shape features and connectivity measures in the brain. Also the processing of large cohorts has been improved and more standardized, improving our understanding of neurodegenerative disease, and providing reference data for clinical use. In the onco image analysis research line focus has been on the development and validation of MRI based non-invasive biomarkers for (i) the characterization of tumor tissue, and (ii) the evaluation of cancer treatment response and outcome. In the model based medical image analysis research line several techniques for the quantitative analysis of a variety of medical images, have been developed with a focus on statistical learning and modeling. Applications include (i) Segmentation and shape analysis of brain structures from MR images, (ii) Characterization of blood vessel plaque components in MR and CT images using registration and classification, (iii) Shape and motion modeling including landmark uncertainty estimates, and (iv) Quantification of pulmonary disease by lung density, texture, and airway measuresWithin the cellular and molecular image analysis group the focus has been on the development and evaluation of improved methods for (i) particle detection and tracking in fluorescence microscopy, (ii) model-based cell segmentation, tracking, and lineage reconstruction for the study of embryogenesis, and (iii) super-resolution image reconstruction with particular application to molecular magnetic resonance imaging. Within the image guidance in interventions research line, the focus has been on (i) modeling cardiac and coronary shape and motion from 4D CTA, and (ii) using these dynamic models for registering cardiac CTA to coronary X-ray angiography images

In the next reporting period, within the cardiovascular image analysis research main focus areas will be on the detection and quantification of progression of atherosclerotic disease, and prognostic value of cardiovascular imaging biomarkers, both in the context of large scale longitudinal imaging studies. Within the neuro image analysis line we will focus on further improving and extending the list of biomarkers of neurodegenerative disease, but also on the processing of larger and more heterogeneous patient cohorts. In oncology we will focus on multi-modal image analysis for treatment planning, treatment monitoring and outcome prediction to facilitate individualized patient care. In model-based image analysis the work on computer-aided diagnosis of neurodegenerative, cardiovascular, and pulmonary disease will be extended and there will be a special focus on the investigation of transfer learning techniques to improve analysis of image data acquired with different scanning protocols. In cellular and molecular image analysis we will focus on the development of methods for single molecule analysis, large-scale object tracking, neuronal reconstruction, and learning-based image segmentation. Within the image guidance in interventions research we will continue and extend the current cardiac work, and additionally start research in image guidance in radiology, developing methods to improve guidance for o.a. US guided RFA and X-ray guided RFA and TACE. Furthermore, in 2011, a new research line on image registration has been initiated. In the coming period, this research line will focus mainly on registration of carotid vessel wall imaging data (to combine information from multiple modalities) and on registration of MR brain scans (to facilitate atlas building and automated disease classification).

Main Publications:

1) M. Schaap, C.T. Metz, T. van Walsum, A.G. van der Giessen, A.C. Weustink, N.R.A. Mollet, C. Bauer, H. Bogunović, C. Castro, X. Deng, E. Dikici, T. O'Donnell, M. Frenay, O. Friman, M. Hernández Hoyos, P.H. Kitslaar, K. Krissian, C. Kühnel, M. A. Luengo-Oroz, M. Orkisz, Ö Smedby, M. Styner, A. Szymczak, H. Tek, C. Wang, S. K. Warfield, S. Zambal, Y. Zhang,

ASCI bi-annual Report 2009-2010 43

G. P. Krestin and W.J. Niessen. Standardized Evaluation Methodology and Reference Database for Evaluating Coronary Artery Centerline Extraction Algorithms, Medical Image Analysis, 13(5):701-714, 2009.

2) T. den Heijer, F. van der Lijn, P. J. Koudstaal, A. Hofman, A. van der Lugt,G. P. Krestin, W.J. Niessen and M.M.B. Breteler, A 10-year follow-up ofhippocampal volume on magnetic resonance imaging in early dementia andcognitive decline, Brain, 2010

3) R. de Boer, H.A. Vrooman, M. A. Ikram, M. W. Vernooij, M.M.B. Breteler, A.van der Lugt and W.J. Niessen, Accuracy and reproducibility study ofautomatic MRI brain tissue segmentation methods, NeuroImage, 2010

4) C.T. Metz, S. Klein, M. Schaap, T. van Walsum and W.J. Niessen. Nonrigid registration of dynamic medical imaging data using nD+t B-splines and a groupwise optimization approach, Medical Image Analysis, 2011

5) E. Meijering, O. Dzyubachyk, I. Smal, W. A. van Cappellen. Tracking in Cell and Developmental Biology. Seminars in Cell and Developmental Biology, vol. 20, no. 8, October 2009, pp. 894-902.

6) O. Dzyubachyk, J. Essers, W. A. van Cappellen, C. Baldeyron, A. Inagaki, W. J. Niessen, E. Meijering. Automated Analysis of Time-Lapse Fluorescence Microscopy Images: From Live Cell Images to Intracellular Foci. Bioinformatics, vol. 26, no. 19, October 2010, pp. 2424-2430.

7) L. Alic, M Van Vliet, C. F. van Dijke, A. M. M. Eggermont, J.F. Veenland and W.J. Niessen, Heterogeneity in DCE-MRI parametric maps: a biomarker for treatment response?, Physics in Medicine and Biology, 2011 Mar 21;56(6):1601-16

8) L. Alic, J. Haeck, K. Bol, S. Klein, S. T. van Tiel, P. A. Wielopolski, M de Jong, W.J. Niessen, M Bernsen and J.F. Veenland, Facilitating tumor functional assessment by spatially relating 3D tumor histology and in vivo MRI: Image registration approach, PLoS ONE, PLoS ONE 6(8) :e22835

9) L. Sørensen, S. Shaker, and M. de Bruijne. Quantitative analysis of pulmonary emphysema using local binary patterns. IEEE Transactions on Medical Imaging, 29(2):559-569, 2010.

10) H.C. Achterberg, F. van der Lijn, T. den Heijer, A. van der Lugt, M.M.B. Breteler, W.J. Niessen, and M. de Bruijne. Prediction of dementia by hippocampal shape analysis. In Fei Wang, Pingkun Yan, Kenji Suzuki, and Dinggang Shen, editors, Machine Learning in Medical Imaging, volume 6357 of Lecture Notes in Computer Science, pages 42-49. Springer, 2010.

3.21 Contribution of RUN-UMCR Diagnostic Image Analysis Group (DIAG) NijmegenRadiology Department,Radboud University Nijmegen Medical Centre

Prof Dr N Karssemeijer, Prof Dr B van Ginneken, Dr R Manniesing, Dr C Sanchez, Dr HJH Huisman

Research 2009-2010

The Diagnostic Image Analysis Group aims at developing computer algorithms to aid clinicians in the interpretation of medical images and thereby improve the diagnostic process. Driven by clinically relevant application domains, innovative methods are investigated to address challenging problems in image analysis, computer-aided detection, and computer-aided diagnosis. While in previous years the group had a strong focus on breast and prostate imaging applications, research has expanded to new areas as chest imaging and retinal imaging. An overview per theme is presented below.

Retinal imaging has rapidly grown within ophthalmology in the past twenty years. The availability of cheap cameras to take direct images of the retina, fundus photography, makes it possible to examine the eye for the presence of many different eye diseases with a simple, non-invasive method. Within DIAG, research has focused on automatic early detection of diabetic retinopathy from fundus photographs. We are also applying the computer-aided detection and quantification techniques we have developed to diagnosis and quantification of macular degeneration.

Breast imaging remains an active field of research of DIAG. Research focuses on developing automated image analysis methods aimed at helping radiologists with detecting and diagnosing breast cancer. Analysis of mammograms is the most

ASCI bi-annual Report 2009-2010 44

important topic, as x-ray mammography is the technology by which the vast majority of cancers are detected. Computer aided reading of mammograms is currently the largest application of CAD in practice. However, it has to be improved to increase effectiveness. Research has been performed to improve algorithms, in particular those involving combination of views and exams and exploitation of context. Apart from that, we are also trying to understand how radiologists can best make use of CAD techniques. We performed observer studies to investigate how radiologist are influenced by CAD in their decisions. Optimizing human performance with CAD appears to be far from trivial and requires better understanding of cognitive processes leading to diagnostic decisions. In addition to projects in mammography, projects started in which multi-modal breast imaging applications are developed. These involve combinations of MRI, whole breast ultrasound, tomosynthesis, and mammography.

Lung imaging is a new area for DIAG. It has a large potential for development of clinically relevant applications as lung diseases are among the most important causes of death. In 2020, it is expected that chronic obstructive pulmonary disease (COPD) is the third cause of death, respiratory infections will be the fourth cause of death worldwide, followed by lung cancer, the most deadly cancer. Tuberculosis will rank seven on this list. There are many other diseases of the lung, such as interstitial lung disease, occupational lung disease and cystic fibrosis. For all these diseases, imaging is vital to learn more about their nature, to detect them early and to follow up on their course and treatment. Research in DIAG focuses both on the analysis of chest x-ray and chest CT. For x-ray we are developing tools to detect tuberculosis and lung cancer, and work on algorithms to remove normal anatomical structures from the radiographs and to detect changes between baseline and follow-up exams. For CT we are working on computer-aided detection of nodules, which can be small lung tumors, and we have developed a toolkit for analysis of the lungs, the lobes, airways and vessel trees. With these techniques that are used in many research projects with clinical collaborators, we are quantifying different aspects of various lung diseases.

Prostate imaging with MRI remains a major research themes. A diagnostic workstation under development in DIAG is used in day-to-day clinical practice and clinical research projects. With Nijmegen being one of the world-wide leading centers in prostate imaging the setting is ideal for innovative multi-disciplinary research. Research is aimed at development of automated image analysis tools and computer aided diagnosis to support clinical workflow and quality of image interpretation. Improved localization and staging of prostate cancer for targeted biopsies is a main direction of research. More recently, advanced processing techniques are being developed aimed at screening applications, in which information from multiple MRI sequences is optimally combined. In addition, a project is focusing on helping radiologist to identify and diagnose lymph node metastases using USPIO enhanced MRI.

Publications

B. van Ginneken, S.G. Armato, B. de Hoop, S. van de Vorst, T. Duindam, M. Niemeijer, K. Murphy, A.M.R. Schilham, A. Retico, M.E. Fantacci, N. Camarlinghi, F. Bagagli, I. Gori, T. Hara, H. Fujita, G. Gargano, R. Belloti, F.D. Carlo, R. Megna, S. Tangaro, L. Bolanos, P. Cerello, S.C. Cheran, E.L. Torres and M. Prokop. "Comparing and combining algorithms for computer-aided detection of pulmonary nodules in computed tomography scans: the ANODE09 study", Medical Image Analysis 2010;14:707-722.

M. Samulski, R. Hupse, C. Boetes, R. Mus, G. den Heeten and N. Karssemeijer. "Using Computer Aided Detection in Mammography as a Decision Support", European Radiology 2010;20(10):2323-2330.

P.C. Vos, T. Hambrock, J.O. Barentsz and H.J. Huisman. "Computer-assisted analysis of peripheral zone prostate lesions using T2-weighted and dynamic contrast enhanced T1-weighted MRI", Physics in Medicine and Biology 2010;55(6):1719-1734.

R. Hupse and N. Karssemeijer. "Use of normal tissue context in computer-aided detection of masses in mammograms", IEEE Transactions on Medical Imaging 2009;28(12):2033-2041. N. Karssemeijer, A.M. Bluekens, D. Beijerinck, J.J. Deurenberg, M. Beekman, R. Visser, R. van Engen, A. Bartels-Kortland and M.J. Broeders. "Breast cancer screening results 5 years after introduction of digital mammography in a population-based screening program", Radiology 2009;253(2):353-358.

M. Velikova, M. Samulski, P.J.F. Lucas and N. Karssemeijer. "Improved mammographic CAD performance using multi-view information: a Bayesian network framework", Physics in Medicine and Biology 2009;54(5):1131-1147.

ASCI bi-annual Report 2009-2010 45