image processing on diagnostic workstations 9unix! operating system!. today, most systems are...
TRANSCRIPT
Image Processing on Diagnostic Workstations 123
B. M. ter Haar Romeny, PhDProfessor, Eindhoven University of Technology, Dept. of Biomedical Engineering, Image Analysis and Interpre-tation, PO Box 513, WH 2.106, 5600 MB Eindhoven, The Netherlands
Image Processing on Diagnostic Workstations 9Bart M. ter Haar Romeny
9.1 Introduction
Medical workstations have developed into the super-
assistants of radiologists. The overwhelming pro-
duction of images, hardware that rapidly became
cheaper and powerful 3D visualization and quanti-
tative analysis software have all pushed the devel-
opments from simple PACS viewing into a really
Scientifi c terms marked with ! are explained in Wikipedia: www.wikipedia.org
versatile viewing environment. This chapter gives
an overview of these developments, aimed at radi-
ologists’ readership. Many references and internet
links! are given which discuss the topics in more
depth than is possible in this short paper. This paper
is necessarily incomplete.
Viewing stations are core business in a radiolo-
gist’s daily work. All big medical imaging industries
supply professional and integrated environments
(such as Philips ViewForum, Siemens Syngo X, GE
Advantage, etc.). There are dedicated companies for
viewing software (a.o. Merge eFilm) or OEM solu-
tions (a.o. Mercury Visage, Barco). The application
domain of workstations is increasing. We now see
them regularly employed in PACS and teleradiol-
ogy diagnostic review, 3D/3D-time (4D) visualiza-
tion, computer-aided detection (CAD), quantitative
image analysis, computer-assisted surgery (CAS),
radiotherapy treatment planning, and pathology.
Also the applications for medical image analysis in
the life-sciences research are increasing, due to the
increased attention to small-animal scanning sys-
tems for molecular imaging!, and the many types
of advanced microscopes (such as confocal micros-
copy! and two-photon laser scanning micro-
scopes!), all giving huge 3D datasets. The focus of
this chapter is on image processing (also termed
image analysis or computer vision) applications.
9.2 Hardware
Early systems were based on expensive hardware
platforms, called workstations, often based on the
UNIX! operating system!. Today, most systems are
C O N T E N T S
9.1 Introduction 123
9.2 Hardware 123
9.3 Software 125
9.4 3D Visualization 125
9.5 Computer Aided Detection (CAD) 127
9.6 Atlases 128
9.7 CAD/CAM Design 128
9.8 Diffusion Tensor Imaging! (DTI!) – Tractography! 129
9.9 Registration 129
9.10 RT Dose Planning 129
9.11 Quantitative Image Analysis 130
9.12 Workstations for Life Sciences 131
9.13 Computer-Aided Surgery (CAS) 132
9.14 New Developments 133
9.15 Outlook 133
References 134
NERI_09-ter Haar Romeny.indd 123NERI_09-ter Haar Romeny.indd 123 28.08.2007 08:34:2128.08.2007 08:34:21
124 B. M. ter Haar Romeny
based on readily available and affordable PC and
Mac hardware platforms (running MS-Windows
or Mac-OS respectively), which are still following
Moore’s law! of increasing performance (a doubling
every 24 months) at a stable price level.
The central processor unit (CPU!) is the core of
the system, running today at several Gigahertz, and
performance is expressed in Giga-FLOPS! (109 fl oat-
ing point operations! per second). Famous CPUs are
the Intel Pentium chip, and the AMD Athlon proces-
sor. Today, we see the current 32 bit processors being
replaced by 64 bit processors, which are capable of
processing more instructions simultaneously and
addressing a larger number of memory elements
(232 = 4.2 × 109, so a 32 bit system cannot have more
than 4.2 GB of memory (109 = Giga)). There is also a
trend to have more CPUs (‘dualcore’) on the moth-
erboard!, paving the way to parallel processing,
which is currently still in its infancy.
The memory in the diagnostic workstation is
organized in a hierarchical fashion. From small to
large: the CPU has a so-called cache! on its chip,
as a local memory scratchpad for super-fast access,
and communicates with the main RAM (random
access memory!, today typically 1–4 GB) through
the data bus!, a data highway in the computer. As
the RAM is fully electronic, access is fast (nanosec-
onds), much faster than access to a local hard disk!
(milliseconds). When the RAM is fully occupied, the
CPU starts communicating with the hard disk. This
explains why increasing the RAM of a slow com-
puter can markedly upgrade its performance. In a
PACS system, the disk storage is typically done on
a ‘redundant array of inexpensive disks’ (RAID!),
where many disks in parallel prevent loss of data in
case of failure of a component.
The speed of the network should be able to accom-
modate the network traffi c. Typically the workstation
is part of a local area network (LAN!). Today giga-
bit/second speeds are attained over wired networks,
wireless is slower (30–100 Mbit/s) but convenient for
laptops and ‘person digital assistants’ (PDAs). Many
PACS installations can be serviced remotely through
LAN connections to the supplier, anywhere.
Networks are so fast nowadays that 3D volume
rendering can be distributed from a central pow-
erful computer to simple (and thus low cost) view-
ing stations, called ‘thin clients’! (a.o. Terarecon
Aquarius). A powerful dedicated graphics board (in
this case the VolumePro 1000) with dedicated hard-
ware runs several 3D viewing applications simul-
taneously, and is remotely controlled by the users
of the thin clients. Advantage is the capability to
handle huge datasets (e.g. > 3000 slices) easily, but
scalability (to e.g. dozens of users) is limited.
Interestingly, the power of ‘graphical processing
units’! (GPU!, the processor on the video card!
(or graphics accelerator card!) in the system) has
increased even faster than CPU power, mainly due
to the fact that GPUs form the core of the computer
game industry. The millions of systems needed
for this lucrative market and the high competition
between the market leaders NVIDIA and ATI have
Fig. 9.1a–c. Brain aneurysm (a) and carotids (b): examples
of volume renderings with a computer game graphics card
(3Mensio Inc) (c)
a
c
b
NERI_09-ter Haar Romeny.indd 124NERI_09-ter Haar Romeny.indd 124 28.08.2007 08:34:2328.08.2007 08:34:23
Image Processing on Diagnostic Workstations 125
created a huge performance/price ratio. A GPU has
a 50 times faster communication speed of the data
internally between memory and processor, and has
dedicated hardware for rendering artifi cial environ-
ments, such as texture mapping!, pixel shaders!
and an intrinsic parallel design with pixel pipelines!.
They have fi nally become fully programmable (and
can be instructed by languages as DirectX! and
OpenGL!) and are equipped with 1–1.5 gigabytes of
local memory. These ‘games’ hardware boards are
now increasingly used in 3D medical visualization
applications (a.o. 3Mensio Medical Systems). There
is also a community exploring the use of GPUs for
general processing (DICOM undated a).
The viewing screens of diagnostic workstations
have to be of special diagnostic quality. Excellent
reviews of the important parameters (resolution,
contrast, brightness, 8, 10 or 12 bit intensity range,
homogeneity, stability, viewing angle, speed, etc. are
available in the so-called white papers by a variety
of vendors (a.o. Barco – Barco undated, Eizo – Eizo
undated).
9.3 Software
The revolution in PACS (and teleradiology) viewing
stations was fi red by the standard “Digital Imag-
ing and Communications in Medicine” ! (DICOM)
standard (DICOM undated a), 4000 pages). In the
1990s the ACR (American College of Radiology) and
NEMA (National Electrical Manufacturers Associa-
tion) formed a joint committee to develop this stan-
dard. The standard is developed in liaison with other
standardization organizations including CEN TC251
in Europe and JIRA in Japan, with review also by
other organizations including IEEE, HL7 and ANSI
in the USA. It is now widely accepted. Convenient
short tutorials are available (Barco undated). As the
scanners and viewing software continue to develop,
new features have to be added to the standard con-
tinuously. Vendors are required to make available
their so-called conformity statements (see for exam-
ple Burroni et al. 2004), i.e. a specifi ed list of what
conforms to the current version of the standard.
The second revolution was the standardization
of the internal procedural organization of medi-
cal data handling in the ‘Health Level 7’! standard
(HL7) (DICOM undated a).
The basic function of a viewing station is the con-
venient viewing of the data, with a patient selection
section. The functions are grouped in a so-called
‘graphical user interface’! (GUI!). Versatile PC
based viewing packages are now widely available
(see RSNA 2006 for an extensive list), many also
offering ‘extended ASCI’! character sets for the
Chinese, Japanese and Korean markets.
Basic functions of the GUI include administra-
tive functions as patient and study selection, report
viewing and generation, and visualization func-
tions as cine loop, ‘maximum intensity projec-
tion’! (MIP), ‘multi-planar reformatting’! (MPR!)
including oblique and curved reconstructions, cut
planes, measurement tools for distances and angles,
magnifying glass, annotations, etc.
The development of computer vision algorithms
often follows a hierarchical pathway. The design
process (rapid prototyping) is done in high-level
software (examples are Mathematica!, Maple!,
Matlab!), where very powerful statements and
algebraic functionality make up for very short
code, but his is diffi cult to extent to the huge multi-
dimensional medical images. When the formulas
are understood and stable, the implementation is
made into lower languages, like C, C++, Java. When
ultimate speed (and limited variability) is required,
the code can be implemented in hardware (GPU!,
fi eld programmable gate array’s (FPGA!), dedicated
chips, etc.). Many packages offer software develop-
ment kits for joint development (e.g. MevisLab!
by MEVIS, ‘Insight Segmentation and Registration
Toolkit’ (ITK!) by NLM, etc.).
9.4 3D Visualization
The fi rst breakthrough in the use of workstations
has been by the invention of generating realistic 3D
views from tomographic volume data in the 1980s.
Now 3D volume rendering is fully interactive, at high
resolution and real-time speed, and with a wide vari-
ety of options, making it a non-trivial matter to
use it.
Many dedicated companies are now established
(such as Vital Images with Vitrea, Mercury Computer
Systems with Amira, Barco with Voxar, 3Mensio
with 3Vision, Terarecon with Aquarius, etc.). Often
a third party 3D viewing application is integrated
NERI_09-ter Haar Romeny.indd 125NERI_09-ter Haar Romeny.indd 125 28.08.2007 08:34:2628.08.2007 08:34:26
126 B. M. ter Haar Romeny
in the PACS viewing application, and supplied as
a complete system by such an ‘original equipment
manufacturer’! (OEM!).
The principle of ray tracing (‘rendering’!)
(Nowinski et al. 2005) is actually based on mimick-
ing the physics of light refl ection with the computer:
the value of a pixel in a 2D image of a 3D view (also
called a 2.5D view) is calculated from the refl ected
amount of light from a virtual light source, either
bouncing on the surface of the 3D data (this process
is called ‘surface rendering’!), or as the summation
of all contributions from the inside of the 3D data-
set along the line of the ray in question, composed
with a formula that takes into account the transpar-
ency (or the inverse: the opacity) of the voxels (this
process is called ‘volume rendering’!). The use can
change the opacity settings by manipulating the
so-called ‘transfer function’!, this function giving
the relation between the measured pixel value from
the scanner and the opacity. As there is an infi nite
number of settings possible, users often get con-
fused, and a standard set of settings is supplied, e.g.
for lung vessels, skull, abdominal vascular, etc., or
a set of thumbnails is given with examples of pre-
sets, from which the user can choose. Attempts are
underway to extract the optimal settings from the
statistics of the data itself (Nowinski et al. 2005).
In virtual endoscopy (e.g. colonoscopy) the
camera is positioned inside the 3D dataset. Chal-
lenges for the computer vision application are the
automatic calculation of the optimal path for the
fl y-through through the center of the winding colon,
bronchus or vessel. Clever new visualizations have
been designed to screen the foldings in the colon for
polyps at both the forward as backward pass simul-
taneously: unfolding (ter Haar Romeny 2004) (see
Fig. 9.1) and viewing an unfolded cube (Vos et al.
2003) (see Fig. 9.2).
Segmentation is the process of dividing the 3D
dataset in meaningful entities, which are then visu-
alized separately. It is essential for 3D viewing by,
e.g. cut-away views, and also, unfortunately, one
of the most diffi cult issues in computer vision. It is
discussed in more detail in Section 9.5. When clear
contrasts are available, such as in contrast enhanced
CT or MR angiography and bone structures in CT,
the simple techniques of thresholding and region
growing can be employed, up to now the most often
used segmentation technique for 3D volume visual-
ization.
This also explains the popularity of maximum
intensity projection!, where pixels in the 2.5D view
are determined from the maximum along each ray
from the viewing eye through the dataset (such a
diverging set of rays leads to a so-called ‘perspec-
tive rendering’!). As this may easily lead to depth
ambiguities, often the more appealing ‘closest vessel
projection’! (CVP) is applied, where the local maxi-
mum values closest to the viewer is taken. The sam-
pled points of the (oblique) rays through the dataset
are mostly located in between the regular pixels, and
are calculated by means of interpolation!.Fig. 9.2. Volume rendering of the heart and coronaries
( Terarecon Inc)
Fig. 9.3. Virtual colonoscopy with unfolding enables inspection of folds from all sides. From ter Haar Romeny (2004)
�
NERI_09-ter Haar Romeny.indd 126NERI_09-ter Haar Romeny.indd 126 28.08.2007 08:34:2728.08.2007 08:34:27
Image Processing on Diagnostic Workstations 127
9.5 Computer Aided Detection (CAD)
One of the primary challenges of intelligent soft-
ware in modern workstations is to assist the human
expert in recognition and classifi cation of disease
processes by clever computer vision algorithms.
The often used term ‘computer-aided diagnosis’
may be an overstatement (better: ‘computer-aided
detection’), as the fi nal judgement will remain with
the radiologist. Typically, the computer program
marks a region on a medical image with an anno-
tation, as an attention sign to inspect the location
or area in further detail. The task for the software
developer is to translate the detection strategy of
the expert into an effi cient, effective and robust
computer vision algorithm. Modern techniques are
also based on (supervised and unsupervised) ‘data
mining’! of huge imaging databases, to collect sta-
tistical appearances. E.g. learning the shape and tex-
ture properties of a lung nodule from 1500 or more
patients in a PACS database is now within reach.
Excellent reviews exist on current CAD techniques
and the perspectives for CAD (Doi 2006; Gilbert
and Lemke 2005). The fi eld has just begun, and some
fi rst successes have been achieved. However, there
is a huge amount of development still to be done in
years to come.
Some advances in CAD techniques that have
brought good progress are in the following applica-
tion areas.
Mammography: this has been the fi rst fi eld where
commercial applications found ground, in par-
ticular due to the volume production of the associ-
ated screening, the high resolution of the modality
and the specifi c search tasks. Typical search tasks
involve the automated detection of masses, micro-
calcifi cations, stellate or spiculated tumors, and the
location of the nipple.
How do such algorithms work? Let us look in
some detail to one example: stellate tumor detec-
tion (Hofman et al. 2006). As breast tissue con-
sists of tubular structures from the milk-glands to
the nipple, tumor extensions may preferably follow
these tubular pathways. In a projection radiograph
this leads to a spiculated or star-shaped pattern.
The computer will inspect the contextual environ-
ment of each pixel (say 50 × 50 pixels) on the pres-
ence of lines with an orientation pointing towards
the relevant pixel. In this way a total of 2500 ‘votes’
are collected for each pixel. The pixels with a voting
probability exceeding some threshold are possible
candidates for further inspection.
The location of the nipple is important as a gen-
eral coordinate origin for localization references
with, e.g. previous recordings. The general statis-
tical ‘fl ow’ of line structures points towards the
nipple; the location can reasonably well be found by
modeling the apparent statistical line structure with
physical fl ow models.
The role of MRI in breast screening is rising. As
in regular anatomical scans, too many false nega-
tive detections are found, and current attention now
focuses on dynamic contrast enhanced MRI. The
rationale is the high vascularity of the neoplasm,
leading to a faster uptake and outwash over time of
the contrast medium compared to normal tissue.
Current research focuses on the understanding of
this vascular fl ow pattern (e.g. by two-compartment
modeling) and the optimal timing of the image
sequence.
Polyp detection in virtual colonography: polyps
are characterized by a mushroom-like extrusion of
the colon wall, and can be detected by their shape:
they exhibit higher local 3D curvature! (‘Gauss-
ian curvature’!) properties. These can be detected
with methods from ‘differential geometry’! (the
theory of shapes and how to measure and character-
ize them), and highlighted as, e.g. colored areas as
attention foci for further inspection.
Methods have been developed to carry out an
electronic cleansing! of the colon wall when con-
trast medium is still present. An interesting current
target is possible to reduce strongly the radiation
dose of the CT scan, and still be able to detect the
Fig. 9.4. Unfolded cube projection in virtual colonoscopy.
From Vos et al. (2003)
NERI_09-ter Haar Romeny.indd 127NERI_09-ter Haar Romeny.indd 127 28.08.2007 08:34:3228.08.2007 08:34:32
128 B. M. ter Haar Romeny
polyp structures, despite the deterioration of the
detected colon wall structures. Clever shape smooth-
ing techniques and edge-preserving smoothing of
the colon surface have indeed enabled a substantial
dose reduction.
Thorax CAD: here the focus is on the automated
detection of nodules in the high resolution multi-
slice CT (MSCT) data, on the detection of pulmo-
nary emboli, and of textural analysis by classifi -
cation of pixels, e.g. for the quantifi cation of the
extent of sarcoidosis. See Sluimer et al. (2006) for
a review.
Other CAD applications include calcium scoring,
used to detect and quantify calcifi ed coronary and
aorta plaques, analysis of retinal fundus images for
leaking blood vessels as an early indicator for diabe-
tes, and the inspection of skin spots for melanoma
(of particular attention in Australia).
9.6 Atlases
The use of interactive 3D atlases on medical worksta-
tions is primarily focused on education and surgery.
As an example, K.-H. Höhne’s pioneering Voxel-
Man series of atlases (Hofman et al. 2006) was ini-
tiated by the ‘visible human project’!. Atlases for
brain surgery (e.g. the Cerefy Brain atlas family;
Nowinski et al. 2005) now become probabilistic,
based on a large number of patient studies.
9.7 CAD/CAM Design
Workstations can also assist in the creation of
implants from the 3D scans of the patients. This is
a highly active area in ENT, dental surgery, orthope-
dic surgery and cranio-maxillofacial surgery. Many
design techniques have been developed to create the
new shapes of the implants, e.g. by mirroring the
healthy parts of the patient of the opposite side of the
body, 3D region growing of triangulated ‘fi nite ele-
Fig. 9.5a–c. Virtual colonoscopy with surface smoothing. a Original dose (64 mAs); b 6.25 mAs; c 1.6 mAs. From Peeters
(2006b)
Fig. 9.6. The famous Voxel-Man atlas explored many types of
optimal educational visualization. From Höhne (2004)
ba c
NERI_09-ter Haar Romeny.indd 128NERI_09-ter Haar Romeny.indd 128 28.08.2007 08:34:3328.08.2007 08:34:33
Image Processing on Diagnostic Workstations 129
ment models’! in the assigned space, etc. The ‘stan-
dard tesselation language’! (STL!) is a common
format to describe surfaces for 3D milling equipment
for rapid prototyping!, such as stereolithography!
systems, plastic droplets ditherers, fi ve-axes com-
puterized milling machines, laser powder sinter-
ing systems, etc. Many dedicated rapid prototyping
companies exist (e.g. Materialize Inc., see also www.
cc.utah.edu/~asn8200/rapid.html). In the medical
arena several large research institutes are active in
this area (Ceasar, Berlin; Co-Me, Zürich).
9.8 Diffusion Tensor Imaging! (DTI!) – Tractography!
Three-dimensional (3D) visualization of fi ber tracts
in axonal bundles in the brain and muscle fi ber bun-
dles in heart and skeletal muscles can now be done
interactively. The images are no longer composed of
scalar! (single) values in the voxels, but a complete
diffusion tensor! (a 3 × 3 symmetric matrix!) is
measured in each voxel.
The three so-called eigenvectors! can be calcu-
lated with methods from linear algebra!; they span
the ellipsoid of the Brownian motion! that the water
molecules make at the location of the voxels due to
thermal diffusion. Complex mathematical methods
are being investigated to group the fi bers in mean-
ingful bundles, to segment and register the DTI data
with anatomical data, and fi nd fi ber crossings and
endings automatically. An interesting development
is the photorealistic rendering of the tiny bundle
structures (with specularities and shadows), based
on the physics of the rendering of hair.
9.9 Registration
Registration, or matching, is a classical technique in
image analysis (Hajnal et al. 2001). It is employed
to register anatomical to anatomical, or anatomical
to functional data, in any dimension. Examples are
MRI-CT, PET-CT, etc. The construction of a PET and
a CT gantry in a single system effectively solves the
registration problem for this modality.
The matching can be global (only translation, ori-
entation and zooming of the image as a whole) or
local (with local deformation, also called warping!).
Registration can be done by fi nding correspondence
between (automatically detected) landmarks, or on
the intensity landscape itself (e.g. by correlation!).
There is always an entity (a so-called functional!)
that has to be minimized for the best match: e.g.
the mean squared distance between the landmarks,
a Pierson correlation coeffi cient, or others. In par-
ticular, for multi-modality matching, the mutual
information! (MI) has been found to be an effective
minimizer. As an example, in MRI bone voxels are
black and in CT white; they show as a cluster in the
joint probability histogram of the MR vs CT inten-
sities. The MI is a measure of entropy (disorder) of
this histogram.
9.10 RT Dose Planning
The accuracy of radiotherapy dose calculations,
based on the attenuation values of the CT scan of
the patient, needs to be very high to prevent under-
exposure of the tumor and overexposure of the
healthy tissue. Typically the isodose surfaces are
calculated and viewed in 3D in the context of the
local anatomy. Increasingly the images made in the
linear accellerator with the electronic portal imag-
ing device! (EPID) are used for precise localization
of the beam and repeat positioning of the patient,
Fig. 9.7. Muscle fi bers tracked in a high-resolution DTI
MRI of a healthy mouse heart. Lighting and shadowing of
lines combined with color coding of helix angle (αh). From
Peeters et al. (2006a)
NERI_09-ter Haar Romeny.indd 129NERI_09-ter Haar Romeny.indd 129 28.08.2007 08:34:4128.08.2007 08:34:41
130 B. M. ter Haar Romeny
by precise registration techniques. The low contrast
images (due to the high voltage of the imaging beam)
can be enhanced by such techniques as (adaptive)
histogram equalization!.
9.11 Quantitative Image Analysis
This is the fastest growing application area of medi-
cal workstations. The number of applications is vast,
every major vendor has research activities in this
area, and many research institutes are active. To
quote from the scope of ‘Medical Image Analysis’,
one of the most infl uential scientifi c journals in the
fi eld:
“The journal is interested in approaches that uti-
lize biomedical image datasets at all spatial scales,
ranging from molecular/cellular imaging to tissue
/ organ imaging. While not limited to these alone,
the typical biomedical image datasets of interest
include those acquired from: magnetic resonance,
ultrasound, computed tomography, nuclear medi-
cine, X-ray, optical and confocal microscopy, video
and range data images.
The types of papers accepted include those that
cover the development and implementation of algo-
rithms and strategies based on the use of various
models (geometrical, statistical, physical, func-
tional, etc.) to solve the following types of problems,
using biomedical image datasets: representation of
pictorial data, visualization, feature extraction, seg-
mentation, inter-study and inter-subject registra-
tion, longitudinal / temporal studies, image-guided
surgery and intervention, texture, shape and motion
measurements, spectral analysis, digital anatomical
atlases, statistical shape analysis, computational
anatomy (modeling normal anatomy and its varia-
tions), computational physiology (modeling organs
and living systems for image analysis, simulation
and training), virtual and augmented reality for
therapy planning and guidance, telemedicine with
medical images, telepresence in medicine, telesur-
gery and image-guided medical robots, etc.”
See also the huge amount of toolkits for computer
vision: http://www.cs.cmu.edu/~cil/v-source.html.
Important conferences in the fi eld are MICCAI,
CARS, IPMI, ISBI and SPIE MI. In the following
some often-used techniques are shortly discussed.
There are excellent tutorial books (Molecular
visualizations undated; Yoo 2004) and review
papers for the fi eld.
Segmentation! is a basic necessity for many sub-
sequent viewing or analysis applications. Mostly
thresholding and 2D/3D region growing are applied,
but these often do not give the required result.
Proper segmentation is notoriously diffi cult. There
are dozens of techniques, such as model-based seg-
mentation, methods based on statistical shape vari-
ations (‘active shape models’!), clustering methods
in a high-dimensional feature space (e.g. for tex-
tures), histogram-based methods, physical models
of contours (‘snakes’, level sets!), region-growing!
methods, graph partitioning! methods, and multi-
scale segmentation!.
The current feeling is that fully automated seg-
mentation is a long way off, and a mix should be
made between some (limited, initial) user-interac-
tion (semi-automatic segmentation).
Feature detection! is the fi nding of specifi c land-
marks in the image, such as edges, corners, T-junc-
tions, highest curvature points, etc. The most often
used mathematical technique is multi-scale differ-
ential geometry! (Ter Haar Romeny 2004). It is
interesting that the early stages of the human visual
perception system seem to employ this strategy.
Image enhancement! is done by calculating spe-
cifi c properties which then stand out relative to the
(often noisy) background. Examples are the likeness
of voxels to a cylindrical structure by curvature rela-
tions (‘vesselness’!), edge preserving smoothing!,
coherence enhancing!, tensor voting!, etc. Com-
mercial applications are, e.g. MUSICA (‘Multi-Scale
Image Contrast Amplifi cation’, by Agfa), and the
Swedish ContextVision (http://www.contextvision.
se/). Enhancement is often used to cancel the noise-
increasing effects of substantially lowering the X-ray
dose, such as in fl uoroscopy and CT screening for
virtual colonoscopy.
Quantitative MRI is possible by calculating the
real T1 and T2 fi gures from the T1 and T2 weighted
acquisitions, using the Bloch equation! of MRI
physics. Multi-modal MRI scans can be exploited
for tissue classifi cation: when different MRI tech-
niques are applied to the same volume, each voxel
is measured with a different physical property, and
a feature space can be constructed with the physical
units along the dimensional axes: e.g. in the charac-
terization of tissue types in atherosclerotic lesions
with T1, T2 and proton density weighted acquisi-
tions, fat pixels tend to cluster, as do blood voxels,
muscle voxels, calcifi ed voxels, etc. Pattern recogni-
NERI_09-ter Haar Romeny.indd 130NERI_09-ter Haar Romeny.indd 130 28.08.2007 08:34:4228.08.2007 08:34:42
Image Processing on Diagnostic Workstations 131
tion techniques like neural networks! and Bayesian
statistics! may fi nd the proper cluster boundaries.
Shape can be measured with differential geomet-
ric properties, such as curvature!, saddle points!,
etc. It is often applied when, e.g. in the automated
search for (almost) occluded lung vessels in pul-
monary emboli, polyps on the colon vessel wall,
measuring the stenotic index, spiculated lesions in
mammography, etc. A popular method is based on
‘active shape models’!, where the shape variation is
established as so-called shape eigenmodes! by ana-
lyzing a large set of variable shapes and perform-
ing a ‘principal component analysis’!, a well known
mathematical technique. The fi rst eigenmode gives
the main variation, the second the one but larg-
est, etc. Fitting an atlas or model-based shape on a
patient’s organ or segmented structure becomes by
this means much more computationally effi cient.
Temporal analysis is used for bolus tracking
(time-density quantifi cation), functional maps of
local perfusion parameters (of heart and brain),
contrast-enhanced MRI of the breast, cardiac output
calculations by measuring the volume of the left ven-
tricle over time, multiple sclerosis lesion growth /
shrinkage over time, regional cardiac wall thickness
variations and local stress/strain calculations, and
in fl uoroscopy, e.g. the freezing of the stent in the
video by cancellation of the motion of the coronary
vessel.
9.12 Workstations for Life Sciences
In life sciences research a huge variety of (high
dimensional) images is generated, with many new
types of microscopy! (confocal!, two-photon!,
cryogenic transmission electron microscopy!,
etc.) and dedicated (bio-) medical small animal
scanners (micro-CT, mini PET, mouse-MRI, etc.).
The research on molecular imaging and molecu-
lar medicine is still primarily done in small animal
models.
There is great need for quantitative image analy-
sis. An example is, e.g. the measurement of quan-
titative parameters that determine the strength of
newly engineered heart valve tissue of the patient’s
own cell line, such as collagen fi ber thickness, local
orientation variation and tortuosity!. The source
images are from two-photon microscopy, where the
collagen is specifi cally colored with a collagen spe-
cifi c molecular imaging marker.
Another example is the detailed study of the micro-
vascular structure in the goat heart from ultra-thin
slices of a cryogenic microtome! (degree of branch-
ing, vessel diameter, diffusion and perfusion dis-
tances, etc.). Typical resolution is 25–50 micron in
all directions, with datasets of 20003.
Fig. 9.8. a Multimodality MRI of atherosclerotic plaque in the human carotid artery: (w1) T1-weighted 2D TSE, (w2) ECG-
gated proton density-weighted TSE, (w3) T1-weighted 3D TFE, (w4) ECG-gated partial T2-weighted TSE, (w5) ECG-gated
T2-weighted TSE. b Feature space for cluster analysis. c Classifi cation result. From Hofman et al. (2006)
a
cb
NERI_09-ter Haar Romeny.indd 131NERI_09-ter Haar Romeny.indd 131 28.08.2007 08:34:4328.08.2007 08:34:43
132 B. M. ter Haar Romeny
This research arena will benefi t greatly in the
near future from the spectacular developments in
the diagnostic image analysis and visualization
workstations.
9.13 Computer-Aided Surgery (CAS)
In the world of CAS some very advanced simula-
tion and training systems (KISMET, Voxel-Man)
have been created. Especially in dental implants,
craniofacial surgery and laparoscopic surgery there
are many and highly advanced systems today. Surgi-
cal navigation workstations are routinely displaying
the combination of the anatomy and the position
and orientation of the instruments in the operating
theatre.
An interesting development is the use of complex
fl uid dynamics modeling, which enables the predic-
tion of rupture chances in abdominal aorta surgery,
and selecting optimal therapeutic procedures with
bypass surgery in the lower aorta.
In neurosurgery workstations can be employed in
the calculation of an optimal (safest) insert path for
electrodes for deep brain stimulation (DBS), based
on a minimal costs path avoiding blood vessels and
ventricles, and starting in a gyrus. Workstations
assist in inter-operative visualization by warping
the pre-operative imagery to the real situation in
the patient during the operation, by intra-operative
MRI, or ultrasound.
Fig. 9.10. A 3D visualization of a microtome stack
(40 × 40 × 40 μm) of the micro-vasculature of a goat heart
(van Bavel et al. 2006) [Bennink 2006]
Fig. 9.11. Virtual laparoscopy trainer (Origin: Forschungs-
zentrum Karlsruhe KISMET)
Fig. 9.9a,b. Two-photon fl orescence microscopy of collagen fi bers of tissue-engineered heart-
valve tissue. a Result of structure preserving denoising. From Daniels et al. 2006
a b
NERI_09-ter Haar Romeny.indd 132NERI_09-ter Haar Romeny.indd 132 28.08.2007 08:34:4728.08.2007 08:34:47
Image Processing on Diagnostic Workstations 133
9.14 New Developments
The visual perception of depth (when viewing 3D)
data is helped enormously if the viewer can move
the data himself. There are many depth cues (stereo,
depth from motion, depth from perspective), but
depth from motion is the strongest. That is why
maximum intensity projections (MIP) are prefer-
ably viewed dynamically. By self-tracking also the
muscle’s proprioceptors are giving feedback to the
brain, adding to the visual sensation. The combi-
nation with human’s superb eye-hand coordination
has led to the concept of the Dextroscope (www.dex-
troscope.com), where a (computer generated) view
or object can be manipulated under a half-trans-
parent mirror, through which the viewer sees the
display. Displays can also be equipped with haptic
(tactile) feedback systems, which are now commer-
cially available.
Super-large screens, and touch screens are becom-
ing popular; a new trend is the multi-touch screen
(http://cs.nyu.edu/~jhan/ftirtouch/ with movie),
where multiple positions to interact simultaneously
make more complex transformations possible, such
as zooming, multiple simultaneous objects interac-
tions, etc.
Fig. 9.12a–c. Abdominal aorta aneurysm: a color coding of displacement (mm); b Von Mises strain; c Von Mises stress (kPa).
From de Putter et al. (2005)
ba c
Fig. 9.13. Stereo viewing and manipulation with haptic feed-
back
9.15 Outlook
We have actually just started with exploiting the
huge power these super assistants can add, in any
of the fi elds discussed above – hardware, software
and integration. Image processing plays an essential
role, be it for visualization, segmentation, computer-
aided detection, navigation, registration, or quanti-
tative analysis. There will be an ever greater need for
clever and robust algorithms: it is the conviction of
the author that the study of human brain mechanism
for the inspiration for such algorithms has a bright
future to come (ter Haar Romeny 2004). The radi-
ologists will benefi t from these supper-assistants,
and fi nally: the patient has the best benefi t of all.
NERI_09-ter Haar Romeny.indd 133NERI_09-ter Haar Romeny.indd 133 28.08.2007 08:34:5228.08.2007 08:34:52
134 B. M. ter Haar Romeny
References
Barco (undated) White Papers Barco (screens, DICOM): http://www.barco.com/medical/en/downloads/whitepa-pers.asp?dltype=15
Bovik AC (ed) (2000) Handbook of image and video process-ing (communications, networking and multimedia). Aca-demic Press
Burroni M et al. (2004) Melanoma computer-aided diag-nosis: reliability and feasibility study. Clin Cancer Res 10:1881–1886
Daniels F, ter Haar Romeny BM, Rubbens MP, van Assen HC (2006) Quantifi cation of collagen orientation in 3D engineered tissue. In: Ibrahim F (ed) Proc Int Conf on Biomedical Engineering BioMed 2006, Kuala Lumpur, Malaysia, pp 344–348
de Putter S, Breeuwer M, Kosea U, Laffarguec F, Rouet J, Hoogeveen R, van den Bosch H, Buthe J, van de Vosse F, Gerritsen FA (2005) Automatic determination of the dynamic geometry of abdominal aortic aneurysm from MR with application to wall stress simulations. Proc CARS, International Congress Series 1281, pp 339– 344
DICOM (undated a) DICOM standard: http://medical.nema.org/
DICOM (undated b) conformance statement (example): https://www.merge.com/RESOURCES/pdf/dcs/dcs_efi lm21.pdf
Doi K (2006) Diagnostic imaging over the last 50 years: research and development in medical imaging science and technology. Phys Med Biol 51:R5–R27
Eizo (undated) White Papers Eizo (screens): http://www.eizo.com/support/wp/index.asp
Gilbert FJ, Lemke H (eds) (2005) Computer-aided diagnosis. Special issue of British Journal of Radiology, vol. 78, Brit-ish Institute of Radiology
GPGPU (undated) General-purpose computation using graphics hardware, http://www.gpgpu.org/
Hajnal JV, Hill DLG, Hawkes DJ (eds) (2001) Medical image registration. CRC
Health Level 7 (undated) Health Level 7, http://www.hl7.org/. See also regional sites (Australia http://www.hl7.org.au/, UK http://www.hl7.org.uk/, Canada http://www.cihi.ca/hl7, etc.)
Hofman JMA, Branderhorst WJ, ten Eikelder HMM, Cappen-dijk VC, Heeneman S, Kooi ME, Hilbers PAJ, ter Haar Romeny BM (2006) Quantifi cation of atherosclerotic plaque components using in-vivo MRI and supervised classifi ers”, Magn Reson Med 55:790–799
Höhne K-H (2004) VOXEL-MAN 3D-Navigator” (CD-ROM). Springer, Berlin Heidelberg New York
Karssemeijer N (1995) Detection of stellate distortions in mammograms using scale-space operators. Proc. Infor-mation Processing in Medical Imaging, pp 335–346
Kaufman, Müller K (2005) Overview of volume rendering. The visualization handbook. Elsevier
Molecular visualizations (undated) Molecular visualiza-tions: http://molvis.sdsc.edu/visres/index.html
Nowinski W, Thirunavuukarasuu S, Benabid G (2005) The Cerefy clinical brain atlas. Enhanced edition with sur-gical planning and intraoperative support (CD-ROM). Thieme
Peeters THJM, Vilanova A, Strijkers GJ, ter Haar Romeny RBM (2006b) Visualization of the fi brous structure of
the heart. Proc. 11th workshop on Vision, Modeling and Visualization. Aachen, Germany
Peeters THJM, Vilanova A, ter Haar Romeny RBM (2006a) Visualization of DTI fi bers using hair-rendering tech-niques. In: Lelieveldt BPF, Haverkort B, de Laat CTAM, Heijnsdijk JWJ (eds) Proc ASCI 2006. Lommel, Belgium, pp 66–73
Peters JF, Grigorescu SE, Truyen R, Gerritsen FA, de Vries AH, van Gelder RE, Rogella P (2005) Robust automasted polyp detection for low-dose and normal-dise virtual colonoscopy. Proc CARS 2005, International Congress Series 1281, Berlin, Germany, pp 1146–1150
RSNA 2006 Exhibitor list: http://rsna2006.rsna.org/rsna2006/V2006/exhibitor_list/home.cfm
Schaefer-Prokop CM, van Delden OM, Bouma H, Sonnemans JJ, Gerritsen FA, Lameris JS (2006) To assess the added value of a prototype computer-aided detection (CAD) system for pulmonary embolism (PE) in contrast-enhanced multi-detector computed tomography (CT) images. Proc Eur Conf of Radiology, Vienna, Austria, EPOS Poster
Sereda P, Vilanova A, Gerritsen FA (2006) Automating trans-fer function design for volume rendering using hierarchi-cal clustering of material boundaries. In: Sousa Santos B, Ertl T, Joy K (eds) Eurographics/IEEE VGTC Symposium on Visualization (EuroVis),Lisboa, Portugal, pp 243–250
Siggraph (undated) Ray tracing tutorial (SIGGRAPH): http://www.siggraph.org/education/materials/HyperGraph/raytrace/rtrace0.htm
Sluimer IC, Schilham AMR, Prokop M, van Ginneken B (2006) Computer analysis of computed tomography scans of the lung: a survey. IEEE Transactions on Medi-cal Imaging, vol. 25, pp. 385–405
Sonka M, Michael Fitzpatrick J (eds) (2000) Handbook of medical imaging – vol. 2. Medical image processing and analysis. SPIE, Belligham, WA
ter Haar Romeny BM (2004) Front-end vision and multi-scale image analysis. Springer, Berlin Heidelberg New York
ter Haar Romeny BM (2004) Front-end vision and multi-scale image analysis. Springer, Berlin Heidelberg New York
van Bavel E, Bakker EN, Pistea A, Sorop O, Spaan JA (2006) Mechanics of microvascular remodeling. A review. Clin Hemorheol Microcirc 34(1/2):35–41
Vilanova A, Gröller E (2003) Geometric modelling for virtual colon unfolding. In: Brunnett, Harmann, Müller, Lisen (eds) Geometric modeling for scientifi c visualization. Springer, Berlin Heidelberg New York, pp 453–468
Vos FM, van Gelder RE, Serlie IWO, Florie J, Nio CY, Glas AS, Post FH, Truyen R, Gerritsen FA, Stoker J (2003) Three-dimensional display modes for CT colonography: con-ventional 3D virtual colonoscopy versus unfolded cube projection. Radiology 228:878–885
Voxel Man surgical workstation: http://www.uke.uni-ham-burg.de/medizinische-fakultaet/voxel-man/index_ENG.php
Whitby J (2006) The DICOM Standard”, White paper Barco Inc, 2006. URL: http://www.barco.com/barcoview/down-loads/WhitePaper_DICOM.pdf
Yoo TS (2004) Insight into images: principles and practice for segmentation, registration, and image analysis. AK Peters
NERI_09-ter Haar Romeny.indd 134NERI_09-ter Haar Romeny.indd 134 28.08.2007 08:34:5528.08.2007 08:34:55