FOUR-DIMENSIONAL RADIATION TREATMENT PLANNING FOR

LUNG CANCER

_______________

A Thesis

Presented to the

Faculty of

San Diego State University

_______________

In Partial Fulfillment

of the Requirements for the Degree

Master of Science

in

Radiological Health Physics

_______________

by

Ammar Durghalli

Fall 2011

iii

Copyright © 2011

by

Ammar Durghalli

All Rights Reserved

iv

DEDICATION

I dedicate this work to my parents for their never-ending love. I also dedicate this to

my uncle William Durghalli for his love and for being there for me more times than I can

remember. Finally, I dedicate this to all my teachers, past and present who inspired me along

the way.

v

ABSTRACT OF THE THESIS

Four-Dimensional Radiotherapy Treatment

Planning for Lung Cancer

by

Ammar Durghalli

Master of Science in Radiological Health Physics

San Diego State University, 2011

Lung cancer is the second most diagnosed and the leading cause of cancer death in

the US. It accounts for nearly 15% of all new cancer cases, and about 28% of all cancer

deaths. Unfortunately, lung cancer radiotherapy is associated with a poor clinical outcome.

Thus, the need for an aggressive radiation therapy regimen, that is, involving fewer fractions

and higher radiation doses per fraction to tumor targets while increasing healthy tissue

sparing, is evident for increasing local control rates and clinical outcome. Stereotactic body

radiation therapy (SBRT) is among current state of the art techniques that fill this need by

providing highly conformal, high-dose radiation doses to cancerous tumors. Such techniques

as SBRT rely on state of the art imaging systems to provide precise localization of tumor

targets, as well as critical organs at risk, throughout all stages of the radiotherapy process

from treatment simulation and planning, and throughout the radiation delivery.

The main source of uncertainty in radiation delivery of lung cancer is due to the

respiration-induced deformation of the thoracic anatomy during imaging/treatment.

Therefore, the four-dimensional computed tomography (4DCT) imaging is a crucial step in

the design of a highly conformal SBRT plan. 4DCT captures the anatomy at multiple stages

of the respiratory cycle. However, the current SBRT plans are based on a single aggregate

CT set, such as the maximum intensity projection (MIP) or the average intensity projection

(AIP) CT images, which is derived from a 4DCT dataset and represents a motion

encompassing CT image on which treatment planning is based. However, this imaging

method, while saves time, presents a limitation on SBRT since neither MIP nor AIP CT

images correctly represent the moving anatomy. The resulting planned dose and actual

delivered dose may or may not be substantially different depending on each patient case.

Deformable image registration (DIR) is an image processing technique that calculates

the relative motion magnitude and direction of each image voxel between a corresponding

two images of the same anatomy. The result can, in principle, be used to correctly account

for the motion-induced errors in dose calculations, and thus provides means to verify the

accuracy of radiation dose delivery, known as 4D planning. The goal of this thesis is to

pursue the viability of this verification process. The two well-known DIR algorithms were

studied and implemented: (1) Horn-Schunck’s optical flow, and (2) Demons algorithms.

In this thesis, a representative two lung SBRT plans were re-calculated based on the

DIR between all 4DCT image phases, and the resulting "4D doses" were compared to the

original planned doses. Results showed that the current MIP-based SBRT planning doses did

not significantly differ from the full 4D plan doses. Moreover, it was shown that the optical

vi

flow algorithm is faster and more accurate than the Demons algorithm. Further studies are

needed to validate our groundbreaking work in the future.

vii

TABLE OF CONTENTS

PAGE

ABSTRACT ...............................................................................................................................v

LIST OF TABLES ................................................................................................................... ix

LIST OF FIGURES ...................................................................................................................x

ACKNOWLEDGEMENTS ................................................................................................... xiv

CHAPTER

1 GENERAL SURVEY OF RADIATION THERAPY ...................................................1

1.1 Radiation Therapy: Historical Perspective ........................................................1

1.2 Why Radiotherapy Works..................................................................................1

1.3 Radiation Therapy Process ................................................................................3

1.4 Image Guided Radiation Therapy ......................................................................6

1.5 Management of Respiratory Motion ..................................................................8

1.5.1 Respiratory Motion in Lung Cancer .......................................................10

1.5.2 The Mechanics of Breathing ...................................................................10

1.5.3 Problems of Respiratory Motion during Radiotherapy ...........................11

1.5.4 Methods to Account for Respiratory Motion ..........................................12

1.6 Stereotactic Body Radiation Therapy for Lung Cancer ...................................12

1.7 The Role of Image Registration in Radiation Therapy ....................................14

2 DEFORMABLE IMAGE REGISTRATION ..............................................................18

2.1 Basic Components of an Image Registration Algorithm .................................18

2.1.1 Transformation Model ............................................................................18

2.1.2 Registration Metric .................................................................................23

2.1.3 Optimizer and Registration Scheme .......................................................29

2.2 Regularization: Why Is It Needed? ..................................................................30

2.3 Inverse-Consistent Optical Flow and Demons Algorithms .............................35

2.3.1 Inverse Consistency ................................................................................35

2.3.2 Optical Flow Deformable Image Registration ........................................36

2.3.3 Registration in the Inverse Direction and Inverse Consistency ..............38

viii

2.3.4 Symmetric Optical Flow System Cost Function .....................................41

2.3.5 Solving the Asymmetric System Cost Equation .....................................42

2.3.5.1 Case I: Horn-Schunck (HS) Optical Flow Algorithm....................42

2.3.5.2 Case II: Demons Algorithm ...........................................................45

2.3.4 Variational Interpretation of the Demons Approach ..............................49

3 APPLICATION OF DEFORMABLE IMAGE REGISTRATION TO FOUR-

DIMENSIONAL TREATMENT PLANNING OF LUNG CANCER ........................52

3.1 4DCT, Maximum Intensity Projection CT, and Treatment Planning ..............52

3.2 Application of Deformable Image Registration to 4DCT Radiation

Treatment PLanning...............................................................................................53

3.3 Image Registration Results ..............................................................................55

3.4 Four-Dimensional Analysis of MIP-Based Treatment Plans...........................59

3.5 Optical Flow vs. Demons for Lung CT ...........................................................64

4 CONCLUSION ............................................................................................................67

4.1 Results from 4D Analysis of SBRT Lung Cancer Cases .................................67

4.2 Algorithms Performance ..................................................................................67

4.3 Final Discussion and Future Direction ............................................................67

REFERENCES ........................................................................................................................69

APPENDIX

MATLAB PSUEDO CODE ..............................................................................................72

ix

LIST OF TABLES

PAGE

Table 1.1. Comparison of Typical Characteristics of 3D/IMRT Radiotherapy and

SBRT............................................................................................................................15

Table 2.1. Notational Conventions ..........................................................................................37

Table 3.1. DIR Parameters Used in the Registration Shown in Figures 3.2-3.4 (pp. 55-

56) ................................................................................................................................57

Table 3.2. DIR Parameters Used in the Registration Shown in Figures 3.5-3.7 (p. 58) .........59

x

LIST OF FIGURES

PAGE

Figure 1.1. A modern medical linear accelerator (linac). ..........................................................2

Figure 1.2. The various steps in the radiation treatment process are represented by

links in a chain. Source: J. Van Dyk, “Radiation oncology overview,” in The

Modern Technology of Radiation Oncology, edited by J. Van Dyk (Medical

Physics Publishing, Madison, WI, 1999), pp. 1-17. ......................................................4

Figure 1.3. Schematic illustration of ICRU volumes in cross section, as would be

visible on a CT 2D slice, for example. ...........................................................................5

Figure 1.4. A 2D slice of a 3D, seven-field SBRT lung cancer plan. Dose is

represented in RGB color scheme and superimposed upon contours of the

PTV (pink). ....................................................................................................................6

Figure 1.5. Left: Differential DVH corresponding to PTV in Figure 1.4 (p. 6); it

shows the volume of a structure receiving a dose in each dose bin; ideal shape

is a delta function. Right: Cumulative DVH, obtained by integrating the

differential DVH, and indicates the volume receiving less than or equal to the

corresponding dose on the dose axis; ideal shape is, naturally, a step function. ...........7

Figure 1.6.. Variations in respiratory patterns from the same patient taken a few

minutes apart. The three curves in each plot correspond to infrared reflector

measured patient surface motion in the SI, AP, and ML directions, with each

component arbitrarily normalized. Top: the motion pattern is relatively

reproducible in shape, displacement magnitude, and pattern. Bottom: the trace

is so irregular that it is difficult to distinguish any respiratory pattern. Source:

P. J. Keal et al., “The management of respiratory motion in radiation

oncology, report of AAPM Task Group 76,” Med. Phys. 33, 3874-3900

(2006). ............................................................................................................................9

Figure 1.7. A schematic of the 4D CT process. Top: Images are acquired multiple

times at each couch position, for many respiratory phases. A respiratory

signal, driven by input from, say, the patient's abdominal wall height, is

synchronized with image acquisition time. Bottom: This allows all images of a

particular phase (from all couch positions) to be concatenated into a complete

3-D CT image. All of the phases put together make up a 4-D CT data set.

Source: P. J. Keal et al., “The management of respiratory motion in radiation

oncology, report of AAPM Task Group 76,” Med. Phys. 33, 3874-3900

(2006). ..........................................................................................................................13

Figure 2.1. Cubic B-spline deformation model. The displacement Δx as a function of

x is determined by the weighted sum of basis functions Bi. The double arrow

shows the region of the overall deformation affected by the weight factor w7.

xi

3D deformation functions are constructed using 1D deformations for each

dimension. Source: M. L. Kessler, “Image registration and data fusion in

radiation therapy,” Br. J. Radiol. 79, S99–108 (2006). ...............................................20

Figure 2.2. Multi-resolution registration of lung data using B-splines. Both knot

density and image resolution are varied during registration. This can help

avoid local minima and decrease overall registration time. Source: M. L.

Kessler, “Image registration and data fusion in radiation therapy,” Br. J.

Radiol. 79, S99–108 (2006). ........................................................................................21

Figure 2.3. Left: Visualization of a deformation computed between images registered

using B-splines and Right: Fluid flow model. The deformation is known for

every voxel but only displayed for a subset of voxels for clarity. Source: M. L.

Kessler, “Image registration and data fusion in radiation therapy,” Br. J.

Radiol. 79, S99–108 (2006). ........................................................................................22

Figure 2.4. Ambiguity in registration solution, even for the case of rigid

transformations. Top: Template image T. Bottom: Reference image R. A

possible solution is a translation that aligns bottom left corner of T with that of

R. Another is a 90 deg. rotation of T followed by previous translation. But the

later solution matches top left corner of T with bottom left corner of R. Many

other possible solutions are obviously possible as well. Source: J. Modersitzki,

FAIR: Flexible Algorithms for Image Registration (Society for Industrial and

Applied Mathematics, Philadelphia, PA, 2009). ..........................................................32

Figure 2.5. Illustration of inverse consistency errors introduced by asymmetric DIR.

Points A and B are corresponding points in images T, and R, respectively.

DVF V (U) is the result of registering T (R) to R (T). (a): V (U) maps A (B) to

A' (B') in R (T). (b): Using V (U), B' (A') is mapped to B'' (A'') in R (T). The

distances AA'', and BB'' represent the inverse consistency errors. Source: D.

Yang, H. Li, D. Low, J. Deasy, and I. El Naqa, “A fast inverse consistent

deformable image registration method based on symmetric optical flow

computation,” Phys. Med. Biol. 53, 6143–6165 (2008). .............................................39

Figure 2.6. Demonstration of the proposed inversely consistent registration method.

Matching points A and B are in images T, and R, respectively. After n passes,

A (B) is matched with A' (B'). A' and B' can be thought of as belonging to an

average image An= (Tn+ Rn)/2, that Tn and Rn are registered to, by calculating

incremental motion fields Vn+1 and Un+1 after pass n. The incremental fields

are then used to obtain the updated fields Vn+1 and Un+1. In turn, Vn+1 and

Un+1are used to get the updated images Tn+1, and Rn+1 according to (31), and

(32). The algorithm converges when the Vn 's become sufficiently small. The

total, inverse-consistent motion fields V and U are then calculated according

to equations (2.33) and (2.34). Source: D. Yang, H. Li, D. Low, J. Deasy, and

I. El Naqa, “A fast inverse consistent deformable image registration method

based on symmetric optical flow computation,” Phys. Med. Biol. 53, 6143–

6165 (2008). .................................................................................................................40

Figure 2.7. Geometric interpretation of the optical flow equation (1D case). .........................47

xii

Figure 2.8. Thirion demons. Top: The reference image (called scene in Thirion's

terminology) with six demons on its contour (black dots), and the contour of a

deformable model (model or moving image). The arrows indicate movement

direction. The demons push the moving image inward (in direction of ) if

the scene and model overlap, and outward (in direction of if they do not

overlap. Middle and Bottom: I and J represent the moving and fixed images,

respectively. Pushing and pulling force by demon in a one-dimensional

model............................................................................................................................48

Figure 3.1. Pixel intensity-based projection protocols from 4DCT of a moving tumor,

used to detect high to low intensity anatomic structures (a) tumor contours

from all separate 4DCT phases, (b) Maximum intensity projection (MIP)

represents where the tumor is present at sometime in the respiratory cycle, (c)

Minimum intensity projection (MinIP) represents the space, in the respiratory

cycle, which the tumor always occupies, (d) Mean intensity projection.

Source: R. W. M. Underberg, F. Lagerwaard, B. Slotman, J. P. Cuijpers, and

S. Senan, “Use of maximum intensity projections (MIP) for target volume

generation in 4DCT scans for lung cancer,” Int. J. Radiat. Oncol. Biol. Phys.

63, 253-260 (2005).......................................................................................................54

Figure 3.2. A coronal view of lung anatomy at two respiratory phases for Patient 1

from a 4DCT. Left: 10% phase (near EOI). Right: 50% reference phase

(EOE). ..........................................................................................................................55

Figure 3.3. The images of Figure 3.2 (p. 56) registered (10% phase registered to 50%

phase) using inverse-consistent Demons algorithm. Left: the deformed 10%

phase CT image is now very similar to the reference image. Right: 50%

reference phase- image. ...............................................................................................56

Figure 3.4. Difference image before and after DIR, for the images in Figure 3.2 (p.

56), and Figure 3.3. Left: Difference before. Right: Difference after. ........................56

Figure 3.5. A coronal view of lung anatomy at two respiratory phases for Patient 2,

from a 4DCT. Left: 0% phase (EOI). Right: 50% reference phase (EOE). .................58

Figure 3.6. The images of Figure 3.5 (p. 58) registered (0% phase registered to 50%

phase) using inverse-consistent optical flow algorithm. Left: The deformed

0% phase CT image is very similar now to the reference image. Right: 50%

reference phase- image. ...............................................................................................58

Figure 3.7. Difference image before and after DIR, for the images in Figure 3.5, and

Figure 3.6 . Left: Difference before. Right: Difference after. ......................................58

Figure 3.8. Dose volume histogram for Patient 1 case. The green curve is the DVH

for the bilateral lung from the actual treatment plan. The red curve shows the

same DVH according to 4D calculations of delivered dose, based on

deformable image registration of 4DCT images. .........................................................60

Figure 3.9. Dose volume histogram for Patient 2 case. The green curve is the DVH

for the bilateral lung from the actual treatment plan. The red curve shows the

xiii

same DVH according to 4D calculations of delivered dose, based on

deformable image registration of 4DCT images. .........................................................61

Figure 3.10. Dose volume histogram for Patient 2 case. The green curve is the DVH

for the PTV from the actual treatment plan. The red curve shows the same

DVH according to 4D calculations of delivered dose, based on deformable

image registration of 4DCT images. ............................................................................62

Figure 3.11. Dose volume histogram for Patient 2 case. The green curve is the DVH

for the left lung from the actual treatment plan. The red curve shows the same

DVH according to 4D calculations of delivered dose, based on deformable

image registration of 4DCT images. ............................................................................63

Figure 3.12. Dose volume histogram for Patient 2 case. The green curve is the DVH

for the right lung from the actual treatment plan. The red curve shows the

same DVH according to 4D calculations of delivered dose, based on

deformable image registration of 4DCT images. .........................................................64

Figure 3.13. Demons DIR for a slice of 0% phase of a 4DCT of lung. Left: Difference

between CT image slices before DIR (MSE=2.9 x 105); Right: Difference

after DIR (MSE= 1.3 x 105). Computation time was 2.82 hours. ................................65

Figure 3.14. H.S OF DIR for 0% phase. Left: Difference before DIR (MSE=2.9 x

105); Right: Difference after DIR (MSE= 1.4 x 10

5). Computation time was 45

minutes. ........................................................................................................................65

Figure 3.15. Demons DIR for 20% phase. Left: Difference before DIR (MSE=2.1 x

105); Right: Difference after DIR (MSE= 1.2 x 10

5). Computation time was

2.1 hours. ......................................................................................................................66

Figure 3.16. H.S OF DIR for 20% phase. Left: Difference before DIR (MSE=2.1 x

105); Right: Difference after DIR (MSE= 1.1 x 10

5). Computation time was 35

minutes. ........................................................................................................................66

xiv

ACKNOWLEDGEMENTS

First, I would like to thank my advisor, mentor, and friend Dr. William Y Song for his

support, patience, and for believing in me. His countless insights put me on the right track,

and without him, this work would not have been possible.

Secondly, I would like to thank my committee chair and teacher Dr. Usha Sinha for

her continued support and enthusiasm, and for the extraordinary efforts she continues to put

into the medical physics program at SDSU. Without her, our program would not have been

CAMPEP –accredited!

I also would like to thank Dr. Faramarz Valafar for taking the time to read my thesis,

and to be on my thesis committee.

Special acknowledgment is also in order to Dr. Deshan Yang of Washington

University in St. Louis, MO for making available his MATLAB code, and his helpful

discussions.

Finally, I would like to thank all of my fellow researchers at UCSD's Centre for

Advanced Radiotherapy Technologies. Their support and encouragement made my research

experience both fruitful and enjoyable.

1

CHAPTER 1

GENERAL SURVEY OF RADIATION THERAPY

This introductory chapter outlines basic facts and issues in radiation therapy in

general and as applied to lung cancer, in particular. It also introduces many of the common

concepts and nomenclature used in radiotherapy.

1.1 RADIATION THERAPY: HISTORICAL PERSPECTIVE

The history of radiation therapy began with the discovery of x-rays and radioactivity,

by Roentgen in 1895, Becquerel and Curie in 1896 and 1898, respectively, and has since

advanced to the point where the penetrating ability of ionizing radiation is utilized to deliver

a lethal dose to a tumor volume. The first patient cured by radiation therapy was reported in

1899.1 At the beginning, however, inadequate techniques and lack of stable, high –energy

sources, resulting in recurrences of tumors and normal tissue complications, dampened the

initial enthusiasm for this novel therapy. A major change was the discovery of the

importance of fractionation, or multiple daily irradiation to the same tissue site. This resulted

in reduced risk of complications.2

1.2 WHY RADIOTHERAPY WORKS

Cancer is the second leading cause of death in the industrialized countries and the

only major disease for which death rates are increasing. The demand for cancer care will

increase over the decade as the aging of the baby boomer population drives a dramatic

increase in the incidence of many cancers. Approximately 60% of cancer patients are treated

with external beam radiotherapy (EBRT) at some point during management of their disease.

The main goal of radiation therapy (RT) is to maximize the dose to the target while limiting

the dose to nearby healthy organs (“organs at risk”), in order to improve control of tumor

growth and limit side effects.

Radiation therapy is primarily used to treat cancer by locally targeting radiation to the

diseased tissue. Radiation beams are produced by medical linear accelerators (Figure 1.1).

These devices are mounted on a gantry with a rotating couch to allow for many beam

2

Figure 1.1. A modern medical linear accelerator (linac).

3

directions to be focused on the target volume. Sparing of normal tissues is accomplished in

two fundamental ways: geometric avoidance of normal tissues is accomplished by directing

multiple beams at the target, thus delivering a high dose where the beams intersect at the

target, and a relatively lower dose outside of the intersection. Biological sparing of normal

tissue is accomplished by fractionating the therapy over several weeks, irradiating daily. The

tumor tissue lacks repair mechanisms to repair DNA damage from the radiation, whereas

normal tissues can repair minor DNA damage. Therefore, by fractionating the treatment,

normal tissues are provided time to repair, thus biologically sparing the normal tissue.3

1.3 RADIATION THERAPY PROCESS

Radiation therapy is a clinical process in which ionizing radiation, typically in the

range 4-25 MeV, is used to treat a variety of tumors, the majority being malignant, with

intent to cure or palliate.2 The aim is to deliver a highly conformal radiation dose to a target

tumor volume while minimizing the dose given to surrounding healthy tissues. The end is to

generally increase the quantity and/or quality of life.

The radiation therapy process for cancer treatment consists of multiple steps as

illustrated schematically in Figure 1.2.4 The steps are indicated as links in a chain to

demonstrate the dependence of later stages on earlier steps. These steps are briefly

summarized here. First, when a patient is diagnosed with cancer, the most appropriate

treatment strategy is determined. If the patient is selected for radiation treatment, the first

step is to localize the tumor and its surrounding critical normal structures with a three

dimensional (3D) computed tomography (CT) imaging of the anatomy. The attending

physician delineates the gross tumor volume (GTV), which is the visible tumor volume

which is seen in the CT images.5 A second volume, called the clinical target volume (CTV),

is defined to encompass the GTV as well as accounting for possible microscopic extensions

of the disease that is not seen in the images.5 The construction of this volume is based on

clinical experience, disease-specific patterns of spread, and on the initial staging of the

disease. Finally, a margin is added to the CTV to account for the uncertainties associated

with radiation dose delivery. These uncertainties are of two types, internal, or physiological

(e.g. due to respiration, bladder or rectum filling levels, swallowing, heartbeat, bowel

motion…etc), and set-up errors in positioning the patient relative to each beam from one

4

Figure 1.2. The various steps in the radiation treatment process are represented by

links in a chain. Source: J. Van Dyk, “Radiation oncology overview,” in The Modern

Technology of Radiation Oncology, edited by J. Van Dyk (Medical Physics

Publishing, Madison, WI, 1999), pp. 1-17.

treatment day to another. The resulting total volume is collectively called the planning target

volume (PTV) and it's this volume that will receive the prescribed doses of radiation.5

Figure 1.3 illustrates GTV, CTV, PTV and other volumetric therapy planning

concepts, as defined by the ICRU (International Commission on Radiation Units &

Measurements). Once the patient-specific PTV is defined, an isocentre is chosen and its

coordinates are determined such that reference marks (i.e. tattoos) can be placed on the

patient's skin for future treatment set-up. The isocentre is the point of intersection of the

central axes of the treatment beams and is usually made to coincide with a central point in the

GTV. The isocentre is also the point in 3D space about which the treatment head of the linear

accelerator (linac) and the treatment couch rotate, and where the four treatment room laser

cross hairs coincide. The tattoo marks are then identified during the treatment set-up and

5

Figure 1.3. Schematic illustration of ICRU volumes in cross section, as

would be visible on a CT 2D slice, for example.

aligned with the in-room lasers, thus ensuring the treatment isocentre (established during CT

simulation) is coincident in space with the linac isocentre in the treatment room.

In the next step, the CT images, isocentre coordinates, and contour information

(delineated target and critical organs) are imported into a computerized treatment planning

system (TPS) for beam placement, beam shaping (e.g. blocks, shields, and multi-leaf

collimators (MLC)), plan optimization, and three-dimensional (3D) dose distribution

calculations. Organs at risk (OARs), also localized and contoured during the target

delineation stage, are the main dose limiting parameters and are an integral part of the overall

plan optimization process. Figure 1.4 illustrates a seven-field lung treatment plan generated

using the Eclipse TPS (Varian, Palo Alto, CA) at the UCSD's Moores Cancer Center, La

Jolla, CA. Note that the PTV is well covered by the highest dose region (dark red + red) and

the dose then falls off quickly far from the PTV; thus sparing normal lung tissue. At this

stage, the dose-volume histogram (DVH) of the target volume and the OAR's are also

6

Figure 1.4. A 2D slice of a 3D, seven-field SBRT lung

cancer plan. Dose is represented in RGB color scheme

and superimposed upon contours of the PTV (pink).

calculated by the TPS to supplement a graphical evaluation of the plan (Figure 1.5).

Differential DVH is a histogram of the dose in each volume element or voxel, defined by the

size of the dose grid, inside a structure of interest. The vertical axes whether in fractional or

absolute volume units, indicates the volume that receives a dose in a certain range (set by the

size of the dose bins, e.g. 5 cGy). When integrated it yields a cumulative DVH, which

indicates the fractional (or absolute) volume receiving a dose greater than or equal to the

dose defined by each dose bin. With smaller bin sizes, the cumulative DVH line becomes

smoother. The cumulative DVH is typically the one used to assess radiation treatment plans.

1.4 IMAGE GUIDED RADIATION THERAPY

Generally speaking, image-guided radiation therapy (IGRT) is the process of radiation

therapy that uses image guidance procedures for target localization; before and during

treatment. These procedures use imaging technology, such as cone beam CT (CBCT) and

four-dimensional CT (4DCT), to identify and correct problems arising from inter-and intra-

fractional variations in patient setup and anatomy, including shapes and volumes of treatment

target, organs at risk (OAR), and surrounding normal tissue. As the PTVs are more and more

conformal to tumors as in intensity-modulated radiation therapy (IMRT), the accuracy

7

Figure 1.5. Left: Differential DVH corresponding to PTV in Figure 1.4 (p. 6); it

shows the volume of a structure receiving a dose in each dose bin; ideal shape is a

delta function. Right: Cumulative DVH, obtained by integrating the differential

DVH, and indicates the volume receiving less than or equal to the corresponding

dose on the dose axis; ideal shape is, naturally, a step function.

requirements of PTV localization and dosimetric coverage during each treatment fraction

becomes increasingly stringent.

Many imaging systems have now become available in the treatment room, many

mounted directly on the linear accelerator (on-board imagers or, OBIs), to aid in the

visualization of the tumor and surrounding anatomy before and during treatments. Some of

these systems are6

1. Portal and radiographic imagers: Modern accelerators are equipped with two kinds of

imaging systems: kilovoltage x-ray imager and megavoltage (MV) electronic portal

imaging device (EPID). The first has an x-ray tube mounted on the gantry with an

opposing flat panel detector. The second uses the MV beam from the linac and also

has its own flat panel detector. Although neither OBI produces a really good contrast

image, both are useful in determining the PTV's position relative to the bony anatomy

and/or radio-opaque markers (fiducials) implanted in the target tissues.

2. In-room CT scanner: An in-room CT scanner shares the couch with the linac and can

be moved in and out of the way on rails. Its availability provides, in addition to target

localization prior to treatment, the ability to reconstruct the dose distribution as well.

The latter can be compared with the dose distributions and isodose curves obtained in

the original plan before each treatment, or periodically during the course of the

treatment. This allows one to make setup corrections or modify treatment parameters

to minimize variations between planned and actual treatments. This is actually an

example of what is called image-guided adaptive radiation therapy (IGART).

3. Kilovoltage cone-beam CT (CBCT): Kilovoltage OBI systems are usually capable

of simple radiography, fluoroscopy (real time radiography), and cone-beam computed

tomography (CBCT). The x-ray tube is mounted on a retractable arm at 90 degrees

relative to central axis of the linac beam. Images are recorded by the flat panel area

8

detectors mounted opposite the tube. kVCBCT involves acquiring planar projections

as the gantry rotates through 180 degrees or more. The 3D volumetric CT images are

reconstructed by a filtered back-projection algorithm.7 kVCBCT produces images

with good soft tissue contrast, which is helpful in delineating GTVs.

4. The advantages of kVCBCT over MVCBCT, to be discussed next, are:

Better contrast and spatial resolution (about 1mm voxel size at isocentre)

Better soft-tissue visibility at much lower dose

Compatibility of kVCBCT images with the original treatment plan images for

patient setup verification and correction

Combination of radiography, fluoroscopy, and CBCT capabilities from the same

source and detector, which provides great flexibility in implementing the goals

of IGRT.

5. Megavoltage cone-beam CT: MVCBCT uses a traditional EPID with its Si flat-panel

detector. The x-ray source is the linac megavoltage beam. Although the soft-tissue

contrast is reduced from that of kVCBCT, the images are still good enough for 3-D

localization of target. The advantages of MVCBCT over kVCBCT are:

There's less susceptibility to imaging artifacts due to metallic objects such as hip

implants, dental fillings, and surgical clips.

CT numbers obtained from MVCBCT correlate directly with electron density.

The known dose distribution characteristics of the therapeutic beam allow more

accurate calculation of imaging dose in the MVCBCT acquisition process.

Implementation of MVCBCT does not require extensive modification of a

linear accelerator that is already equipped with an EPID.

1.5 MANAGEMENT OF RESPIRATORY MOTION

Respiratory motion plays a negative role in all tumors of the thorax, abdomen, and

pelvis. The most prevalent tumor motion occurs in lung cancer, where tumor can move

several centimeters in any direction. The esophagus, liver, pancreas, prostate, and kidneys,

among other organs are also known to move with breathing, causing degradation in imaging

quality and subsequently in dose delivery. Breathing patterns can vary dramatically even for

the same patient in a matter of minutes, as illustrated in Figure 1.6.8 The top figure shows the

breathing patterns in all three directions (SI, AP, ML) exhibiting very regular patterns that

almost reproducible. The bottom figure shows the breathing patterns for the same patient

measured a few minutes later, showing a much more pronounced lack of regularity.

9

Figure 1.6.

. Variations in respiratory patterns from the same patient taken a few

minutes apart. The three curves in each plot correspond to infrared reflector measured

patient surface motion in the SI, AP, and ML directions, with each component

arbitrarily normalized. Top: the motion pattern is relatively reproducible in shape,

displacement magnitude, and pattern. Bottom: the trace is so irregular that it is

difficult to distinguish any respiratory pattern. Source: P. J. Keal et al., “The

management of respiratory motion in radiation oncology, report of AAPM Task Group

76,” Med. Phys. 33, 3874-3900 (2006).

10

1.5.1 Respiratory Motion in Lung Cancer

Lung cancer accounts for 28 % of all cancer deaths in U.S. The 5 year survival rate

(all stages) is around 15%. Higher dose of radiation treatment correlates with local control

and survival rate (a decrease of 18% in risk of death per 10 Gy increase in BED (biologically

equivalent dose). To achieve a 30 months 50% local progression-free survival, 85 Gy is

required, which is much higher than the current clinically acceptable dose (due to high risk of

lung complications). Thus there is a need for technologies that allow for an increased tumor

dose while increasing the sparing of healthy tissue. Such technologies should allow for

tracking and modeling of tumor motion. It's worth noting that respiratory motion contributes

the least to the error in radiation treatment of lung cancer. The leading sources of error (an

order of magnitude or higher) are set-up errors, and large inter-physician GTV outline-

variations. Studies have shown that lung tumor motion is independent of tumor size and

location, and independent of pulmonary function.8 Along with other similar findings in the

literature, means that a real-time tumor tracking or gating process should be used to manage

respiratory-induced tumor motion.9 This suggests that tumor motion has to be assessed and

modeled individually for each patient.

1.5.2 The Mechanics of Breathing

The primary function of the lung is to allow CO2-O2 exchange between blood and air.

The levels (i.e. pressures) of CO2 and O2 gases and the pH of arterial blood control the

magnitude as well as the frequency of the respiration cycle (through chemo-receptors). The

CO2 pressure plays the most important role, however.

Breathing is involuntary for the most part, although individuals can have, to certain

limits control over the frequency and magnitude of respiration as well as breath holds.

During normal quiet breathing, the contraction of the diaphragm increases the superior-

inferior dimension of the chest cavity. The contraction of the intercostal muscles pulls the rib

cage superiorly and anteriorly, increasing its lateral and anterior dimensions. Exhalation is a

passive process where the elasticity of the lung and chest walls returns them to their pre-

inhalation positions.

The flow of atmospheric air in and out of lungs is determined by what's called

transpulmonary pressure. Transpulmonary pressure is defined as the difference between the

11

air pressure in the alveolar space (intrapulmonary pressure) in the lungs and the interpleural

pressure (the pressure of the fluid inside the pleura; the pleura is the membrane

lining/connecting the lungs to chest wall). The latter is always negative (i.e. less than

atmospheric pressure as well as intrapulmonary pressure), and is responsible for the lungs not

collapsing inwards. The former (Transpulmonary) fluctuates but eventually equalizes with

atmospheric pressure. If the transpulmonary pressure is below atmospheric pressure (as is the

case during inhalation), air flows in the lungs. The situation is reversed for exhalation.

If transpulmonary pressure is plotted against lung volume for both exhalation and

inhalation, one finds that at the same transpulmonary pressure, the exhalation volume is

larger than the inhalation volume. This is an example of hysteresis phenomena, where cause

and effect relationship (lung volume and transpulmonary pressure in this case) is not a simple

one and exhibits past history dependence.

1.5.3 Problems of Respiratory Motion during

Radiotherapy

Some problems arising in radiotherapy due to breathing motion are presented:8

1. Image-acquisition limitations: During image acquisition, respiratory motion can

cause severe artifacts that affect CT scans such as shortening, elongation, splitting,

shifting of midpoints of objects.10

2. Treatment-planning limitations: During treatment planning, and after the GTV and

CTV have been outlined, margins are added to the CTV to form the PTV (planning

target volume). These margins account for intra-faction (during a single session)

errors due to respiration of the patient, inter-fraction and setup errors. These margins

are prescribed by general guidelines, and are, therefore, suboptimal; they're likely to

either lead to increased radiation dose to healthy tissue, or to under-dosing of CTV by

underestimating the tumor's range of motion. 11

3. Radiation-delivery limitations: During the actual delivery of the radiation beam to the

patient's tumor, the combined vector sum of displacements (tumor relative to bone +

bone relative to beam) causes a blurring of the dose distribution that was based on

static assumptions. This is seen mainly around the beam edges (where organs are

moving in and out of beam) and in effect increases the beam's penumbra. Here we are

still assuming an ideal zero-gradient (variation) of the beam intensity in the middle of

the beam field, and thus no effect due to organ/tumor motion in the middle. In the

case of IMRT this is no longer the case, and the blurring effect is even worse.

12

1.5.4 Methods to Account for Respiratory Motion

Methods that help to account for respiratory motion in radiation therapy can be put

into five major categories: motion-encompassing methods, respiratory-gating techniques,

breath hold techniques, forced shallow-breathing techniques, and respiration-synchronized

techniques.8 Motion-encompassing methods are CT imaging methods that include the

entire range of tumor motion during respiration. They are slow CT scanning, inhale and

exhale breath-hold CT, and four dimensional CT (4DCT).

Slow CT scanning is useful mostly for lung cancer, and it captures the full range of

tumor motion from respiration (at time of scan). The CT scanner is operated slowly (on the

scale of respiratory cycle time) and/or multiple CT scans are averaged, during each couch

position, as the patient breathes freely. Thus, the image shows the full extent of respiratory

motion, or a tumor-encompassing volume. A limitation is that the breathing pattern may

change between imaging and treatment, thus additional margins are needed to account for

these variations. The treatment is, therefore, planned on a more realistic geometry that

represents the entire respiration cycle, and the overall treatment process does not increase in

complexity over that of an ordinary, free-breathing CT scan. The disadvantages of slow CT

scans are loss of resolution due to the blurring effect, which increases errors in tumor and

organ delineation, and the increased dose to the patient.

Inhale and exhale breath-hold CT obtains two CT scans at exhale and inhale

positions. The maximum intensity projection (MIP, c.f. Section 3.1) tool can be used to

obtain the tumor-motion-encompassing volume, provided there's no mediastinal tumor

involvement. This approach does not suffer from the blurring effect of a slow CT scan.

Finally, 4DCT provides high quality CT images and can afford detailed information

on respiratory motion, such as tumor trajectory and mean position. Variations in breathing

pattern during the image acquisition process does affect 4DCT quality and breathing training

techniques need to be practiced to ensure reproducibility. Figure 1.7 shows a schematic

outlining the process of acquiring a 4DCT. 8

1.6 STEREOTACTIC BODY RADIATION THERAPY FOR

LUNG CANCER

Currently, nearly all lung cancer patients at UCSD's Moores Cancer Centre are given

radiation treatment using the stereotactic body radiation therapy (SBRD) technique. SBRT

13

Figure 1.7. A schematic of the 4D CT process. Top: Images are acquired multiple

times at each couch position, for many respiratory phases. A respiratory signal,

driven by input from, say, the patient's abdominal wall height, is synchronized with

image acquisition time. Bottom: This allows all images of a particular phase (from

all couch positions) to be concatenated into a complete 3-D CT image. All of the

phases put together make up a 4-D CT data set. Source: P. J. Keal et al., “The

management of respiratory motion in radiation oncology, report of AAPM Task

Group 76,” Med. Phys. 33, 3874-3900 (2006).

14

refers to a radiotherapy procedure that is highly effective in controlling early stage primary

and oligometastatic (limited metastasis) cancers at locations throughout the abdominopelvic

and thoracic cavities, and at spinal and paraspinal (adjacent to the spinal column) sites. The

major distinction between SBRT and conventional 3D conformal radiation therapy is the

delivery of large doses in a few fractions, which results in a high biological effective dose

(BED). Moreover, the practice of SBRT requires a high level of confidence in the accuracy

of the entire radiation delivery process. This means conformation of high doses to the tumor

and rapid fall-off of doses away from tumor. This is accomplished by the integration of

modern imaging, simulation, treatment planning, and delivery technologies into all phases of

the treatment process; including throughout beam delivery. Additional distinctions of SBRT

are increase in the number of beams used, the frequent use of non-coplanar beam

arrangements, small or no beam margins for penumbra, and the use of inhomogeneous dose

distributions and dose-painting techniques (including IMRT). Table 1.1 illustrates the

features that characterize SBRT and 3D/IMRT techniques.12

1.7 THE ROLE OF IMAGE REGISTRATION IN RADIATION

THERAPY

Despite the extended time frame of fractionated radiotherapy (4–6 weeks), RT

planning is carried out based on information that is currently limited to a single 3D

anatomical computed tomography (CT) image data set acquired at the onset of treatment

design. The patient is marked (tattooed) for repeated alignment with localization lasers in the

treatment room. The treatment planning is then performed on the CT scan where beam

geometries, energies, and collimation are determined, and the resultant dose distribution is

computed. This concept may result in severe treatment uncertainties, resulting in irradiation

of risk organs and reduced tumor coverage.

Natural processes in the body and response of normal and target tissue to the

treatment result in significant inter- and intra-fractional geometrical changes. Intra-fractional

(during a single treatment fraction) geometric change occurs during radiation delivery due to

breathing, cardiac motion, rectal peristalsis and bladder filling. Inter-fractional (day-to-day)

geometric change occurs over the weeks of therapy, due to digestive processes, change of

breathing patterns, difference in patient setup, and treatment response like growth or

shrinkage of the tumor or nearby risk organs (e.g., the parotids in head and neck treatment).

15

Table 1.1. Comparison of Typical Characteristics of 3D/IMRT Radiotherapy and SBRT

Characteristic 3D/IMRT SBRT

Dose/fraction 1.8-3 Gy 6-30 Gy

No. of fraction 10-30 1-5

Target definition CTV/PTV :tumor may not

have a sharp boundary

GTV/CTV/ITV/PTV

Well-defined tumors: GTV

= CTV

Margin Centimeters Millimeters

Physics/dosimetry

monitoring

Indirect Direct

Required setup accuracy TG40, TG142 TG40, TG142

Primary imaging

modalities

CT CT/MRI/PET-CT

Redundancy in geometric

verification

No Yes

Maintenance of high

spatial targeting accuracy

for the entire treatment

Moderately enforced

(moderate patient position

control and monitoring)

Strictly enforced (sufficient

immobilization and high

frequency position

monitoring through

integrated image guidance)

Need for respiratory

motion management

Moderate-Must be at least

considered

Highest

Staff training Highest Highest + special SBRT

training

Technology

implementation

Highest Highest

Radiological

understanding

Moderately well

understood

Poorly understood

Interaction with systemic

therapies

Yes Yes

Source: S. H. Benedict et al., “Stereotactic body radiation therapy: The report of AAPM task group 101,” Med.

Phys. 37, 4078-101 (2010).

16

As mentioned previously, these changes are taken into account by population-based

“uncertainty” margins around the target area, which may be excessive or conservative and

are applied to the structure identified before the therapy begins.

Repeat 3D imaging with single or multiple imaging modalities acquired at various

time intervals during and after a radiation course provides the opportunity to increase

treatment accuracy and precision by optimizing treatment in response to anatomical changes;

to improve target delineation through modality-specific complementary tumor

representations, to quantify patient specific physiological motion, and to assess treatment

response. The exploitation of integrated imagery may allow both dose escalation to the tumor

and reduction of dose given to organs at risk. This has the potential to allow for dose

escalation using larger fractions size hypo-fractionated regimes increasing the chance of local

control without increasing toxicity.

The concepts of adaptive radiotherapy (ART) and image-guided radiotherapy (IGRT)

provide methods to monitor and adjust the treatments to accommodate the changing patient.

ART is an off-line approach where the anatomical and biological changes are monitored over

the course of treatment, and the treatment is modified when significant changes are

identified. IGRT is typically an on-line concept where the patient or treatment plan is shifted

or modified for each treatment. Both concepts require advanced image processing tools in

order to be successful in clinical practice.3

The goal of deformable image registration is to resolve differences in geometry

while maintaining modality-specific differences in information content by means of

estimating the spatial relationship between the volume elements (i.e., the image voxels) of

corresponding structures across image data sets. The solution of this task in turn allows for

the geometrically corrected transfer of target and organ at risk contours (or regions of

interest, ROI) between images, quantitative description of physiological motion patterns,

measurement of image-based surrogates of treatment response, and the design of dose

patterns and determination of their effect in deforming anatomy on a patient-specific basis.3

The next chapter of this body of work reviews the basic algorithmic components of a

deformable image registration (DIR) algorithm. We concentrate in this work on non-

parametric deformable image registration; specifically, two of the ones belonging to the

optical flow family of DIR algorithms (H.S optical flow, and Demons), commonly used in

17

RT planning and their applications to treatment of lung cancer where geometric changes are

most prominent, due to respiratory motion. We also compare the two algorithms according to

how they perform on the same lung CT images, and determine which one is more suitable for

this application.

18

CHAPTER TWO

DEFORMABLE IMAGE REGISTRATION

In this Chapter, the basic components of a deformable image registration algorithm

are discussed, and non-parametric image registration theory is investigated in detail; in

particular, the optical flow and Demons algorithms are discussed at length.

2.1 BASIC COMPONENTS OF AN IMAGE REGISTRATION

ALGORITHM

The basic task of image registration is to compute the coordinate transformation that

maps the coordinates of anatomically corresponding points (e.g. tissue voxels) in two

images. No matter what approach is followed to carry this out, it will involve three basic

components; transformation model, registration metric, and optimization and registration

scheme.13

2.1.1 Transformation Model

The transformation model will depend on the clinical site (e.g. lung, abdomen, brain

…etc), imaging conditions and the particular application. In an ideal situation, the

corresponding pixels in the two images share the same image coordinates, with the same

orientation, scale and centre for the coordinate system. In such a case the transformation from

one image coordinates to the other is simply the identity transform I. This ideal situation is

almost realized for dual modality images produced by the same machine; examples are PET-

CT or SPECT-CT machine; provided physiological motion (e.g. respiration, bladder

filling…etc) is controlled or can be ignored. Generally there will be at least a rigid

transformation between two image sets. For brain images, where the anatomy's position and

orientation are defined by the rigid skull, a simple rotation and translation model can map the

two images correctly. More generally, an affine transformation is the most general rigid

transformation model with 12 degrees of freedom (loosely speaking, 3 parameters for each of

translation, rotation, shearing and scaling). Affine transformations preserve collinearity

("parallel lines remain parallel"). The DICOM imaging standard uses affine transformations

19

to specify the spatial relationship between two coordinate systems (the reference coordinate

system, or RCS, is the machine's defined system where X direction is along LR (left-right), Y

is along AP (anterior-posterior), and Z is along SI (superior-inferior) direction, and the image

coordinate system).14

Although integration of more sophisticated transformation models into

planning systems is being researched, currently most commercial treatment planning systems

only support image registration using affine transformations.

In most applications, affine transformations are not very useful as the anatomy of a

patient is rarely only rigidly deformed. In some situations where one can use cropping to

isolate a structure of interest, such as the prostate, from its surroundings as it moves rigidly,

and thus ignore the non-rigid deforming of the bladder and rectum, that structure can be

registered between two images using only a rigid or affine transformation model. But these

are special cases, and in most cases the motion of an organ's surroundings causes non-rigid

deformations on it. For example, the liver is deformed according to the filling of the stomach,

and the lungs are deformed in size and shape during the breathing cycle. Therefore one is

soon faced with the reality that a non-rigid or deformable transformation model must be used

to accurately map the two images.

Deformable transformation models vary in complexity from being simple extensions

of affine transformations with limited number of parameters (e.g. using polynomials or some

other bases functions to expand the transform function) to a completely local or free-form,

non-parametric model where each point or voxel in the image domain is deformed

independently, and the number of degrees of freedom is three times the number of voxels in

the image. Between these two extremes are parametric registration models designed to

handle various degrees of semi-local deformations using a moderate number of parameters,

such as splines models (global and piecewise polynomials).

Global polynomials have been used successfully to model and remove image

distortions in MR and other images as a preprocessing step for image registration, but they

are not suitable as transformation models for deformation of anatomy; i.e. to approximate the

coordinate transformation that describes the deformation. The reason is that polynomials

exhibit increasingly oscillating behavior between the interpolated data, and the oscillations

even get worse with increasing the degree of the polynomial. Spline-based transformations

(piece-wise polynomials) such as B-splines avoid this problem by building up the

20

transformation function using a set of weighted basis functions defined over only a limited

region. In other words, each basis function is, for example, non-zero only between two

successive data points (image pair voxels coordinates). Figure 2.1 illustrates such an

approach for a one-dimensional cubic B-spline.13

Figure 2.1. Cubic B-spline deformation model. The displacement Δx as a

function of x is determined by the weighted sum of basis functions Bi. The

double arrow shows the region of the overall deformation affected by the

weight factor w7. 3D deformation functions are constructed using 1D

deformations for each dimension. Source: M. L. Kessler, “Image registration

and data fusion in radiation therapy,” Br. J. Radiol. 79, S99–108 (2006).

The displacement Δx at a given point, as shown in Figure 2.1, is computed as the

weighted sum of basis functions centered at a series of locations called knots. The aim is to

find the weights, or parameters, wi that parameterize the transformation. A common cost

function to be minimized, to optimize the B-spline coefficients will be the sum of squared

differences (SSD). With B-splines, the basis functions Bi have compact support (i.e. they are

zero outside of a compact set). Thus, changing the weight of a basis function affects only a

specific portion of the deformation. Additionally, by increasing the number of knots more

complex and localized deformations can be modeled. Figure 2.2 shows an example of using

B-splines for image registration of lung data.13

This example also illustrates the use of the

multi-resolution technique where the two images are registered on increasing levels of multi-

resolution technique where the two images are registered on increasing levels of

21

Figure 2.2. Multi-resolution registration of lung data using B-splines. Both knot

density and image resolution are varied during registration. This can help avoid

local minima and decrease overall registration time. Source: M. L. Kessler, “Image

registration and data fusion in radiation therapy,” Br. J. Radiol. 79, S99–108

(2006).

resolution (four or five levels are typical). As will be discussed later, by down-sampling the

images in this manner, local minima solutions to the registration problem are avoided.

Another spline based transformation model is thin-plate splines. Thin-plate splines

defines a set of landmarks on both images and tries to minimize a bending energy between

them to determine the transformation parameters. A thin plate spline is the 2D analog of the

cubic spline in 1D. It is the fundamental solution to the bi-harmonic differential

equation that describes the shape of a thin steel plate whose displacement is

above the -plane, and has the form

,

where . This basis function is the natural generalization to two dimensions of

the function 3 that underlines the familiar one-dimensional cubic spline.

15 Given a set of

data points, a weighted combination of thin plate splines centered about each data point gives

the interpolation function that passes through the points exactly while minimizing the so-

called "bending energy". For a thin plate subjected to only sleight bending, the bending

22

energy at a point is proportional to the quantity at that point,

and the desired minimizes:

Unlike B-splines, the location of each landmark (control point) does affect the deformation

globally. Using more control points reduces the influence of each point but comes at a higher

computational cost.

Finally, free-form or non-parametric transformation models are represented using

vector fields of the displacement's magnitude and direction at each voxel in the image data

set (Figure 2.3).13

There are different algorithms for solving for the deformation vector field

(DVF) using these non-parametric models. They all rely on defining some external local

force function to drive the deformation with a regularizing internal force that maintains

smoothness and boundedness on the DVF. Common models include fluid flow, optical flow

(based on intensity values and intensity gradients), and finite element method. 16-20

The focus

of this body of work is the second of these approaches, namely, optical flow-based

algorithms and its application to deformable image registration of 4D lung CT image data

sets.

Figure 2.3. Left: Visualization of a deformation computed between images

registered using B-splines and Right: Fluid flow model. The deformation is

known for every voxel but only displayed for a subset of voxels for clarity.

Source: M. L. Kessler, “Image registration and data fusion in radiation

therapy,” Br. J. Radiol. 79, S99–108 (2006).

23

2.1.2 Registration Metric

Once a transformation model is chosen, one must incorporate into it a particular

metric which, in a certain sense, measures the similarity between the two images. This metric

is incorporated into a cost function that is maximized or minimized to arrive at the solution

parameters of the model, for parametric registration models, or find a numerical solution for

non-parametric models. For all practical purposes, registration metrics are classified as either

geometry-based or intensity-based.

Geometry-based metrics make use of features extracted from the images, such as

anatomic or artificial landmarks and organ boundaries. More specifically, most geometry-

based metrics involve the use of points, lines, or surfaces. 21-25

For a parametric

transformation model using point matching, the coordinates of corresponding points in the

two images are used to define the registration metric. These points are usually either

anatomic landmarks (e.g. bronchial bifurcations in thoracic CT) delineated in both images by

a physician or other expert, or implanted fiducial markers. The registration metric in this case

is the Euclidian distances (sum of squared distances, SSD) between corresponding points.

For a rigid transformation between two Cartesian coordinate systems, the transformation is

composed of a rotation and a translation. A translation of a point in 3D has three degrees of

freedom, and a rotation about an axis has another three (the direction of the axis of rotation

and the angle through which a point is rotated about that axis). For image deformation we

also introduce a scaling factor in each dimension and a shearing force as well to make the

general affine transform. Thus for a rigid transformation, a minimum of three pairs of points

are required and for affine transformations, a minimum of four pairs of non-coplanar points

are required in the coordinate systems (images) in order to determine the transformation.

Using more pairs of points reduces the bias in estimating the transformation parameters

introduced by errors in the delineation of any pair of points. This is, of course, not always

easy to accomplish in multimodality registration as different modalities of imaging (e.g.

MRI vs. CT) produce different tissue contrasts and implanting a large number of markers

may not always be desirable.

Line and surface matching metrics try to maximize the overlap between

corresponding lines and surfaces extracted from the two images, such as the brain or skull

surface or pelvic bones. These are relatively easier to delineate using automated techniques,

24

but may still pose a challenge in multimodality images. Additionally, these extracted

geometric features are implicitly used as surrogates for the whole image and, therefore, when

registering the images based on them, any deformations or machine-based distortions away

from these features are not detected during the registration process.

Intensity-based metrics make direct use of the intensity information already

available in the image voxels (numerical grey scale information) to measure how well the

two images are registered. Intensity-based metrics are often also called similarity measures

since they effectively measure the similarity between the intensity distribution between

spatially corresponding voxels in one image (fixed image) and the voxels in the deformed

version of it, or the second image (moving image). The most common similarity measures in

clinical use are: sum-of-squared intensity differences (SSD) and cross-correlation for image

registration from CT to CT data, and mutual information for both similar and multimodality

images. 26- 29

SSD metrics are sensitive to large intensity differences between voxels. Thus, it

is implicitly assumed that, other than image deformations, the images to be registered only

differ by background noise. Therefore SSD is only applicable for single-modality image

registration.

Mutual information between two images is maximum when they are perfectly

registered and aligned, and there's no dependence on the absolute values of intensity.

Therefore the mutual information metric is suitable for handling a situation in image

registration where some object is missing or is not clearly visible. For example, a tumor

might show up clearly on an MR image but be indistinct on a corresponding CT image. Over

the tumor volume, where the tumor is not well defined in the CT image, mutual information

has a low value, but the higher values at surrounding voxels of corresponding healthy tissue

dominates and drives the registration.

Sum-of-square differences measure, is defined by.30, 31

Definition 1 Let The sum of squared difference (SSD)

distance measure is defined by

(2.1)

25

Where, for a general coordinate transformation we define

(2.2)

and for a parametric transformation we define

(2.3)

Here, is a parameterization of such that for each spatial component , we

have

(2.4)

and the parameters are grouped into a vector,

(2.5)

An image is defined as a mapping which assigns every spatial point a

gray value b(x).30

The dimention of the spatial domain is

Definition 2 Let

1.

2.

3.

The set of all images is denoted by

We note that compares it is thus implicitly assumed that

gray values of corresponding points are equal. Cross correlation, on the other

hand, allows for a linear relationship between intensities.

Cross-correlation metrics are based on normalized correlation between the two

images T (template, or moving image) and R (reference image). First, we define correlation

between two images.30

26

Definition 3 Let

We need a modified version of correlation that can be used as a measure of similarity

between R and T. As it stands in Definition 3, correlation can be viewed as the -inner

product between If are normalized, then the correlation is the

cosine of the angle between them, in the sense of the inner product over a vector space.

Maximization of the normalized correlation with respect to gives an image which

is close to in the sense that are maximally linearly dependent.

The normalization is done by subtracting the expectation value (mean) and dividing

by the standard deviation.30

Definition 4 Let be an image. The expectation value and

the standard deviation of B are defined by

Where | .

Definition 5 Let . The correlation coefficient is defined by

,

Where and and are defined in Definition 4.

We can see that (by the Cauchy-Schwarz inequality). is thus just the cosine

of the angle between and . Since now measures the correlation

between two vectors of unit length, it's insensitive to scaling changes in the intensities of R

27

and T. Thus, unlike cross correlation can be used when there's a linear dependence

between T and R intensities. Using this normalization, the cross correlation distance measure

can now be defined30

(dropping the y-subscript):

Definition 6 Let . The correlation-based distance measure

is defined by

where µ and σ are defined by Definition 3. For a transformation

Since

(2.6)

(2.7)

Thus we see that there's a strong connection between the minimization of and the

maximization of

Mutual Information is an entropy-based distance measure that is widely used in

information theory. The definitions for mutual information and mutual information-distance

measure are:30

Definition 7 Let be a density (probability) on

where

28

Definition 8 Let . The mutual information (MI) distance

measure is defined by :

where denote the gray-value densities of R, T, and the joint gray-value

density distribution, respectively. Alternatively, we may write:

For a transformation :

.

Intuitively, Definition 7 for the entropy can be understood by considering an event (e.g. gray

value I for a voxel at position x) with probability Then , and the event

contributes zero to the entropy, since there's no uncertainty, or information gained by

communicating that the event has occurred (it always occurs). If an image is viewed as a

"sample" of an imaginary "intensity source, "Definition 6 defines the entropy as the average

information per image.

Mutual information essentially measures the entropy of the joint density. It is

maximal if the two images are maximally related (i.e. registered). In this case the joint

probability density (e.g. joint histogram) is very sharp, in the sense that the joint probability

is zero except for certain image intensity pairs at corresponding spatial positions. These gray

value pairs are determined by the statistical relationship that exists between significant gray

value structures. This relationship does not have to be known explicitly, which makes MI a

very powerful method than can be applied to same modality or multi-modality images. When

significant gray-value structures are properly aligned, it automatically leads to a peak in the

joint gray value distribution detected as maximum of MI, or minimum in .

29

2.1.3 Optimizer and Registration Scheme

The most intuitive way of approaching the registration problem is to minimize the

chosen distance measure In other words,

minimize the distance between with respect to u,

(2.8)

where it is convenient to split the transformation into the trivial identity part and the so-

called deformation or displacement part ,

(2.9)

where it should be mentioned that this is the so-called Eulerian viewpoint, or Eulerian

coordinates. The Eulerian viewpoint follows the image particles (e.g. tissue voxels) in terms

of their new coordinates. This is in contrast to the Lagrange coordinates which follow the

particles in terms of their old coordinates. This terminology is borrowed from fluid

dynamics. For image registration there are two reference frames, the Lagrange coordinates

and the Euler coordinates. If we assume the transformation to be invertible, we can write

are called the coordinates, deformation, and transformation with

respect to the Lagrange (Euler) frame of reference, respectively. In other words, the Lagrange

viewpoint follows the tissue points, whereas the Euler viewpoint transforms the image

domain. We follow the convention in most of literature and adopt the Euler viewpoint.

However, the corresponding transformation of the images can be counterintuitive as the

image , since after transforming the

coordinates , we require

(2.10)

(2.11)

From a computational point of view, we can treat digital images as continuous mappings by

choosing an appropriate interpolation scheme (e.g. tri-linear interpolation). So we may

assume that images are arbitrarily smooth, and exploit fast numerical schemes that depend on

derivatives of any order. For an image I, we have:30

30

(2.12)

(2.13)

Computationally, the Euler frame is preferable.30

2.2 REGULARIZATION: WHY IS IT NEEDED?

Most iterative image registration algorithms use an optimization scheme such as

gradient descent or in general a first order derivative-based approach. For free-form or non-

parametric registration one cannot obtain a closed-form solution since the problem statement

is usually ill-posed, and we cannot speak of "the solution". One needs to introduce additional

constraints to arrive at a 'reasonable' solution (for example, one does not want deformations

that result in warping of bones or folding of tissue). In other words, directly minimizing (2.8)

will lead to a solution that is not regular; since small changes in the input data (intensity

values) may lead to large changes of the output data (the DVF), the solution is not unique

since the problem is almost always non-convex, and the deformation may not even be

continuous. Thus, it's not possible to construct an appropriate scheme for a numerical

solution. The usual approach is to regularize the cost function by introducing a regularizer

term.

The idea of regularization is to measure the quality of candidate transformations and

choose the best candidate according to the chosen regularity, or smoothness measure. In

many applications the desired properties of the transformation are not known a priori.

Therefore, different smoothing techniques have to be used.

To illustrate why the registration problem is hard, and why regularization is needed, a

few simple examples may help:31

Example 1: this example illustrates the problem of the uniqueness of the solution.

Suppose that the sum of three numbers , and we need to find the

values for . Clearly, the solution is not unique and one must make some assumptions about

the solution. For example,

is uniquely defined.

31

Image registration is inherently ill-posed: For every position , one is

asking for a vector quantity , but generally only a scalar quantity I(y) is available.

So it becomes necessary to regularize the problem by modifying it so that it becomes

solvable. The analogue for image registration is to restrict the transformation to a certain

subspace, such as the space of rigid transformations, , and the problem becomes

to find the proper weights . However, even in this restricted space, it's not a valid

assumption that a unique transformation exists. Figure 2.4 illustrates this point.

Example 2: This example illustrates the problem of ill-posedness. Suppose ,

and . If the forward problem is to compute , for ,

then: . The inverse problem is to compute

Since , we can write a unique solution But a slight

perturbation of T, say, , results in: , which is

quite different from , in terms of both magnitude and the change of sign of the second

component. This issue of ill-posedness can lead to serious errors in image registration.

Example 3: This example continues Example 1 to illustrate the regularization

process, even though the answer is obvious in this case. One approach would be to model

and to solve the data fitting term

. We could also introduce the regularizer term

and try the one of the following strategies:

1. Minimize subject to

2. Minimize subject to . This picks the solution with minimum variation.

32

Figure 2.4. Ambiguity in registration solution, even for the

case of rigid transformations. Top: Template image T.

Bottom: Reference image R. A possible solution is a

translation that aligns bottom left corner of T with that of

R. Another is a 90 deg. rotation of T followed by previous

translation. But the later solution matches top left corner

of T with bottom left corner of R. Many other possible

solutions are obviously possible as well. Source: J.

Modersitzki, FAIR: Flexible Algorithms for Image

Registration (Society for Industrial and Applied

Mathematics, Philadelphia, PA, 2009).

33

3. Compromise between data fitting (as in 1) and regularization (as in 2) to solve

The regularization parameter can be varied to put more

emphasis either on data fitting or on regularization.

all three approaches result in the same solution

. If the forward problem is to compute , for

, then: . The inverse problem is to compute

Since , we can write a unique solution

But a slight perturbation of T, say, , results in:

, which is quite different from , in terms of both magnitude

and the change of sign of the second component. This issue of ill-posedness can lead to

serious errors in image registration.

Example 3: This example continues Example 1 to illustrate the regularization

process, even though the answer is obvious in this case. One approach would be to model

and to solve the data fitting term

. We could also introduce the regularizer term

and try the one of the following strategies:

1. Minimize subject to

2. Minimize subject to . This picks the solution with minimum variation.

3. Compromise between data fitting (as in 1) and regularization (as in 2) to solve

The regularization parameter can be varied to put more

emphasis either on data fitting or on regularization.

all three approaches result in the same solution

34

The regularizer is often based on an of the derivatives30, 31

(usually 1st or

2nd

order) of the displacement vector (x):

Definition 6 Regularization is based on of the derivatives of the

displacement (x):

(2.14)

where is a differential operator, and α is a regularization parameter.

Thus for image registration, the minimization problem (2.8) is replaced by30

The registration problem: Given two images R, T, and a positive regularizing parameter

find a deformation u(x), such that

(2.15)

As we will see later, the optical flow and Demons algorithms make use of the diffusion

regularizer. For example, in two dimensions, , and

(2.16)

The diffusion regularizer measures the variation in the displacement. It is therefore used to

impose a "smoothness" constraint on the DVF, when (2.15) is minimized. In other words, the

regularizer term penalizes deviations from smoothness, so that neighboring points have

similar velocities, or rate of change of intensity values.

As mentioned earlier, the multi-resolution scheme becomes essential in non-

parametric image registration, and is used to start the registration on a coarse scale, and the

results are used to initialize the next registration stages at progressively higher resolutions.

The solution to (2.15) is obtained by numerically solving the resulting Euler-

Lagrange equations. The Euler-Lagrange equations for (2.15) are given by the following

theorem:30

Theorem 1 The Euler-Lagrange equations for

35

is the diffusion regularizer defined by (or, equivalently,

by (14) above), are

(2.17)

Where Neumann boundary conditions are satisfied

( ,

and is called

the force term and is given by

(2.18)

The proof of Theorem 1 is straight forward and is given in Modesitzki30

(c.f. Theorem 8.1,

and Theorem 11.1). The Euler-Lagrange equations are defined only on the interior of the

image domain; to solve them, boundary conditions must be specified. Neumann boundary

conditions state that the directional derivative of the displacement field in the normal

direction (along gradient), at the image boundary, is zero.

The numerical solution to (2.17) can be obtained using a fixed-point iteration scheme

(e.g. Gauss-Seidel iteration method), or a variety of regularizer-specific, specialized methods.

For Example, Modesitzki30

solves (2.17) for the diffusion regularizer using the additive

operator splitting (AOS) technique.

2.3 INVERSE-CONSISTENT OPTICAL FLOW AND DEMONS

ALGORITHMS

In this project, two closely related DIR algorithms were used to register 4DCT images

of the lung; the original optical flow (OF) proposed by Horn and Schunck, and the so-called

Demons algorithm proposed by Thirion.18, 32, 33

Additionally, we implement the symmetric,

inverse-consistent approach to image registration based on the work of Yang et al.34

Much of

the exposition below is based on Yang and Horn with additional details and steps in proofs

filled in where needed. 32, 34

2.3.1 Inverse Consistency

Whatever the registration algorithm, the accuracy of the registration is the most

important and desirable feature that decides its clinical applicability. The problem with

measuring accuracy is that one does not have an absolute reference (the so-called "ground

36

truth") against which to measure the registration algorithm; such as the case for artificially

predefined transformations. However, this is not possible with real patient images as a

ground truth is not available. Other validation methods, primarily landmark and volume

matching are possible. An example of landmark matching is in the case of lung CT images

which lend themselves well to landmark-based validation due to the abundance of high

contrast, anatomical landmarks such as vessel and bronchial bifurcations. But the accuracy of

these methods is limited because they are not based on a voxel-by-voxel comparison, and do

not usually cover the entire image. They are also time consuming and demand dedication of

expert physicians to delineate.

Inverse consistency of a registration algorithm means that the registration is

consistent in either direction. In other words, the registration maps the same voxels whether

image 1 is registered to image 2 or vice versa. Inverse consistency is considered as a reliable

way to judge the accuracy of the registration algorithm. This can be intuitively clear, since an

accurate registration algorithm should not depend on which image is called image 1 and

which is image 2. However, inverse consistency does not by itself guarantee, or imply

accuracy, as an algorithm could in principle result in an inaccurate registration albeit being

inverse consistent. Inverse consistency is nevertheless a way to improve accuracy (Accuracy

can also be increased numerically, for example, in a GPU computational implementation

where number of iterations can be very large). Moreover, inverse consistency is desirable in

the clinical setting for doing image-guided (IG) and adaptive radiation therapy (ART), where

treatment planning contours, volumes…etc are defined on the planning CT (kVCT), while

daily doses, contours, …etc are referenced to daily images such as cone beam CT (CBCT) or

mega voltage CT (MVCT). Inverse consistency allows voxel mapping in both directions so

that it becomes easy and reliable to register daily images to the planning CT or the planning

CT to the planning CT.

2.3.2 Optical Flow Deformable Image Registration

OF algorithms originated from the original HS (Horn-Schunk) algorithm.32

The

Demons algorithm also belongs to the OF family. They rely on image intensity and gradient

information. In general, for two images, T (moving image) and R (reference, or fixed image),

to be registered, we are looking for a transformation such that is

37

similar to in a certain sense (e.g. ) The transformation is usually split into the

trivial identity part and the so-called deformation or displacement part

(2.19)

is often referred to as the optical flow field, and describes how each tissue voxel (in the

moving image) moves to be brought in correspondence with its original position (in the fixed

image). It's defined on the coordinate grid of the fixed image, and points from the moving

image (T) to the fixed image (R); so that:

(2.20)

The notation/conventions are described in Table 2.1.34

Table 2.1. Notational Conventions

T The moving (template) Image (Image 1)

R The fixed (reference) image (Image 2)

The difference image,

The sum image R + T

Image domain (a subset of )

x Coordinate of points in image domain

V, U The deformation vector fields ("pull-back" vector fields) ; the

vector displacements that deform T ( R ) into R ( T )

The incremental deformation vector field

the image T deformed by V

(V1 applied 1

st)

Inverse of V field.

Source: D. Yang, H. Li, D. Low, J. Deasy, and El Naqa, “A fast inverse consistent deformable image

registration method based on symmetric optical flow computation,” Phys. Med. Biol. 53, 6143–6165 (2008).

Physically, V is a vector that can be resolved into two components; one along the

direction of the image intensity gradient, and the other perpendicular to it. So (2.8) cannot by

itself give "the" solution V, and must be supplemented by another equation/condition to

generate a solution. We impose a smoothness constraint (regularizer) on V, so the

optimization system cost function becomes

38

(2.21)

where is the image domain, (V) is the diffusion regularizer, and α is a parameter to

be adjusted. Most optical flow algorithms use

+…, where ( ) is the

trace operator. For small |V|, equation (2.20) is simplified by a 1st-order Taylor

approximation as follows:

(

(2.22)

Again, we should emphasize that this is a small-motion-model approach to the algorithm, and

thus it's assumed that |V| is sufficiently small.

2.3.3 Registration in the Inverse Direction and

Inverse Consistency

The registration problem in the opposite direction (R to T) is formulated separately in

an exactly similar manner, to calculate a motion field U such that

(2.23)

Thus we write a cost function of the same form as (2.21) for U:

(2.24)

As is evident in Figure 2.5, even if V is computed, U still needs to be computed

independently because there's no direct dependence among their solutions in the algorithms.

Therefore the results are not in exact correspondence; i.e. not inverse consistent.

As mentioned earlier, it is very desirable to have the DVF's U and V be inversely

consistent, so that registration could start with either image and the results are consistent;

ideally, we want the inverse consistency error, ICE = 0, or

(2.25)

The inverse consistency errors could then be defined as:

(2.26)

(2.27)

39

a b Figure 2.5. Illustration of inverse consistency errors introduced by asymmetric DIR.

Points A and B are corresponding points in images T, and R, respectively. DVF V (U) is

the result of registering T (R) to R (T). (a): V (U) maps A (B) to A' (B') in R (T). (b):

Using V (U), B' (A') is mapped to B'' (A'') in R (T). The distances AA'', and BB''

represent the inverse consistency errors. Source: D. Yang, H. Li, D. Low, J. Deasy, and

I. El Naqa, “A fast inverse consistent deformable image registration method based on

symmetric optical flow computation,” Phys. Med. Biol. 53, 6143–6165 (2008).

If V and U are inverse consistent, and will both equal zero; otherwise

as shown in Figure 2.5 (b). The combined inverse consistency error ICE is

defined by:

(2.28)

As shown in Figure 2.6, the two images T and R are symmetrically deformed toward

each other at every pass.

At

pass n, an incremental motion field is computed by minimizing a symmetric cost

function (it will turn out to have the same form as (2.21) and (2.24) for the asymmetric

registration), using a slight modification to the HS and demons algorithms. The two total

motion fields, for image T and for image R, are updated by accumulating and

, as:

(2.29)

(2.30)

40

Figure 2.6. Demonstration of the proposed inversely consistent registration method.

Matching points A and B are in images T, and R, respectively. After n passes, A (B)

is matched with A' (B'). A' and B' can be thought of as belonging to an average

image An= (Tn+ Rn)/2, that Tn and Rn are registered to, by calculating incremental

motion fields Vn+1 and Un+1 after pass n. The incremental fields are then used to

obtain the updated fields Vn+1 and Un+1. In turn, Vn+1 and Un+1are used to get the

updated images Tn+1, and Rn+1 according to (31), and (32). The algorithm converges

when the Vn 's become sufficiently small. The total, inverse-consistent motion fields

V and U are then calculated according to equations (2.33) and (2.34). Source: D.

Yang, H. Li, D. Low, J. Deasy, and I. El Naqa, “A fast inverse consistent deformable

image registration method based on symmetric optical flow computation,” Phys.

Med. Biol. 53, 6143–6165 (2008).

So, for example:

and are updated as

(2.31)

(2.32)

The two updated deformed Images are then used for another pass n+1, until

for some predefined threshold; e.g. voxel.

Initially, The subsequent incremental fields are defined

on the incremental, updated image domains of , respectively . Therefore,

are defined on different domains, or image coordinates, at each pass. Thus (As

is also clear from the composition rule for in Table 2.1), even

41

though . Therefore it is necessary to update individually using (2.29)

and (2.30).

To ensure invertibility and smoothness of is forced to be less than

0.4 voxels in magnitude.34

The justification for this is based on a previous investigation by

Reuckert et al.35

It's clear that if are interchanged, will also be swapped, and

thus the registration direction is reversed. As is clear in Figure 2.6, the final motion field, VTR

(URT) registers T (R) to R (T), and is (c.f. Figure 2.6)

(2.33)

(2.34)

It's easy to see that VIJ and UJI are inversely consistent.

2.3.4 Symmetric Optical Flow System Cost Function

As explained before, is deformed by and generates

(2.35)

Similarly

(2.36)

In analogy with (2.21), we introduce the functional

(2.37)

We impose the constraint

(2.38)

and select the regularizer such that

(2.39)

We also choose Then using (2.35), (2.36), and (2.38) into (2.37), we get

(2.40)

or more compactly:

(2.41)

Where .

42

Equation (2.41) and (2.22) are of exactly the same form. Therefore we can solve

(2.41) for using the same algorithm that solves (2.22).

2.3.5 Solving the Asymmetric System Cost Equation

First, equation (2.27) is solved using Horn & Schunck method and Thirions Demons

method. Then the resulting solutions are modified for inverse consistency.

2.3.5.1 CASE I: HORN-SCHUNCK (HS)

OPTICAL FLOW ALGORITHM

Equation (2.22),

(2.22)

may be solved by the standard methods of the calculus of variations. For simplicity, let's

solve (2.22) for the two-dimensional ( case, and then easily generalize to . Let

The functional

to be minimized is

The goal is to

find Using basic results from

calculus of variations36

and applying the Euler-Lagrange equations, we have (dropping the

subscript from E):

(2.42)

(2.43)

Where, for now, subscripts indicate partial derivatives; for example

expanding the terms in (2.42):

(2.44)

(2.45)

43

(2.46)

Combining (2.42), (2.44), (2.45), (2.46)

(2.47)

in an exactly analogous manner, we get from (2.43):

(2.48)

We use a finite difference approximation for the Laplacian,

where . For a

discussion of the Laplacian, see Morse & Feshbach,37

or Byron & Fuller.38

The value for

depends on the discretization of ; for convenience, we let

Solving for u and v:

and similarly

Rearranging the last two equations, we can write

44

So finally

(2.49)

and

(2.50)

To calculate the velocity for each image pixel, the averages of the velocity (deformation)

field is taken over a limited neighborhood of that pixel. All other quantities are evaluated at

the pixel location. Therefore the resulting matrix for the system is very sparse (a large

percentage of entries are zero). The matrix will also have three times (for 3D-images) the

number of rows and columns as there are pixel elements in the image. Thus a numerical

solution scheme is computationally much more efficient. The Gauss-Siedel method

converges rapidly. The new set of velocity estimates ( is computed from the

previous averages and the estimated gradients:

(2.51)

(2.52)

There are a few important things to note about the solutions (2.49), and (2.50). First, the

estimate at a point at iteration (n+1) does not depend on the estimate from the previous

iteration, n.

Second, at parts of the image where the gradient is zero, the velocity estimate will

only get contribution from the averages at neighboring points. There will be no local

information (provided by the gradient) to constrain the velocity of motion of brightness

pattern. Thus the values from the surrounding points will propagate into the zero-gradient

region. Thus, after a sufficient number of iterations, velocity estimates are filled in from the

boundary of a region of constant brightness. This is just the solution of Laplace's equation

with given boundary conditions. This is the same situation as in the propagation effects in the

solution of the diffusion equation for heat in a uniform flat plate.

45

Third, regarding the tightness of the smoothness constraint, when the brightness

(intensity values) of the image in a region is a linear function of image coordinates, the

gradient will be constant in that region, and motion along the direction at right angles to the

gradient will not be obtained correctly, but will be filled in from the boundary as in the zero-

gradient case. In general, the solution is most accurate in regions where the intensity gradient

is not too small and varies in direction from point to point. In that case information which

constrains both components of the optical flow velocity is then available in a relatively small

neighborhood. On the other hand, too abrupt of variations in intensity values will lead to

large errors in the estimates of the gradient due to under-sampling and aliasing.

Now we are ready to generalize the solution of (2.22) for general dimension d:

(2.53)

Where the motion is field at iteration k, and is averaged for each voxel in its

neighborhood. To solve the symmetric system cost function (2.41), for the incremental

motion fields, we need only replace in (2.53):

(2.54)

After all iterations are done, is , the desired solution for (2.41).

2.3.5.2 CASE II: DEMONS ALGORITHM

The Demons algorithm starts from the optical flow equation, which assumes temporal

constancy of brightness, or intensity values of the image sequence. I.e. grey values of objects

in the image don't change over time. Thus

(2.55)

This is equivalent to the 1st order Taylor expansion used above to arrive at (2.22), namely

(2.56)

but with replaced by , the gradient of the reference image. Here we interpret the

intensity difference as a partial time derivative:

(2.57)

for

46

We assume for simplicity that the image T is a deformed version of image R, i.e. we

assume a deformation process generating images at time such that:

. Thus any particle P in the image domain follows a path

, where denotes the location of the particle at time t, and where

the particle is identified by its position Returning to the optical flow equation

(2.55), and (2.57), and setting we obtain:

..

In particular for we obtain

Assuming that , the general solution , is given by:

(2.58)

where Thirion, in his demons approach, suggests using the

velocity vector with smallest magnitude. In other words, ignoring the component at right

angles to the image gradient ( :

(2.59)

Figure 2.7 illustrates geometrically the idea behind the Demons force (velocity).

But the computation of V becomes delicate whenever is close to zero; small

perturbations may lead to large (and inaccurate) values of V. To avoid this, an additional

regularization parameter is introduced, such that

(2.60)

In order to reduce the number of parameters, Thirion suggests taking

; ignoring the fact that occasionally (in such a case V= 0, but

is not well defined by equation (2.60)).

47

R

T

Slope = R-T / V

intensity

space

-V

x

v . grad (R) = R - T

grad(R)

Figure 2.7. Geometric interpretation of the optical flow equation (1D case).

An important point to note here is that the desired DVF we need to register I to J is

(2.61)

As in Figure 2.8, this is the displacement due to force applied by each demon to the moving

image. In the image domain , a "demon" is situated at each spatial position in the

reference image, where . Depending on the gradient and the image

difference.

, the demons induce a pushing force . The demon pushes the image T

at in the direction of , if (in this case, the two images do not overlap at

d, and the corresponding point in T is labeled an "outside" point) and according to

if (in this case, the two images overlap at d, and the corresponding point in T

is labeled as an "inside" point).

48

Figure 2.8. Thirion demons. Top: The reference image (called scene in Thirion's

terminology) with six demons on its contour (black dots), and the contour of a

deformable model (model or moving image). The arrows indicate movement

direction. The demons push the moving image inward (in direction of ) if the

scene and model overlap, and outward (in direction of if they do not overlap.

Middle and Bottom: I and J represent the moving and fixed images, respectively.

Pushing and pulling force by demon in a one-dimensional model.

49

The resulting motion field from (2.61) is typically not smooth. Thirion's Demons

approach is to use a low-pass filter (Gaussian) with a fixed variance The demons algorithm

exploits the fact that the Gaussian kernel ,

is the Green function (i.e. impulse response, or a fundamental solution) for the diffusion

equation, and approximates the stationary (steady-state) solution by successive convolution

with a Gaussian filter of suitable length. This will be clarified a little more below. 39, 40

2.3.4 Variational Interpretation of the Demons

Approach

If we consider the particular distance measure30

This term enters the system cost function along with a similar term from the regularizer (c.f.

equation (2.15)). The minimization problem for each term involves the vanishing of its 1st

variation, or perturbation along any direction . Thus for the distance measure ,

we calculate its Gateaux derivative:

(2.62)

to arrive at the 2nd

equation above, we expand, to 1st order,

where and is the inner

product on .

50

The term within the integral, , by Theorem 1, equation

(2.18), can now be seen as the force term in the variational setting. From (2.17), the Euler-

Lagrange equations are now

(2.63)

The Laplacian term arises from the smoother (regularization) term. Equation (2.63) is of the

form

(2.64)

where is a differential operator (e.g. Laplacian). This is a semi-linear partial differential

equation. A convenient numerical scheme to solve (2.64) is to put it in a fixed-point form:

. Start from an initial guess , and arrive at the solution iteratively by

(2.65)

Modifying (2.65) slightly,

(2.66)

Introducing an artificial time t, making the displacement

and setting where is a fixed time step, (2.66) can be put in the form of

the reaction-diffusion PDE: where is a differential operator (e.g. Laplacian). This is a

semi-linear partial differential equation. A convenient numerical scheme to solve (2.64) is to

put it in a fixed-point form

(2.67)

The steady state solution of (2.67) is the minimizer of the registration problem.

A numerical scheme may be based on (2.66). If we want to solve (2.67) with

respect to the whole space, i.e., then under mild conditions on the force

51

, it would be possible to obtain an analytic solution:40

if ,

then the convolution

(2.68)

is well defined almost everywhere, and is a distribution solution of (2.66), with

Here,

,

is the Gaussian kernel, and is a fundamental solution (Green's function, or impulse response)

of the heat (diffusion) equation . In order to solve (2.66) with respect to the

discrete, and bounded region of our image registration , we may approximate the Gauss

kernel by a Gauss filter of characteristic width .

This Gauss filter approach is what Thirion calls "Demons 1:a complete grid of

demons."18

52

CHAPTER THREE

APPLICATION OF DEFORMABLE IMAGE

REGISTRATION TO FOUR-DIMENSIONAL

TREATMENT PLANNING OF LUNG CANCER

Traditionally, the intra-fraction (between different fractions to same patient)

movement of non-small cell lung cancer (NSCLC) tumors is not measurable by CT scans.

3D-CTs are often acquired while the patient breathes freely, and therefore don't capture the

extent of respiratory motion and tissue deformation. In addition, breathing motion-induced

CT artifacts can lead to incorrect estimation of tumor position, size and shape.10

During

planning for radiation therapy, this movement is accounted for with a generic, population

based margin (e.g. 3 mm) added to the CTV (or ITV) during CT simulation scans. However,

without precise knowledge of radiation target size, position, and motion, the prescribed

margins might be either too large for some patients, or insufficient for others. Four-

dimensional computed tomography scans (4DCT) have recently been introduced and are

becoming widely used in clinical practice. 4DCT provides valuable tumor motion

information and significant reduction in image artifacts.

3.1 4DCT, MAXIMUM INTENSITY PROJECTION CT, AND

TREATMENT PLANNING

As introduced previously in Chapter 1, a four-dimensional computed tomography

image data set (4DCT) is a CT scan that, in addition to spatial information, incorporates

temporal information of the anatomy. Thus, it provides information on the motion of tissue

voxels during the scan. In acquiring a 4DCT, at each couch position, the anatomy is scanned

10 - 15 times throughout the respiratory cycle. The scanning times are synchronized with a

respiratory signal that tracks the motion of the anterior abdominal wall during respiration.

During the image reconstruction process, images from corresponding respiratory positions

(based on the respiratory timing signal) from all couch positions are binned together. This

results in multiple CT images each representing different a breathing phase, closely spaced in

53

time and thus capturing, to a good approximation, the motion of the internal anatomy.

Usually, 10 or more images are selected to make a 4DCT, and are numbered in increments of

10 from 0 to 90. So, for example, the 0% phase CT corresponds to end of inhale (EOI) and

50% phase corresponds to end of exhale (EOE) position, and so on.

The most obvious, and robust method of creating an ITV from a 4DCT dataset, is to

create 10 CTVs from the 10 phases of the 4DCT, and then an ITV is created from the union

of the individual single-phase CTVs. Despite its advantage, this method is very time

consuming as one must contour the CTV and other organ contours on 10 images instead of

one. Another alternative method, used currently in clinical practice, is to create a Maximum

Intensity Projection (MIP) image. 41- 43

MIP is a single image created from the 4DCT dataset,

and exploits the fact that tumor tissue is of higher density than normal lung tissue. It is

simply the image created by choosing for each image pixel's intensity value (i.e. CT number),

the maximum intensity pixel from the corresponding 10 pixels in the 4DCT. This reflects the

highest density value encountered in each pixel throughout the respiratory cycle (Figure 3.1).

It is a one CT scan that should take no longer to contour than an ordinary 3D helical CT scan.

Figure 3.1 shows a schematic outlining the contour of a tumor, and illustrates the definition

of MIP as well as Minimum Intensity Projection, and Mean Intensity Projection (Projection

here implies a certain angle, or ray, along which the CT volume is viewed and max (min,

mean..Etc) intensities are selected).

The generation of MIP scans has been used frequently to generating ITVs from 4DCT

datasets, and also for contouring planning volumes for organs at risk. The purpose of this

thesis work is to investigate the reliability of this procedure by analyzing treatment plans of

actual patients and recalculating the received dose using deformable image registration (DIR)

algorithms. By exploiting DIR between the different 4DCT phase images, we can arrive at

better estimate of the actual radiation dose to the patient than the one calculated based on

MIP planning, as outlined next.

3.2 APPLICATION OF DEFORMABLE IMAGE

REGISTRATION TO 4DCT RADIATION TREATMENT

PLANNING

Once a conventional patient plan is done on the treatment planning system, based on

a MIP CT dataset, 10 additional plans, based on the 4DCT data set, are created. This is done

54

Figure 3.1. Pixel intensity-based projection protocols from 4DCT of a moving tumor,

used to detect high to low intensity anatomic structures (a) tumor contours from all

separate 4DCT phases, (b) Maximum intensity projection (MIP) represents where the

tumor is present at sometime in the respiratory cycle, (c) Minimum intensity

projection (MinIP) represents the space, in the respiratory cycle, which the tumor

always occupies, (d) Mean intensity projection. Source: R. W. M. Underberg, F.

Lagerwaard, B. Slotman, J. P. Cuijpers, and S. Senan, “Use of maximum intensity

projections (MIP) for target volume generation in 4DCT scans for lung cancer,” Int.

J. Radiat. Oncol. Biol. Phys. 63, 253-260 (2005).

by keeping all the MIP plan parameters (beams, gantry angles, dose, MU's..etc) the same,

and applying them to each phase 3DCT of the 4DCT dataset, to create multiple plans. This

will produce dose distribution images, one for each phase plan. These dose distributions

share the same external parameters, but doses at corresponding spatial locations correspond

to doses to different tissue voxels due to respiration movement. Thus they need to be

deformed back to some reference position of the anatomy, before they can be added together

to produce the total dose.

Next, all 4DCT images are registered to the reference phase CT (usually the EOE, or

50% phase) using our DIR algorithm. The algorithm was written and run in MATLAB. The

result of the registration, for each phase of the 4DCT, is a set of vectors (one for each voxel

element) that indicate the coordinate displacement in each direction that will bring the

corresponding image into congruence with the reference image, as defined in Equation (2.20)

of Chapter 2 above:

where R refers to the 50% phase (fixed, or reference) image, T to the phase image being

registered (moving image), and V is the resulting displacement, or deformation, vector field

(DVF).

55

Once the DVF is obtained, it can be used to deform the resulting dose distribution

images from all 4DCT individual phase plans. For example, if the 4DCT has (n) CT volume

datasets, R, T1 , …..Tn-1 , then n dose distribution images, , are generated

using the original plan parameters. The DIR results in (n-1) DVFs, and the

total dose at position x is calculated as follows:

This total dose can now be used to calculate the dose delivered to any predefined anatomic

structure such as the PTV, lungs, heart, spinal cord….etc. Since the contours of these

structures are known, and can be extracted easily from the original plan's DICOM files (files

ending in .RS), as well as from the reference phase plan DICOM file. The contour files

contain the x, y, and z-coordinates of all the points that make up a certain contour. These

coordinates are then used to create a binary mask that can be used to extract the dose to that

structure from the total dose D(x). Dose volume histograms (DVH) are then calculated and

plotted for each structure of interest, and the 4D plan is compared to the MIP plan.

3.3 IMAGE REGISTRATION RESULTS

Some results of deformable image registration are shown for a lung cancer patient in

Figures 3.2, 3.3, and 3.4.

Figure 3.2. A coronal view of lung anatomy at two respiratory phases for Patient 1

from a 4DCT. Left: 10% phase (near EOI). Right: 50% reference phase (EOE).

56

Figure 3.3. The images of Figure 3.2 (p. 55) registered (10% phase registered to 50%

phase) using inverse-consistent Demons algorithm. Left: the deformed 10% phase CT

image is now very similar to the reference image. Right: 50% reference phase- image.

Figure 3.4. Difference image before and after DIR, for the images in Figure 3.2 (p. 55),

and Figure 3.3. Left: Difference before. Right: Difference after.

Table 3.1 shows the values set for the different registration parameters used in the

algorithm. So, for instance, 5 levels of resolution were used, which means that the original

images were halved in size and resolution successively five times. The registration starts

from the lowest (i.e. coarsest) level to the highest. The number of iterations/passes is

decreased as the registration goes to higher resolution to keep computational time

manageable. The stopping criteria for the registration iterations is when the DVF magnitude

is less than or equal to 0.002 voxels. The Gaussian kernel used to convolve with the DVF has

a standard deviation 3 voxels.

Figures 3.5, 3.6, and 3.7 show similar results for a second lung cancer patient. The

registration shown was done using the original optical flow algorithm of Horn & Schunck.

57

Table 3.1. DIR Parameters Used in the Registration Shown in Figures 3.2-3.4

(pp. 55-56)

Stages to

use: 5

Multigrid Stage

High resolution……………………………………low resolution

1 2 3 4 5

# of passes 2 2 2 2 3

10 30 80 100 80

Stop

condition 1

0.002 OK

Stop

condition 2

0.0001 Cancel

σiter (1 to 5) 3

σpass (after

each pass, 0

to 5)

0.5

Multigrid Filter (1=Gaussian, 2=Max, 3=Min, 4=Max Abs): 1

58

Figure 3.5. A coronal view of lung anatomy at two respiratory phases for Patient 2,

from a 4DCT. Left: 0% phase (EOI). Right: 50% reference phase (EOE).

Figure 3.6. The images of Figure 3.5 (p. 58) registered (0% phase registered to 50%

phase) using inverse-consistent optical flow algorithm. Left: The deformed 0% phase

CT image is very similar now to the reference image. Right: 50% reference phase-

image.

-

Figure 3.7. Difference image before and after DIR, for the images in Figure 3.5,

and Figure 3.6 . Left: Difference before. Right: Difference after.

59

Table 3.2 shows the registration parameters used for the above registration results.

Table 3.2. DIR Parameters Used in the Registration Shown in Figures 3.5-3.7

(p. 58)

Stages to

use: 5

Multigrid Stage

High resolution……………………………………low resolution

1 2 3 4 5

# of passes 5 5 10 10 10

30 60 80 100 200

Stop

condition 1

0.002 OK

Stop

condition 2

0.0001 Cancel

σiter (1 to 5) 3

σpass (after

each pass, 0

to 5)

0.5

Multigrid Filter (1=Gaussian, 2=Max, 3=Min, 4=Max Abs): 1

3.4 FOUR-DIMENSIONAL ANALYSIS OF MIP-BASED

TREATMENT PLANS

As outlined in the beginning of the chapter, 4-D analysis of the MIP plan for each

patient was carried out using the DVF's resulting from deformable image registration. Dose

volume histograms (DVHs) for important structures are shown in Figure 3.8-3.12. Each

DVH figure shows two DVH plots; one plot is the original MIP plan DVH and the other is

the DVH that results from 4D analysis of the MIP plan. Figure 3.8 shows that the lungs are

receiving a slightly higher dose than indicated by the original plan. For example, the 4D plan

shows 20% of lung volume is receiving at least a 1000 cGy dose, whereas the MIP plan has

16% of lung volume receiving at least this dose. Figure 3.9 is a similar plot for another

patient. Here the difference between the two DVH curves is even less significant. Figure 3.10

compares the two DVH plots for the PTV. The two DVH curves are almost the same except

60

Figure 3.8. Dose volume histogram for Patient 1 case. The green curve is the

DVH for the bilateral lung from the actual treatment plan. The red curve shows

the same DVH according to 4D calculations of delivered dose, based on

deformable image registration of 4DCT images.

61

Figure 3.9. Dose volume histogram for Patient 2 case. The green curve is the DVH for

the bilateral lung from the actual treatment plan. The red curve shows the same DVH

according to 4D calculations of delivered dose, based on deformable image registration

of 4DCT images.

62

Figure 3.10. Dose volume histogram for Patient 2 case. The green curve is the DVH

for the PTV from the actual treatment plan. The red curve shows the same DVH

according to 4D calculations of delivered dose, based on deformable image

registration of 4DCT images.

63

Figure 3.11. Dose volume histogram for Patient 2 case. The green curve is the DVH

for the left lung from the actual treatment plan. The red curve shows the same DVH

according to 4D calculations of delivered dose, based on deformable image

registration of 4DCT images.

64

Figure 3.12. Dose volume histogram for Patient 2 case. The green curve is the DVH

for the right lung from the actual treatment plan. The red curve shows the same

DVH according to 4D calculations of delivered dose, based on deformable image

registration of 4DCT images.

for slight under-dosage to the target volume in the dose range 3600-4000 cGy. Figures 3.11

(p. 63) and 3.12 are for the left/right lungs, respectively.

3.5 OPTICAL FLOW VS. DEMONS FOR LUNG CT

The two algorithms that were used in this study, namely the Demons and H.S optical

flow, were compared quantitatively to determine which is more suitable for registration of

lung CT images.

The results show that the original optical flow algorithm of Horn & Schunck (HS)

was much faster (by up to a factor of 4) and produced comparable or better accuracy than

Demons based on comparison of the subtracted images and MSE between reference and

registered images. The algorithms were allowed to run the same total number of iterations.

The multiple pass approach is only beneficial in the OF algorithm, but is redundant in the

65

Demons and does not result in any improvement since there really is no distinction between

pass and iteration; images are deformed, and thus re-sampled, per iteration and per pass

(c.f. Appendix). The improved speed in OF is due to the fact that the deformed image is not

re-sampled and updated at each iteration (only at each pass), while as mentioned earlier this

is done at each iteration or pass in the Demons algorithm. The improvement in accuracy is

due to the very different ways smoothness of the DVF is implemented in each algorithm.

Figures 3.13-16 show the results for one CT slice along with summaries of pertinent

parameters.

Figure 3.13. Demons DIR for a slice of 0% phase of a 4DCT of lung. Left: Difference

between CT image slices before DIR (MSE=2.9 x 105); Right: Difference after DIR

(MSE= 1.3 x 105). Computation time was 2.82 hours.

Figure 3.14. H.S OF DIR for 0% phase. Left: Difference before DIR (MSE=2.9 x 10

5);

Right: Difference after DIR (MSE= 1.4 x 105). Computation time was 45 minutes.

66

Figure 3.15. Demons DIR for 20% phase. Left: Difference before DIR (MSE=2.1 x 10

5);

Right: Difference after DIR (MSE= 1.2 x 105). Computation time was 2.1 hours.

Figure 3.16. H.S OF DIR for 20% phase. Left: Difference before DIR (MSE=2.1 x 10

5);

Right: Difference after DIR (MSE= 1.1 x 105). Computation time was 35 minutes.

Thus for registering 4DCT lung images, the original H.S. optical flow algorithm is

recommended over the Demons algorithm due to increased speed of computation and better

accuracy.

67

CHAPTER 4

CONCLUSION

In this thesis, I have presented a detailed look at deformable image registration,

particularly in theory, and applied it to actual clinical cases of lung cancer. I have also

investigated the applicability of this image processing technique to dose calculations and

accumulation in the clinic. The results showed the potential clinical importance of this

application for adaptive radiation therapy.

4.1 RESULTS FROM 4D ANALYSIS OF SBRT LUNG

CANCER CASES

The preliminary results show that current SBRT plans for lung cancer, which are

based on MIP CTs, are able to provide good dose conformity to the target compared with full

4D-planning based on the entire 4DCT data set. However, not enough patient cases were

studied, and a more detailed study is needed as well as large number of patient cases.

4.2 ALGORITHMS PERFORMANCE

The two algorithms used to carry out deformable image registration; the Demons and

the original Horn-Schunck (H.S) were both well suited for this application. However, the

results show that H.S algorithm works better for lung CT cases both in terms of computation

time and registration accuracy.

4.3 FINAL DISCUSSION AND FUTURE DIRECTION

Deformable image registration is difficult to implement and is not yet accepted

clinically. The research in this field is still ongoing, and it's hoped that this study contributes

in a small way to the improvement of clinical implementation of DIR, and ultimately to

patients clinical outcome.

MATLAB has been used in the implementation of our algorithms, and this poses a

limitation in both speed and memory. But since this work is in the academic research stage,

and was not aimed at clinical application, MATLAB provided an easy and intuitive access to

68

technical computing. The next logical step is to implement deformable image registration in

C or C++, for example.

69

REFERENCES

1 C. A. Perez, L. Brady, and J. L. Roti, “Overview,” in Principles and Practice of Radiation

Oncology, edited by C. A. Perez, L. W. Brady (Lippincott-Raven, Philadelphia, PA,

1998), pp. 1-78.

2 E. J. Hall, Radiobiology for the Radiologist, 5th ed. (Lippincott, Williams & Wilkins,

Philadelphia, PA, 2000).

3 C. T. Leondes, Biomechanical Systems Technology: Computational Methods (World

Singapore Scientific Publishing Company, Singapore, 2007).

4 J. Van Dyk, “Radiation oncology overview,” in The Modern Technology of Radiation

Oncology, edited by J. Van Dyk (Medical Physics Publishing, Madison, WI, 1999),

pp. 1-17.

5 J. Van Dyk and J. S. Taylor, “CT simulators,” in The Modern Technology of Radiation

Oncology, edited by J. Van Dyk (Medical Physics Publishing, Madison, WI, 1999),

pp. 131-168.

6 F. M. Khan, The Physics of Radiation Therapy, 4th ed. (Lippincott, Williams & Wilkins,

Philadelphia, PA, 2010), pp. 501-514.

7 I. A. Feldkamp, L. C. Davis and J. W. Kress, “Practical cone-beam algorithm,” J. Opt. Soc.

Am. A. 1, 612-619 (1984).

8 P. J. Keal et al., “The management of respiratory motion in radiation oncology, report of

AAPM Task Group 76,” Med. Phys. 33, 3874-3900 (2006).

9 C. W. Stevens et al., “Respiratory-driven lung tumor motion is independent of tumor size,

tumor location, and pulmonary function,” Int. J. Radiat. Oncol. Biol. Phys. 51, 62-68

(2001).

10 J. H. Lewis and S. B. Jiang, “A theoretical model for respiratory motion artifacts in free-

breathing Ct-Scans,” Phys. Med. Biol. 54, 745–755 (2009).

11 International Commission on Radiation Units and Measurements (ICRU), Report 62:

Prescribing, Recording and Reporting Photon Beam Therapy (Supplement to ICRU

Report 50) (International Commission on Radiation Units and Measurements,

Bethesda, MD, 1999).

12 S. H. Benedict et al., “Stereotactic body radiation therapy: The report of AAPM task group

101,” Med. Phys. 37, 4078-101 (2010).

13 M. L. Kessler, “Image registration and data fusion in radiation therapy,” Br. J. Radiol. 79,

S99–108 (2006).

14 National Electrical Manufacturers Association, Digital Imaging and Communications in

Medicine, Part 3: Information Object Definitions (National Electrical Manufacturers

Association, Rosslyn, VA, 2009).

70

15 F. Bookstein, “Principal warps: Thin-plate splines and the decomposition of

deformations,” IEEE Trans. Pattern Anal. Machine Intell. 11, 567–85 (1989).

16 W. Lu et al., “Fast free-form deformable registration via calculus of variations,” Phys.

Med. Biol. 49, 3067–87 (2004).

17 G. E. Christensen et al., “Image-based dose planning of intracavitary brachytherapy:

Registration of serial-imaging studies using deformable anatomic templates,” Int. J.

Radiat. Oncol. Biol. Phys. 51, 227–43 (2001).

18 J. P. Thirion, “Image matching as a diffusion process: An analogy with Maxwell’s

demons,” Med. Image Anal. 2, 243-260 (1998).

19 H. Wang et al., “Implementation and validation of a three-dimensional deformable

registration algorithm for targeted prostate cancer radiotherapy,” Int. J. Radiat. Oncol.

Biol. Phys. 61, 725–35 (2005).

20 K. K. Brock et al., “Accuracy of finite element model (FEM)-based multi-organ

deformable image registration,” Med. Phys. 32, 1647–59 (2005).

21 M. L. Kessler, S. Pitluck, P. L. Petti, and J. R. Castro, “Integration of multimodality

imaging data for radiotherapy treatment planning,” Int. J. Radiat. Oncol. Biol. Phys.

21, 1653–67 (1991).

22 J. M. Balter, C. A. Pelizzari, and G. T. Chen, “Correlation of projection radiographs in

radiation therapy using open curve segments and points,” Med. Phys. 19, 329–34

(1992).

23 K. A. Langmack, “Portal imaging,” Br. J. Radiol. 74, 789–804 (2001).

24 C. A. Pelizzar, G. T. Chen, D. R. Spelbring, and R. R. Weichselbaumm, “Accurate three-

dimensional registration of CT, PET, and/or MR images of the brain,” J. Comput.

Assist. Tomogr. 13, 20–6 (1989).

25 M. van Herk and H. M. Kooy, “Automatic three-dimensional correlation of CT-CT, CT-

MRI, and CT-SPECT using chamfer matching,” Med. Phys. 21, 1163–78 (1994).

26 J. Kim and J. A. Fessler, “Intensity-based image registration using robust correlation

coefficients,” IEEE Trans. Med. Imaging 23, 1430–44 (2004).

27 C. R. Meyer et al., “Demonstration of accuracy and clinical versatility of mutual

information for automatic multimodality image fusion using affine and thin-plate

spline warped geometric deformations,” Med. Image Anal. 1, 195–206 (1997).

28 P. Viola and W. M. Wells, “Alignment by maximization of mutual information,” Int. J.

Comp. Vision 24, 137–54 (1997).

29 F. Maes et al., “Multimodality image registration by maximization of mutual information,”

IEEE Trans. Med. Imaging 16, 187–98 (1997).

30 J. Modersitzki, Numerical Methods for Image Registration (Oxford University Press, New

York, NY, 2004).

31 J. Modersitzki, FAIR: Flexible Algorithms for Image Registration (Society for Industrial

and Applied Mathematics, Philadelphia, PA, 2009).

71

32 B. K. P. Horn and B. G. Schunck, “Determining optical flow,” Artif. Intell. 17, 185–203

(1981).

33 J. P. Thirion, Fast Non-Rigid Matching of 3D Medical Images. (Institut National de

Recherche en Informatique et en Automatique, Antibes, France, 1995).

34 D. Yang, H. Li, D. Low, J. Deasy, and I. El Naqa, “A fast inverse consistent deformable

image registration method based on symmetric optical flow computation,” Phys.

Med. Biol. 53, 6143–6165 (2008).

35 D. Rueckert, P. Aljabar, R. Heckemannn, J. Hajnal, and A. Hammers, “Diffeomorphic

registration using BSplines,” MICCAI 2006 Lect. Notes Comp. Sci. 4191, 702–9

(2006).

36 J. Mathews and R. L. Walker, Mathematical Methods of Physics, 2nd ed. (Benjamin

Cummings, Menlo Park, CA, 1970).

37 P. M. Morse and H. Feshbach, Methods of Theoretical Physics, vol 1. (McGraw Hill, New

York, NY, 1953).

38 J. R. Byron and R. W. Fuller, Mathematical of Classical and Quantum Physics (Dover

Publications, Mineola, NY, 1992).

39 G. B. Folland, Introduction to Partial Differential Equations, 2nd ed. (Princeton

University Press, Princeton, NJ, 1995).

40 P. Dennery and A. Krzywicki, Mathematics for Physicists (Dover Publications, Mineloa,

NY, 1996).

41 R. Muirhead, S. McNee, C. Featherstone, K. Moore, and S. Muscat, “Use of maximum

intensity projections (MIPs) for target outlining in 4DCT radiotherapy planning,” J.

Thoracic Oncol. 3, 1433-1438 (2008).

42 R. W. M. Underberg, F. Lagerwaard, B. Slotman, J. P. Cuijpers, and S. Senan, “Use of

maximum intensity projections (MIP) for target volume generation in 4DCT scans for

lung cancer,” Int. J. Radiat. Oncol. Biol. Phys. 63, 253-260 (2005).

43 M. Ezhil et al., “Determination of patient-specific intra-fractional respiratory motion

envelope of tumors from maximum intensity projections of 4DCT datasets,” Int. J.

Radiat. Oncol. Biol. Phys. 69, S2511 (2007).

72

APPENDIX

MATLAB PSUEDO CODE

73

Multi-Resolution Registration Scheme

Output:

DVF vector arrays: (e.g. DVF.x is the x-component of DVF, and indicates the x-

component of the movement at each image pixel (x,y,z), ….etc.)

IDVF: the inverse of DVF.

Dimg1: deformed input image, img1, by applying DVF to img1.

Input:

img1: the target image (input image to be deformed to match the reference image)

img2: the reference image

steps : the number of resolution levels, set default to 4.

loops_in_step: number of passes in each step. Set default to [6 6 6 4 2]

maxiter: number of iterations in each loop. Set default to [20 40 60 80 100]

stop: the DVF threshold to stop iterations. set default to 0.002 (voxels)

method: 1 (optical flow), or 2 (demons)..

Step 1:

max_motion = 0.4; (to later limit the DVF magnitude to 0.4 voxels, as discussed in Yang)

generate reduced versions of images 1 and 2, up all needed levels, using, for example, the

Gaussian pyramid scheme. For example, if using 4 levels of resolution for the registration,

generate img1_2, img1_4, img1_8, img2_2, img2_4, img2_8, by successively running

Gaussian pyramid rourine.

Step 2: starting the multi-resolution registration

for step = 1 : steps, do the following:

real_step = steps + 1 - step; (this gives the starting coarser resolution level)

print (starting step step);

switch real_step

case 5

im1 = img1_16; im2 = img2_16;

case 4

im1 = img1_8; im2 = img2_8;

case 3

im1 = img1_4; im2 = img2_4;

74

case 2

im1 = img1_2; im2 = img2_2;

case 1

im1 = img1; im2 = img2;

end

im1 = single (im1); im2 = single (im2);

normalize images:

max_voxel = max(max(im1(:)), max(im2(:)));

im1 = im1/max_voxel; im1 = im2/max_voxel;

dim1 = size(im1); dim2 = size(im2);

if length(dim1= 2)

dim1 = [dim1 1];

end

if length(dim2= 2)

dim1 = [dim2 1];

end

if step ==1 (when step =1, it's the start of the registration, and no DVF has been

calculated and stored yet), initialize DVF and inverse DVF structures to zero:

print(initializing motion field)

DVF.y = zeros(dim2,'single'); DVF.x = zeros(dim2,'single'); DVF.z = zeros(dim2,'single');

IDVF.y = DVF.y; IDVF.x = DVF.x; IDVF.z = DVF.z;

Dimg1 = im1; Dimg2 = im2;

else (else the DVF and inverse DVF from the last step need to be double-sampled (and

multiplied by 2) for use in the next level, and also deform target and reference images using

the resulting DVF and IDVF, respectively)

print('double-sampling motion field by interpolation….')

upsample(DVF); upsample(IDVF);

if ~isequal(size(DVF),dim2)

DVF = DVF = DVF(1:dim2(1), 1:dim2(2), 1:dim2(3));

75

end

if ~isequal(size(IDVF),dim2)

IDVF = IDVF(1:dim2(1), 1:dim2(2), 1:dim2(3));

end

print('double-sampling motion fields is finished')

Dimg1 = DeformImage(im1, DVF/2); Dimg2 = DeformImage(im2, IDVF/2);

print('image deformation is finished')

end

for loop = 1 : loops_in_step (real_step) START OF MULTI-PASS LOOP

maxiter = maxiter(real_step);

switch method

case 1 (Horn-Schunck Optical Flow method)

print ('Starting Horn_Schunck original optical flow method…')

DVF1 = OpticalFlow(Dimg1, Dimg2, voxel_size, maxiter, stop);

case 2 (Demons method)

print ('Starting Demons method…')

DVF1 = Demons(Dimg1, Dimg2, voxel_size, maxiter, stop);

end

DVF1 = CheckMagnitude1(DVF1, max_motion); (function

CheckMagnitude1 restricts the magnitude of the DVF to be less than the

specified threshold, max_motion)

DVF1 = GaussianLowpass(DVF1, sigma); (smoothing the DVF with a

Gaussian filter, OPTIONAL)

if loop = 1 (1st pass)

DVF_this_step = DVF1; IDVF_this_step = - DVF1;

else (compose just-computed DVF1 with previous DVF_this_step)

print('computing result motion field for this pass…')

DVF_this_step = DVF1 + DeformImage(DVF_this_step, DVF1);

IDVF_this_step = - DVF1 + DeformImage(IDVF_this_step, - DVF1);

end

76

DVF_this_step = GaussianLowpass(DVF, sigma); (smoothing the DVF

with a Gaussian filter, OPTIONAL)

IDVF_this_step = GaussianLowpass(IDVF, sigma); (smoothing the IDVF

with a Gaussian filter, OPTIONAL)

print('deforming images….')

Dimg1 = DeformImage(single(im1), (DVF + DVF_this_step));

Dimg2 = DeformImage(single(im2), (IDVF + IDVF_this_step));

print('deforming images is finished')

[MI NMI CC MSE] = CompareImages(Dimg1, Dimg2); (obtain

statistical information like mutual information, cross correlation…etc)

if method == 2

break; ( break out of the multi-pass loop, since it's redundant in Demons

method)

end

end (END OF MULTI-PASS LOOP)

if step == 1

DVF = DVF_this_step; IDVF = IDVF_this_step;

else

print ('computing total motion field for current step…..')

DVF = DVF_this_step + DeformImage(DVF, DVF_this_step);

IDVF = IDVF_this_step + DeformImage(IDVF, IDVF_this_step);

end

OPTIONAL: Smooth with another Gaussian low pass filter:

DVF = GaussianLowpass(DVF, sigma);

IDVF = GaussianLowpass(IDVF1, sigma);

END END OF MULTI-RESOLUTION LOOP

Step 3 Compute final motion fields

IDVF0 = InvertMotionField(IDVF); DVF0 = InvertMotionField(DVF);

DVF = IDVF0 + DeformImage(DVF, IDVF0); clear IDVF0;

77

IDVF = DVF0 + DeformImage(IDVF, DVF0); clear DVF0;

Step 4 Compute deformed image 1 and deformed image 2

Dimg1 = DeformImage(single(im1), DVF);

Dimg2 = DeformImage(single(im2), IDVF); OPTIONAL

Return

function DVF = CheckMagnitude(DVF, threshold) This function limits the magnitude of

DVF vector to be less than a specified threshold.

dvf = sqrt(DVF.x^2 +DVF.y^2 +DVF.z^2);

if max(dvf(:)) <= threshold

return;

end

dvf2 = min(dvf,threshold);

factor = dvf2./(dvf + (dvf == 0)); this factor is always < 1

DVF.x = DVF.x .* factor; DVF.y = DVF.y .* factor; DVF.z = DVF.z .* factor;

return

78

Optical Flow Image Registration Algorithm

Output: DVF.y, DVF.x, DVF.z; the three spatial components of the displacement vector

field which describes the image deformation.

Input: R (reference, or fixed image), T (template, or moving image), voxel_size, max_iter,

stop)

Step one:

initialize some variables:

max_intensity = max(max(R(:)), max(T(:)));

if ~exist('max_iter', 'var') || isempty(max_iter)

max_iter = 20;

end

if ~exist('stop', 'var') || isempty(stop)

stop = 0.002;

end

if ~exist('voxel_size', 'var') || isempty(voxel_size)

voxel_size = InputVoxelSize(); (InputVoxelSize is a custom function that takes

voxel dimensions and returns voxel size in units of the smallest voxel dimension)

end

voxel_size = voxel_size/min(voxel_size);

dim1 = size(R); dim2 = size(T);

if length(dim1= 2)

dim1 = [dim1 1];

end

if length(dim2= 2)

dim1 = [dim2 1];

end

y = 1 : dim1(1); x = 1 : dim1(2); z = 1:dim1(3);

DVF.y = zeors(dim1, 'single'); DVF.x = zeors(dim1, 'single'); DVF.z = zeors(dim1, 'single');

Step 2

calculate the image gradients using (ImaeGradientByMask):

[dy1, dx1, dz1] = ImageGradientByMask(R);

79

[dy2, dx2, dz2] = ImageGradientByMask(T);

Next, it's necessary to multiply the computed gradients with the appropriate ratios, if voxels

are not of the same dimension in all directions x, y, and z.

dy1 = dy1 * voxel_size(1); dx1 = dx1 * voxel_size(2); dz1 = dz1 * voxel_size(3);

dy2 = dy2 * voxel_size(1); dx2 = dx2 * voxel_size(2); dz1 = dz2 * voxel_size(3);

dy = (dy1 + dy2); dx = (dx1 + dx2); dz = (dz1 + dz2);

Id = R - T;

alpha = 0.2 * max_intensity;

sum1 = alpha*aplha + (dx.^2 + dy.^2 + dz.^2);

max_motion_per_iteration = 0.5;

Step 3

Start optical flow calculation:

for I = 1 : max_iter

[vy vx vz] = HSVelocityAverage(DVF.y, DVF.x, DVf.z);

(HSVelocityAverage calculates the average DVF around a voxel)

vy0 = DVF.y; vx0 = DVF.x; vz0 = DVF.z;

sum2 = dy .* vy + dx .* vx + dz .* vz;

Idy = Id .* dy; Idx = Id .* dx; Idz = Id .* dz;

DVF.y = vy - (dy .* sum2 + Id .* Idy)./sum1;

DVF.x = vx - (dx .* sum2 + Id .* Idx)./sum1;

DVF.z = vz - (dz .* sum2 + Id .* Idz)./sum1;

clear sum2;

deltaV = sqrt((vy0-DVf.y)^2 + (vx0-DVF.x)^2 + (vz0-DVf.z)^2); limiting the DVF update

per iteration

if max(deltaV(:)) > max_motion_per_iteration

indx = find (deltaV > max_motion_per_iteration);

deltaVy = DVF.y(indx) – vy0(indx);

deltaVx = DVF.x(indx) – vx0(indx);

deltaVz = DVF.z(indx) – vz0(indx);

deltaVy = deltaVy * max_motion_per_iteration/deltaV(indx);

deltaVx = deltaVx * max_motion_per_iteration/deltaV(indx);

80

deltaVz = deltaVz * max_motion_per_iteration/deltaV(indx);

DVF.y(indx) = vy0(indx) + deltaVy;

DVF.x(indx) = vx0(indx) + deltaVx;

DVF.z(indx) = vz0(indx) + deltaVz;

deltaV(indx) = max_motion_per_iteration;

clear deltaVy deltaVx deltaVz indx;

end

clear vy0 vx0 vz0;

maxV = max(deltaV(:));

print('iteration iter : motion mean = mean(deltaV(:)), maximum motion = maxV')

if maxV <= stop break out of main for loop if DVF is less than stop threshold

break

end

end End of optical flow calculation

81

The Gaussian Pyramid Scheme

Introduction

The scheme is based on the work of Burt & Adelson and summarized in their 1983

paper (Laplacian Pyramid as a compact image code). In that paper, Gaussian pyramids are

used as part of a proposed image compression scheme, the details of which does not terribly

concern us here; albeit it's very profitable to read the entire paper. We only need to generate

the Gaussian pyramid levels according to equation (1), page 533, of that publication, by

repeated local averaging using a 5x5 filter, w:

Here, is the reduced version of image , where,

. N is the number of levels in the pyramid, and the

numbers are such that the original dimensions of the image are such

that:

Put more simply, we are assuming that C and R are odd integers.

To implement in Matlab, we need to slightly adjust the above equation, since we can

only index arrays with indices . Since we want to construct our weighting

filter w(m,n) such that m=n=5 (or in 3D, m=n=q=5, for w(m,n,q)), we need to offset the m=-

2 term (0 -2 =-2) so that it is =1. Thus we write instead:

for .

82

Additionally, to keep the averaging process correct at the boundaries, we pad the

original image at the boundaries with two additional rows and columns on each side of the

boundary, in each dimension.

So, for the 2D case, along the rows dimension, we set:

image = [image(1, :); image(1,:); image; image(R,:); image(R,:)]; and along the columns

direction:

image = [image(:,1) image(:,1) image image(:,C) image(:,C)]

For 3D the syntax is a little different, but the idea is exactly the same (see code

below).

The choice for the 1D weighting filter parameters are chosen as derived in the paper:

w1 = [0.05 0.25 0.4 0.25 0.05].

For higher dimensions, we make the filer separable, composed of multiple 1D filters.

So for 2D,

w2 = w1' * w1, which is a 5x5 matrix, so w2(m,n) = w1(m).w1(n)

For 3D, w3 is a 5x5x5 array:

w3 (m,n,q)=w1(m).w1(n).w1(q).

Matlab pseudo code (3D images):

Input: original image, I1

Output: Reduced image, I2

Step 1:

define the 1D weighting filter w1 = [0.05 0.25 0.4 0.25 0.05];

define input image dimension: dim1 = size(I1);

define output image dimension: dim2 = ceil (dim1 * 0.5);

Initialize the output image array to zeros: I2 = zeros(dim2,class(I1));

m = [-2 : 2]; n = m;

Step 2:

Initialize 3D filter: w3 = ones(5,5,5);

for i = 1 : 5, do the following:

w3(i,:,:) = w3(i,:,:)*w1(i);

w3(:,i,:) = w3(:,i,:)*w1(i);

83

w3(:,:,i) = w3(:,:,i)*w1(i);

Step 3

Pad the boundaries:

Initialize the padded image IP = zeros(dim1+4,class(I));

Padded image less the padding on the lower end (left, bottom, back), is set equal to

original image I1:

IP(3:2+dim(1),3:2+dim(2),3:2+dim(3)) = I1;

Now set the padding on the lower ends: IP(1,:,:)=IP(3,:,:); IP(2,:,:)=IP(3,:,:);

IP(:,1,:)=IP(:,3,:);IP(:,2,:)=IP(:,3,:); IP(:,:,1)=IP(:,:,3);IP(:,:,2)=IP(:,:,3);

Set the padding on the higher ends: IP(end,:,:) = IP(end-2,:,:); IP(end-1,:,:) = IP(end-

2,:,:);

IP(:,end,:) = IP(:,end-2,:); IP(:,end-1,:) = IP(:,end-2,:);

IP(:,:,end) = IP(:,:,end-2); IP(:,:,end-1) = IP(:,:,end-2);

I1 = IP; clear IP;

Step 4 implementing the Gaussian pyramid :

for k = 0 : dim2(3)- 1

for j = 0 : dim2(2) -1

for i = 0 : dim2(1) -1

A = I1(2*i+m+3,2*j+m+3,2*k+m+3).*Wt3; note: this is

84

element-wise (.*), not matrix multiplication (*)

I2(i+1,j+1,k+1) = sum(A(:));

end

end

end

Calculating the Gradient of a 3D Image

The gradient of a 3-dimensional image is calculated by using a simple convolution

with one-dimensional filters, or masks, in each dimension. The mask chosen by default is

[-1 8 0 -8 1]/12.

Output: the three components of the gradient vector of image intensity, or if image is

2D, the two dimensional gradient.

Input: image, mask.

If ~ exist ('mask', 'var')

mask = [-1 8 0 -8 1]/12;

end

dim1= size(image);

if length(dim = 2)

dim = [dim 1];

end

c = 1 : dim(1); r = 1 : dim(2); f = 1 : dim(3);

dy = mask(1) * image(max(c-2, 1), : , :) + mask(2) * image(max(c-1,1), : , :) + mask(4) *

image(min(c+1, dim(1)), : , :) + mask(5) * image(min(c-2,dim(1)), : , :);

dx = mask(1) * image(: , max(r-2, 1), : ) + mask(2) * image(: , max(r-1,1), : ) + mask(4) *

image(: , min(r+1, dim(2)), :) + mask(5) * image(: , min(r-2,dim(2)), :);

if dim(3) == 1

dz = zeros(dim,'single');

else

dz = mask(1) * image(: , : , max(f-2, 1)) + mask(2) * image(: , : , max(f-1,1)) + mask(4) *

image(: , : , min(f+1, dim(3))) + mask(5) * image(: , : , min(f-2,dim(3)));

end