Real-time 3D Segmentation of the Left Ventricle Using Deformable SubdivisionSurfaces

Fredrik OrderudNorwegian University of Science and Technology

Trondheim, Norwayfredrik.orderud@idi.ntnu.no

Stein Inge RabbenGE Vingmed Ultrasound

Oslo, Norwaystein.rabben@med.ge.com

Abstract

In this paper, we extend a computationally efficientframework for real-time 3D tracking and segmentation tosupport deformable subdivision surfaces. Segmentation isperformed in a sequential state-estimation fashion, using anextended Kalman filter to estimate shape and pose param-eters for the subdivision surface. As an example, we haveintegrated Doo-Sabin subdivision surfaces into the frame-work. Furthermore, we provide a method for evaluatingbasis functions for Doo-Sabin surfaces at arbitrary param-eter values. These basis functions are precomputed duringinitialization, and later used during segmentation to quicklyevaluate surface points used for edge detection.

Fully automatic tracking and segmentation of the leftventricle is demonstrated in a dataset of 21 3D echocardio-graphy recordings. Successful segmentation was achievedin all cases, with limits of agreement (mean±1.96SD) forpoint to surface distance of 2.2±0.8 mm compared to man-ually verified segmentations. Real-time segmentation at arate of 25 frames per second consumed a CPU load of 8%.

1. Introduction

The emergence of volumetric image acquisition withinthe field of medical imaging has attracted a lot of scientificinterest over the last years. In a recent survey, Noble etal. [1] presented a review of the most significant attemptsfor 3D segmentation within the field of ultrasonics. How-ever, all of these attempts are limited to being used as postprocessing tools, due to extensive processing requirements,even though volumetric acquisition may be performed inreal-time with the latest generation of 3D ultrasound tech-nology. Availability of technology for real-time trackingand segmentation in volumetric data sets would open uppossibilities for instant feedback and diagnosis during dataacquisition.

Orderud et al. has recently presented a framework forreal-time tracking and segmentation in volumetric data [2].This framework treats the tracking problem as a state es-timation problem, and uses an extended Kalman filter torecursively track global pose and local shape parametersusing a combination of state predictions and measurementupdates. In [2], a deformable spline model was used totrack left ventricular (LV) shape segmentation 3D echocar-diography. Later, in [3] the framework was combined witha trained active shape model with predefined deformationmodes to improve segmentation accuracy.

This state estimation approach is based on prior work byBlake et al. [4] in 2D, who used a Kalman filter to trackB-spline contours deformed by linear transforms within amodel subspace referred to as shape space. Later, thesame approach was applied to real-time LV tracking in 2Dechocardiography by Jacob et al. [5]. Similar efforts havelater been published by Comaniciu er al. [6], who focusedon the information fusion problem encountered in state-space tracking. A state-based approach for cardiac defor-mation analysis has also recently been published by Liu &Shi in [7].

Usage of spline models for shape segmentation does,however, imply some inherent topological restrictions,since the control vertices of a spline surface are restrictedto form a regular quadrilateral structure. The LV model in[2] were for instance based on a cylindrical topology, witha hole at both the apex and base that required ad-hoc stepsto form a closed surface. Polygonal models coupled withsubspace deformation [3] does not suffer from any of thesetopological restrictions, but instead requires much highermesh resolution in order to form smooth surfaces, whichimplies higher computational complexity and a more com-plex surface description.

1.1. Contributions

In this paper, we extend the Kalman tracking frameworkfrom [2] to support a wider class of smooth deformable

1978-1-4244-2243-2/08/$25.00 ©2008 IEEE

surfaces known as subdivision surfaces [8, 9], that general-ize spline surfaces to support meshes of arbitrary topology.This allows for more flexible modeling of arbitrary meshstructures, without the inherent topological restrictions as-sociated with spline surfaces. We focus on a specific type ofsubdivision surfaces known as Doo-Sabin surfaces, whichhave some properties that make them suitable for low res-olution cardiac modeling. Details for exact evaluation ofsurface points, as well as partial derivatives, for Doo-Sabinsubdivision surfaces is also presented, to provide a meansof evaluating basis functions for arbitrary surface points re-quired for the edge detection process.

A low resolution Doo-Sabin subdivision model is thenused to model the left ventricle of the heart. This model hasadjustable control vertices to allow alteration of the shape,and is coupled with a global pose transform to position andorient it within the volume. Successful real-time LV seg-mentation in 3D echocardiography is finally demonstratedusing the proposed subdivision model in conjunction to theKalman filter based tracking framework.

1.2. Nomenclature

Scalars are expressed in italic, vectors in boldface andmatrices in uppercase boldface. Control vertices are de-noted ’q’, displacement directions ’d’, surface points ’p’and surface normal vectors ’n’. State vectors are denoted’x’. Discrete time is denoted with subscript ’k’ for theKalman filter, and control vertex or surface point indicesare denoted with subscript ’i’.

2. Evaluation of Doo-Sabin Surfaces

Doo-Sabin surfaces [8] is a type of subdivision sur-face that generalize bi-quadric B-spline surfaces to arbi-trary topology. Following the same approach as Stam forCatmull-Clark surfaces [10], we define Doo-Sabin subdi-vision as a matrix operation. Each surface patch can besubdivided into four new sub-patches by multiplying theNq × 3 control vertex matrix Q0 with the (Nq + 7) × Nq

subdivision matrix S, as is shown in Fig. 1. The con-tent of this matrix originates from the regular Doo-Sabinsubdivision rules, which are outlined in appendix A. Con-trol vertices for the region of support for each sub-patchk ∈ 0, 1, 2, 3 of choice can then be extracted from thesubdivided control vertices using a picking matrix Pk, suchthat Qn+1,k = PkSQn.

Regardless of the topology of Qn, all sub-patchesQn+1,k will at most consist of a single irregular face in ad-dition to three quadrilaterals. Successive subdivision oper-ations on Qn+1,k will then yield a single irregular patch,while the three others becomes regular bi-quadric splinepatches that can be evaluated directly. By assuming, with-out loss of generality, that the irregular face in Qn+1 is

located top-left, then the picking matrix Pk gives an reg-ular 3 × 3 bi-quadric control vertex mesh when k 6= 0,and a irregular mesh consisting of Nq control vertices whenk = 0. This relation can be exploited by performing re-peated subdivisions n times until the desired surface point isno longer within an extraordinary patch (k 6= 0). DenotingS0 = P0S, we can express this as Qn,k = PkSSn−1

0 Q0.The number of subdivision steps n required depends on

the logarithm of (u,v), while the sub-patch to pick after thefinal subdivision is determined using the following criteri-ons:

n = b− log2 (maxu, v)c (1)

k =

1 if 2nu > 1/2 and 2nv < 1/22 if 2nu > 1/2 and 2nv > 1/23 if 2nu < 1/2 and 2nv > 1/2

(2)

Direct evaluation of surface points can then be performedfor any patch location (u, v) except (0, 0), by subdividingsufficient number of times, until the new subdivided patchbelow (u, v) no longer contains a extraordinary face, andtreating the resulting sub-patch as a ordinary bi-quadricspline surface. For locations near (0, 0), an approximatesurface evaluation can be obtained by perturbating (u, v)slightly to prevent n from growing beyond a predefined up-per limit. Basis functions with regards to the original non-subdivided control vertices can similarly be computed usingthe same approach:

b(u, v)|Ωnk

=(PkSSn−1

0

)Tb(tk,n(u, v)) , (3)

where Ωnk is the subdivision mapping function described

above, that determines the number of subdivision steps re-quired based on (u, v) [10]. b is the regular bi-quadric B-spline basis functions defined in appendix B, and tk,n is adomain mapping function used to map the parametric inter-val (u,v) to the parametric interval within the desired sub-patch:

tk,n(u, v) =

(2n+1u− 1, 2n+1v ) if k = 1(2n+1u− 1, 2n+1v − 1) if k = 2(2n+1u, 2n+1v − 1) if k = 3

(4)

Partial derivatives of the basis functions, bu and bv , aresimilarly computed by replacing b(u, v) with the respectivederivatives of the B-spline basis functions in the formula.Surface positions can then be evaluated as an inner productbetween the control vertices and the basis functions

p(u, v) = QT0 b(u, v) . (5)

Note that this approach is not dependent on diagonaliza-tion of the subdivision matrix, as in [10]. Instead, repeated

Qn+1

= S Qn

Qn Q

n+1

P0Q

n+1P

1Q

n+1P

2Q

n+1 P3Q

n+1

u

v

Figure 1. Illustration of the Doo-Sabin subdivision process. Thecontrol vertices Qn that define the initial surface patch (upper left)are subdivided into new control vertices Qn+1 (upper right) bymultiplying Qn with the subdivision matrix S. Application of thepicking matrix Pk on Qn+1further divides the subdivided meshinto four sub-patches that together span the same surface area asthe original patch.

matrix multiplication performed n times will result in ex-actly the same result. The associated increase in computa-tional complexity associated with this repeated multiplica-tion will not be a burden if evaluation of basis functions isperformed only once, and later re-used to compute surfacepoints regardless of movement of the associated control ver-tices.

3. Deformable Subdivision ModelThis section explains how deformable subdivision mod-

els, such as the LV model shown in in Fig 2, can be incorpo-rated into the Kalman tracking framework. The subdivisionmodels consists of control vertices qi for i ∈ 1 . . . Nqwith associated displacement direction vectors di that de-fines the direction in which the control vertices are allowedto move. Displacement directions are typically based onsurface normals, since movement of control vertices in thisdirection results in the greatest change of shape. In additionto the control vertices, the topological relationships betweenthe control vertices have to be defined in a list C(c). Thislist maps surface patches c ∈ 1 . . . Nc to enumerated listsof control vertex indices that define the region of supportfor each surface patch.

We denote the local deformations Tl(xl) to our de-formable model as the deformations obtainable by mov-ing the control vertices of the subdivision model. Theselocal deformations are combine with a global transformTg(xg,pl) to position, scale and orient the model withinthe image volume where the tracking takes place.

After creation of the model, a set of surface points haveto be defined, which are to be used for edge detection mea-surements. This set consists of parametric coordinates (in-cluding patch number) for each of the points (u, v, c)l,

and are typically distributed evenly across the model surfaceto ensure robust segmentation. By restricting the distribu-tion of these edge profiles to fixed parametric coordinatesthroughout the tracking, then basis functions for each edgeprofile can be precomputed during initialization. These ba-sis functions are independent of the position of the controlvertices, and can therefore be re-used during tracking to ef-ficiently generate surface points regardless of local shapedeformations.

3.1. Calculation of Surface Points

The Kalman filter framework requires the creation of aset of surface points pl with associated normal vectors nl

and Jacobi matrices Jl, based on a predicted state vector xl.The creation of these objects can be performed efficientlyfollowing the steps below:

1. Update position of control vertices qi based on thestate vector: qi = qi + xidi, where qi is the meanposition of the control vertex and xi is the parame-ter corresponding to this control vertex in the statevector for each control vertex. di is the displace-ment direction for control vertex qi. The full statevector for the model then becomes the concatenationof the state parameters for all control vertices xl =[x1, x2, . . . , xNl

]T . One can here chose to force cer-tain vertices to remain stationary during tracking with-out altering the overall approach. This would both re-duce the deformation space, as well as the number ofparameters to estimate.

2. Calculate surface points pl as a sums of control ver-tices weighted with their respective basis functionswithin the surface patch of each surface point: pl =∑

i∈C(cl)biqi.

3. Calculate surface normals nl as the cross product be-tween the partial derivatives of the basis functions withregards to parametric values u and v within the surfacepatch of each surface point: nl =

∑i∈C(cl)

(bu)iqi ×∑i∈C(cl)

(bv)iqi.

4. Calculate Jacobian matrices for the local deformationsJl by concatenating the displacement vectors mul-tiplied with their respective basis functions: Jl =[

bi1di1 , bi2di2 , . . .]i∈C(cl)

. The Jacobian ma-trix will here be padded with zeros for columns corre-sponding to control vertices outside the region of sup-port for the surface patch of each surface point.

Precomputation of basis functions enables the operationsabove to be performed very quickly, which is crucial forenabling real-time implementations.

(a)

Figure 2. Orthogonal views of the initial undeformed subdivisionsurface (left), as well as the same model deformed to fit the LVafter tracking in some frames (right). The model consists of 24surface patches, that each are outlined by a bold black border andsubdivided into a 5 x 5 quadrilateral grid for visualization. Theencapsulating wire-frame mesh illustrates the control vertices (cir-cles) that define the surface.

3.2. Global Transform

The composite object deformation T(x) =Tg(Tl(xl), xg) is obtained by combining the localdeformations of the subdivision model with a globaltransform to create a joint model. This leads to a compositestate vector x = [xT

g , xTl ]T consisting of Ng global and

Nl local deformation parameters. This separation betweenlocal and global deformations is intended to ease modeling,since changes in shape are often parametrized differentlycompared to deformations associated with global position,size and orientation.

We denote pl, nl and Jl for the surface points cre-ated from the subdivision surface with local deformationsTl(xl). These points are subsequently transformed bymeans of a global pose transform Tg , that translates, rotatesand scales the model to align it correctly within the imagevolume. Surface points are trivially transformed using Tg ,whereas normal vectors must be transformed by multiply-ing with the normalized inverse spatial derivative of Tg toremain surface normals after the global transform [11]:

pg = Tg(pl,xg) (6)

ng =∣∣∣∣∂Tg(pl,xg)

∂pl

∣∣∣∣ (∂Tg(pl,xg)

∂pl

)−T

nl (7)

The Jacobian matrices for the composite deformations thenbecomes the concatenation of both global and local state-space derivatives. The local part is created by multiplyingthe 3 × 3 spatial Jacobian matrix for the global transformwith the 3 × Nl local Jacobian matrix for the deformablemodel, as follows from the chain-rule of multivariate calcu-lus:

Jg =[∂Tg(pl,xg)

∂xg,

∂Tg(pl,xg)∂pl

Jl

]. (8)

ModelKalmanPredict

KalmanUpdate

x,P-

x,P^

Measure

T(x)

Assimilate

∑p, n v, r

H R v,H R H

T

T

-1

-1

h

Figure 3. Overview over the processing chain in the Kalman filterbased tracking framework. All five steps are performed only oncefor each new frame.

4. Kalman Tracking FrameworkThe tracking framework is decomposed into the 5 sepa-

rate steps shown in Fig. 3.

4.1. State Prediction

Incorporation of temporal constraints is accomplished inthe prediction stage of the Kalman filter by augmenting thestate vector to contain the last two successive state esti-mates. A motion model which predicts state x at time stepk + 1, with focus on deviation from a mean state x0, canthen be expressed as:

xk+1 − x0 = A1 (xk − x0) + A2 (xk−1 − x0) , (9)

where xk is the estimated state from time step k. Tuningof properties like damping and regularization towards themean state x0 for all deformation parameters can then beaccomplished by adjusting the coefficients in matrices A1

and A2. Prediction uncertainty can similarly be adjusted bymanipulating the process noise covariance matrix B0 that isused in the associated covariance update equation for Pk+1.The latter will then restrict the rate of which parameter val-ues are allowed to vary.

4.2. Evaluation of Deformable Model

Creation of surface points p, normals n and Jacobianmatrices J, based on the predicted state xk. This is per-formed as described in section 3.

4.3. Edge Measurements

Edge measurements are used to guide the model towardthe object being tracked. This is done by measuring the dis-tance between points on a predicted model inferred from themotion model, and edges found by searching in normal di-rection of the model surface. This type of edge detection isrefereed to as normal displacement [4], and is calculated asthe normal projection of the distance between a predictededge point p with associated normal vector n and a mea-sured edge point pobs:

v = nT (pobs − p) . (10)

Each normal displacement measurement is coupled witha measurement noise r value that specifies the uncertainty

associated with the edge. This value is typically depen-dent on edge strength or other measure of uncertainty. Thischoice of normal displacement measurements with associ-ated measurements noise enables usage of a wide range ofpossible edge detectors. The only requirement for the edgedetector is that it must identify the most promising edgecandidate for each search profile, and assign an uncertaintyvalue to this candidate.

Linearized measurement models [12], which are re-quired in the Kalman filter for each edge measurement, areconstructed by transforming the state-space Jacobi matricesthe same way as the edge measurements, namely taking thenormal vector projection of them:

hT = nT J . (11)

This yields a separate measurement vector h for each nor-mal displacement measurement, that relates the normal dis-placements to changes in the state vector.

4.4. Measurement Assimilation

State-space segmentation forms a special problem struc-ture, since the number of measurements typically far ex-ceeds the number of state dimensions. Ordinary Kalmangain calculation will then be computationally intractable,since they involve inverting matrices with dimensions equalto the number of measurements.

An alternative approach is to assimilate measurementsin information space [12] prior to the state update step.This enables very efficient processing if we assume thatthe measurements are uncorrelated, since uncorrelated mea-surements lead to a diagonal measurement covariance ma-trix R. All measurement information can then be summedinto an information vector and matrix of dimensions invari-ant to the number of measurements:

HT R−1v =∑

i hir−1i vi (12)

HT R−1H =∑

i hir−1i hT

i . (13)

4.5. Measurement Update

Measurements in information filter form require somealterations to the state update step in the Kalman filter.This can be accomplished by utilizing that the Kalman gainKk = PkHT R−1, and reformulating the equations to ac-count for measurements in information space. The updatedstate estimate x for time step k then becomes:

xk = xk + PkHT R−1vk . (14)

The updated error covariance matrix P can similarly be cal-culated in information space to avoid inverting unnecessarylarge matrices:

P−1k = P−1

k + HT R−1H . (15)

Figure 4. Volume correlation plot for the proposed segmenta-tion against the reference method at end-diastole (EDV) and end-systole (ESV) in each of the 21 recording.

Distance [mm] EDV [ml] ESV [ml] EF [%]2.2± 0.8 3.6± 21.4 9.0± 17.4 −5.9± 11.1

Table 1. Bland-Altman analysis of the segmentation results com-pared to the reference segmentation. Results are expressed asmean difference ±1.96SD.

This form only requires inversion of matrices with dimen-sions equal to the state dimension.

5. Results

The proposed tracking framework were evaluated byperforming fully automatic tracking in 21 unselected 3Dechocardiography recordings of the heart, recorded with aVivid 7 ultrasound scanner (GE Vingmed Ultrasound, Nor-way) using a matrix array transducer (3V). Exactly the sameinitialization were used in all of the recordings, and theresulting segmentations were compared to semi-automaticsegmentation using a custom made segmentation tool (GEVingmed Ultrasound, Norway). The reference segmenta-tions were conducted by an expert, and whenever neededmanually adjusted to serve as a validated reference compa-rable to manual tracing.

A manually constructed Doo-Sabin model consisting of24 surface patches, as shown in Fig 2, were used as basis forthe LV segmentation. This deformable model were com-bined with a global pose transform that featured parametersfor translation, rotation and isotropic scaling of the modelto position and orient it within the image volume. 384 edgeprofiles distributed evenly over the surface were used usedfor edge detection. Each of these profiles consisted of 30image samples spaced 1 mm apart. Edge detection was thenperformed in each profile, based on a transition criterion [2],with edge weighting based on the transition height. Theseedge measurements were combined with a outlier removalstep which discarded edges with normal displacement valuedifferencing significantly for that of its neighbors.

Figure 5. Example segmentation results from four of the record-ings (rows). Orthogonal intersection slices through each volumeshows the segmentation result at end-diastole (left) and end sys-tole (right). The red intersection lines show the proposed Kalmansegmentation, and the yellow lines the reference segmentation.

Figure 4 and table 1 shows the results from the compar-ison with the reference segmentation, using Bland-Altmananalysis of the average point to surface distance betweenthe meshes, difference in end diastolic volumes (EDV), endsystolic volumes (ESV) as well as differences in the com-puted ejection fraction (EF)1. Fig. 5 shows orthogonal inter-section slices of the segmentations at end diastole and endsystole from four of the recordings to illustrate the typicalsegmentation quality obtained.

Compared to previous efforts on real-time Kalman seg-mentation [2, 3], this indicates slightly more accurate seg-mentations with smaller distances between the meshes andless bias in the ejection fraction numbers. The tracking andsegmentation was performed using a C++ implementationon a 2.16 GHz Intel Core 2 duo processor, were real-timesegmentation at a rate of 25 frames per second consumedapproximately 8% CPU power.

The tracking converged in 2-4 frames after initializationin most of the recordings. A typical convergence for oneof the recordings is shown in Fig. 6. A single recordingdid, however, require a half heartbeat to converge, becauseto the basal edge profiles were not long enough to reach the

1Ejection fraction is the ratio of LV contraction. It is commonly usedfor assessment of global ventricular function, and is computed as (EDV-ESV)/EDV.

Figure 6. Intersection slices through the volume showing the con-vergence rate for the tracking. The initial mesh before trackingis started is shown top-left (red for proposed and yellow for ref-erence), followed by deformed meshes after tracking in the first6 frames. A plot of the average surface to surface distance be-tween the deformed meshes and the reference segmentation foreach frame is shown bottom-right.

base of the heart before the heart was maximally contractedat end systole.

6. DiscussionIn this paper, we have extended prior work [2, 3] to en-

able fully automatic and real-time LV tracking and segmen-tation in dense volumetric data using deformable subdivi-sion surfaces.

6.1. Subdivision Model

Usage of subdivision models for segmentation has somedesirable properties that makes them suitable for model-ing of cardiac structures, in that they combine the inherentsmoothness and continuity of surface derivatives of splinesurfaces with the support of arbitrary topology from flatpolygonal meshes. This enables more flexible modeling,since control vertices of the surface models are no longerrestricted to the quadrilateral structure known from splinesurfaces. Subdivision models that form closed surfaces andsurfaces with complex geometries can therefore be con-structed in a simple and intuitive fashion.

The inherent smoothness also makes it possible to repre-sent cardiac geometries using low resolution models, con-sisting of few control vertices, as shown in this paper. Thismakes for more robust segmentation compared to high res-olution models, since fewer shape parameters have to beestimated during tracking. Papillary muscles and trabecularstructures, that are known to disrupt segmentation [1], arealso handled robustly, since the low-resolution model areunable to represent the local sharp discontinuities in shapethese structures represents.

Doo-Sabin is chosen in favor of the more commonCatmull-Clark surfaces, that generalize bi-cubic B-splinesurfaces, because bi-quadric surfaces has a narrower regionof support compared to bi-cubic surfaces. This makes themmore suitable for segmentation with low resolution models,where the range of support for each surface patch should berestricted to a local area, and not be so large that it coversa significant portion of the entire model. The proposed seg-mentation approach is, however, not restricted to Doo-Sabinsurfaces in particular, so other subdivision schemes, such asCatmull-Clark and Loop could also be used without alter-ing the overall approach. The only requirement is that thesubdivision process can be expressed as a linear operationon matrix form.

Usage of subdivision models for cardiac tracking havealso been presented in [13], but this paper focuses on imageregistration using iterative gradient descent algorithms, andnot on segmentation per se. It therefore depends on man-ual surface initialization, and is thus not suitable for fullyautomatic behavior.

6.2. Segmentation Results

The results shown in table 1 indicate that usage of sub-division surfaces leads to improved segmentation accuracycompared to spline and active-shape models [2, 3], eventhough fewer edge profiles were used for edge detection.This is believed to be caused by the subdivision model ismore capable of capturing the shape and deformation pat-tern of the LV.

The segmented ventricles showed good overall agree-ment with the reference segmentations, both with respectto to point to surface distances and for the computed vol-umes. Segmentation accuracy is primarily limited by thedifficulty of edge detection in echocardiography recordings,which suffers from inherently poor image quality. It is dif-ficulty, even for experts, to accurately determine the endo-cardial border in such recordings. Perfect agreement withreference segmentations might therefore never be achieved.The simple transition criteria used in this paper is chosenprimarily because it behaves robustly and has a long radiusof convergence. More advanced criterias might very wellyield more accurate segmentation in areas of weak and un-clear edges, but state of the art edge detection is not themain focus of this paper.

Tracking convergence seems to primarily be limited bythe length of the edge profiles. There is, however, an in-herent trade-off between convergence speed, and edge de-tection robustness/outlier frequency here, since longer edgeprofiles might lead to the detection of more outlier edgesthat might disrupt the tracking. When disregarding the firstfew frames during tracking, the Kalman tracker seemed torespond fast enough to capture changes in pose and shapebetween successive frames, even though it only used a sin-

gle refinement iteration per frame.Precalculation of basis functions for the subdivision

model during initialization also lead to a more computa-tionally efficient implementation compared to [2, 3], evenwhen compensating for the reduction in the number of edgeprofiles used.

6.3. Kalman Filter Approach

Most segmentation approaches used in medical imaging,such as active shape segmentation and simplex mesh seg-mentation [1], are based on modeling of forces acting upona deformable model with semi-realistic physical properties.Segmentation is then performed by using iterative optimiza-tion algorithms to determine an equilibrium state betweeninternal shape forces and external image forces. Segmen-tation typically requires hundreds of iterations to convergeusing this approach, which makes real-time 3D segmenta-tion using this approach computationally intractable.

State-estimation based tracking instead uses a non-iterative algorithm, based on a Bayesian least squares so-lution of the linearized tracking problem, namely the ex-tended Kalman filter. The model is segmented by com-puting a solution to a system of equations to fit the modelto the detected edges, while at the same time regularizingthe fitting by incorporating a kinematic model to restrict therate of change for shape and pose parameters. This leads tooutstanding computational performance compared to itera-tive algorithms, and enables real-time tracking and segmen-tation in volumetric datasets. Usage of extended Kalmanfilters for segmentation does, however, imply the makingof some assumptions with regards to Gaussian distributionsand linearity:

Firstly, the framework assumes that the normal displace-ment values are independent and follow a Gaussian distribu-tion. This, however, only accounts for the normal displace-ment values only, and not to the shape of the underlyingedge profiles or details in the edge detector, in which arenot assumed to follow any given distribution in the Kalmanfilter. Secondly, the extended Kalman filter assumes all de-formations to be linear. Except for global rotation, whichis inherently non-linear, every other mode of deformation islinear. This includes global translation, scaling and the lo-cal shape deformations, which due to the tensor product for-mulation of the polynomial basis functions are linear in theposition of the control vertices. There is very little changein global rotation of the heart between successive frames,so the linearization approximation penalty is believed to besmall.

7. ConclusionIn this paper, we have proposed a Kalman filter based

framework for fully automatic real-time tracking and seg-

mentation in volumetric data, using deformable subdivi-sion surfaces. Usage of subdivision surfaces enables sim-ple modeling of closed surfaces, and surfaces with com-plex topology, without any of the limitations associated withspline surfaces. In addition, a method for exact evalua-tion of surface points at arbitrary parameter values for Doo-Sabin surfaces is provided, to enable efficient precalculationof basis functions used to extract edge profiles.

The results indicate that usage of subdivision surfacesleads to improved segmentation accuracy compared tospline and active-shape models [2, 3]. Precalculation of ba-sis functions also significantly reduces the computationalcomplexity. The combination of subdivision models with aKalman filter tracker thus enables 3D segmentation that isboth robust and capable of operating in real-time.

A. Doo-Sabin Subdivision MatrixThe subdivision weights used for faces consisting of n

vertices are used as defined by Doo & Sabin [8]:

αjn =

δj,0

4+

3 + 2 cos(2πj/n)4n

, (16)

where δi,j is the Kronecker delta function which is one fori = j and zero elsewhere. Subdivision of the control ver-tices within a single face can then be expressed as a linearoperation using a subdivision matrix Sn:

Sn =

α0

n α1n α2

n . . . α−1n

α−1n α0

n α1n . . . α−2

n

α−2n α−1

n α0n . . . α−3

n

. . . . . . . . . . . . . . .α1

n α2n α3

n . . . α0n

. (17)

Subdivision of whole patches is accomplished by com-bining Sn for all four faces in a patch into a composite sub-division matrix S. The structure of this matrix depends onthe topology and control vertex enumeration scheme em-ployed, but construction should be straightforward.

B. Basis functions for Quadratic B-splinesThe 9 tensor product quadratic B-spline functions can be

expressed as a product of two separable basis polymonialsfor the parametric value u and v (i = 0, . . . , 8):

bi(u, v) = Pi%3(u) Pi/3(v) , (18)

where “%” and “/” denotes the division remainder and divi-sion operators respectively. Pi(t) are the basis polynomialsfor quadratic B-splines with uniform knot vectors:

2P0(t) = 1− 2t + t2 (19)

2P1(t) = 1 + 2t− 2t2 (20)

2P2(t) = t2 (21)

References[1] J. A. Noble and D. Boukerroui, “Ultrasound image segmen-

tation: A survey,” Medical Imaging, IEEE Transactions on,vol. 25, no. 8, pp. 987–1010, 2006.

[2] F. Orderud, J. Hansegard, and S. I. Rabben, “Real-time track-ing of the left ventricle in 3D echocardiography using astate estimation approach,” in Medical Image Computing andComputer-Assisted Intervention – MICCAI 2007, vol. 4791of LNCS, pp. 858–865, Springer, 2007.

[3] J. Hansegard, F. Orderud, and S. Rabben, “Real-time activeshape models for segmentation of 3D cardiac ultrasound,” inComputer Analysis of Images and Patterns - CAIP, pp. 157–164, 2007.

[4] A. Blake and M. Isard, Active Contours: The Applicationof Techniques from Graphics, Vision, Control Theory andStatistics to Visual Tracking of Shapes in Motion. Secaucus,NJ, USA: Springer-Verlag New York, Inc., 1998.

[5] G. Jacob, J. A. Noble, C. Behrenbruch, A. D. Kelion, andA. P. Banning, “A shape-space-based approach to trackingmyocardial borders and quantifying regional left-ventricularfunction applied in echocardiography,” Medical Imaging,IEEE Transactions on, vol. 21, no. 3, pp. 226–238, 2002.

[6] D. Comaniciu, X. S. Zhou, and S. Krishnan, “Robustreal-time myocardial border tracking for echocardiography:An information fusion approach,” Medical Imaging, IEEETransactions on, vol. 23, no. 7, pp. 849–860, 2004.

[7] H. Liu and P. Shi, “State-space analysis of cardiac motionwith biomechanical constraints,” Image Processing, IEEETransactions on, vol. 16, no. 4, pp. 901–917, April 2007.

[8] D. Doo and M. Sabin, “Behaviour of recursive division sur-faces near extraordinary points,” Computer-Aided Design,vol. 10, pp. 356–360, Nov. 1978.

[9] E. Catmull and J. Clark, “Recursively generated b-spline sur-faces on arbitrary topological meshes,” Computer-Aided De-sign, vol. 10, pp. 350–355, Nov. 1978.

[10] J. Stam, “Exact evaluation of catmull-clark subdivision sur-faces at arbitrary parameter values,” in SIGGRAPH ’98:Proceedings of the 25th annual conference on Computergraphics and interactive techniques, (New York, NY, USA),pp. 395–404, ACM Press, 1998.

[11] A. H. Barr, “Global and local deformations of solid prim-itives,” in SIGGRAPH ’84: Proceedings of the 11th an-nual conference on Computer graphics and interactive tech-niques, (New York, NY, USA), pp. 21–30, ACM Press, 1984.

[12] Y. Bar-Shalom, X. R. Li, and T. Kirubarajan, Estimationwith Applications to Tracking and Navigation. Wiley-Interscience, 2001.

[13] R. Chandrashekara, R. Mohiaddin, R. Razavi, and D. Rueck-ert, “Nonrigid image registration with subdivision lattices:Application to cardiac mr image analysis,” in Medical ImageComputing and Computer-Assisted Intervention – MICCAI2007, vol. 4791 of LNCS, pp. 335–342, Springer, 2007.