1

Stephen Pizer, Sarang Joshi, Guido GerigMedical Image Display & Analysis Group (MIDAG)

University of North Carolina, USA

with credit to P. T. Fletcher, C. Lu, M. Styner, A. Thall, P.

Yushkevich And others in MIDAG

This set of slides can be found at the website midag.cs.unc.edu/pubs/presentations/SPIE_tut.htm

Tutorial:Tutorial: Anatomic Object Ensemble RepresentationsAnatomic Object Ensemble Representations forfor Segmentation & Statistical CharacterizationSegmentation & Statistical Characterization

17 February 2003

2

Segmentation

Objective: Extract the most probable target object geometric conformation z given the image data I

Requires prior on object geometry p(z)

Requires a measure of match p(I|z) of the image to a particular object conformation, so the image must be represented in reference to the object geometric conformation

3

Statistical Geometric Characterization

Requires priors p(class) and likelihoods p(z|class) Uses

Medical science: determine geometric ways in which pathological and normal classes differ

Diagnostic: determine if a particular patient’s geometry is in the pathological or the healthy class

Educational: communicate anatomic variability in atlases Priors p(z) for segmentation Monte Carlo generation of images

4

Part I

Multiscale Geometric Primitives,Especially M-reps

Multiscale Deformable Model Segmentation

Stephen Pizer

Tutorial: Anatomic Object Ensemble Representations for Segmentation & Statistical Characterization

17 February 2003

5

Relation of this object instance to other instances Representing the real world Basic entities: object ensembles & single objects Deformation while staying in statistical entity class Discrimination by shape class and by locality Mechanical deformation within a patient: interior primitive

Relation to Euclidean space/projective Euclidean space Matching image data

Multiple object-oriented scale levels Yields efficiency in segmentation: coarse to fine Yields efficiency in number of training samples

for probabilities

Object Representation Objectives

6

Object Ensembles & Single Objects

Object descriptionsIntuitive, related to anatomic understandingMathematically correct

Object interrelation descriptionsAbutment and non-interpenetration

Large scale Smaller scale

7 Multiple Object-oriented Scale Levels -- For Efficiency

Scale based parents and neighborsIntuitive scale levels

Ensemble Object Main figure Subfigure

Slab through-section Boundary vertex

8 Multiple Object-oriented Scale Levels -- For Efficiency

Scale based parents and neighbors Statistics via Markov random fields [Lu]

Residue from parent: zki = ith residue at scale

level k Difference from neighbors’ prediction p(zk

i relative to P(zki), zk

i relative to N(zki))

Efficiency of training from low dimension per probability

Features with position and level of locality (scale) Feature selection [Yushkevich]

9

Discussion of Scale

Spatial aspects of a geometric feature Position Scale: 3 different types

Spatial extent Region summarized

Level of detail captured Residues from larger scales

Distances to neighbors with which it has a statistical relationship

Markov random field Consider point distribution model,

landmarks, spherical harmonics, dense Euclidean positions, m-reps

Large scale Smaller scale

10

Scale Situations in Various Statistical Geometric Analysis Approaches

Coarse

Fine

Location

Leve

l of D

etai

l

Location Location

Global coef for Multidetail feature Detail residues each level of detail

Examples: boundary spherical boundary points, m-rep object harmonics, global dense position hierarchy, principal components displacements wavelets

11

Atlas voxels with a displacement at each voxel: x(x), label(x)

Set of distinguished points {xi} with a displacement at each Landmarks Boundary points in a mesh

With normal b = (x,n) Loci of medial atoms: m =

(x,F,r,) or end atom (x,F,r,)

(show on Pablo)

Object Representations: Atoms

u

v

t

Hub

Spoke Spoke

12Multiscale Object Representation via Interiors: M-reps

Interiors (medial) at all but smallest scale levels Boundary displacement at smallest scale level

Allows fixed structure in medial part Residues from previous scale level

At each level recognizes invariances associated with shape

Provide correspondence Across population & Across comparable

structures Provides prediction by neighbors

Translation, rotation, magnification Structure trained from population [Styner] Basis for deformable model segmentation

Continuous vs. sampled rep’nsboundary trad’l medial medial atom

13

M-rep Gives Multiscale Intrinsic Coord’s for Nonspherical & Nontubular Objects

Here single-figure On medial locus

(u,v) in r-proportional metric v along medial curve of

medial sheet u across medial sheet t around crest

Across narrow object dimension along medial spokes Proportion of medial width r

u

v

u

v

t

14Discrete M-rep Multifigure Objects and Multiobject Ensembles

Meshes of medial atoms Objects connected as host,

subfigures Hinge atoms of subfigure

on boundary of parent figure Blend in hinge regions Special coordinate system

(u,w,t) for blend region Multiple such objects, inter-

related via neighbor’s figural coords o

o

o o o

o

o o o

o o

w

15

M-rep Intrinsic Coordinates

Within figure One medial atom provides a coordinate

system for its neighbor atoms Position, Orientation, Metric

Between subfigure and figure Host atoms’ coordinate systems provides

coordinate system for protrusion or indentation hinge

Between figures or between objects One object provides coordinate system for

neighbor object boundary

u

v

t

16

Interpolating Boundaries in a Figure

Interpolate x, r via B-splines [Yushkevich] Trimming curve via r<0 at outside control points

Avoids corner problems of quadmesh Yields continuous boundary

Via modified subdivision surface [Thall] Approximate orthogonality at spoke ends Interpolated atoms via boundary and distance

At ends elongation needs also to be interpolated

Need to use synthetic medial geometry [Damon]

Medial sheet

Implied boundary

17

Sampled medial shape representation: M-rep tube figures

Same atoms as for slabs r is radius of tube spokes are rotated about b Chain rather than mesh

b

x n

x+rRb,n()bx+rRb,n(-)b

18

Segmentation by Deformable M-reps

w

For each scale level k, coarse to fineFor all residues i at scale level k: zk

iMaximize [log

p(zki relative to P(zk

i), zki relative to N(zk

i)) + log p(Image|{zj

i, j>=k, all i})] i.e., maximize geometric typicality + geometry to image match

(show on Pablo)

19

Intensity Profiles Template Used in Geometry to Image Match

Mean profile image along red meridian line, from training or as analytic function of /r

Inside

Outside

Left Hippocampus

Template to target image correspondence via figural coordinates

20 3-Scale Deformation of M-reps [Pizer, Joshi, Chaney, et al.]

Segmentation of Kidney from CT

Optimal movement

Optimal warp

Refined boundary

Hand-placed

21

Three Stage - Single Figure Segmentation of Kidney from CT

Axial, sagittal, and coronal target image slicesAxial, sagittal, and coronal target image slicesGrey curve: before step. White curve: after stepGrey curve: before step. White curve: after step

Optimal movement Optimal warp

Refined boundary

22 Segmentation by Deformable M-repsControlled Validations

w

KidneysHuman segmented

Robust over all 12 kidney pairsAvg distance to human segn’s boundary: <1.7mm

Clinically acceptable agreement with humansMonte Carlo produced

Robust against initialization Other anecdotal validations

Liver, male pelvis ensemble, caudate, hippocampus

23

For a copy of the slides in this talk see website: midag.cs.unc.edu/pubs/presentations/SPIE_tut.htm

For background to this talk see tutorial at website: midag.cs.unc.edu/projects/object-shape/tutorial/index.htm

or papers at midag.cs.unc.edu

24

References: Non-M-reps Voxel displacements and labels: Grenander, U and M Miller (1998).

Computational anatomy: an emerging discipline. Quarterly of Applied Mathematics, 56: 617-694. Christensen, G, S Joshi, and M Miller (1997). Volumetric transformation of brain anatomy. IEEE Transactions on Medical Imaging, 16(6): 864-877.

Landmarks: Dryden, I & K Mardia, (1998). Statistical Shape Analysis. John Wiley and Sons (Chichester).

Point distribution models: T Cootes, A Hill, CJ Taylor (1994). Use of active shape models for locating structures in medical images. Image & Vision Computing 12: 355-366.

Spherical harmonic models: Kelemen, A, G Székely, G Gerig (1999). Elastic model-based segmentation of 3D neuroradiological data sets. IEEE Transactions of Medical Imaging, 18: 828-839.

25

References: M-reps Overview: Pizer, S, G Gerig, S Joshi, S Aylward (2002). Multiscale

medial shape-based analysis of image objects. Proc. IEEE, to appear. http://midag.cs.unc.edu/pubs/papers/IEEEproc03_Pizer_multimed.pdf

Deformable m-reps segmentation: Pizer, S, et al. (2002). Deformable m-reps for 3D medical image segmentation. Subm. for IJCV special UNC-MIDAG issue. http://midag.cs.unc.edu/pubs/papers/IJCV01-Pizer-mreps.pdf

Figural coordinates: Pizer S, et al. (2002). Object models in multiscale intrinsic coordinates via m-reps. Image & Vision Computing special issue on generative model-based vision, to appear. http://midag.cs.unc.edu/pubs/papers/GMBV02_Pizer.pdf

Forming m-rep models: Styner, M et al., Statistical shape analysis of neuroanatomical structures based on medial models. Medical Image Analysis, to appear spring 2003. http://midag.cs.unc.edu/pubs/papers/MEDIA01-styner-submit.pdf

26

References: M-reps Continuous m-reps: Yushkevich, P et al. (2002). Continuous Medial

Representations for Geometric Object Modeling in 2D and 3D. Image & Vision Computing special issue on generative model-based vision, to appear. http://midag.cs.unc.edu/pubs/papers/IVC02-Yushkevich

Implied boundaries via subdivision surfaces: Thall, A (2002). Fast C2 interpolating subdivision surfaces using iterative inversion of stationary subdivision rules. UNC Comp. Sci. Tech. Rep. TR02-001. http://midag.cs.unc.edu/pubs/papers/Thall_TR02-001.pdf

Markov random fields: Lu, C, S Pizer, S Joshi (2003). A Markov Random Field approach to multi-scale shape analysis. Subm. to Scale Space. http://midag.cs.unc.edu/pubs/papers/ScaleSpace03_Conglin_shape.pdf

Math of m-reps --> boundaries: Damon, J (2002), Determining the geometry of boundaries of objects from medial data. UNC Math. Dept. http://midag.cs.unc.edu/pubs/papers/Damon_SkelStr_III.pdf