volume illumination and segmentationhomepages.inf.ed.ac.uk/tkomura/cav/presentation13.pdf · taku...
TRANSCRIPT
Taku Komura
Volume Illumination and Segmentation
Computer Animation and Visualisation – Lecture 13
Taku Komura
Institute for Perception, Action & BehaviourSchool of Informatics
Taku Komura
Overview
Volume illumination Segmentation
Taku Komura
Volume Illumination Why do we want to
illuminate volumes? illumination helps us to
better understand 3D structure
—displays visual cues to surface orientation
—highlight significant gradients within volume
Taku Komura
Light Propagation in Volumes Lighting in volume
− In ray-casting (the method in the last lecture) only transmission and emission considered
− can also:— reflect light— scatter light into different directions
Taku Komura
Illumination of Volumes
For every voxel ray intersects, need to consider:
• Light absorbed.• Light emitted.• Light scattered out of the ray.• Light scattered into the ray.
Taku Komura
Illumination of Volumes
For every voxel ray intersects, need to consider:
• Light absorbed.• Light emitted.• Light scattered out of the ray.• Light scattered into the ray.
Taku Komura
Example : multiple scattering
Light scattered multiple times to produce simulation of a cloud
Taku Komura
Example : sub-surface scattering
Varying how light is scattered inside a surface affects perception
− hence useful in visualisation
− why ? - prior visual experience, perception of distance ?
Very large statue
Medium sized
Small statue.
Taku Komura
Simple Tricks: Subsurface Scattering
The more the light travels in a material, the more it gets scattered and absorbed Reduce the intensity of the light which has travelled longer
Taku Komura
Volume Illumination - ?
Scattering is too costly so we usually do not take them into account when doing volume rendering
But we still can add slight shadows to the volume by illuminating them
Taku Komura
What are we illuminating ?
embedded (iso-) surface sharp gradients in the scalar value
Taku Komura
Shading an Embedded iso-surface
Classify volume with a step function Use regular specular / diffuse surface shading Remember for lighting equations require illumination direction
− camera model (position)
− surface orientation
− need to calculate and store surface normal
Taku Komura
Estimating the surface normal from the depth
Use distance map to the iso-surface value
1. Determine the threshold value2. Determine the surface voxels based on the
threshold3. Compute the normal vectors based on centred
difference method
For example, if we sample the centre of the voxels,
We can extract the normal vectors of the region where
the scalar values are changing significantly,
i.e. boundary of tissues
Taku Komura
Result : illuminated iso-surface
Surface normals recovered from depth map of surface
MIP technique
Shaded embedded iso-surface.
Taku Komura
Illuminating Opacity (Scalar) Gradient
Illuminate “scalar gradient” instead of iso-surface
− requirement : estimate and store gradient at every voxel
CompositeShaded opacity gradient(shades changes in opacity)
Taku Komura
Illumination : storing normal vectors
Visualisation is interactive
− compute normal vectors for surface/gradient once
− store normal
− perform interactive shading calculations
Storage :
− 2563 data set of 1-byte scalars ~16Mb
− normal vector (stored as floating point(4-byte)) ~ 200Mb!
− Solution : quantise direction & magnitude as small number of bits
Taku Komura
Illumination : storing normal vectors
Subdivide an octahedron into a sphere.
Number the vertices.
Encode the direction according to the nearest vertex that the vector passes through.
For infinite light sources, only need to calculate the shading values once and store these in a table.
Quantize vector direction into one of N directions on a sub-divided sphere
Taku Komura
Overview
Volume illumination Segmentation
Taku Komura
Medical Segmentation
Segmenting the data into different tissues
The data can be either volume / image
Taku Komura
Segmentation of CT imaging CT value corresponds to density → easier to segment
Sometimes, you might want to segment tissues of the same CT values
− Segmenting muscles
Taku Komura
How to segment MRI data?
Liver
Trachea
Lungs
Gall Bladder
How to extract an organ here?
A Dog’s thorax
MRI values are not directly proportional to density
Taku Komura
Image Processing on MRI Data From image processing / computer vision
− segment image slices— noise removal (smoothing)— edge detection (changes in colour)— Region growing— thresholding – all scalar values in
between upper and lower limits
Liver outlined in red
Image processing
− not strictly visualisation
− additional tool in visualisation pipeline
− see Advanced Vision course
Taku Komura
Region Growing
start from initial image patch grow outwards including similar regions stop when boundary or distinct change in
value reaching
Taku Komura
Example : MRI segmented knee
Original MRI slice (top, left), Segmentation (top, right), Overlay (bottom).
Courtesy : Brigham & Women's Hospital
Taku Komura
“Segmentation” of dog organs
Segmentation of organ in each image slice (2D)
− visualisation of segmented 2D image stack as 3D volume
3D model of Liver.Liver outlined in red
Taku Komura
Triangulated 3D model of dog liver
Vertices in one contour need to be
matched with those in the next slice to
produce triangulated mesh.
Problem - different number of vertices in
each curve
Taku Komura
Problem 1 : Grow “hole”
High slice
Middle slice
Taku Komura
Problem 2 : split contours
Known as the “branching problem”.
Middle slice Low slice
Taku Komura
[Geiger '93]
OR OR
Possible Ambiguities Contour connection method
needs to handle these cases
Solution: NUAGES [Geiger '93]
− fill convex hull of contour with tetrahedra
− discard those outside the contours or with only one edge on each level
− 3D contour connection = remaining exterior tetrahedra faces
Taku Komura
NUAGES : Triangulate Contour Triangulate contour vertices using Delaunay Triangulation
− no inside / outside constraint
− forms planar convex hull of each contour
[Geiger '93]
Taku Komura
Delaunay triangulation
Input : vertices
Output : triangles composed by the vertices
The triangulation DT(P) of P such that no point in P is inside the circumcircle of any triangle in DT(P)
maximizes the minimum angle of all the angles of the triangles in the triangulation
Gives nice set of triangles for finite element analysis
Connecting the centres of the circumcircles produce the Voronoi diagram
Taku Komura
NUAGES : Form Tetrahedra
Form tetrahedra by joining closest vertex on opposite contour
− tetrahedra t1 and t2 formed with closest opposite vertex enclosed within area of triangle
− The vertex on the opposite side that is closest to the circumcenter is selected
− tetrahedra t12 formed by two edges that cross
− Vertices are added if necessary
[Geiger '93]
Taku Komura
NUAGES : Form Tetrahedra
Form tetrahedra by joining closest vertex on opposite contour
− t1 : A triangle on Plane 1 and point on t2
− t2 : A triangle on Plane 2 and point on t1
− t12 : connecting two edges each on t1 and t2
[Geiger '93]
Taku Komura
NUAGES : Remove Tetrahedra
Remove : Tetrahedra with an edge outside of the contour.
Remove : Non-solid ones
[Geiger '93]
Non-solid : tetrahedra not connected to t1 nor t2
Taku Komura
Problems The closest point on the other side might be too far
away We insert points by projecting the Voronoi skeleton of
the top contour to the bottom
Taku Komura
Voronoi diagram − We are given a set of points P.
− Each point p has a Voronoi cell V(p).
− V(P) consists of all points closer to p than any other points in P
− Created by connecting the center of circumcircles of the triangles produced by Delaunay triangulation
Taku Komura
Creating skeletons by the Voronoi diagrams
By connecting the edges of the Voronoi diagram of the points sampled over a closed polygon, we can produce its skeleton
The skeleton inside the polygon is called internal Voronoi skeleton (IVS), and that outside is called external Voronoi skeleton (EVS)
Taku Komura
Whether we insert points We check if the IVS of one side intersects with the
EVS of the other side If they do, the EVS is projected to the other side
and points are sampled over it
Taku Komura
NUAGES : Results
Use exterior triangles of remaining tetrahedra to form outside boundary of the shape
Upper Contour
Lower Contour
Connected : top down view
Connected : side view
[Geiger '93]
Taku Komura
NUAGES : Results
Multiple medical image slices joined by NUAGES
− http://www-sop.inria.fr/prisme/fiches/Medical/
Taku Komura
Dog Liver : Final Model
Completed model – despite presence of contour holes/ branches (NUAGES)
Taku Komura
Fully automatic segmentation is difficult
although you apply all great computer vision techniques
Segmentation is a low level procedure
− Based on colour information
− But users have high level knowledge
Taku Komura
Volume Catcher We need to give some knowledge to the system
− A good user interface can improve the efficiency of segmentation
− Sketch-based interface , Owada et al. I3D 2005
http://www.youtube.com/watch?v=2Utj9L6dLrE
Taku Komura
Specifying the Region of Interest The user specifies which area he/she is interested in The system finds out the color information inside and
outside the volume data The color information is used for region growing /
segmentation
Taku Komura
Contour-based Interface for Refining Volume Segmentation
An interface to refine the contours directly in 3D http://www.riken.jp/brict/Ijiri/movies/contourrefine.wmv
http://www.youtube.com/my_videos_upload?forceui=3
Taku Komura
Summary Subsurface Scattering
Volume Illumination
Segmentation
Readings Jensen et al. “A Practical Model for Subsurface Light Transport”,
SIGGRAPH 2001
http://http.developer.nvidia.com/GPUGems3/gpugems3_ch14.html
Marc Levoy, “Display of Surfaces from Volume Data”, IEEE Computer Graphics and Applications,Vol. 8, No. 3, May, 1988, pp. 29-37.
S Owada, et al. “Volume Catcher”, I3D 2005
Takashi Ijiri and Hideo Yokota: Contour-based Interface for Refining Volume Segmentation Computer Graphics Forum, 29(2007), 7, 2153-2160. (Pacific Graphics2010)