Volume VisualizationVolume Visualization

Visualization IIVisualization IIMSIM 842, CS 795/895MSIM 842, CS 795/895

Instructor:Instructor:Jessica CrouchJessica Crouch

Volume Viz Problem:Volume Viz Problem:

Data points fill a 3D volumeData points fill a 3D volume

You can only display one 2D image at a You can only display one 2D image at a timetime

Naïve rendering of a volume will just let Naïve rendering of a volume will just let you see the surfaceyou see the surface

How can you visualize the volume data How can you visualize the volume data in a way that lets you understand the in a way that lets you understand the internal structure of the data?internal structure of the data?

Structured vs. Scattered Structured vs. Scattered DataData

The best approach will depend on the The best approach will depend on the characteristics of the datacharacteristics of the data

Data points arranged in rows, columns, Data points arranged in rows, columns, and layers (3D image) can be referred to and layers (3D image) can be referred to as voxels (volumetric-pixels)as voxels (volumetric-pixels)

Data points that follow a non-gridded Data points that follow a non-gridded pattern are called “scattered”pattern are called “scattered”

Wikipedia image

Volume Viz & Data Volume Viz & Data DimensionalityDimensionality

Each data point hasEach data point has 3D coordinates (x,y,z), specifying a 3D coordinates (x,y,z), specifying a

position in the volumeposition in the volume

and one of the following:and one of the following:– Scalar – a single number, 0D dataScalar – a single number, 0D data– Vector – a list of numbers, 1D dataVector – a list of numbers, 1D data– Tensor – a matrix or ND array of numbers, for Tensor – a matrix or ND array of numbers, for

N>1N>1

Data examplesData examples Scalar DataScalar Data

– CT Scan CT Scan →→ x-ray opacity for each voxel x-ray opacity for each voxel– PET Scan PET Scan → positron radiation for each voxel→ positron radiation for each voxel– Atmospheric simulation (or measurement) Atmospheric simulation (or measurement) →→ pollutant concentration pollutant concentration– Ocean temperature distributionOcean temperature distribution

Vector DataVector Data– Velocity vectors at each voxelVelocity vectors at each voxel

Fluid flow (river water, arterial blood, etc.)Fluid flow (river water, arterial blood, etc.)– Displacement vectors at each voxelDisplacement vectors at each voxel

Provides mapping between different deformed configurations of the same Provides mapping between different deformed configurations of the same objectobject

Tensor dataTensor data– Higher dimensional data is more complicated:Higher dimensional data is more complicated:

Stress and strain matricesStress and strain matrices Gradients of 3D functionsGradients of 3D functions

Simplest CaseSimplest Case

Let’s consider continuous, scalar, voxel Let’s consider continuous, scalar, voxel datadata– Geometrically regular arrangement (3D grid)Geometrically regular arrangement (3D grid)– Just 1 data value per voxelJust 1 data value per voxel

Think of CT data (3D x-ray)Think of CT data (3D x-ray) What would you be interested in seeing?What would you be interested in seeing? How would you construct a visualization?How would you construct a visualization?

Simplest possible Simplest possible visualization is to just look at visualization is to just look at the slicesthe slices– Use color mapping (or Use color mapping (or

grayscale mapping) to see slice grayscale mapping) to see slice valuesvalues

Slices can be usefulSlices can be useful

We can do more interesting We can do more interesting thingsthings

Scalar Voxel Viz.Scalar Voxel Viz.

Scalar Voxel Viz.Scalar Voxel Viz.

Bone voxels will be bright (have large values), other Bone voxels will be bright (have large values), other voxels will be darkvoxels will be dark

How could you visualize the surface of the bones?How could you visualize the surface of the bones?– Several alternatives exist, including:Several alternatives exist, including:

1. Render an isosurface: Use Marching Cubes Algorithm1. Render an isosurface: Use Marching Cubes Algorithm

2. Ray Casting: Integrate scalar variable along the view 2. Ray Casting: Integrate scalar variable along the view directiondirection

3. Splatting3. Splatting

Isosurfaces from 3DIsosurfaces from 3D How to construct a polygonal surface from a 3D How to construct a polygonal surface from a 3D

dataset?dataset?

Find points on the isosurface of a scalar variableFind points on the isosurface of a scalar variable– 3D analog of 2D contour maps3D analog of 2D contour maps– You pick a meaningful threshold valueYou pick a meaningful threshold value

Voxels with data values below the threshold sit on one side of Voxels with data values below the threshold sit on one side of the isosurface, voxel with data values above the threshold sit on the isosurface, voxel with data values above the threshold sit on the otherthe other

Simplifying assumption is that threshold value = 0 Simplifying assumption is that threshold value = 0 – To force this, just subtract the desired threshold value from all data To force this, just subtract the desired threshold value from all data

pointspoints

Tessellate points on the isosurface to create a Tessellate points on the isosurface to create a polygonal representation of the surfacepolygonal representation of the surface– The “Marching Cubes” algorithm does thisThe “Marching Cubes” algorithm does this

This is one of the famous, older (>20 yrs) viz. algorithmsThis is one of the famous, older (>20 yrs) viz. algorithms

Render the tessellated isosurface using normal 3D Render the tessellated isosurface using normal 3D graphics methodsgraphics methods

Isosurfaces & OpacityIsosurfaces & Opacity

The Visible Man: Image The Visible Man: Image DataData

Visible Man IsosurfacesVisible Man Isosurfaces

Marching Cubes Marching Cubes ConsideredConsidered

How effective is the visualization How effective is the visualization perceptually?perceptually?– What can you see?What can you see?– What can’t you see?What can’t you see?

How is this in terms of efficiency?How is this in terms of efficiency?– Phase IPhase I– Phase IIPhase II– RenderingRendering

Compare other approaches…Compare other approaches…

Ray Casting Volume DataRay Casting Volume Data

Review Ordinary Ray Review Ordinary Ray CastingCasting

How does it work?How does it work?

Ray Casting with a VoxelRay Casting with a Voxel

Ray Casting forRay Casting for Approximation of Light Integral Approximation of Light Integral

ExamplesExamples

Ray Casting ConsideredRay Casting Considered

How effective?How effective?– How are the images different from How are the images different from

those produced by Marching Cubes?those produced by Marching Cubes?

How efficient?How efficient?

Another alternative…Another alternative…

Volume SplattingVolume Splatting

Ray casting is an Ray casting is an image spaceimage space methodmethod– Ray are generated per pixel, and travel Ray are generated per pixel, and travel

out into the volumeout into the volume

Volume splatting is an Volume splatting is an object spaceobject space methodmethod– Data points in the volume are mapped Data points in the volume are mapped

((splattedsplatted) onto the image plane) onto the image plane

Splatting: A Feed Forward Splatting: A Feed Forward ProcessProcess

Feed forward: Splatting

Splat!

Splatting (feed-forward) Splatting (feed-forward)

?

Fill the holesFill the holes

We need to fill the pixel values between the volume projection samples

?

3D Kernel for Splatting3D Kernel for Splatting

Need to know the 3D extent of each voxel, and Then project the extent to the image plane

This is called ‘footprint’

VisibilityVisibility

How does occlusion work?How does occlusion work?

Splatting uses back to front Splatting uses back to front compositing compositing

Samples in front are added on top Samples in front are added on top of and partially obscure previously of and partially obscure previously splatted (further back) samplessplatted (further back) samples

Gaussian kernel Gaussian kernel

A popular kernel is the Gaussian A popular kernel is the Gaussian function (think bell curve)function (think bell curve)

Sigma controls the widthSigma controls the width Gaussians have circular “splat” Gaussians have circular “splat”

footprintsfootprints

Ray Casting vs. SplattingRay Casting vs. Splatting

What are the differences in the images What are the differences in the images produced?produced?– Think about aliasingThink about aliasing

Efficiency considerations?Efficiency considerations?– In what order are the voxel data points In what order are the voxel data points

accessed?accessed? For ray casting?For ray casting? For splatting?For splatting?

Summary of TechniquesSummary of Techniques

Marching cubes: Marching cubes: – Construct polygonal isosurfacesConstruct polygonal isosurfaces– Render with regular graphics pipelineRender with regular graphics pipeline

Ray casting:Ray casting:– Image space algorithmImage space algorithm– Shoot rays from image into volume, integrate color Shoot rays from image into volume, integrate color

& opacity along each ray& opacity along each ray

Volume splatting:Volume splatting:– Object space algorithmObject space algorithm– Project each voxel onto the image planeProject each voxel onto the image plane

These are good ways of looking directly at the These are good ways of looking directly at the recorded data. What could you do to focus recorded data. What could you do to focus attention on significant features in the data?attention on significant features in the data?

Feature ExtractionFeature Extraction

What is a significant feature?What is a significant feature?– Answer is application dependentAnswer is application dependent– Often, regions with a high gradient Often, regions with a high gradient

are significantare significant

What is the gradient of a voxel What is the gradient of a voxel volume?volume?– Look at the gradient of a 2D image…Look at the gradient of a 2D image…

1010 5050 252500

101000

1010

1010 5050 252500

101000

1010

252500

252500

252500

252500

252500

101000

101000

252500

101000

101000

1010 1010 252500

1010 1010

0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

5.5

2525 120120 2525 -120-120 -50-50

2525 120120 2525 -120-120 -50-50

125125 00 00 00 -125-125

5050 7575 00 -75-75 -50-50

55 120120 00 -120-120 -5-5

-.5-.5 00 .5.5

-.5-.5 00 .5.5

-.5-.5 00 .5.5

Derivative in X direction

1010 5050 252500

101000

1010

1010 5050 252500

101000

1010

252500

252500

252500

252500

252500

101000

101000

252500

101000

101000

1010 1010 252500

1010 1010

0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

5.5-5-5 -25-25 -125-125 -50-50 -5-5

-120-120 -100-100 00 -75-75 -120-120

-45-45 -25-25 00 00 -45-45

120120 120120 00 120120 120120

5050 5050 125125 5050 5050

.5.5 .5.5 .5.5

00 00 00

-.5-.5 -.5-.5 -.5-.5

Derivative in Y direction

Gradient Direction VectorsGradient Direction Vectors

2525 120120 2525 -120-120 -50-50

2525 120120 2525 -120-120 -50-50

125125 00 00 00 -125-125

5050 7575 00 -75-75 -50-50

55 120120 00 -120-120 -5-5

-5-5 -25-25 -125-125 -50-50 -5-5

-120-120 -100-100 00 -75-75 -120-120

-45-45 -25-25 00 00 -45-45

120120 120120 00 120120 120120

5050 5050 125125 5050 5050

dI/dx: dI/dy:

0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

5.5

Volume gradientVolume gradient

Is a 3x1 vectorIs a 3x1 vector [dI/dx dI/dy dI/dz][dI/dx dI/dy dI/dz]

Vector points “uphill” in direction of Vector points “uphill” in direction of increasing data valueincreasing data value

Vector length (magnitude) indicates Vector length (magnitude) indicates how quickly data values in the how quickly data values in the neighborhood are changingneighborhood are changing

Gradient MagnitudeGradient Magnitude

Given scalar data, the gradient Given scalar data, the gradient magnitude can be computed magnitude can be computed – Result is Result is another scalar data setanother scalar data set– Each voxel value gets the length of Each voxel value gets the length of

the gradient vector computed for the gradient vector computed for the original datathe original data

Large gradient magnitude values Large gradient magnitude values in a region indicate in a region indicate sharp sharp changeschanges in the original data in the original data

Gradient Magnitude Viz.Gradient Magnitude Viz.

Sharp changes are often significantSharp changes are often significant– May indicate an edge or surface May indicate an edge or surface

Large gradient in CT data indicate a voxel is on Large gradient in CT data indicate a voxel is on bone surfacebone surface

– May indicate where something important is May indicate where something important is happening happening Large temperature gradient in atmosphere can Large temperature gradient in atmosphere can

indicate a weather “front”indicate a weather “front”

Can apply the three visualization Can apply the three visualization methods already discussed to the methods already discussed to the gradient magnitude datagradient magnitude data

Other feature extraction Other feature extraction methodsmethods

Can apply other types of pre-processing Can apply other types of pre-processing to the input data to find areas of interestto the input data to find areas of interest– Search for blobs of a certain sizeSearch for blobs of a certain size– Search for areas with a certain texture or Search for areas with a certain texture or

patternpattern– Filter out high frequency noise from input Filter out high frequency noise from input

datadata

Create new data set by assigning voxel Create new data set by assigning voxel values based on how well original data values based on how well original data matches the search criteriamatches the search criteria

Have to know what you’re looking for, Have to know what you’re looking for, what defines “significant” for a what defines “significant” for a particular applicationparticular application

Still to come…Still to come…

Consideration of volume viz. for Consideration of volume viz. for higher dimensional datahigher dimensional data– VectorsVectors– TensorsTensors– Time-varying sequencesTime-varying sequences– Data with probabilistic uncertaintyData with probabilistic uncertainty

StreamlinesStreamlines

Follow the flow of a vector field, tracing the path a particle would take

Tensor GlyphsTensor Glyphs

Ellipsoids, Ellipsoids, Cuboids, Cuboids, Superquadratics Superquadratics (see right), & (see right), & other 3D other 3D polyhedra and polyhedra and implicit surface implicit surface shapesshapes

For 3x3 matrices, For 3x3 matrices, illustrates 3 axes illustrates 3 axes (or eigenvectors)(or eigenvectors)