scalable visualization toolkits for bays to brains

NPACI: National Partnership for Advanced Computational Infrastructure

SDSC

The Scripps Research Institute

U. Texas, Austin

U.C.L.A.

SIO/MIT

U. California, Davis

Mississippi State U.

Scalable Visualization Toolkits for Bays to Brains


Motivation• Support analysis, filtering, and compositing

– Larger-than-core (and swap) volumetric data sets

– Multi-modal and time-varying data

– Multiple data sets simultaneously

• And...– Do efficient data movement

– Execute well on parallel architectures

– Integrate easily w/existing applications & toolkits

– Provide interactive user interfaces for volume exploration

CT

Cryosection

Classification


Motivating Applications• Develop state-of-the-art visualization of:

– Brain data• 1789 TIF color images

• 1472 x 1152 x 1789 x 3 (bytes) = 8.5 GB

– Ocean data• 2160 x 960 x 30 x 4 (bytes) = 0.2 GB

• 237.3 (MB) x 128 (time-steps) = 29.7 GB

• Surface height, salinity, temperature, velocity

• Demonstrate use beyond these fields


Toolkits

Function libraries providing scalable visualization features for new and existing volumetric applications


Layered Toolkit Architecture• Lowest layers provide access

to larger-than-core data sets– Automatic paging of

N-dimensional mesh data

– Local and remote (SRB) files

• Higher layers providelarge-data visualization– Optimized for best data movement

– Multi-data set scenes

– On-the-fly isocontouring during rendering

Volume Scene Toolkit

SRB & Local I/O

Isocontouring Toolkit

Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications


Mesh Toolkit• Manage a paged mesh (array-like)

– An N-dimensional grid of values

– Spatial data & time-series

– Arbitrary data content

– Multiple file formats

– Volume metadata

– Operations like filtering, sub-sampling, ...

• Layered atop local I/O and SRB


SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications


Mesh Toolkit• Intelligent pre-fetch by encapsulating common

access patterns– Random access (slow)

– Structured access (faster)

– Storage-order access (fastest)

• Neighborhood-aware paging– Page-in nearby data in N-dim.

– Convolution filtering, ...


SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications

5 13 24679 8

51 32 4 6 7 8 9

51 32 4 6 7 8 9

0 1 2 3 4

5 6 7 8 9

10 11 12 13 14

15 16 17 18 19

20 21 22 23 24

Current center Current center elementelement

Keep neighboring Keep neighboring elements paged-in as wellelements paged-in as well

Filter Filter windowwindow


Mesh Toolkit• Status

– C++ and Java implementations

– Fully documented

– SRB integration in progress

– In active use for large data manipulation

– Software release at 2001 All-Hands (with SRB)


SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications


Volume Scene Toolkit• Define a scene containing multiple data sets

– Leaf nodes are data sets, functions, ...

– Interior nodes composite, filter, ...

– Transforms align overlapping data sets

• Evaluate it to generate results– Client defines the scene

– Server on big iron evaluates it

– Layered atop Mesh Toolkit

SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications


ClientClient

ServerServer


Volume Scene Toolkit• Cut, Segment, Filter, and Combine data

– Manage multiple data sets of arbitrary data content

SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications



Volume Scene Toolkit• Status

– C++ prototype implementation

– In active use for multiple data sets• Head and brain data

• Tissue and cell data (Science cover)

• Other (Orion, ...)

– Software release in 2001

SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications



VolRen Toolkit• Data-efficient isocontouring

– Isocontours produce surfaces at chosen data values• Large data produces very large isocontour polygon sets

– Toolkit uses a fast seed-based approach• Small amount of pre-computation speeds up

isocontouring on later user requests

– Works from compressed data• Parallel visualization of compressed streams

SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications



SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications


VolRen Toolkit• Merged isocontouring and rendering

– Isocontour extraction during volume rendering• Avoids creation & storage of large polygon sets

– Raycasting rays intersect volume &isosurfaces, properly mixing colors and shading

• Parallel out-of-core algorithm– Image-space partitioning

– Master-slave structure

– Low data communication cost due todata pre-processing and reorganization


VolRen Toolkit• Available as a toolkit and a stand-alone system

– Can integrate into applications, or

– Client-server system for rendering of high-resolution videos

– Client-server system for interactive exploration of time-dependent data

SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications


ClientClient

ServerServer

Param


VolRen Toolkit• Status

– C++ implementation

– Additional features in development (incorporation of streaming images and geometry- progressive)

– In active use for multiple data sets (structured and unstructured data)

– will layer atop mesh-toolkit

– Software release 1st quarter 2001 (toolkit)

– remote access (client-server) release 2001

SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications



Rendering Toolkit• High-performance perspective volume renderer

– Optimized for parallel architectures

– Efficient data movement for larger-than-core rendering

• Available as a toolkit and a stand-alone renderer– Can integrate into applications, or

– Client sends parameters

– Server on big iron renders usingserver-side high-resolution data

SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications


ClientClient

ServerServer

VISTA


Rendering Toolkit• Render volume meshes

– Perspective ray-casting of individual volume data sets

– Scenes voxelized first at arbitrary resolution

SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications

Volume Scene Toolkit VISTA


Rendering Toolkit• Render volume scenes

– Perspective ray-casting of volume scenes

– Only evaluates scene where needed

– Particularly good for time-varying scenes

SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications



Rendering Toolkit• Status

– C++ prototype implementation (based on Orion renderer)

– In active use for multiple data sets

– Software release mid-2001

SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications



3D GUI Toolkit• Common GUI components for 3D

– Generic parts for quick GUI development

– 3D drawing windows

– 3D manipulators (shape handles)

– 3D input device handlers

– Point cloud & textured volume shapes

SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications



3D GUI Toolkit• Status

– Java partial implementation

– Additional features in development

– In active use in multiple 3D viewers


SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications



Tools

Stand-alone interactive applications and serversbuilt using the scalable visualization toolkits


Multi-Mesh Viewer• Interactive multi-mesh viewer

– Built atop Mesh & 3D GUI Toolkits

– Client-server access to Rendering Toolkit

– Uses standard 3D graphics hardware

– Single and multiple volume viewing

SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications


Interactive

Rendered


Multi-Mesh Viewer• Data-efficient visualization

– Adjustable viewing resolution

– Movable region-of-interest

– Client-side proxy volumes forhigh resolution data

– Client-side interaction withserver-side rendering

SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications


Interactive Rendered


Multi-Mesh Viewer• Status

– Java implementation

– Portable: PC, SGI, …

– SRB integration in progress

– In active use to interactively view meshes

– Software release at 2001 All-Hands (with SRB)

SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications



SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications


Python Volume Viewer• Hardware accelerated visualization

– Built atop Mesh Toolkit +DejaVu, VLI, & Python language

– Uses VolumePro rendering hardware• Renders 2563 volume in real time

– Interactively adjust color, lighting, ...

PVV


Python Volume ViewerSRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications


• Data-efficient multi-resolution visualization– Interactive zoom between low & high resolution data

PVV


Python Volume Viewer• Status

– Python using C++ toolkit implementation

– Portable: PC, Sun, SGI

– In active use to interactively visualize meshes and explore lighting, color-table, opacity and viewing parameters

– Software release in 2nd quarter 2001

SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications

Volume Scene Toolkit PVV


Java Mesh Visualizer• Integrated visualization tool

– Built atop Mesh & 3D GUI toolkits +Shastra collaboration suite

– Client-server access to data servers,renderers, and isocontouring

– Web-based applet orstand-alone application

SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications

Volume Scene Toolkit JVisTool


Java Mesh Visualizer• Flexible visualization

– Adjustable cutting planes

– Isocontouring control

– Integrated collaboration features

– Client-side proxy volumes forhigh resolution data

– Client-side interaction withserver-side rendering & isocontouring

SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications



Java Mesh Visualizer• Status

– Java implementation

– Portable: PC, SGI, …

– In active use to visualize meshes and isocontours


SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications



Volume Scene Editor• Interactive volume scene editor

– Built atop Mesh, Scene, & 3D GUI Toolkits

– Client-server access to Rendering Toolkit

– Diagramatic scene structure editing

– Client-side interaction and editing withlow-resolution proxy volumes

– Server-side rendering withhigh-resolution volumes

SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications



Volume Scene Editor• Data-efficient multi-mesh visualization

– Efficient compositing of multiple large meshes

– Multi-modal & time-varying visualization

– Interactive volume scene layout

– Interactive control of filters, compositing, lighting, …

– Scene and viewpoint animation control

SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications



Volume Scene Editor• Status

– C++ early prototype complete

– Prototype in active use

– Java re-design in progress


SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications



Visualization Results

New techniques to better visualize large, complex data sets - from Bays to Brains and beyond...


Brain Visualization• Cleaning cryosection image data

– Slices show ice and clutter around the brain

– Must remove this clutter before visualization• No real surface (cavities!)

– Auto-segmentation uses YIQ color model, filtering, region growing, dilation, segmentation

Remove exterior Detect cavities


Brain Visualization• Compressing data using wavelets

– Enables multi-resolution visualization with same data size

– Each step separates low and high frequency components

original (256 x 256) Step 1 Step 2


Brain Visualization• Visualizing the cleaned brain data set

– Cutting, segmenting, classifying, lighting, rendering, ...


Oceanography Visualization• Visualizing temperature

– Temperature mapped to color

– Isocontours + volume rendering


Oceanography Visualization• Visualizing velocity

– Vector fields atop volume rendering


Oceanography Visualization• Visualizing velocity

– Vector direction mapped to color

– Vector magnitude mapped to intensity


SDSC

The Scripps Research Institute

U. Texas, Austin

UCLA

SIO/MIT

U. California, Davis

Mississippi State U.

Scalable Visualization Toolkits for Bays to Brains

scalable visualization toolkits for bays to brains

Documents

chosen data valueslarge

data preprocessing

nearby data

overlapping data sets

multiple data setshead

efficient data movementexecute

multiple data setsleaf

mesh toolkitstatusc