Supporting Phenotyping through Visualization and Image Analysis

Raghu Machiraju,Computer Science & Engineering,

Bio-Medical InformaticsThe Ohio State University

About Myself Associate Professor, Computer Science and

Engineeering, BioMedical Informatics 7th Year at OSU Research Interests – Imaging, Graphics and

Visualization Notable Points

Co-Chair of Visualization 2008 Conference, Columbus OH Alumni in video gaming/animation industry (Pixar, EA), National

Government Labs (Lawrence Livermore), Industrial Research (Samsung, IBM, Mitsubishi Electric), Medical Schools (Harvard Medical School)

Research Activities

Medical, Biological Imaging and Visualization

Optical Microscopy

In-vivo, fluorescence imaging

Structural/Functional Magnetic Resonance Imaging

Diffusion Tensor Imaging

•Mostly interested in:

• Segmentation, Registration, Tracking

• Applications: phenotyping, longitudinal studies

Stained (H&E) Light Microscopy Stack

Confocal Microscopy Stack

Reconstruction of Microscopic Architecture

Cellular structures near mammary gland of a female mouse Source: Dr. Leone, Cancer Genetics, OSU

Embryonic Structure of Zebra Fish, Source: Dr. Sean Megason, Harvard Medical School

My Colleagues …

Kishore Mosaliganti, 5th yearBioinformatics/Cancer Genetics

Kun Huang, Biomedical Informatics

Gustavo Leone, Mike Ostrowski

Human Cancer Genetics Program

The Usual Imaging Pipeline

Harvest RbHarvest Rb- - && RbRb+ + micemice Sectioning - 5 micronsSectioning - 5 microns ImagingImaging

VisualizationVisualization

An Advanced Role for Imaging Support Mouse Placenta

Role of Rb tumor suppressor gene Changes in placental morphology Fetal death and miscarriages

Large data size High resolution image (~1 GB) 800~1200 slides/dataset

Quantification Surface area/volume of different tissue

layers Infiltration between tissue layers

Need More - Morphometric Differences

Labyrinth-Spongiotrophoblast Interface

Wild Type (Top) vs. Mutant (Bottom)

Yet Another (A)Typical Example

Mouse Mammary Gland PTEN phenotyping Data characteristics

High resolution 20X images (~1 GB) 500 slides/dataset

Mammary duct segmentation and 3D reconstruction

Digging In - Tumor Micro-Environment

Mouse Mammary Gland More comprehensive system biology study Data characteristics

Confocal, multi-stained 50 slides/dataset

Multi-channel segmentation and 3D reconstruction

The Last One - Zebrafish Embryogenesis

Identifying and tracking development in the embryo Presence of salient structures

3D cell segmentations and tracking required Different in-plane and out-plane resolutions 800 Time steps available

A 2D image plane Final 3D segmentation

The Underlying Premise

Is there an unified way to visualize and analyze the various microscopic image modalities ?

The Essentials Of Microstructure

Premise - you can measure, visualize and analyze cellular structures if you characterize and build virtual microstructure

ComponentDistributionsPackingArrangements

Material Interfaces

Essential I- Component Distributions & Packing

Tissue layers differ in spatial distributions Characteristic packing of RBCs, nuclei, cytoplasm - phases Differ in porosity, volume fractions, sizes and arrangement NOT JUST ANOTHER TEXTURE ! Use spatial correlation functions !

Essential II - Component Arrangements

Arrangements Complex tessellations which can better characterize changes. A step ahead of looking at only nuclei their packing

Complex geometry Concentric arrangement of epithelial cells Torturous 3D ducts and vasculature

Essentials III – Material Interfaces

Labyrinth-Spongiotrophoblasts Interface

The Holy Grail – Virtual Cellular ReconstructionsBefore using cellular segmentation Using N-pcfs and cellular segmentations

Pipelines

1 TeraByte1 TeraByte

1Gb x 1 Gb x 9001Gb x 1 Gb x 90020 x magnification20 x magnification

Image Registration (3-D alignment) Feature extraction

Image Segmentation

3-D Visualization

Quantification

NIH Insight Tool Kit (ITK), NA-MIC Tools (microSlicer3)

Conclusions Highly multi-disciplinary approach. Need scalability and robustness Useful workflows need to be constructedMuch application-domain knowledge has to be

embedded in algorithmsValidation of methods and proving robustness is a

pre-occupation.The final goal of a virtual cellular architecture is

not that elusive

Destroying The Amazon Rain Forest K. Mosaliganti and R. Machiraju et al. An Imaging Workflow for Characterizing Phenotypical

Change in Terabyte Sized Mouse Model Datasets. Journal of Bioinformatics, 2008 (to appear) K. Mosaliganti and R. Machiraju et al. Visualization of Cellular Biology Structures from Optical

Microscopy Data. IEEE Transactions in Visualization and Computer Graphics, 2008 (to appear) K. Mosaliganti, R. Machiraju et al. Tensor Classification of N-point Correlation Function features

for Histology Tissue Segmentation. Journal of Medical Image Analysis, 2008 (to appear) K. Mosaliganti and R. Machiraju et al. Geometry-driven Visualization of Microscopic Structures in

Biology. Workshop on Knowledge-Assisted Visualization, Proceedings of EuroVis2008 (to appear).

K. Mosaliganti, R. Machiraju et al. “Detection and Visualization of Surface-Pockets to Enable Phenotyping Studies”. IEEE Transactions on Medical Imaging, volume 26(9), pages 1283-1290, 2007.

R. Sharp, K. Mosaliganti et al. “Volume Rendering Phenotype Differences in Mouse Placenta Microscopy Data”. Journal of Computing in Science and Engineering, volume 9 (1), pages 38-47, Jan/ Feb 2007.

P. Wenzel and K. Mosaliganti et al. Rb is critical in a mammalian tissue stem cell population. In Journal of Genetics and Development, volume 21 (1), pages 85-97, Jan 2007.

K. Mosaliganti and R. Machiraju et al. Automated Quantification of Colony Growth in Clonogenic Assays. Workshop on Medical Image Analysis with Applications in Biology, 2007, Piscatway, Rutgers, New Jersey, USA.

R. Ridgway, R. Machiraju et al. Image segmentation with tensor-based classification of N-point correlation functions. In MICCAI Workshop on Medical Image Analysis with Applications in Biology, 2006.

O. Irfanoglu, K. Mosaliganti et al. “Histology Image Segmentation using the N-Point Correlation Functions”. International Symposium of Biomedical Imaging, 2006.

Acknowledgements

Joel Saltz, BMI Richard Sharp, Okan Irfanoglu, Firdaus Janoos,

CSE OSUWeiming Xia, Sean Megason, Harvard Medical

school Jens Rittscher, GE Global ResearchNIH, NLM Training GrantNSF ITR grant

Thank You !

Questions ?