NCBO/NCOR Image Ontology Mtg - 3/2006

BIRN Image Ontology Requirements

Maryann Martone & Jeff Grethe (UCSD)

Bill Bug (Drexel U. College of Medicine)

A shared biomedical IT infrastructure to hasten the

derivation of new understanding and treatment

of disease through use of distributed knowledge

Biomedical Informatics Research Network

• Collaboration between groups with different expertise and resources (technical, scientific, social and political)

• Technical infrastructure to support collaboration (designed to be extensible to other biomedical communities)

• Open access and dissemination of data and tools (i.e. Open Source)

• Bringing transparent GRID Computing to Biomedical Research

BIRN Image Related Ontology Use

Critical to BIRN-wide:

•data integration

human experts & machine-based

•data reduction

across BIRN sites (29 locations - multiple labs/location) and with the rest of the world

•analysis

•knowledge extraction

•data pooling/binning/sorting/modes

•automated data validity & integrity

BIRN Image Related Ontology Use

A work in progress

• BIRN IT Infrastructure maturing

• Broad spectrum of Image-related tools & data

• software & APIs for handling files - many image-specific

• spatially-mapped neuro-data sets

• visualization, spatial normalization & analysis

• Intra-BIRN and external collaborative practice maturing

• workflow - models of computation (e.g., Ptolemy / Kepler)

BIRN Image Related Ontology Use

A work in progress• Ontology tools & ontology use rapidly evolving

• BONFIRE (tool)

• BIRNLex (lexicon)

• MIND (ontology) - only what we cannot re-use

• BIRN Ontology use• expect to make mistakes

• have a huge task - must start yesterday

• BIRN ontology “best practices” - social engineering

• BIRN Project - comprehensive Use Case set• research use of biomedical image related ontologies

finish

BIRN Image Related Ontology Use

• Image Ontology Consortium (burgeoning)

We need your help

• Beat us up

BIRN will need

• triage

• chronic Rx

• BUT

BIRN

Image Ontology Consortium

BIRN Image-Related Ontology Requirements

Biological Specimen

Imaging real-world bio-entities: a taxonomy of parts

• molecules

• small : pre-promelanocortin, etc.• multi-subunit macromolecular complexes

• large: transcriptional apparatus, etc

• sub-cellular components

• Purkinje cell body layer in the cerebellar lobes, gomeruli of the olfactory bulb, etc.

• cells• mesoscopic cellular complexes

• small: Golgi apparatus, lysosomes, dendritic spines, transmitter vesicles, etc.• large: dendrites, axon hillock, nodes of Ranvier, presynaptic compartment, etc.

• brain nuclei• macroscopic brain regions• organs & body regions• whole subject

BIRN Image-Related Ontology Requirements

Imaging the Biological Specimen

material entity indirectly observed via the detector

• Resulting image combination of:

•physical scale of the object imaged

•physical laws governing interaction between:

•physical scale of the imaging detector

• incident radiation source and applied fields (EMF)

• device components effecting incident fields (lenses, filters)

• specimen

• device components downstream of specimen (lenses, filters, detector(s) )

BIRN Image-Related Ontology Requirements

Imaging the Biological Specimen

Specimen Physical Properties• interaction with incident/ stimulus EMF

Contrast Enhancement •Better living through chemistry

•Histochemical treatment• differentially alter specimen light-scattering properties to enhance reflected or

transmitted light differential

•Emission-based probes• highly selective affinity for specimen molecular determinant

• radio-isotopes

• fluorescent probes

•Effect NMR profile• MRI contrast agents

BIRN Image-Related Ontology Requirements

Imaging Modalities

• MRI• standard structural MRI• diffusion-tensor imaging MRI (DTI-MRI)• functional MRI (fMRI)

• PET• optical Imaging

• ultrasound/echo = not in BIRN yet

• transmittance/absorbance

• reflectance• fluorescence

• brightfield, phase-contrast

• standard wide-field, confocal (optical/LSCM, digital deconvolution), multi-photon• electron microscopy

• TEM• 2D TEM & 3D TEM (EM tomography)

• SEM• video

• vital cellular imaging• behaving subjects

• provenance data varies widely by technique• typical lab likely producing only 1 or 2 of these

BIRN Image-Related Ontology Requirements

Cross Image Comparison

intra-modality comparison

• image acquisitionProvenance (KEY)

• image manipulation

•? - need ontologies beyond Image & Computation

•calibration via controlled specimen mimics• standards: Air Force target, uranium glass, fluorescent beads (size & intensity)

• phantoms

• device characteristics & image protocol parameters

• algorithm(s) / workflow

• models of computation

background vs. signal•only a subset of specimen determinants are under study

inter-modality comparison

•highly controlled assay / protocol

use Image Ontology

use Computation/SW Ontology

Ways of Knowing Neuroanatomy

Neuroscience Lexicon

Ontologies

Voxel-based data analysis & abstraction

• NeuroNames, BIRNLex, RadLex, NIF, etc.• “control” vocabulary use?• manage lexical exceptions

• FMA, RadIO, etc.• support mereotopological reasoning

• feature (fiducial)-based & voxel-based spatial normalization

• BIRN-wide morphological & stereological analysis

• analysis of spatially-mapped neuro-data sets

• NLP KR extraction

Ways of Representing Neuroanatomy

• across sites, specimens, scales & imaging modalities

• homographic homonyms, synonyms, eponyms, etc.

Ways of Representing Neuroanatomy

Must integrate these 3 informatic threads - and eventualy

• literature informatics (The Bibliome)

• biological models

Integrate Images Across• distinct spatial scales

• distinct imaging modalities

• distinct subject image sets (species/strain/sub-strain/…/subject)

• BIRN sponsored meeting?

• parameterized view of 3D data sets can link ontology view to voxel view

• contact Maryann Martone or Jeff Grethe

BIRN Image-Related Ontology Requirements

BIRN Use Cases for

Image-Related Ontology

fBIRN: Functional MRI

Reference Anatomical ScanfMRI Scans from 10 Different Sites

• Same Subject, Registered, Same Slice

Calibration

Phase I Traveling Calibration Phase I Traveling Calibration Subject Dataset AvailableSubject Dataset Available

Phase I Traveling Calibration Phase I Traveling Calibration Subject Dataset AvailableSubject Dataset Available

Image-related Ontology Entities

Multi-site fMRI imaging variability using the same subject

• subject/anatomy

• Functional view of anatomy changing over time

• device

• assay

• algorithm

General Issues

• The same real-world biological entity when assayed under the same conditions should give consistent results

• Despite their being imaged on different devices in different labs

CorrectedUncorrected

Image intensity variability onsame subject scanned at 4 sites

• Common acquisition protocol, distortion correction, and evaluation to improve reproducibility, within- and across-sites.

mBIRN: Structural MRI Controlling for distortions caused by field gradient nonlinearities

Siemens - Whole-BodySymphony/SonataMax displ. 2.5/3.2mm

GE - Whole-BodyCRM NVi/CVi

Max displ. 4.2/8.6mm

Siemens - Head-OnlyAllegra/AC-44

Max displ. 5.7/20.2mm

Image-related Ontology Entities

General Issues

• subject/anatomy

• device

• assay

• algorithm

• applied field

• orientation/extent

• Distortion depends on size and orientation of subject & idiosyncrasies of field propagation

• Imaging canonical sample must give near identical results:

• tightly controlled acquisition conditions

• device-specific distortion correction

• proper normalization

• Protocol has implied logical consequences

• Scan human phantoms multiple times at each site

Siemens

GE

NO DISTORTION CORRECTION

Same Subject Co-registered

DISTORTION CORRECTION

Distortion correction does improve cortical surface co-registration

Distortion correction does improve cortical surface co-registration

Alignment of Surfaces Needs to be accurate and reproducible

Cortical Estimates (size, shape, & opacity)

Image-related Ontology Entities

General Issues

• subject/anatomy

• device

• assay

• algorithm

• applied field

• Separating biological effects from artefactual variability and systematic error

• Significant biological effects MUST be repeatable in a statistically significant way

Parcellation • Freesurfer (MGH)

Tractography• DoDTI (H.J. Park)

Visualization• Slicer (BWH)

Developing a White Matter Atlas Using Diffusion Tensor Imaging MRI (DTI-MRI)

Image-related Ontology Entities

General Issues

• subject/anatomy• boundaries

• device• DTI-specific

• assay

• algorithm

• Defining tract boundaries & spatial relations very difficult

• Need to relate to functional brain maps

• DTI - specific hypothesis about biological reality postulated based on assumptions about relative H2O mobility

Raw data De-faced data

•Automated de-facing without brain removal

MRI De-identification Required for HIPAA compliance

Image-related Ontology Entities

General Issues

• subject/anatomy• boundaries

• subject/species• human only

• Defacing critically sensitive to definition of boundaries.

• Must maintain link to data related to individual subject without including personal identifiers.

Pipeline for bulk network file management

Automated Upload Pipeline Automatic network-based, processing and validation

Image-related Ontology Entities

General Issues

• subject/anatomy• boundaries

• robust & automated• upload:

• MRI Storage Request Broker (SRB) network file storage

• metadata SRB & database warehouse• integration & interoperability:

• diverse inputs

• sharable network repository

• BIRN-wide common data access stack (BIRN Rocks)

warehouseBIRN

SRB

Generate and/or validate:• De-identification (HIPAA)

• Quality Assurance (QA)

• Image format

• BIRN ID

• distributed GRID architecture (BIRN Racks)

• QA• based on real-world

criteria

• Automated data handling

• QA criteria:

• reliably represent biological entities across sites, devices and subjects

• can have very complex assumptions with significant ontological component

Mouse BIRN Animal models of human neurological disorders

Study animal models to test hypotheses about human neurological disorders

• across dimensional scales• across imaging modalities

Experimental Allergic Encephalomyelitis (EAE) mouse model

Multiple Sclerosis (MS)

Dopamine Transporter (DAT) knockout mouse

schizophrenia, attention-deficit hyperactivity disorder (ADHD), Tourette’s disorder, and substance abuse

Alpha-synuclein mouse model

symptoms/pathology of Parkinson’s Disease

Cancer animal models consortium with astrocytoma mouse model

NCI supported with Terry Van Dyke @ Duke

Cal Tech - Duke - UCLA - UCSD - U.Tenn./Drexel U.

4. Use integration

environment to navigate and query

across data sources

1. Create site-specific databases

3. Situate the data in a common spatial

framework

BIRN Data Integration Example of multi-scale, multi-modal data federation from Mouse BIRN

Image-related Ontology Entities

General Issues

• subject/anatomy• spatial reasoning

• subject/species• environment• phenotype attributes

• Must perform complex cross-correlations between spatial location, subject genotype, environmental treatment, phenotype attributes

• Across species, subjects, imaging modalities, laboratories

2. Create relevant links to a shared

ONTOLOGY

Brain volume spatial registration processing stream

Spatial Registration Workflow

Image-related Ontology Entities

General Issues

• subject/anatomy• spatial reasoning

• device

• assay

• algorithm / workflow

• Algorithms must be specified via models of computation (e.g., Ptolemy) to support full automation

• Gray-scale voxel-based analysis (feature-based & numerical) must seamlessly combine with mereotopological & histo-cellular models of neuroanatomy

Semi-automatic spatial normalization across a large,

multifarious neuroimage repository

• pipelines:

Register:• image slice series and volumetric atlases

Correlate:• cellular and sub-cellular changes with non-invasive imaging

• LONI• NeuroTerrain

• GRID:• GRIDSphere• Kepler

BIRN SMART Atlas

• Ilya Zaslavsky• Joshua Tran• Haiyun He• Amarnath Gupta

GIS-like tool spatially integrating multi-scale distributed brain data.

Image-related Ontology Entities

General Issues

• subject/anatomy

• spatial reasoning• normalized

coordinate space

• molecular entities

• Present data and query interface commensurate with a neuroscientist’s sense of biological reality

• integrate across spatial scale

SRB

BIRN SMART Atlas

MediatorMediator

UCSD

wrapper

UCLA

wrapper

Cal Tech

wrapper

Duke

wrapper

UT/Drexel

wrapper

Mediated database interoperability and distributed query architecture

Image-related Ontology Entities

General Issues

• subject/anatomy

• distributed spatial reasoning• normalized coordinate space

• integrate across spatial scale

• Data mediation infrastructure must be ontology-centric to support sematically-based queries

• Source database data models must be mapped ontologically in order to:• determine field semantic

equivalency• search content in

appropriate, shared semantic context

• gene/protein• spatially-mapped mRNA &

protein distribution

BIRN Image-Related Ontology Use

HELP!

In closing