QueryAtlasSteve Pieper, Greg Brown, David Kennedy,

Maryanne Martone, Jyl Boline, Burak Ozyurt,

Wendy Plesniak, Michael Halle,

Anna Tang, and Florin Talos

QueryAtlas: a Slicer3 module

The Query Atlas is a 3D Slicer module that uses atlas-based anatomical representations, linked to underlying semantic descriptions, as an interactive 3D spatial reference for viewing experimental results and for performing information searches to study and interpret them.

Slicer3

Slicer3:

• Cross-Platform Integrated Visualization and Analysis: Structure / Function

• Multi-Site Development: BWH, MIT, GE, Kitware, Isomics, UCSD, JHU, MGH…

• Multi-Project Scope: fBIRN, mBIRN, NAC, NA-MIC, NCIGT, RO1s…

Impact:

• fBIRN and mBIRN Informatics (XNAT) and Data (XCEDE) Compatibility

• Leverage Active Dissemination and Training Efforts of Partner Grants

Slicer3

Slicer3: Complete re-design using “Community-centered” approach

Considers “user” experience for diverse community:

• End-users of the desktop application

• Developers working on core software, using different programming styles

• Contributors of algorithms who want a light-weight API

Slicer3

GUI designed with input from application scientists for improved usability

Toolbar with module

navigation

Module GUI panel

Slice viewer control

3D viewer control

Status & progress

QueryAtlas Overview

Use the QueryAtlas to:

•Load volumetric analyses from XCEDE compliant or XNAT databases;

•Visualize volumes, models and parametric map overlays.

•Annotate models and labelmaps with anatomy labels

•Formulate queries in the context of particular study results (fMRI activations or group statistics, for example)

•View and manage collections of query results

QueryAtlas Overview

Target Audiences:

• Neuroscientists: Address “Information Overload” with Ontology-Aware Tools

• Tool Builders: Provide Reference Implementation of XCEDE Compliant Application Software

• Non-Specialists: Educational Tool and Compelling Demonstration of Data Richness

•Leverages Cross-Testbed BIRN Resources

QueryAtlas Overview

Complete QueryAtlas Xcede 2.0 Catalog:

To support BIRN research, the QueryAtlas is Xcede 2.0 catalog compliant. A complete QueryAtlas Xcede catalog describes and contains uris for:

• an anatomical volume (brain.mgz);

• results of a FreeSurfer morphological analysis ( a label map, aparc+aseg.mgz and models, lh.pial and/or rh.pial);

• the model's annotataion overlay (lh.aparc.annot and/or rh.aparc.annot) derived from the anatomical;

• results of the FIPS processing (an example_functional.nii dataset and brain activation statistics volume datasets of interest, zstatX.nii, zfstatX.nii); and

• a FreeSurfer matrix (anat2exf.register.dat) to register the anatomical to the example functional volume.

** The anatomical volume, the example functional volume and the matrix that registers them in FreeSurfer are required when the catalog is parsed in order to compute the registration matrix in Slicer's coordinate system that correctly aligns all statistics volumes with the scene's anatomical information.

QueryAtlas Overview

Relationship to BIRN informatics efforts

XNAT

acquisitionprotocols

distortion correction

tools

local databases

workflows

analysis tools

...

PubMed

IBVD

BrainInfo

visualization&

Interpretation(Slicer3/QueryAtlas)

(.xcat)(.xar)

Supported data

Currently, the QueryAtlas supports:

• Annotated FreeSurfer datasets (aparc+aseg.mgz, lh/rh.aparc.annot)

• Xcede 2.0 catalog .xcat files for fBIRN FIPS/FreeSurfer data

• XNAT archive .xar files for mBIRN Qdec data

• Can be extended to include other atlas-based annotated label maps or model files: (requires extending readers and data handling methods, and the controlled vocabulary)

Overview

Supported workflows:

Interface Overview

1. Scene loading andAnnotation setup

2. Annotation andDisplay options

3. Ontology Mappingand Browsing

4. Define Search Terms

5. Build queries, ViewManage and Save

Results

Bookmark file read/write for Firefox

Set Mozilla Firefox 2 location through the Application Settings Interface View -> ApplicationSettings -> Slicer Settings

Loading Qdec data

• Load an XNAT .xar archive from command-line, or load a .qdec file from the GUI

• Set up annotations

• Select amongpopulation statistics overlays

Interactive annotations on Qdec data

“way-finding” with interactive annotations on the FreeSurfer inflated brain mapped with different population statistics

Does the correlation betweenthickness and Left NonWMHypointensities differ from zero?

Does the average thicknessdiffer from zero?

Does the average thicknessdiffer from zero?

click to play movie

Loading FIPS/FreeSurfer data

• Load an XCEDE catalog (.xcat file)from command-line or from the GUI

(example phII BIRN SIRP paradigm, combined morphology and functional analyses)

• Set up Slice layers for convenient display

• Set up annotations

(XCEDE 2.0 catalog descriptions)

Loading FIPS/FreeSurfer data

• Select among brain activation overlays included in the catalog (zstat.nii, zfstat.nii…)

(XCEDE 2.0 catalog descriptions)

C7

C14C17

Loading FIPS/FreeSurfer data

(Go to Volumes Module to control data display)

• Select “brain” as the active volume, and set its threshold to “Auto”

• Select each statisticsfile as the activevolume and set itsthreshold and window/level to adjust thedisplay appropriately

• Navigate back to the QueryAtlas

Interactive annotations

Combined structural image data and functional statistics informed by local or formal anatomical annotations

• In the annotation and Display Options GUI Panel, select the termset used for annotations

• Toggle LH and RH model visibility

• In the three Slice Viewers, toggle Slice Plane visibility

click to play movie

Controlled vocabulary

Extensible mechanism for linking atlas-based labels in various local naming systems to symantically compatible terms used in the larger biomedical informatics community

• CSV format files, leverages current BIRNLex ontology and mapping to other termsets and NeuroNames resources from BrainInfo

• Specifically provides mapping between local termsets (FreeSurfer) and BIRNLex, NeuroNames, UMLS, IBVD

• Extensible to include other local and formal termsets

• Distributed with Slicer’s QueryAtlas module

• TBD: Authoring/editing policy and mechanisms, means of tracking BIRNLex changes

Features: Quick translation & search

3D Viewer’s Context menu (right click) provides a quick link to ontology mapping and quick search.

OntologyMapping &

structure term selection panel

Select and translateOntology Browser (BIRNLex)Ontology Browser (NeuroNames)

Local: ctx-lh-superiortemporalBIRN: Superior_temporal_gyrusNeuroNames: superior temporal gyrusUMLS: Superior temporal gyrus structure (Body structure)IBVD: Superior Temporal GyrusBrainInfo: superior temporal gyrus

close

Features: SPL Ontology browser

3D Viewer’s Context menu launches SPL Ontology Browser if local term maps to BIRNLex term or NeuroNames term

(requires Java 1.5+)

Interactive tree view of BIRNLex and NeuroNames structures

entry

matching terms

info

f.o.v

Tree view: pan, zoom, expand, collapse

Features: SPL Ontology browser

SPL Ontology browser:

• information visualization tool • written in Java using Jython scripting language • based on open-source Prefuse toolkit

• contains subset of BIRNlex ontology (root = Regional part of Organ) • contains all of NeuroNames, including synonyms • converted to JSON format for simplified parsing and better human readability: NeuroNames CSV file to JSON is python script BIRNlex to JSON uses Jython and Jena ontology toolkit

• Slicer sends commands using a socket-based mechanism • (Currently, communication is one-way)

Features: Detailed search

Select terms of interest and pass to structure search terms in the Search Terms GUI panel…

Features: Detailed search

• Select among structure terms to use

• Specify additional (group, species, other) terms

• Select search target, and inspect resulting links

• Reserve and accumulate useful links

Features: Detailed search

• Save useful links to Firefox bookmark file (or load past results)

Example data & more information

• FIPS/FreeSurfer example dataset:

http://wiki.na-mic.org/Wiki/index.php/Slicer3:Module:QueryAtlas#Example_data

• Qdec example dataset:

http://www.na-mic.org/Slicer/Download/Data/TEST4.qdec

• Simple tutorial describing features and use:

http://wiki.na-mic.org/Wiki/index.php/Slicer3:Module:QueryAtlas

Future work

• Ability to query richer information repositories and infrastructure using available web services

• Information display tools for intuitive inspection and integrated comprehension of search results (images, data tables, text)

• Tools for transacting with BIRN HID and other XCEDE-compliant data resources

• Extending software and controlled vocabulary to include new atlas information

• Adding experimental natural language parsing tools for query formulation

• Developing mechanisms for tracking BIRNLex and approach to community authoring and editing of controlled vocabulary

Acknowledgement

This research was supported by Grant 5 MOI RR 000827 to the FIRST BIRN and Grant 1 U24 RR021992 to the FBIRN Biomedical Informatics Research Network (BIRN, http://www.nbirn.net), that is funded by the National Center for Research Resources (NCRR) at the National Institutes of Health (NIH). This work was also supported by NA-MIC, NAC, NCIGT. NeuroNames ontology and URI resources are provided courtesy of BrainInfo, University of Washington.