3d slicer architecture and implementation
DESCRIPTION
3D Slicer Architecture and Implementation. Steve Pieper, PhD. Goals. NA-MIC Kit: Software and Methodologies for Medical Image Computing Facilitate Research Promote Interoperability Stable, Cross-Platform Run Time Environment Full Set of Core Features Avoid Duplicated Effort - PowerPoint PPT PresentationTRANSCRIPT
NA-MICNational Alliance for Medical Image Computing http://na-mic.org
3D SlicerArchitecture andImplementation
Steve Pieper, PhD
National Alliance for Medical Image Computing http://na-mic.org
2
Goals
• NA-MIC Kit: Software and Methodologies for Medical Image Computing– Facilitate Research– Promote Interoperability
• Stable, Cross-Platform Run Time Environment– Full Set of Core Features– Avoid Duplicated Effort
• Flexible Module Architecture– Plug-ins should be As Simple As Possible
National Alliance for Medical Image Computing http://na-mic.org
Overview
• MRML, Logic, GUI
• Core Libraries and Dependencies
• Some Details of MRML
• Modules
3
National Alliance for Medical Image Computing http://na-mic.org
4
Slicer3 “Observer MVC” Pattern
• MRML (Model)– For Scene Description and Application State– MRML Nodes are Persistent and Undoable– Scene and Nodes are Observable
• Logic Encapsulate VTK and ITK Pipelines (Controller)– Observe MRML to Configure Pipelines– Help Create/Manage Nodes– No UI Components (no Widgets, Actors, Mappers,
Renderers or RenderWindows)• GUI (View)
– Observe and Edit MRML– Interact with User and Display Hardware
• Modules Should Follow Same Conventions
Logic
MRML Nodes
GUI
Widgets Renderers
EditObserve
Observe
Observe
Edit
Edit
“Observe” means generic event mechanisms are used to pass information.“Edit” means code can directly call methods.
Example: GUI can call methods in Logic classes,but Logic cannot call GUI methods.MRML cannot call Logic or GUI methods.
There can be many observers for any event.
National Alliance for Medical Image Computing http://na-mic.org
5
Example: EM in Slicer3
MRML Logic GUI
•Global Parameters
•Hierarchical parameters
•Image data
•Segmentation output
•Manage MRML nodes
•API for access to parameters
•Manage hierarchy
•Segmentation algorithm
•Window to parameter set
•Wizard
•Interaction with images
National Alliance for Medical Image Computing http://na-mic.org
6
Example: EM Batch Tool
MRML Logic CLI
•Global Parameters
•Hierarchical parameters
•Image data
•Segmentation output
•Manage MRML nodes
•API for access to parameters
•Manage hierarchy
•Segmentation algorithm
•Command Line Parsing
•Load Parameter Description MRML Scenes
National Alliance for Medical Image Computing http://na-mic.org
Library Dependencies
7
ITKVTKTcl/Tk Teem
KWWidgets vtkTeem vtkITK
MRML
SlicerBaseLogic
SlicerBaseGUI
Slicer3
National Alliance for Medical Image Computing http://na-mic.org
vtkITK, vtkTeem
8
ITKVTKTcl/Tk Teem
KWWidgets vtkTeem vtkITK
MRML
SlicerBaseLogic
SlicerBaseGUI
Slicer3 • Teem: Multidimensional Raster Image Library (Gordon Kindlmann)• Includes NRRD File Format• ExtensiveTensor Manipulation
Utilities
• vtkTeem, vtkITK: Create VTK Compatible Filters using ITK and Teem Code for Implementation• Includes vtkITKArchetype*
Readers and Writers Implemented with ITK I/O Factories
National Alliance for Medical Image Computing http://na-mic.org
MRML
9
ITKVTKTcl/Tk Teem
KWWidgets vtkTeem vtkITK
MRML
SlicerBaseLogic
SlicerBaseGUI
Slicer3 • MRML: Medical Reality Markup Language
• Library Provides Central Data Representation for Slicer3• Application State is Explicitly
Described in MRML• GUI and Modules:
• Observe MRML to Learn of Changes to State
• Manipulate State to Reflect User Interaction and Calculation Results
National Alliance for Medical Image Computing http://na-mic.org
SlicerBaseLogic
10
ITKVTKTcl/Tk Teem
KWWidgets vtkTeem vtkITK
MRML
SlicerBaseLogic
SlicerBaseGUI
Slicer3 • Utility Code for Implementing Base Application Functionality• Input/Output• Volume Reslicing• Image Filters
• No Rendering, and No GUI Code
National Alliance for Medical Image Computing http://na-mic.org
SlicerBaseGUI
11
ITKVTKTcl/Tk Teem
KWWidgets vtkTeem vtkITK
MRML
SlicerBaseLogic
SlicerBaseGUI
Slicer3 • KWWidgets Subclasses• vtkSlicerApplication• Custom KWWidgets
• Rendering Utility Code
• VTK 3D Widget Subclasses
National Alliance for Medical Image Computing http://na-mic.org
12
Scene Description
Provided by D. Gering
National Alliance for Medical Image Computing http://na-mic.org
MRML Concepts
• Node– Unit of Organization for MRML– Group of Variables Representing an Object or Concept State
• Scene– Slicer Application has Single Scene Instance that has Collection
of Nodes and API for Access/Create/Delete• Undo/Redo & Scene Snapshots
– Scene Swaps Nodes in to / out of Current State– Logic/GUI Auto-Update through Observers
• Serialization / Deserialization– Nodes Responsible for Read/Write of XML Version of State– Scene Read/Write Analogous to Snapshots
13
National Alliance for Medical Image Computing http://na-mic.org
MRML Concepts (cont)
• Observers / Events– Scene and Nodes Invoke Custom Observable Events for
Specific Actions (e.g. NodeAddedEvent) or ModifiedEvents for General Updates
– Node API Allows Disabling ModifiedEvent Invocation to Allow Groups of Operations with Single ModifiedEvent
– Node API Includes MRMLObserverManager Class to Simplify Addition/Removal of Multiple Event Observers
– Scene Provides EventBroker to Support Observer Introspection, Asynchronous Invocation, Logging, and Event Compression
14
National Alliance for Medical Image Computing http://na-mic.org
MRML Concepts (cont)
• IDs & References– Each Node is Given a Unique ID for Retrieval in Scene– Nodes Can Refer to Other Nodes by ID
• E.g. TransformableNodes can Refer to a TransformNode by ID
• E.g. ModelNode can Refer to ModelDisplayNode by ID– Scene Maintains ReferencedIDs and ReferencingNodes Lists to
Maintain One-to-One Relationship During Scene Import
15
National Alliance for Medical Image Computing http://na-mic.org
Node Types
• Displayable/Storable/Transformable• Display/Storage/Transform• Hierarchy• Parameter• Selection• Slice• SliceComposite• Color• Fiducial• Snapshot• Command Line Module• Other Module-Specific Nodes
16
National Alliance for Medical Image Computing http://na-mic.org
17
“Observer MVC” Example
Logic
MRML Nodes
GUI
Widgets Renderers
EditObserve
Observe
Observe
Edit
Edit
Notes: Event invocation happens synchronously in Slicer 3.2, meaning volume is read and updated before AddArchetypeVolume method returns. Asynchronous event processing may become the default in the future.
GUI Currently propagates selection to each slice composite node in the scene. In the future the slice GUIs should be observing the scene and optionally display newly loaded volumes.
• Event: User Picks Add Volume in Volumes GUI (KWWidget)
• Volumes GUI calls AddArchetypeVolume in VolumesLogic
• Volumes Logic Creates VolumeNode and VolumeStorageNode, Reads Data, and Adds to Scene
• Scene Invokes NodeAddedEvent– NodeSelector Widgets Update Menus
• Volumes GUI Sets Active Volume on SelectionNode– Slice Viewers Updated to New Volume
National Alliance for Medical Image Computing http://na-mic.org
Coordinate Systems
• IJK (Index Coordinates of Volume)– vtkMRMLVolumeNode uses vtkImageData to store voxels, with
Origin 0,0,0 and Spacing 1,1,1• RAS (Right-Anterior-Superior)
– vtkMRMLVolumeNode::GetIJKToRASMatrix provides vtkMatrix4x4 that includes origin, spacing, and direction vectors to map to patient space
– Origin is at the center of the voxel– Vectors Point in Specified Direction (i.e. R goes from Left to Right)
• LPS (Left-Posterior-Superior)– Used by ITK code and DICOM– Slicer Automatically Converts RAS->LPS
18
National Alliance for Medical Image Computing http://na-mic.org
Transforms
• TransformToParent– vtkMRMLTransformableNode Instances Contain TransformNodeID
Reference to a vtkMRMLTransformNode – Allows Many Nodes Inside Each Transform and Transform
Hierarchies– TranfrormNode Defines how to Go from Child to Parent Space (e.g.
a ModelNode points to a TransfromNode that Scales it to Global RAS Coordinates)
– Transforms Represented by vtkGeneralTransforms (linear/nonlinear)
• (TransformFromParent – not yet implemented)– Nonlinear Registration Results Often Better Represented as
Resampling Transforms, Opposite of Current Modeling Transforms
19
National Alliance for Medical Image Computing http://na-mic.org
Modularity Goals
• Keep the base package “lean and mean”
• Modules have individual identity
– Per-module web site, svn, downloads, mailing lists, wiki…
• Allow users to assemble their own set of tools
– Customized ‘Bundles’ by task or application
• Easy to download compatible extensions
– Analogous to Firefox extensions
– Integrate extension builds into developer/nightly/release processs
• NITRC Supplement to NA-MIC helping to pay for needed infrastructure (Neuroimaging Informatics Tools and Resources Clearinghouse)
– NITRC can host neuroimaging projects (gforge implementation)
20
National Alliance for Medical Image Computing http://na-mic.org
21
Base Features vs. ModulesFeatures Base ModulesVisualization MRML: Models, Volumes, Volume Rendering, Fiducials,
Continuum Meshes, Labeled DataCan create custom MRML Nodes and behavior
Filtering None Implemented as Modules using ITK or other Libraries
Registration Transform Display and Edit, Save/Restore Calculate Transforms, Resample Data
Segmentation Label Maps, Parcellated Surfaces Segmentation Algorithms in ITK or other Libraries
Quantification Label, Image, Volume Statistics; Numpy access to MRML
Applications in Python or MATLAB
Real-time Integration VTK Rendering, KWWidgets framework, Tracker Support (as Transforms)
Direct Manipulation of the MRML Scene; 2D/3D Widgets; Device Interfaces, OpenIGTLink Module
Diffusion Imaging DWI, DTI, Fiber Bundles Tractography, Clustering, Atlases
Applications “Bundles” of Modules in Distribution: Registration, Editor, some Filters,
Customized Extensions, Domain-specific code, Optimized Interfaces…
National Alliance for Medical Image Computing http://na-mic.org
Module Terminology• Built-In Modules
– Few Core Features Linked to Slicer3.cxx– Non-Core Modules becoming Loadable
• Loadable Modules– Dynamically Discovered and Loaded– Can Access Application’s MRML Scene Instance and Call SlicerBaseGUI
and Related APIs (vtkRendering, KWWidgets…)• Scripted Modules
– Like Loadable, but Written in Tcl or Python• Command Line Modules
– Can Run as Stand Alone Executables– Provide Standard Command Line Parsing, which Allows Slicer to Generate
GUI Automatically– Can be Compiled as Shared Library for Dynamic Link to Slicer
22
National Alliance for Medical Image Computing http://na-mic.org
Loadable Modules
• http://www.slicer.org/slicerWiki/index.php/Slicer3:How_to_implement_an_Interactive_Module_GUI
• http://www.slicer.org/slicerWiki/index.php/Slicer3:Loadable_Modules:HOWTO
• Create vtkSlicerModuleGUI Subclass– Override Virtual Methods for CreateGUI, ProcessMRMLEvents,
etc.• Provide Custom vtkMRMLNode Subclass(es) to Represent, Save
and Restore state• Create Logic Classes to Implement Module Functionality• Create CMake Configuration Files to Build and Install Shared
Libraries for Runtime Discovery and Packaging
23
National Alliance for Medical Image Computing http://na-mic.org
Scripted Modules
• A Generic ScriptedModule Module Provides Mapping of C++ Method Entry Points to a Corresponding Naming Convention for Scripts
• Scripts are Dynamically Discovered at Runtime• A Generic ScriptedModuleNode Allows Save/Restore of Keyword /
Value Pairs to Represent Module State• Full API of Slicer, MRML, KWWidgets, VTK, available (except where
not wrappable; these APIs have been designed for scriptability)• Editor is extensive Tcl-based Module• Several Python examples are included in current svn trunk
– Python modules can also access Numpy routines for Numerical processing (Volumes and other DataArrays are directly mapped to Numpy arrays for easy manipulation)
24
National Alliance for Medical Image Computing http://na-mic.org
Command Line Modules
• Use the GenerateCLP command (provided with Slicer) to convert XML description of program arguments into a C++ .h file
• Main program of module uses PARSE_ARGS macro provided by .h file to instance variables corresponding to parameters defined in XML file
• PARSE_ARGS also defines a --xml argument that returns original XML string on stdout
• Slicer parses the XML to create GUI for each module• Are Covered Extensively in the HelloWorld Tutorial
• Note: Command Line Modules can be written in any language. C++ utilities are provided, but can be emulated / recreated as needed.
25