3D modeling workshop3D modeling workshopElectron microscopy dataElectron microscopy data

Helena Vihinen (D.Sc.)Electron tomography

Ilya Belevich (Ph.D.)Computing, SBF-SEM

EM UnitEM UnitInstitute of BiotechnologyInstitute of Biotechnology

University of HelsinkiUniversity of Helsinki

Head of EM UnitEija Jokitalo

Introduction to modeling on EM dataModeling schemeBasic features of following software:

IMOD, Amira, Imaris and MatLab

Short break

Hands-on with Amira and MatLabDataset acquired with SBF-SEM (Huh7 cell, ER cytochemically stained)

Content of this workshop

• Life sciences has undergone a rapid technology evolution producing an ever-growing number of multidimensional data sets

• In addition to ET, also “serial section tomography by SEM” has been automated (SBF-SEM and FIB-SEM)

• To interpret the 3D-data sets, they need to be processed, visualized, analyzed and quantified

• Due to complicity of data from biological material, modeling of tomograms is still done (semi)-manually

3D imaging at EM level

• Target object in the context of the cell• S/N ratio can be poor (low dose)• Missing wedge in tomograms• Electron tomography vs. poly tomography (e.g., SBF-SEM)

3D electron microscopy data

• IMOD (Boulder Laboratory for 3-D Electron Microscopy of Cells, USA)

• Amira (Visage Imaging Inc.)• Imaris (Bitplane AG, Andor Technology)• MATLAB (Mathworks)• DigiETC (Digisens)• Xplore3D, ARGOS (FEI Company)• UCSF Chimera• Others

Softwares for 3D modeling

Template matching, fuzzy logic, oriented filters, watershed transform, eigenvector analysis, local differential structure, Gaussian-like membrane model etc.

DATASETstack of slices

2D or 3D filtering

Volume rendering (not in IMOD)

SegmentationBitmap (Amira, MatLab)Contouring (IMOD)

AligningVisualization of the model

Isosurface

Measurements

Generating the model

Modeling scheme

http://bio3d.colorado.edu/imod/doc/3dmodguide.html

IMOD 3dmodBoulder Laboratory for 3-D Electron Microscopy of Cells

Image display and modeling program, part of IMOD packagePlatforms: Linux, Windows, Mac OS

Image alignment

Automatic image alignment

# xfalign (align adjacent images)# xftoxg (global transformation list)# newstack (apply transformations)

Interactive image alignmentfor serial thin section tomography

Midas enables correction of distortions (stretching / free transform)

Purple-green overlay

Model generation = skinning: Contours are used to place a surface in 3D

Objects consist of countours (or group of contours), contours consist of points

Objects are closed, open or scattered

Scattered point object can be given different sizes

IMOD – model organization

Ghost line

Copy

Contour lines pushed to fit the structures

Contours are copied only to slices where shape is altered

Segmentation =Draw contours

Contouring according to threshold (seeding or thresholding whole slice/volume)

Special drawing Tools by Dr. Andrew Noske

Works like brush, shortcut keys for everything

Drawing contours

Interpolator (Andrew Noske)

Line TrackerContour along a linear feature such as a membrane filling in points between two points placed along the feature

Copy a contour from one section to the next and adjust its position to fit changes

• Slicer provides more flexible view of structures because it can display a slice at an arbitrary orientation through the image volume.

• Provides a convenient way to track trajectories in 3D

Drawing in XYZ and slicer windows

Mesh of triangles

Tubes for e.g., microtubules

Number of triangles controlled by Z inc

Smoothing by Tol (points will be skipped, but the surface defined by the mesh will always be within Tol (pixel) of the original data

Creating a surface with IMOD

#imodmesh and #smoothsurf provides few more options not in the GUI

Isosurfaces

Restricted volume424x424x186 pixelsat a time

Measurements: #imodinfo

Length

Volume

Surface area

# imodcurvature : Analyze radius of curvature of contours or surfaces and encode in model

# fenestra : Calculate projected area of holes (fenestrae) in a 3D model.

# mtk : Analyze distances between model objects in 3D

Other programs in IMOD

# imodauto : Generate a contour based model from a 3D image using a threshold based algorithm

# imodfillin : Fill in missing contours using information in a mesh# nda : Do neighbor density analysis of point positions in plane # sda : Do surface density analysis of point positions on a

surface # smoothsurf : Smooth a surface defined by model contours

# imodmop : Cut out image data inside of model closed contours, circles, spheres or along lines

Image filtering in 2D and 3D e.g, NAD

Special programs used for microtubule (MT) studies.

# checkmtmod : Check various features of a MT model.# howflared : Measure microtubule end flaring and curvature.# fiberpitch : Measure pitch of fibers and position relative to an axis.# mtdetect : Detect circularly symmetric features.# mtlengths : Compute lengths of MTs in a model.# mtmodel : Build a MT model out of individual points.# mtoverlap : Display and analyze overlap of spindle MTs.# mtpairing : Analyze pairing between MTs.# mtrotlong : Extract MTs rotated into longitudinal orientation.# mtsmooth : Smooth a model of MTs.# mtteach : Develop criteria for feature recognition.# reducemtmod : Reduce the number of points in a MT model.# selfalign : Refine alignment of a MT model based on MT trajectories.

Other programs in IMOD

Snapshots from slices (ZaP window) and models (movie window)Videos made by other program e.g., VideoMach (www.Gromada.com)

Videos – image series

Clipping planes

Videos – image series

IMOD collaborating with other programs

Shape (The Boulder 3-D Electron Microscopy of Cells at the University of Colorado at Boulder).

• Filters for enhancing the contrast of tomograms of cells

• Identify and label biological structures in 3D and export the resulting model to IMOD. Based on the Line Filter Transform (LFT) and the Orientation Filter Transform (OFT) (Sandberg et al., 2007)

• “The program undergoes continuous development…(2008)…In the near future we also plan to release Shape3D”

IMOD can convert models to VRML 1.0 file format RIB file format list of patch positions and displacementsmeta file format

+ Free+ Made for electron tomographic data+ Made for biological specimens+ Man –pages in web + Help files in program+ Help available in IMOD mailing list

- Visualization restricted compared to Amira/Imaris (no volume rendering)

- Other program required for making videos- GUI not covering all the commands

IMOD : Pros and Cons

Software for 3D data visualization, analysis and modeling

Platforms : Windows, Mac OS, Linux

In addition to electron microscopic data Amira is designed to handle various other kind of data such as:

clinical or preclinical image data from CT, MR, nuclear imaging, ultrasound, optical microscopy

molecular models, vector and flow data, simulation data on finite element models, and all types of multidimensional image, vector, tensor, and geometry data

The Amira software is organized in functional modules, consisting of base package plus optional add-on packages

More Amira optional add-on packages

• Process– 2D and 3D image filtering– Interactive and automatic

segmentation– Interactive and automatic

slice alignment– Surface generation– Interactive surface editing

• Analyze– Volume, area, distance

in 3D etc.– Curvature– Density map– Arithmetic operations

on images, vector fields and unstructured grids

– Direct integration with MatLab

• Visualize– Orthogonal and oblique slicing– Volume rendering – Isosurfaces– Stereo views– Animations

Base version of Amira

Modular and object oriented (>270 object and module types)

Modules used to visualize data objects or to perform some computational operations on them

Components are represented by little icons in the Pool

Lines indicates processing dependencies : which modules are to be applied to which objects

Properties area

Pool

3D viewer

Console

Parameter = port of module

Amira window

Tcl script language

Segmentation editor

• Three slice views and a 3D viewer

• Brush, lasso, magic wand, thresholding, intelligent scissors, contour fitting, contour interpolation and extrapolation, wrapping

Each voxel belongs to only one material

Selected/assigned voxels in 3D

Segmentation

• Smoothing and de-noising filters • Fill area/volume• Interpolate

• Target ROI segmented, generate corresponding surface model

• Surface simplification : reduce number of triangles however topological correctness is preserved. Quality of the resulting mesh can be controlled: triangles with long edges can be suppressed.

Surface generation and simplifier

Surface editor

smoothing or refining surface, cutting and copying part of surfaces, defining boundary conditions

• Thresholding• Combined with

segmented volume

Isosurfaces

SBF-SEM

tomogram

• Every voxel of the viewed volume is assumed to emit and absorb light (VolTex and VolRen)

• Heavy use of OpenGL texture mapping, 2D/3D texture modes

Volume rendering

Spherules of SFV VLP, G. Balistreri

Combined with segmented volume

MIM protein deformed liposome membrane, Pieta Mattila

Landmarks (designed for manual alignment)

One click in the center of the particle

Spherical objects (e.g., ribosomes)

• Versatile• Control all objects

+ available modules with their ports e.g., slice number, clipping plane, camera rotation, camera path (key frames)

Animation

+ Versatile+ Tutorials + help files+ Help desk available ( - if you pay)

- Costs 3,600 € + 720 € yearly (-30% for academics) add-on modules 1,800 € + 360 € yearly

- Documentation not-so-great- Needs good computer = enough swap memory in

addition to physical memory

Amira : Pros and Cons