fast algorithms for the reconstruction and analysis of...
TRANSCRIPT
Fast Algorithms for the Reconstruction and
Analysis of Basic Circuits in the Mouse
Cortical MapsMinisymposium on High-throughtput Microscopy and Analysis
November 18, 2008
Yoonsuck Choe
Texas A&M University
Collaborators: Bruce H. McCormick, Louise C. Abbott, John Keyser, David
Mayerich, Jaerock Kwon, Zeki Melek, Donghyeop Han, Pei-San Huang, Stephen
J. Smith, Kristina Micheva.
Sponsors: NIH/NINDS (#1R01-NS54252), NSF (#0079874), Texas HECB
(ATP#000512-0146-2001)
1
In Memory of Bruce H. McCormick
Bruce H. McCormick (1928–2007)
• Designer of the Knife-Edge Scanning Microscope (Mayerich et al.
2008)
2
Long-Term Goal
http://mouseatlas.org http://nervenet.org
• Image the whole mouse brain in submicrometer
resolution.
• Extract the connectome, the complete structural
description of the connection matrix.
• Analyze the basic circuits.
3
Plan of Attack
Shepherd (2003)
• Imaging: high-throughput, whole-brain
microimaging (KESM, ATLUM, Array Tomo.)
• Connectome: from raw image stacks to a complete
structural description of brain connectivity (tracing).
• Basic circuits: from geometric structure to
functional modules of the brain network (mining).
4
Challenges
Shepherd (2003)
• Imaging: staining/labeling, speed, volume limit
• Connectome: amount of data (TB to PB), density of
objects, missing info, noise in data, validation.
• Basic circuits: combinatorial explosion in search,
limited theoretical insight on circuit organization.
5
Overview
1. Background: Geometric reconstruction approaches
2. Fast tracing in 2D
3. Fast tracing in 3D
4. Fast interactive visualization and filtering
5. Discussion: validation, basic circuit mining
6
Background
Approaches in geometric tracking and reconstruction:
• Segment then connect
• Supervised learning
• Vector tracing
7
Background: Seg. then Conn.
Segment Reconstruct
Est. StructureImage Stack Segmented Image Stack
• Register images, segment, and then connect
(Chklovskii 2008).
• Most straight-forward and popular approach (e.g.,
ITK).
• In many cases done manually (Brown et al. 2007;
Fiala 2005) (Ascoli lab, Harris lab)
8
Background: Supervised Learning
Input:Data volume
Output:Restoration
Supervised learning with 3D convolutional neural
networks (Jain et al. 2007) (Seung lab):
• Train weights with manually segmented data set as
the target.
• Test with unlabeled raw data set.
9
Background: Vector Tracing
(Al-Kofahi et al. 2002) (Roysam lab)
• Trace along the natural flow of the fibrous object.
• Use steerable templates for computational efficiency.
• Used in commercial tracing software.
10
Overview of Our Methods
1. Fast tracing in 2D
2. Fast tracing in 3D
3. Fast interactive visualization and filtering
11
Fast Tracing in 2D
*
inte
nsity
position
• Examine along border of
moving window.
• Find cross section (black
part) and interpolate.
• Check the interpolation
against data pixels.
12
Fast Tracing in 2D
ci
ci+1 ci+1
ci
ci+2
step i step i+1
ci+21
2
Han and Choe
• Move window in the direction of fiber (dendrite, axon,
vasculature) direction.
• Can effectively handle branches.13
Tracing Results
Seed Can et al. (1999)
Haris et al. (1999) Our method14
Robustness
�: Ours; ♦ Can et al.; �: Harris et al.
Accuracy tested based on synthetic data (by varying
fiber width).
• Ours much more robust to varying fiber width,
compared to competing approaches such as Can
et al. (1999) or Haris et al. (1999).
15
Performance
0
1000
2000
3000
4000
5000
6000
7000
8000
1 2 3 4 5 6
Tota
l tra
ce le
ngth
Data set
Can et al.Harris et al.Our method
0 10 20 30 40 50 60 70 80 90
100 110
1 2 3 4 5 6
Tim
e/Un
it Le
ngth
Tra
ced
Data set
Can et al.Harris et al.Our method
(a) Total trace length (b) time/unit trace length
Performance measured on vascular data set:
• Compared to Can et al. and Haris et al., our method
(green) traces (a) longer distances, (b) faster.
• Accuracy needs to be checked.
16
Fast Tracing in 3Dpredict
correct
Predictor–Corrector
Mayerich and Keyser (2008), Busse et al. (2006)
• Predictor–Corrector approach.
17
Fast Tracing in 3D
Match!
t = 3
t = 2
t = 1
Template matching Tangential slices Templates
• Template matching.
• Use graphics hardware (GPU) for fast matrix
operations.
18
Tracing Results
• Spinal cord vasculature (mouse).
19
Tracing Results
• Neurons (Array Tomography, zebrafish tectum)
• Tracing (left) and cleaned data based on trace
(right).
20
Performance
0.1
1
10
100
1000
10000
1 10 100 1000 10000
Tim
e (m
s)
Number of Samples
Single Core 2.0GHzQuad Core 2.0GHz
CPU with GPU SamplingFull GPU GeForce 7300
0
5
10
15
20
25
1 10 100 1000 10000
Spee
dup
Fact
or
Number of Samples
GPU (Sampling Only)Single Core 2.0GHz
Run time Speedup (Full GPU)
• Use of GPU gives an order-of-magnitude reduction
in computation time.
21
Fast Interactive Visualization/Filtering
A
B
C
For fast, interactive visualization (Melek et al. 2006):
• Use self-orienting surfaces.
• Use GPU acceleration.
22
Fast Interactive Visualization/Filtering
(a) Wire (b) SOS (c) Ori. filtered
• Fast interactive rendering in realtime.
• Interactive filtering for geometric properties.
• Important for interactive editing.23
Discussion• Uses: constrain models (Blue brain, Neuroconstruct, NeuGen,
Netmorph, L-neuron); provide statistics; simulation (NEURON,
GENESIS); etc.
• Reconstruction: validation is a huge issue (Warfield et al. 2004)
(digital phantom: Koene); fast editing support (proof-reading);
high-performance computing support (Rao)
• Connectivity estimation: need to estimate/infer connectivity
(Kalisman et al. 2003); array tomography (Micheva/Smith)
• Basic circuit mining: NP-complete problem
• Other issues: connection strength, delay, sign (inh/exc), degree
of structure/function coupling
24
Acknowledgments
• People:
– Texas A&M: Y. Choe, B. McCormick, J. Keyser, L. C. Abbott, D.
Mayerich, D. Han, J. Kwon, Y. H. Bai, D. C.-Y. Eng, H.-F. Yang, G.
Kazama, K. Manavi, W. Koh, Z. Melek, J. S. Guntupalli, P.-S. Huang, A.
Aluri, H. S. Muddana
– Stanford: S. J. Smith, K. Micheva, J. Buchanan, B. Busse
– UCLA: A. Toga
– Special thanks: T. Huffman (Arizona State U), R. Koene (Boston U),
Bernard Mesa (Micro Star Technologies)
• Funded by: NIH/NINDS (#1R01-NS54252); NSF (MRI #0079874 and ITR
#CCR-0220047), Texas Higher Education Coordinating Board (ATP
#000512-0146-2001), the Department of Computer Science, and the Office
of the Vice President for Research at Texas A&M University.
25
Concluding Remarks
KESMATLUMArray Tomo.SBF−SEM
EditingGeometric Desc.
Data stack
NEURON, GENESIS, NeuGen, Blue Brain, Neuroconstruct
Image processing
Digital Phantom
Reconstruction Validation
NetmorphNeuGenNeuroconst.
HPC
26
Open Discussion
• How can the resulting structural data be used?
– What kind of scientific questions can be addressed?
– How can it be used to answer medical/clinical questions?
– How to bridge structure and function?
• How much detail is needed?
• How can we validate the results?
• What are the technological obstacles?
• How can we utilize power of high-performance computing?
27
ReferencesAl-Kofahi, K. A., Lasek, S., Szarowski, D. H., Pace, C. J., Nagy, G., Turner, J. N., and Roysam, B. (2002). Rapid automated
three-dimensional tracing of neurons from confocal image stacks. IEEE Transactions on Information Technologyin Biomedicine, 6:171–187.
Brown, K. M., Donohue, D. E., D’Alessandro, G., and Ascoli, G. A. (2007). A cross-platform freeware tool for digitalreconstruction of neuronal arborizations from image stacks. Neuroinformatics, 3:343–359.
Busse, B., Smith, S. J., Taylor, C. A., and Arakaki, R. Y. (2006). Development of a semi-automated method for three-dimensional neural structure segmentation. In Society for Neuroscience Abstracts. Washington, DC: Society forNeuroscience. Program No. 834.13. Online.
Can, A., Shen, H., Turner, J. N., Tanenbaum, H. L., and Roysam, B. (1999). Rapid automated tracing and featureextraction from retinal fundus images using direct exploratory algorithms. IEEE Transactions on InformationTechnology in Biomedicine, 3:125–138.
Chklovskii, D. (2008). From neuronal circuit reconstructions to principles of brain design. In Proceedings of the 5thComputational and Systems Neuroscience Meeting (COSYNE 2008 Abstracts), 331.
Fiala, J. C. (2005). Reconstruct: A free editor for serial section microscopy. Jounral of Microscopy, 218:52–61.
Haris, K., Efstratiadis, S., Maglaveras, N., Pappas, C., Gourassas, J., and Louridas, G. (1999). Model-based morphologi-cal segmentation and labeling of coronary angiograms. IEEE Trans. Med. Imag., 18:1003–1015.
27-1
Jain, V., Murray, J. F., Roth, F., Seung, H. S., Turaga, S., Briggman, K., Denk, W., and Helmstaedter, M. (2007). Usingmachine learning to automate volume reconstruction of neuronal shapes from nanoscale images. In Society forNeuroscience Abstracts. Washington, DC: Society for Neuroscience. Program No. 534.7. Online.
Kalisman, N., Silberberg, G., and Markram, H. (2003). Deriving physical connectivity from neuronal morphology. Biologi-cal Cybernetics, 88:210–218.
Mayerich, D., Abbott, L. C., and McCormick, B. H. (2008). Knife-edge scanning microscopy for imaging and reconstructionof three-dimensional anatomical structures of the mouse brain. Journal of Microscopy, 231:134–143.
Mayerich, D., and Keyser, J. (2008). Filament tracking and encoding for complex biological networks. In Proceedings ofACM Symposium on Solid and Physical Modeling, 353–358.
Melek, Z., Mayerich, D., Yuksel, C., and Keyser, J. (2006). Visualization of fibrous and thread-like data. IEEE Transactionson Visualization and Computer Graphics, 12(5):1165–1172.
Warfield, S. K., Zou, K. H., and Wells, W. M. (2004). Simultaneous truth and performance level estimation (STAPLE): Analgorithm for the validation of image segmentation. IEEE Transactions on Medical Imaging, 23:903–921.
27-2