© s. c. strother, 2006 managing and optimizing fmri pipelines stephen c. strother, ph.d. rotman...

© S. C. Strother, 2006© S. C. Strother, 2006

Managing and Optimizing

fMRI Pipelines

Stephen C. Strother, Ph.D.Rotman Research Institute, Baycrest Centre

http://www.rotman-baycrest.on.ca/rotmansite/home.php

& Medical Biophysics, University of Toronto


OverviewOverview• Background

−data-driven statistics, pipelines and meta-models

• fMRI File Management: NIfTI and the DFWG• Why optimize pipeline meta-models;

−the Functional Imaging Analysis Contest (FIAC) experience?

• Seven meta-model optimization frameworks• Results with the 7th framework: NPAIRS• Data-analysis choices in pipeline meta-models:

−General linear model (GLM)−Canonical variates analysis (CVA)

• Pipeline-driven, between-subject heterogeneity• Recap: What have we learnt?


The Statistician and the ScientistThe Statistician and the Scientist

When asked, the statistician says that he’d like to When asked, the statistician says that he’d like to give one give one last lecture on his theory of statistics. lecture on his theory of statistics.

When the scientist is asked, he says, “I’d like to be When the scientist is asked, he says, “I’d like to be shot first!”shot first!”

Rob Tibshirani @ S-Plus Users conference Oct., 1999. Rob Tibshirani @ S-Plus Users conference Oct., 1999.

http://www-stat.stanford.edu/~tibshttp://www-stat.stanford.edu/~tibs

A statistician and a scientist are going to be executed, and the executioner asks each for their last request.


Why Bother with Data-Driven Statistics?Why Bother with Data-Driven Statistics?

Philosophy• “All models (pipelines) are wrong, but some are useful!”

− “All models are wrong.” G.E. Box (1976) quoted by Marks Nester in, “An applied statistician’s creed,” Applied Statistics, 45(4):401-410, 1996.

• A goal is to quantify and optimize utility!

• “I believe in ignorance-based methods because humans have a lot of ignorance and we should play to our strong suit.”

− Eric Lander, Whitehead Institute, M.I.T.

• Minimize the number of modeling assumptions and/or test multiple hypotheses, i.e., strong inference!


What are the most, 2nd most etc., important steps?

We need more research across multiple data sets!

fMRI Pipelines and Meta-modelsfMRI Pipelines and Meta-modelsReconstructed

fMRI Data

B0 Correction

Slice TimingAdjustment

MotionCorrection

Non-LinearWarping

Spatial & TemporalFiltering

Statistical AnalysisEngine

StatisticalMaps

Some Preprocessing

Steps

ExperimentalDesignMatrix

Rendering of Results on Anatomy

Data Modeling/Analysis

Optimisation Metrics– ROCs– p-values– AIC, BIC– Replication– Prediction– NPAIRS

Why are we not using more modern analysis techniques?

Why are we still focused on null-hypothesis testing?

We need better tools and education!Automated Software

Frameworks

Fiswidget GUIFiswidget GUIXNAT, LONI, Fiswidgets





−the FIAC experience?

• Seven optimization frameworks• Results with the 7th framework: NPAIRS• Data-analysis choices in pipeline meta-models:

−General linear model (GLM)−canonical variates analysis (CVA)

• Pipeline-driven, between-subject heterogeneity• What have we learnt?


NIfTI-DFWG-NIfTI-1.1NIfTI-DFWG-NIfTI-1.1


NIfTI-DFWG-NIfTI-1.1NIfTI-DFWG-NIfTI-1.1

Leading candidates for NIfTI-2 are:MINC-2.0, Multi-frame DICOMXCEDE XML schema


Why Optimize Pipeline Meta-models?Why Optimize Pipeline Meta-models?Practice New insights into human brain function may

be obscured by poor and/or limited choices in the data-processing pipeline!

We don’t understand the relative importance of meta-model choices.

“Neuroscientific plausibility” of results is used to justify the meta-model choices made.

Systematic bias towards prevailing neuroscientific expectations, and against new discoveries and/or testing multiple hypotheses.


The Functional Image Analysis Competition 1 The Functional Image Analysis Competition 1

Examine the perisylvian Examine the perisylvian language network using a language network using a repetition-priming design with repetition-priming design with spoken sentencesspoken sentences

3T whole-body Bruker3T whole-body Bruker

T2-weighted EPI, TR=2.5s, 30 T2-weighted EPI, TR=2.5s, 30 x 4 mm slicesx 4 mm slices

Epochs of 20s ON & 9s OFFEpochs of 20s ON & 9s OFF

2 x 2 design for 4 conditions2 x 2 design for 4 conditions

4 epochs/condition x 2 runs4 epochs/condition x 2 runs

Same Same sentence, sentence, same same speakerspeaker

Same sentence, Same sentence, different speakerdifferent speaker

Different Different sentence, sentence, same same speakerspeaker

Different sentence, Different sentence, different speakerdifferent speaker


The Functional Image Analysis Competition 2The Functional Image Analysis Competition 2

Poline JB, Strother SC, Dehaene-Lambertz G, Egan GF, Lancaster JL. Poline JB, Strother SC, Dehaene-Lambertz G, Egan GF, Lancaster JL. Motivation and synthesis of Motivation and synthesis of the FIAC experiment: The reproducibility of fMRI results across expert analyses.the FIAC experiment: The reproducibility of fMRI results across expert analyses. (in press, special (in press, special issue Hum Brain Mapp)issue Hum Brain Mapp)


The Functional Image Analysis Competition 3The Functional Image Analysis Competition 3

Abstract:Abstract: “… the FIAC … helped identify new activation “… the FIAC … helped identify new activation regions in the test-base data, and …, regions in the test-base data, and …, it illustrates the it illustrates the significant methods-driven variability that potentially exists significant methods-driven variability that potentially exists in the literature.in the literature. Variable results from different methods Variable results from different methods reported here should provide a cautionary note, and reported here should provide a cautionary note, and motivate the Human Brain Mapping community to explore motivate the Human Brain Mapping community to explore more thoroughly the methodologies they use for analysing more thoroughly the methodologies they use for analysing fMRI data.”fMRI data.”

Poline JB, Strother SC, Dehaene-Lambertz G, Egan GF, Lancaster JL. Motivation and synthesis of the FIAC experiment: The reproducibility of fMRI results across expert analyses. (in press, special issue Hum Brain Mapp)


The Functional Image Analysis Competition 4The Functional Image Analysis Competition 4z=-12 z=2 z=5

3

1,4

21

3 3 31

3

The main effects of sentence repetition (in red) and of speaker repetition (in blue). 1: Meriaux et al, Madic; 2: Goebel et al, Brain voyager; 3: Beckman et al, FSL; and 4: Dehaene-Lambertz et al, SPM2.


Optimization Metric FrameworksOptimization Metric Frameworks

Simulation & ROC curvesSimulation & ROC curves1.1. Skudlarski P., et al., Skudlarski P., et al., NeuroimageNeuroimage. 9(3):311‑329, 1999.. 9(3):311‑329, 1999.2.2. Della-Maggiore V., et al., Della-Maggiore V., et al., NeuroimageNeuroimage 17:19–28, 2002. 17:19–28, 2002.3.3. Lukic AS., et al., Lukic AS., et al., IEEE Symp. Biomedical ImagingIEEE Symp. Biomedical Imaging, 2004., 2004.4.4. Beckmann CF & Smith SM. Beckmann CF & Smith SM. IEEE Trans. Med. Img.IEEE Trans. Med. Img. 23:137-152, 2004. 23:137-152, 2004.

Data-Driven:Data-Driven:

1.1. GLM DiagnosticsGLM Diagnostics1.1. SPMd, SPMd, Luo W-L, Nichols T. Luo W-L, Nichols T. NeuroImageNeuroImage 19:1014-32, 2003 19:1014-32, 2003

2.2. Minimize p-valuesMinimize p-values1.1. Hopfinger JB, et al., Hopfinger JB, et al., Neuroimage,Neuroimage, 11:326-333, 2000. 11:326-333, 2000.

2.2. Tanabe J, et al. Tanabe J, et al. Neuroimage,Neuroimage, 15:902-907, 2002. 15:902-907, 2002.

3.3. Model Selection:Model Selection: Classical hypothesis testing, maximum likelihood, Classical hypothesis testing, maximum likelihood, Akaike’s information criterion (AIC), Minimum DescriptionLength, Bayesian Akaike’s information criterion (AIC), Minimum DescriptionLength, Bayesian Information Criterion (BIC) & Model Evidence, Information Criterion (BIC) & Model Evidence, Cross ValidationCross Validation

4.4. Replication/ReproducibilityReplication/Reproducibilitya.a. Empirical ROCsEmpirical ROCs – mixed multinomial model – mixed multinomial model

1.1. Genovese CR., et al., Genovese CR., et al., Magnetic Resonance in MedicineMagnetic Resonance in Medicine, 38:497–507, 1997., 38:497–507, 1997.2.2. Maitra, R., et al., Maitra, R., et al., Magnetic Resonance in Medicine, 48, Magnetic Resonance in Medicine, 48, 62 –70, 2002.62 –70, 2002.3.3. Liou M., et al., Liou M., et al., J. Cog. NeuroscienceJ. Cog. Neuroscience, 15:935-945, 2003., 15:935-945, 2003.

b.b. Empirical ROCsEmpirical ROCs – lower bound on ROC – lower bound on ROC1.1. Nandy RR & Cordes D. Nandy RR & Cordes D. Magnetic Resonance in MedicineMagnetic Resonance in Medicine 49:1152–1162, 2003. 49:1152–1162, 2003.

5.5. Prediction Error/AccuracyPrediction Error/Accuracy1.1. Kustra R & Strother SC. Kustra R & Strother SC. IEEE Trans Med ImgIEEE Trans Med Img 20:376-387, 2001. 20:376-387, 2001.2.2. Carlson, T.A., et al., Carlson, T.A., et al., J Cog NeuroscienceJ Cog Neuroscience, 15:704–717, 2003., 15:704–717, 2003.3.3. Hanson,S.J., et al., Hanson,S.J., et al., NeuroImageNeuroImage 23:156– 166, 2004 23:156– 166, 2004

6.6. NPAIRS: Prediction + ReproducibilityNPAIRS: Prediction + Reproducibility1.1. Strother SC, et. al., Strother SC, et. al., NeuroimageNeuroimage 15:747-771, 2002. 15:747-771, 2002.

2.2. Kjems U, et al., et al., Kjems U, et al., et al., NeuroimageNeuroimage 15:772-786, 2002. 15:772-786, 2002.

3.3. Shaw ME, et. al. Shaw ME, et. al. NeuroimageNeuroimage 19:988-1001, 2003. 19:988-1001, 2003.

4.4. LaConte S, et. al. LaConte S, et. al. NeuroimageNeuroimage 18:18:10-23, 2003.10-23, 2003.

5.5. Strother SC, et. al., Strother SC, et. al., Neuroimage Neuroimage 23S1:S196-S207, 2004.23S1:S196-S207, 2004.

6.6. LaConte S, et. al., LaConte S, et. al., NeuroimageNeuroimage 26:317-329, 2005 26:317-329, 2005


Optimization via SimulationsOptimization via SimulationsReceiver Operating Characteristic (ROC) Curves

PA = P(True positive)

= P(Truly active voxel

is classified as active)

= Sensitivity

PI = P(False positive)

= P(Inactive voxel

is classified as active)

= False alarm rateSkudlarski P, Skudlarski P, NeuroimageNeuroimage. 9(3):311‑329, 1999.. 9(3):311‑329, 1999.Della-Maggiore V, Della-Maggiore V, NeuroimageNeuroimage 17:19–28, 2002. 17:19–28, 2002.Lukic AS, Lukic AS, IEEE Symp. Biomedical ImagingIEEE Symp. Biomedical Imaging, 2004., 2004.Beckmann CF, Smith SM. Beckmann CF, Smith SM. IEEE Trans. Med. Img.IEEE Trans. Med. Img. 23:137-152, 2004. 23:137-152, 2004.

pAUC


Optimization via SimulationsOptimization via Simulations


Optimization Framework 1 (SPMd)Optimization Framework 1 (SPMd)

Massively univariate testing of GLM assumptions and data exploration:

• Luo W-L, Nichols T. Diagnosis and exploration of massively univariate neuroimaging models. NeuroImage 19:1014-32, 2003

• Zhang H, Luo W-L, Nichols TE. Diagnosis of Single Subject & Group fMRI Data with SPMd. Hum Brain Mapp (in press, special FIAC issue)

Example: The impact of high-pass filtering in a phantom.


Optimization Framework 1 (SPMd)Optimization Framework 1 (SPMd)

Lund TE, Madsen KH, Sidaros K, Luo W-L, Nichols TE. Non-white noise in fMRI: Does modelling have an impact? Neuroimage 29:54 – 66, 2006


Optimization Framework 2Optimization Framework 2

Minimize p-values or maximize SPM values, e.g.,

• Hopfinger JB, Buchel C, Holmes AP, Friston KJ, A study of analysis parameters that influence the sensitivity of event related fMRI analyses, Neuroimage, 11:326-333, 2000.

• Tanabe J, Miller D, Tregellas J, Freedman R, Meyer FG. Comparison of detrending methods for optimal fMRI preprocessing. Neuroimage, 15:902-907, 2002.

Does not imply a stronger likelihood of getting the same result in another replication of the same experiment!


Optimization Framework 3Optimization Framework 3Model Selection: An attempt to formulate some

traditional problems in the methodology of science in a rigorous way.

Standard methods: (Classical hypothesis testing, maximum likelihood, Akaike’s information criterion (AIC), Minimum DescriptionLength, Bayesian Information Criterion (BIC) & Model Evidence, Cross Validation) compensate for errors in the estimation of model parameters.

All tradeoff fit with simplicity (least # parameters), but give simplicity different weights.

All favor more complex (less simple) models with more data.

Forster MR. Key concepts in model selection: Performance and Generalizability. J Math Psych 44:205-231, 2000


Quantifying replication/reproducibility because:

• replication is a fundamental criterion for a result to be considered scientific;

• smaller p values do not necessarily imply a stronger likelihood of repeating the result;

• for “good scientific practice” it is necessary, but not sufficient, to build a measure of replication into the experimental design and data analysis;

• results are data-driven and avoid simulations.



Optimization Framework 4aOptimization Framework 4aData-Driven, Empirical ROCs:Data-Driven, Empirical ROCs:

• Genovese CR, Noll DC, Eddy WF. Genovese CR, Noll DC, Eddy WF. Estimating test-retest reliability Estimating test-retest reliability in functional MR imaging. I. Statistical methodology.in functional MR imaging. I. Statistical methodology. Magnetic Magnetic Resonance in MedicineResonance in Medicine, 38:497–507, 1997., 38:497–507, 1997.

• Maitra, R., Roys, S. R., & Gullapalli, R. P. Maitra, R., Roys, S. R., & Gullapalli, R. P. Test–retest reliability Test–retest reliability estimation of functional MRI data.estimation of functional MRI data. Magnetic Resonance in Magnetic Resonance in Medicine, 48, Medicine, 48, 62 –70, 2002.62 –70, 2002.

• Liou M, Su H-R, Lee J-D, Cheng PE, Huang C-C, Tsai C-H. Liou M, Su H-R, Lee J-D, Cheng PE, Huang C-C, Tsai C-H. Bridging Functional MR Images and Scientific Inference: Bridging Functional MR Images and Scientific Inference: Reproducibility Maps.Reproducibility Maps. J. Cog. NeuroscienceJ. Cog. Neuroscience, 15:935-945, 2003., 15:935-945, 2003.

V VV V(M - R ) (M - R )R R

A A I IV

Mλ P 1- P + 1- λ P 1- P

R


Optimization Framework 4bOptimization Framework 4b

Data-Driven, Empirical ROCs:• Nandy RR, Cordes D. Novel ROC-Type Method for Testing the

Efficiency of Multivariate Statistical Methods in fMRI. Magnetic Resonance in Medicine 49:1152–1162, 2003.

P(Y) = P(voxel identified as active) P(Y/F) = P(inactive voxel identified as active) P(Y) vs. P(Y/F) is a lower bound for true ROC

• Two runs: −standard experimental AND −resting-state for P(Y/F).

• Assumes common noise structure for accurate P(Y/F).


Is Replication a Sufficient Metric?Is Replication a Sufficient Metric?

A silly data analysis approach produces the value 1.0/voxel regardless of the input data!

Results are perfectly replicable; • no variance;

• completely useless because they are severely biased!

Must consider such bias-variance tradeoffs when measuring pipeline performance.



Prediction/CrossvalidationResampling

Stone, M. Cross-validatory choice and assessment of statistical predictions. J. R. Stat. Soc. B 36: 111–147. 1974

Hastie T, Tibshirani R, Friedman J. Hastie T, Tibshirani R, Friedman J. The The elements of statistical learning theory.elements of statistical learning theory. Springer-Verlag, New York, 2001Springer-Verlag, New York, 2001


Optimization Framework 5Optimization Framework 5Prediction/Crossvalidation Resampling Papers

Principal Component AnalysisL. K. Hansen, et al. Neuroimage, vol. 9, no. 5, pp. 534-44, 1999.Prediction via GLM, Split-Half ReproducibilityJ. V. Haxby, et al. Science, vol. 293, no. 5539, pp. 2425-30, 2001.Linear Discriminant Analysis/Canonical Variates AnalysisR. Kustra and S. Strother, IEEE Trans Med Imaging, vol. 20, no. 5, pp. 376-87, 2001.T. A. Carlson, et al. J Cogn Neurosci, vol. 15, no. 5, pp. 704-17, 2003.J. D. Haynes and G. Rees, Nat Neurosci, vol. 8, no. 5, pp. 686-91, 2005.Y. Kamitani and F. Tong, Nat Neurosci, vol. 8, no. 5, pp. 679-85, 2005.A.J. O'Toole, et al. J Cogn Neurosci, vol. 17, no. 4, pp. 580-90, 2005.Support Vector Machines (and LDA)D. Cox and R. L. Savoy, Neuroimage, vol. 19, no. 2 Pt 1, pp. 261-70, 2003.S. LaConte, et al. Neuroimage, vol. 26, no. 2, pp. 317-29, 2005.J. Mourao-Miranda, et al. Neuroimage, vol. 28, no. 4, pp. 980-95, 2005.Artificial Neural NetworksB. Lautrup, et al. in Proceedings of the Workshop on Supercomputing in Brain Research: From

Tomography to Neural Networks, H. J. Hermann, et al., Eds. Ulich, Germany: World Scientific, pp. 137-144, 1994.

N. Morch, et al. Lecture Notes in Computer Science 1230, J. Duncan and G. Gindi, Eds. New York: Springer-Verlag, pp. 259-270, 1997.

S. J. Hanson, et al. Neuroimage, vol. 23, no. 1, pp. 156-66, 2004.S. M. Polyn, et al. Science, vol. 310, no. 5756, pp. 1963-6, 2005.


Optimization Framework 6: NPAIRSOptimization Framework 6: NPAIRS

NPAIRS Uses NPAIRS Uses “split-half” resampling to combine: resampling to combine:• Prediction & Reproducibility Metrics

• PCA-based reproducibility measures of:PCA-based reproducibility measures of:− uncorrelated signal and noise SPMs;uncorrelated signal and noise SPMs;− reproducible SPMs (rSPM) on a Z-score scale;reproducible SPMs (rSPM) on a Z-score scale;− multivariate dimensionality.multivariate dimensionality.

• Combined prediction and reproducibility metrics for:Combined prediction and reproducibility metrics for:− data-driven ROC-like curves;data-driven ROC-like curves;− optimizing bias-variance tradeoffs of pipeline interactions.optimizing bias-variance tradeoffs of pipeline interactions.

• Other Measures:Other Measures:− empirical random effects correction;empirical random effects correction;− measures of individual observation influence.measures of individual observation influence.


NPAIRS Metrics in Functional NPAIRS Metrics in Functional Neuroimaging StudiesNeuroimaging Studies

PETPET Strother SC, et. al., Hum Brain Mapp, 5:312-316, 1997. Strother SC, et. al., Hum Brain Mapp, 5:312-316, 1997. Frutiger S, et. al., Neuroimage 12:515-527, 2000.Frutiger S, et. al., Neuroimage 12:515-527, 2000. Muley SA, et. al., Neuroimage 13:185-195, 2001.Muley SA, et. al., Neuroimage 13:185-195, 2001. Shaw ME, et. al., Neuroimage 15:661-674, 2002.Shaw ME, et. al., Neuroimage 15:661-674, 2002. Strother SC, et. al., Neuroimage 15:747-771, 2002.Strother SC, et. al., Neuroimage 15:747-771, 2002. Kjems U, et al., et al., Neuroimage 15:772-786, 2002.Kjems U, et al., et al., Neuroimage 15:772-786, 2002.

fMRIfMRI Tegeler C, et. al. Hum Brain Mapp, 7:267-283, 1999.Tegeler C, et. al. Hum Brain Mapp, 7:267-283, 1999. Shaw ME, et. al. Neuroimage Shaw ME, et. al. Neuroimage 19:988-1001, 2003., 2003. LaConte S, et. al. Neuroimage LaConte S, et. al. Neuroimage 18:10-23, 2003. Strother SC, et. al., Neuroimage 23S1:S196-S207, 2004. LaConte S, et. al., Neuroimage 26:317-329, 2005. Chen X, et. al., Hum Brain Mapp (in press, special FIAC Chen X, et. al., Hum Brain Mapp (in press, special FIAC

issue)issue)


NPAIRS: Split-half reSampling for NPAIRS: Split-half reSampling for Activation-Pattern Reproducibility MetricsActivation-Pattern Reproducibility Metrics

1 1 1 11 r 1+r 02 2 2 2r 1 1 1 0 1-r 1 1

2 2 2 2


NPAIRS Split-Half Prediction and NPAIRS Split-Half Prediction and Reproducibility ResamplingReproducibility Resampling


PCA of data matrix:

Canonical Variates Analysis (CVA):

• Design matrix (G) “brain states” = discriminant classes.−prediction metric = posterior probability of class membership.−maximizes a multivariate signal-to-noise ratio:

(between-class, B)/(pooled within-class, W) covariance;

A Multivariate Model for NPAIRSA Multivariate Model for NPAIRS

Tt

T 1/ 2 T 1/ 2 1t t( ) ( ) svd G G EG WE E B

t vt x v

svd X E SU


Optimization of fMRI Static Force fMRI Optimization of fMRI Static Force fMRI Sixteen subjects with 2 runs/subject Sixteen subjects with 2 runs/subject Acquisition:

• Whole-brain, interleaved 1.5T BOLD-EPI;• 30 slices = 1 whole-brain scan;• 1 oblique slice = 3.44 x 3.44 x 5 mm3;• TR/TE = 4000 ms/70 ms

Experimental Design:

Analyzed with NPAIRS, GLM and PCA/CVA:• Dropped initial non-equilibrium and state-transition scans;Dropped initial non-equilibrium and state-transition scans;• 2-class single-subject;2-class single-subject;• 11-class 16-subject, group analysis;11-class 16-subject, group analysis;• NPAIRS/CVA, GLM-CVA comparison across preprocessing pipelines.NPAIRS/CVA, GLM-CVA comparison across preprocessing pipelines.


Preprocessing for Static ForcePreprocessing for Static Force All runs/subject(s) passed initial quality control:All runs/subject(s) passed initial quality control:

• movement (AIR 5) < 1 voxel;movement (AIR 5) < 1 voxel;• no artifacts in functional or structural scans;no artifacts in functional or structural scans;• no obvious outliers in PCA of centered data matrix.no obvious outliers in PCA of centered data matrix.

Alignment (AIR 5):Alignment (AIR 5):• Within-Subject:Within-Subject: across runs to 1st retained scan of run one; across runs to 1st retained scan of run one;

• Between-Subject:Between-Subject: 1 1st (Affine)st (Affine), 3, 3rdrd, 5, 5thth and 7 and 7thth order polynomials; order polynomials;

• Tri-linear and sinc (AIR 05) interpolation.Tri-linear and sinc (AIR 05) interpolation.

Temporal Detrending using GLM Cosine Basis (SPM):Temporal Detrending using GLM Cosine Basis (SPM):• None, None, • 0.5, (0.5,1.0), (0.5-1.5), (0.5-2.0), (0.5-2.5), (0.5-3.0) cosines/run.0.5, (0.5,1.0), (0.5-1.5), (0.5-2.0), (0.5-2.5), (0.5-3.0) cosines/run.

− (0.5-1.5) includes three GLM columns with 0.5, 1.0 and 1.5 cosines/run(0.5-1.5) includes three GLM columns with 0.5, 1.0 and 1.5 cosines/run

Spatial Smoothing with 2D Gaussian:Spatial Smoothing with 2D Gaussian:• None;None;• FWHM = 1, 1.5, 2, 3, 4, 6, 8 pixels (3.44 mm)FWHM = 1, 1.5, 2, 3, 4, 6, 8 pixels (3.44 mm)

− FWHM = 1.5 voxels = 0.52 mm; FWHM = 6 voxels = 21 mm.FWHM = 1.5 voxels = 0.52 mm; FWHM = 6 voxels = 21 mm.


ROC-Like: Prediction vs. Reproducibility ROC-Like: Prediction vs. Reproducibility 2-Class Static Force, Single Subject A Bias-Variance Tradeoff.

As model complexity increases (i.e., As model complexity increases (i.e., #PCs 10 →100), prediction of #PCs 10 →100), prediction of design matrix’s class labels design matrix’s class labels improves and reproducibility improves and reproducibility

(i.e., activation SNR) decreases.(i.e., activation SNR) decreases.

Optimizing Performance.Like an ROC plot there is a single Like an ROC plot there is a single point, (1, 1), on this prediction vs. point, (1, 1), on this prediction vs. reproducibility plot with the best reproducibility plot with the best performance; at this location the performance; at this location the model has perfectly predicted the model has perfectly predicted the design matrix while extracting an design matrix while extracting an infinite SNR.infinite SNR.

LaConte S, et. al. LaConte S, et. al. Evaluating preprocessing Evaluating preprocessing choices in single-subject BOLD-fMRI choices in single-subject BOLD-fMRI studies using data-driven performance studies using data-driven performance metricsmetrics. . Neuroimage Neuroimage 18:10-23, 2003


Prediction, Reproducibility, Dimensionality and Prediction, Reproducibility, Dimensionality and Canonical Variates (1.5 cos)Canonical Variates (1.5 cos)


Differences in Scanner SmoothnessDifferences in Scanner Smoothness

Courtesy Lee Friedman, UNM & Functional BIRNCourtesy Lee Friedman, UNM & Functional BIRN


Pipeline Meta-models: Data Analysis 1Pipeline Meta-models: Data Analysis 1 Bias-variance tradeoffs as a function of finite sample size are a

critical issue because:• Traditional, inferential, statistical parameter estimation is only asymptotically

unbiased & minimum variance.• Non-traditional estimation may = better signal detection!

− smaller parameter variance in non-asymptotic samples;− asymptotically-biased, often no asymptotic, inferential framework leading to resampling

techniques!

Resampling• Favour parameter estimation with Bootstrap over null hypothesis testing with

permutations!• Bootstrap’s advantage: it can be combined with cross-validation to

simultaneously obtain prediction and parameter estimates.• Boostrap’s disadvantage: requires iid samples; more restrictive than

permutations.


Pipeline Meta-models: Data Analysis 2Pipeline Meta-models: Data Analysis 2 Part of science by Strong Inference:Part of science by Strong Inference:

• for a scientifically interesting observation enumerate all alternative for a scientifically interesting observation enumerate all alternative hypotheses that can account for the observation, based on present hypotheses that can account for the observation, based on present knowledgeknowledge

− Jewett DL. What’s wrong with a single hypothesis. The Scientist, 19(21):10, 2005Jewett DL. What’s wrong with a single hypothesis. The Scientist, 19(21):10, 2005− Platt JR. Strong inference. Science, 146:347-353, 1964Platt JR. Strong inference. Science, 146:347-353, 1964

Comparing univariate GLM versus multivariate CVA Comparing univariate GLM versus multivariate CVA data analysis is a simple means of implementing multi-data analysis is a simple means of implementing multi-hypothesis tests in neuroimaging:hypothesis tests in neuroimaging:• test localizationist versus network theories of brain function!test localizationist versus network theories of brain function!• account for differences in data-analysis sensitivity and specificity!account for differences in data-analysis sensitivity and specificity!• test different interactions with preprocessing pipeline choices!test different interactions with preprocessing pipeline choices!


Simple Motor-Task Replication at 4.0TSimple Motor-Task Replication at 4.0T

t-test Fisher Linear Discriminant = 2-class CVA

L R

C. Tegeler, S. C. Strother, J. R. Anderson, and S. G. Kim, "Reproducibility of BOLD-based functional MRI obtained at 4 T," Hum Brain Mapp, vol. 7, no. 4, pp. 267-83, 1999.


Testing Meta-Model Differences: Static ForceTesting Meta-Model Differences: Static Force

Apply 9 different pipelines to each run of 16 static-force subjects:

− 4 x NPAIRS.CVA (optimised detrending, smoothing, # PCs);− 3 x NPAIRS.GLM (same paramters as 3 x NPAIRS.CVA);− FSL3.2.GLM (high-pass filtering, prewhitening, default HRF)− SPM2.GLM (high-pass filtering, prewhitening, default HRF)

9 statistical parametric images (SPI) x 2 runs = 18 SPIs/subject. perform NPAIRS, splitting on subjects for 18 x 16 SPIs.


Testing Pipeline Differences: Static ForceTesting Pipeline Differences: Static Force


Group-Preprocessing InteractionsGroup-Preprocessing Interactions








• Pipeline-driven, between-subject heterogeneity• Reacp: What have we learnt?


Subject-Specific Pipeline OptimizationSubject-Specific Pipeline Optimization

Shaw ME, et. al., Neuroimage 19:988-1001, 2003


RecapRecap• Background



−the Functional Imaging Analysis Contest (FIAC) experience?

• Seven meta-model optimization frameworks• Results with the 7th framework: NPAIRS• Data-analysis choices in pipeline meta-models:




AcknowledgementsAcknowledgements

Rotman Research InstituteRotman Research InstituteXu Chen, Ph.D.Xu Chen, Ph.D.

Anita Oder, B.Sc.Anita Oder, B.Sc.

Wayne Lee, B.Eng.Wayne Lee, B.Eng.

Cheryl Grady, Ph.D.Cheryl Grady, Ph.D.

Randy McIntosh, Ph.D.Randy McIntosh, Ph.D.

Principal Funding Sources: NIH Human Brain Project, P20-EB02013-Principal Funding Sources: NIH Human Brain Project, P20-EB02013-10 & P20-MH072580-01.10 & P20-MH072580-01.


AcknowledgementsAcknowledgementsUniversity of MinnesotaUniversity of Minnesota

International Neuroimaging Consortium & International Neuroimaging Consortium & VAMC: http://neurovia.umn.edu/incweb

Jon R. Anderson, M.Sc., Sally Frutiger, Ph.D., Kelly Rehm, Ph.D., David Rottenberg, M.D., Jon R. Anderson, M.Sc., Sally Frutiger, Ph.D., Kelly Rehm, Ph.D., David Rottenberg, M.D.,

Kirt Schaper, M.Sc., John Sidtis, Ph.D., Jane Zhang, Ph.D.Kirt Schaper, M.Sc., John Sidtis, Ph.D., Jane Zhang, Ph.D.

Seong-Ge Kim, Ph.D., Essa Yacob, Ph.D., Seong-Ge Kim, Ph.D., Essa Yacob, Ph.D., CMRR & Biomed. Eng.CMRR & Biomed. Eng.

James Ashe, M.D., Ph.D., James Ashe, M.D., Ph.D., Neurology & VAMCNeurology & VAMC

Suraj A. Muley, M.D., Suraj A. Muley, M.D., Neurology & VAMCNeurology & VAMC

Emory UniversityEmory University University of TorontoUniversity of Toronto

Xiaoping Hu, Ph.D.Xiaoping Hu, Ph.D. Rafal Kustra, Ph.D.Rafal Kustra, Ph.D.

Stephen LaConte, Ph.D.Stephen LaConte, Ph.D.

Technical University of DenmarkTechnical University of Denmark Melbourne UniversityMelbourne University

Lars Kai Hansen, Ph.D.Lars Kai Hansen, Ph.D. Gary Egan, Ph.D.Gary Egan, Ph.D.

Finn Arup Nielsen, Ph.D.Finn Arup Nielsen, Ph.D. Marnie Shaw, Ph.D.Marnie Shaw, Ph.D.

Principal Funding Sources: NIH Human Brain Project, P20-EB02013-10 & P20-MH072580-01.Principal Funding Sources: NIH Human Brain Project, P20-EB02013-10 & P20-MH072580-01.


Consensus-Model ROC ResultsSimple Signal Complex Signal

Hansen LK, Nielsen FA, Strother SC, Lange N. Consensus Inference in Neuroimaging. Neuroimage 13:1212-1218, 2001


NPAIRS-CVA Static Force Results: f(Preprocessing)NPAIRS-CVA Static Force Results: f(Preprocessing)

© s. c. strother, 2006 managing and optimizing fmri pipelines stephen c. strother, ph.d. rotman...

Documents

pipeline metamodels

npairsdataanalysis choices

nifti candidates

philosophyall models

applied statistics

dataprocessing pipeline

theory of statistics

fmri pipelines stephen