brain maps like mine

Brain Maps Like Minesemantic and computational image comparison

methods for meta-analysis and reproducibility of brain statistical maps

Vanessa SochatResearch In Progress

October 20, 2015

Outline

BackgroundWhy do we want to compare images?

Outline

BackgroundWhy do we want to compare images?

Computational Image ComparisonImpact of Image Thresholding on Similarity Metrics

Outline

BackgroundWhy do we want to compare images?

Computational Image ComparisonImpact of Image Thresholding on Similarity Metrics

Semantic Image ComparisonOntological and Graph Based Methods

Meta-analysis to synthesize understanding of human cognition and reproducibility of brain statistical maps.

Why Compare Images?

Are we good at reproducibility?

Why Compare Images?

1. For ReproducibilityWe do not not what a replication looks like.

Why Compare Images?

1. For ReproducibilityWe do not not what a replication looks like.

2. For Meta AnalysisWhat does all the research say about “anxiety?”

Outline

BackgroundWhy do we want to compare images?

Computational Image ComparisonImpact of Image Thresholding on Similarity Metrics

Semantic Image ComparisonOntological and Graph Based Methods

Results

What if there is data missing?Should I tranform the images first?What am I trying to optimize?

How similar are these results?

Goal: assess influence of different degrees of image thresholding on the outcome of pairwise image comparison

Goal: assess influence of different degrees of image thresholding on the outcome of pairwise image comparison

VARIABLESthresholdsmetricsoptimization

Methods Results

1. Define thresholds

Z = +/- [0,13]

Methods Results

GIVEN

SINGLE VALUE IMPUTATION

COMPLETE CASE ANALYSIS

2. Define comparison method

Methods Results

3. Define our similarity metrics

Pearson’s R

where

"Correlation coefficient" by Kiatdd - Own work. Licensed under CC BY-SA 3.0 via Wikimedia Commons

Spearman’s Rank Correlation Coefficient

Methods Results

4. What are we optimizing?

Methods Results

Data

465 single subjects7 tasks47 contrast images

working memoryanimalsgamblinglanguagerelationalemotionsocialmotor

“cat” vs. “dog”

Methods Results

Subsampling Procedure

For each of 500 subsamples: Subset data to unrelated groups A and B For each unthresholded map, A i in A Apply each threshold in Z = +/- 0:13, and Z = + 0:13 to all of B Calculate similarity for each of B to A i with CCA or SVI Assign correct classification if contrast A i most similar to equivalent contrast in B

Methods Results

Results

Highest classification accuracy for our dataset: Pearson Complete Case Analysis +/- 1.0

0.984

Results

...

Methods Results

https://github.com/vsoch/image-comparison-thresholding

Results

Methods Results

Conclusions

1. More data is not always better minimum degree of thresholding improves accuracy random field theory may be too much

2. Question the choice of metric, threshold, etc. complete case analysis, pearson, worked for us...

3. “What is the quantitative language that we should use to compare two images?”

Outline

BackgroundWhy do we want to compare images?

Computational Image ComparisonImpact of Image Thresholding on Similarity Metrics

Semantic Image ComparisonOntological and Graph Based Methods

cat dog

Semantic Image Comparison

Is semantic comparison of images useful to classify cognitive states?

The Approach:- “graph” based similarity- “probabilistic” based similarity- compare the two to spatial similarity

Semantic Similarity: OverviewIs semantic comparison of images useful to classify cognitive states?

Goals:- completely automated- assess predictive power of semantic similarity- relate to computational (spatial) similarity

cat

dog

Semantic Similarity: OverviewIs semantic comparison of images useful to classify cognitive states?

cat

dog

Semantic Similarity: OverviewIs semantic comparison of images useful to classify cognitive states?

ONTOLOGY

Tools

Semantic Similarity: MethodIs semantic comparison of images useful to classify cognitive states?

Goals:- completely automated

- Cognitive Atlas, NeuroVault, Pybraincompare

catdog

graph similarity( , )

IMAGE DATA METHODS

Data:- 93 brain maps tagged in NeuroVault with contrast → concept- programatically retrieve data, run methods, and output result

and visualization.

General Workflow- publish interesting results- tag with a contrast, and associated cognitive concepts- assess semantic similarity

- graph based- probabilistic

Semantic Similarity: MethodIs semantic comparison of images useful to classify cognitive states?

Semantic SimilarityGraph Based Methods

cat dog

feline canine

mammal

graph similarity( , )

Graph Similarity: Method

visual feline recognition

visual canine recognition

Wang’s Method

- aggregates semantic contributions of ancestor terms

1. We start with associated concepts.

visual canine recognition

animal recognitionis a kind of

is part ofis a kind of recognition

canine fear response

Wang’s Method

- aggregates semantic contributions of ancestor terms

2. We then take weights at intersection

visual canine recognition

animal recognitionis a kind of

is part ofis a kind of recognition

canine fear response

visual feline recognition

animal recognitionis a kind of

is part ofis a kind of recognition

feline fear response

S( , ) = sum(intersected weights) sum(all weights)

Graph Similarity: Method

Semantic

Graph Similarity: First Round

Semantic SimilarityProbabilistic Methods

Reverse InferenceP(cognitive process | activation)

database of images a new result

Reverse Inferencefor image classification and concept validation

a new result

P( | )

P( | )

P(cognitive process | a spatial map)

P(node mental process|activation) = P(activation|mental process) * P(mental process)

P(activation|mental process) * P(mental process) + P(A|~mental process) * P(~mental process)

P( | )

What does a high score say?about the cognitive concept?

P( | )contributes evidence for

Data:93 brain maps tagged in NeuroVault with contrast → conceptprogramatically retrieve data, run methods, and output result and visualization.

For each of 93 brainmaps, as query image: For each of 140 concept nodes, node, in Cognitive Atlas: calculate P(node|query image) Assign correct classification if P(node|query image) > 0.5

Probabilistic Similarity Method

Probabilistic Similarity Preliminary Results

Probabilistic Similarity Area Under the Curve Across Concepts

Summary

Image comparison is essential formeta-analysis and reproducibility

Summary

Image comparison is essential formeta-analysis and reproducibility

A small amount of image thresholdingaids to find images of similar contrast

Summary

Image comparison is essential formeta-analysis and reproducibility

A small amount of image thresholdingaids to find images of similar contrast

Semantic Image Comparisonis a promising strategy to assess reproducibility

Summary

Image comparison is essential formeta-analysis and reproducibility

A small amount of image thresholdingaids to find images of similar contrast

Semantic Image Comparisonis a promising strategy to assess reproducibility

Acknowledgements

INCF/ NidashSatra GhoshNolan NicholsJessica TurnerTom NicholsJB PolineDavid Keator

CollaboratorsTal Yarkoni

Nipy

FundingMicrosoft ResearchSGF and NSF

PoldracklabRuss PoldrackChris GorgolewskiCraig MoodieSanmi KoyejoPatrick BissettJoke DurnezIan EisenbergMac ShineJoe Wexler

BMIDaniel RubinRuss AltmanMark MusenRebecca SawyerMary JeanneNancySteven BagleyJohn DiMario

Thank You!vsochat@stanford.edu

vsoch.github.io

Coordinate-Based Approaches

column 1: raw data: big black dots showing the local maxima that are reported, dotted line is “true” simulated signal, black thick line is that signal with added noise.column 2: shows the results of ALE: the result is more of a curve because the “ALE statistic” reflects a probability value that at least one peak is within r mm of each voxel, so the highest values of course correspond to actual peaks.column 3: kernel density analysis (KDA) gives us a value at each voxel that represent the number of peaks within r mm of that voxel. If we divide by voxel resolution we can turn that into a “density”column 4: is MULTI kernel density analysis, which is the same as KDA, but the procedure is done for each study. The resulting “contrast indicator maps” are either 1 (yes, there is a peak within r mm) or 0 (nope).

ALE

KDA

MKDA

Pairwise Image

Comparison

Animals paradigm

vsoch.github.io/experiment

visual canine recognition

CONCEPTS

visual feline recognition

visual feline recognition

RELATIONSHIPS

is a kind of

animal recognition

visual canine recognition

is a kind of

GRAPH SIMILARITY

0.8

Pearson Correlation(rho)