temporal and cross-subject probabilistic models for fmri prediction tasks
DESCRIPTION
Temporal and Cross-Subject Probabilistic Models for fMRI Prediction Tasks. Alexis Battle Gal Chechik Daphne Koller Department of Computer Science Stanford University. PBAI Competition. Provided rich data set Interesting interactions across time, subjects, and stimuli - PowerPoint PPT PresentationTRANSCRIPT
Temporal and Cross-Subject Probabilistic Models for fMRI Prediction Tasks
Alexis BattleGal ChechikDaphne Koller
Department of Computer ScienceStanford University
PBAI Competition
● Provided rich data set– Interesting interactions across time, subjects, and
stimuli
● Challenged us to come up with reliable techniques
● Thanks to the organizers!
Key Points
● Predictive voxels selected from whole brain
● Probabilistic model makes use of additional correlations– Subjects’ ratings across time steps
– Ratings between subjects
● Learn strength of each relationship
Modeling the fMRI Domain
A joint distributionin high dimension
Voxels across time
BOLD signal
Ratings across time
User Ratings
funnytimbodylanguage
Modeling the fMRI Domain
A joint distributionin high dimension
Voxels across time
BOLD signal
Ratings across time
User Ratings
funnytimbodylanguage
Training:Use two movies to learn the relations between voxels and ratings
Modeling the fMRI Domain
A joint distributionin high dimension
Voxels across time
BOLD signal
Ratings across time
User Ratings
funnytimbodylanguage
Training:Use two movies to learn the relations between voxels and ratings
Testing:Use the learned relations to predict ratings from fMRImeasurements
Probabilistic Model
• Each voxel measurement• Each rating to predict
Language
fromVox1 Vox2 Vox3
Probabilistic Model
• Each voxel measurement• Each rating to predict• Rating predicted from voxel measurements
– Linear regression model (Gaussian distribution)
Language
fromVox1 Vox2 Vox3
Probabilistic Model
• Each voxel measurement• Each rating to predict• Rating predicted from voxel measurements
– Linear regression model (Gaussian distribution)
Language
fromVox1 Vox2 Vox3• Selected predictive
voxels from whole brain• Regularize (Ridge,
Lasso) to handle noise
Probabilistic Model
Language
Vox1 Vox2
Language
Vox1 Vox2
…T =1 T =2
Probabilistic Model
● Ratings correlated across time– Language at time 1 makes language at time 2 likely
Language
Vox1 Vox2
Language
Vox1 Vox2
…T =1 T =2
Probabilistic Model
● Ratings correlated across time– Language at time 1 makes language at time 2 likely
Language
Vox1 Vox2
Language
Vox1 Vox2
…T =1 T =2
Probabilistic Model
● Ratings correlated across time– Language at time 1 makes language at time 2 likely
– Weight A – how correlated?
Language
Vox1 Vox2
Language
Vox1 Vox2
…T =1 T =2
A*Lang (1)*Lang(2)
A
Probabilistic Model
Subject 1
Subject 2
Language
Vox1 Vox2
Language
Vox1 Vox2
T =1 T =2
Language
Vox1 Vox2
Language
Vox1 Vox2
…
Probabilistic Model
● Ratings likely to be correlated between subjects
Subject 1
Subject 2
Language
Vox1 Vox2
Language
Vox1 Vox2
T =1 T =2
Language
Vox1 Vox2
Language
Vox1 Vox2
…
Probabilistic Model
● Ratings likely to be correlated between subjects– Weighted correlation, NOT equality
Subject 1
Subject 2
Language
Vox1 Vox2
Language
Vox1 Vox2
T =1 T =2
Language
Vox1 Vox2
Language
Vox1 Vox2
…
B B
Probabilistic Model
Joint model over all time points:
Time
…Sub1
Sub2
Gaussian Markov Random Field – joint Gaussian over all rating nodes conditioned on voxel data
Voxel Parameters
● Regularized linear regression for voxel parameters
Language
Vox1 Vox2 Vox3
Voxel Parameters
● Regularized linear regression for voxel parameters
Language
Vox1 Vox2 Vox3
Voxel Parameters
● Regularized linear regression for voxel parameters
β1 = 0.45
β2 = 0.55
Language
Vox1 Vox2 Vox3
Inter-Rating parameters
L(1)
Vox1 Vox2
L(2)
Vox1 Vox2B
● Other weights also learned from data– Example: cross-subject weights
Inter-Rating parameters
L(1)
Vox1 Vox2
L(2)
Vox1 Vox2B
● Other weights also learned from data– Example: cross-subject weights
Attention Faces
Inter-Rating parameters
L(1)
Vox1 Vox2
L(2)
Vox1 Vox2B
B = 0.3 B = 0.7
● Other weights also learned from data– Example: cross-subject weights
Attention Faces
Prediction Results
● Use full learned model, including all weights
● Predict ratings for a new movie given fMRI data
Prediction Results
● Use full learned model, including all weights
● Predict ratings for a new movie given fMRI data
Prediction Results
● Comparison to models without time or subject interactions
Voxel Selection
● Voxels selected by correlation with rating– Number of voxels determined by cross-validation
Voxel Selection
● Voxels selected by correlation with rating– Number of voxels determined by cross-validation
Selected Voxels
Faces Language
L L
Selected Voxels
Motion Arousal
L L
Voxel Selection
● Voxels selected for Language included some in ‘Face’ regions:
L
Voxel Selection
● Voxels selected for Language included some in ‘Face’ regions:
L
● Language and face stimuli correlated in videos
● Complex, interwoven stimuli affect voxel specificity
Voxel Selection
0.330.38
● Voxel selection extension – “spatial bias”– Prefer grouped voxels
* after competition submission
Voxel Selection
* after competition submission
0.330.38
● Voxel selection extension – “spatial bias”– Prefer grouped voxels
Voxel Selection
* after competition submission
0.330.38
● Voxel selection extension – “spatial bias”– Prefer grouped voxels
– Additional terms in linear regression objective:
● |β1| |β2| D(Vox1, Vox2)
D
|| Vox1 –Vox2||2
Adding Spatial Bias
Faces
L L
Conclusions
● Reliable prediction of subjective ratings from fMRI data
● Time step correlations aid in prediction reliability
● Cross-subject correlations also beneficial
● Individual voxels selected from whole brain– Reliability from regularization
– Some found in expected regions
– Some “cross-over” for correlated prediction tasks
