Nonparametric Hierarchical Bayesian Model for Functional Brain Parcellation

Danial Lashkari† Ramesh Sridharan† Edward Vul‡ Po-Jang Hsieh‡

Nancy Kanwisher‡ Polina Golland†

†Computer Science and Artificial Intelligence Laboratory, MIT‡ Department of Brain and Cognitive Sciences, MIT77 Massachusetts Avenue, Cambridge, MA 02139

Abstract

We develop a method for unsupervised analysis of func-tional brain images that learns group-level patterns of func-tional response. Our algorithm is based on a generativemodel that comprises two main layers. At the lower level,we express the functional brain response to each stimulusas a binary activation variable. At the next level, we de-fine a prior over the sets of activation variables in all sub-jects. We use a Hierarchical Dirichlet Process as the priorin order to simultaneously learn the patterns of responsethat are shared across the group, and to estimate the num-ber of these patterns supported by data. Inference based onthis model enables automatic discovery and characteriza-tion of salient and consistent patterns in functional signals.We apply our method to data from a study that explores theresponse of the visual cortex to a collection of images. Thediscovered profiles of activation correspond to selectivity toa number of image categories such as faces, bodies, andscenes. More generally, our results appear superior to theresults of alternative data-driven methods in capturing thecategory structure in the space of stimuli.

1. IntroductionFunctional MRI studies are typically driven by a priori

hypotheses. Typically, an experiment is designed based ona hypothesis and significance tests are used to localize therelevant functional regions in the brain. However, findinga good functional hypothesis is not always straightforward,especially in the presence of multiple patterns of functionalspecificity. For instance, the studies of visual object recog-nition assume that a certain category of images activates se-lective areas in the brain, and look for such regions usingsignificance tests. This approach has been successfully usedto discover areas selective for categories such as faces, bod-ies, and scenes in the cortex [5]. Considering the number ofpossible image categories, however, we face an exceedinglylarge space of likely selectivity patterns if we go beyond theobvious categories studied before. The same problem arises

in other domains, e.g., the studies of language or auditorynetworks, if we aim to investigate the functional specificitystructure in detail.

An alternative approach is to present a variety of stim-uli relevant to the network under study and to apply data-driven fMRI analysis to generate appropriate hypotheses.Data-driven methods decompose the data into a number ofcomponents, each describing one temporal (functional) pat-tern and its corresponding spatial extent. A popular methodfor this decomposition is spatial Independent ComponentAnalysis (ICA) [2, 16] wherein the goal is to make the com-ponents spatially independent. Beyond exploratory analy-sis, ICA is also aimed at automatic denoising of the data.Therefore, it assumes an additive model for the data andallows spatially overlapping components. However, nei-ther of these assumptions is appropriate for studying func-tional specificity. For instance, an fMRI response that is aweighted combination of a component selective for scissorimages and another selective for hammer images may bebetter described by selectivity for tools. In this case, ratherthan adding several components, we should associate eachvoxel with only one pattern of response. Moreover, com-mon extensions of ICA to fMRI analysis require voxel-wisespatial normalization while some functional areas appear inhighly variable locations across subjects [24].

Clustering, another data-driven method, has also beenused to segment the fMRI data based on the time coursesor protocol-related features [1, 8, 9]. Clustering is morenaturally suited to the studies of functional specificity sinceit assigns each voxel to only one cluster. Recent work hasshown that, by representing the data as a set of voxel re-sponses to different conditions, clustering can reveal mean-ingful functional patterns in the brain [13, 14, 23]. However,the current methods for functional segmentation either lacka proper model for group analysis, or do not determine howto choose the number of clusters.

In this paper, we present a novel nonparametric hierar-chical model for functional brain segmentation that appliesto group fMRI data and automatically determines the num-

1978-1-4244-7028-0/10/$26.00 ©2010 IEEE

Figure 1. Schematic diagram illustrating the concept of a system.System k is characterized by vector [φk1, · · · , φkL]T that speci-fies the level of activation induced in the system by each of the Lstimuli. This system describes a pattern of response demonstratedby collections of voxels in all J subjects in the group.

ber of clusters. Our model builds upon the basic frameworkof Hierarchical Dirichlet Processes (HDP) [20] for sharingstructure in a group of datasets. Our structures of interestare salient patterns of functional specificity, i.e., groups ofvoxels with similar functional responses. The nonparamet-ric aspect of the model allows automatic search in the spaceof models with different numbers of systems. Moreover, weprovide a model for fMRI responses that removes the needfor the heuristic normalization schemes commonly used asa preprocessing step [14]. Notably, this model transformsfMRI responses into a set of activation probabilities. There-fore, the activation profiles of systems can be naturally in-terpreted as signatures of functional specificity: they de-scribe the probability that any stimulus or task activates agiven functional parcel. This approach uses no spatial in-formation other than the original smoothing of the data andtherefore does not suffer from the drawbacks of voxel-wisespatial normalization. Based on the model, we derive a scal-able algorithm using a variational Bayesian approximation.

Nonparametric Bayesian models have been previouslyemployed in fMRI data analysis, particularly in modelingthe spatial structure in localization maps found via sig-nificance tests [12, 25]. Our probabilistic model is moreclosely related to a recent application of HDPs to DTI datawhere anatomical connectivity profiles of voxels are clus-tered across subjects [11]. In contrast to all these methodsthat apply stochastic sampling for inference, we take ad-vantage of a variational scheme that is known to have fasterconvergence rate and greatly improves the speed of the re-sulting algorithm [21].

This paper is organized as follows. We begin by describ-ing the two layers of the model and our variational infer-ence procedure in Sec. 2. We present experimental resultsin Sec. 3 and compare them with results found by tenso-rial group ICA [3] and a finite mixture-model clusteringmodel [14]. Finally, we conclude in Sec. 4.

2. Nonparametric Hierarchical BayesianModel for fMRI Data

Consider an fMRI experiment with a relatively largenumber of different tasks or stimuli, e.g., passive viewingof L distinct images in a visual study. The set of raw timecourses {bit} describe the acquired BOLD signals at ac-quisition times t in voxels i. We can estimate the fMRIresponses of voxels to each of the L experimental condi-tions using the standard General Linear Model for BOLDtime courses [7]. As a result, the data is represented as aset of values yil that describe the fMRI response of voxeli to stimulus l. Commonly, studies include data from sev-eral subjects; therefore, we sometimes denote the fMRI re-sponses by yjil where index j identifies different subjects inthe group.

Our generative model explains the fMRI responses ofvoxels assuming a clustering structure that is shared acrosssubjects. To distinguish these functionally-defined clustersfrom the traditional cluster analysis used for the correctionof significance maps [6], we follow the terminology used in[14] and refer to these clusters as functional systems. Fig. 1illustrates the idea of a system as a collection of voxelsacross all subjects that share a coherent pattern of functionalresponse. We assume an infinite number of group level sys-tems; each system k comes with a weight πk that specifiesthe prior probability that it includes any given voxel. Inthis way, different draws from the same distribution poten-tially yield different finite numbers of clusters. Due to inter-subject variability and noise, the group-level system weightπ is independently perturbed for each subject j to generate asubject-specific weight vector βj . System k is also charac-terized by a vector [φk1, · · · , φkL]T where L is the numberof stimuli. Here, φkl ∈ [0, 1] is the probability that system kis activated by stimulus l. Based on the weight βj and thesystem probabilities φ, we generate binary activation vari-ables xjil ∈ {0, 1} that express whether or not voxel i insubject j is activated by stimulus l.

Up to this point, our model has the structure of a stan-dard HDP. The next layer of this hierarchical model defineshow activation variables xjil generate response values yjil.Our model for the fMRI response employs a set of voxel-specific response parameters µji = (µji, aji, λji) to ex-press this connection. Table 1 presents a summary of all thevariables and parameters and Fig. 2 shows the structure ofour graphical model. Next, we discuss the details of the twolayers in our model.

2.1. Model of fMRI Responses

We assume that response yjil in each voxel to each stim-ulus can be described as a mixture of two modes: activeand non-active. The expected response of the active mode isstrictly greater than the expected response of the non-active

Figure 2. The graphical model for the joint distribution implied bythe fMRI response model in Sec. 2.1 and the HDP prior in Sec. 2.2.

mode. This mixture distribution depends on the binary acti-vation variable xjil that specifies the mode of the response,and variables µji = (µji, aji, λji) that describe the param-eters of the mixture. Formally, we have

yjil | (xjil = 0,µji) ∼ Normal(µji, λ

−1ji

), (1)

yjil | (xjil = 1,µji) ∼ Normal(µji + aji, λ

−1ji

), (2)

where µji is the expected value of the non-active response,aji > 0 is the increase in the expected response due to acti-vation, and λji is the reciprocal of the variance of the i.i.d.white noise.

The goal of this layer of the model is to transform theresponse values so that we can effectively compare themacross voxels. To show why this matters, Fig. 3 (top) showsthe elements of the fMRI response vectors [yi1, · · · , yiL]T

for a number of voxels detected to be selective for the samecategory of stimuli through conventional analysis. In or-der to detect these voxels, we form a contrast that assumesthat the responses to the preferred stimuli are on averagehigher than the responses to the rest. However, since the re-sponses of different voxels have different ranges, we cannotdirectly compare response vectors. This phenomenon canbe explained by the effect of excessive noise or the inac-curacy of the models used in the preprocessing stage. Ourmodel encodes the relevant information as binary activationvariables that describe the states of voxels relative to theirown dynamic ranges. We note that the idea of describingthe response of active and non-active voxels by a mixture ofdistributions has been used before in the conventional de-tection framework [15]. Here, we couple this signal modelwith the HDP prior on activation variables as described laterin this section.

We also assume the following priors on the dis-tribution of voxel response variables parameterized byθµ = (θµ, θa, θλ):

µji | θµ ∼ Normal(θµ,1θ

−1µ,2 , θ

−1µ,2

), (3)

aji | θa ∼ Normal+(θa,1θ

−1a,2 , θ

−1a,2

), (4)

λji | θλ ∼ Gamma (θλ,1 , θλ,2) , (5)

yjil fMRI response of voxel i in subject j to stimulus lxjil binary fMRI activation of voxel i in subject j for stimulus lµji non-activated fMRI response of voxel i in subject jaji activation increase in fMRI response of voxel i in subject jλji variance reciprocal of fMRI response of voxel i in subject jzji system membership of voxel i in subject jφkl Bernoulli activation parameters of system k for stimulus lβj system prior weight in subject jπ group-level system prior weightα, γ HDP scale parametersθφ parameters of the prior over system parameters φθµ parameters of the priors on voxel response variables (µ, a, λ)θα, θγ parameters of the priors over scale parameters α and γ

Table 1. Variables and parameters in the model.

!! !" !# !$ % $ # " ! &%

%

$

#

"

!

&%

&$

&#

&"

&!

$%

!! !" !# !$ % $ # " ! &%

%

$

#

"

!

&%

&$

&#

&"

&!

$%

i = 1

i = 10y

!! !" !# !$ % $ # " ! &%

%

$

#

"

!

&%

&$

&#

&"

&!

$%

µjiaji

!"#$%&'(')!"#$%&!!!1

ji

Figure 3. (Top) The values of fMRI responses yil for 10 differentvoxels detected to be face-selective in the conventional analysisof a visual experiment in one subject. Each row contains the re-sponses of one voxel. We show the responses to stimuli l thatcorrespond to faces in red and the responses to all non-face stim-uli in blue. (Bottom) Schematic diagram illustrating the meaningof each of voxel response parameters µji = (µji, aji, λji) in themodel.

where Normal+(θa,1θ−1a,2, θ

−1a,2) is the conjugate prior de-

fined as a normal distribution restricted to positive real val-ues:

p(a) ∝ e−a2 θa,2+a θa,1 , for a ≥ 0. (6)

Positivity of variable aji simply reflects the constraint thatthe expected value of fMRI response in the active state isgreater than the expected value of response in the non-activestate.

2.2. HDP Prior for fMRI Activations

We choose a model to simultaneously infer the numberand characterization of group-level systems using Hierar-chical Dirichlet Processes (HDP) [20]. More specifically,given the set of probabilities of system activation for differ-ent stimuli φ = {φkl} and system memberships of voxels

z = {zji}, zji ∈ {1, 2, · · · }, the model assumes

xjil | zji,φi.i.d.∼ Bernoulli(φzjil). (7)

This implies that all voxels within a system have the sameprobability of being activated by a particular stimulus l. Weuse the stick-breaking formulation of HDP and define theprior for system memberships as follows:

zji | βji.i.d.∼ Mult(βj), (8)

βj | πi.i.d.∼ Dir(απ), (9)

π | γ ∼ GEM(γ), (10)

where GEM(γ) is a distribution over infinitely long vec-tors π = [π1, π2, · · · ]T , named after Griffiths, Engen andMcCloskey [18], defined as follows:

πk = vk

k−1∏k′=1

(1− vk′) , vk | γi.i.d.∼ Beta(1, γ). (11)

The components of the generated vectors π sum to one withprobability 1. Hence, they provide a meaningful interpre-tation as the group-level expected values for the subject-specific multinomial system weights βj in Equation (9).With this prior over the system memberships of voxels z,the model in principle allows an infinite number of systems;however, for any finite set of voxels, a finite number of sys-tems is sufficient to include all voxels.

We let the prior distribution for system-level activationprobabilities φ be

φkli.i.d.∼ Beta(θφ,1, θφ,2). (12)

By introducing bias towards 0, the non-active state, in theparameters of this distribution, we can induce sparsity in theresults. The graphical model for the joint distribution of thismodel and the fMRI response model of Sec. 2.1 is shown inFig. 2.

Equations (7), (8), (9), (10), and (12) together define ourHDP prior over fMRI activation variables x. We furtherassume Gamma priors for the hyperparameters:

α ∼ Gamma(θα,1, θα,2) , γ ∼ Gamma(θγ,1, θγ,2).(13)

2.3. Variational Bayesian Approximation

Several different Gibbs sampling schemes for inferencein HDPs are discussed in [20]. Despite theoretical guar-antees of their convergence to the true posterior, sam-pling techniques generally require a time-consuming burn-in phase. Because of the relatively large size of our prob-lem, we choose an alternative variational approach to infer-ence called Collapsed Variational HDP approximation [21],which is known to yield faster algorithms. Due to spaceconstraints, we cannot provide the derivations in this paper,

ζj ∼ Beta (E[α], Nj)

θ̃rjk = exp“E[logα] + E[log vk] +

Pk′<kE[log(1− vk′)]

”E[rjk] = θ̃rjkEz[Ψ(θ̃rjk + njk)−Ψ(θ̃rjk )]

α ∼ Gamma(θ̃α,1 , θ̃α,2) γ ∼ Gamma(θ̃γ,1 , θ̃γ,2)

θ̃α,1 = θα,1 +Pjk E[rjk] θ̃γ,1 = θγ,1 +K

θ̃α,2 = θα,2 −Pj E[log ζj ] θ̃γ,2 = θγ,2 −

Pk E [log (1− vk)]

vk ∼ Beta(θ̃vk,1 , θ̃vk,2)

θ̃vk,1 = 1 +Pj E[rjk] θ̃vk,2 = E[γ] +

Pj,k′>kE[rjk]

φkl ∼ Beta(θ̃φkl,1 , θ̃φkl,2)

θ̃φkl,1 = θφ,1 +Pj,i q(zji = k)q(xjil = 1)

θ̃φkl,2 = θφ,1 +Pj,i q(zji = k)q(xjil = 0)

µji ∼ Normal(θ̃µji,1 θ̃−1µji,2

, θ̃−1µji,2

)

θ̃µji,1 = θµ,1 + E[λji]`P

l yjil − E[aji]Pl q(xjil = 1)

´θ̃µji,2 = θµ,2 + E[λji]L

aji ∼ Normal+(θ̃aji,1 θ̃−1aji,2

, θ̃−1aji,2

)

θ̃aji,1 = θa,1 + E[λji]Pl q(xjil = 1) (yjil − E[µji])

θ̃aji,2 = θa,2 + E[λji]Pl q(xjil = 1)

λji ∼ Gamma(θ̃λji,1, θ̃λji,2)

θ̃λji,1 = θλ,1 + 12

Pl ( yjil − E[µji]− q(xjil = 1)E[aji] )2

+L2V [µji] + V [aji]

Pl q(xjil = 1) + E[aji]

2Pl V [xjil]

θ̃λji,2 = θλ,2 + L2

Table 2. Update rules for computing the posterior q over the unob-served variables.

but for completeness, here we provide a brief overview ofthe steps and the final update rules of the resulting algorithm(see the Supplementary Material for details).

We first marginalize our distribution over the subject-specific system weights β = {βj} and add a set of aux-iliary variables r = {rji} and ζ = {ζj} to the model tofind closed-form solutions for the inference update rules.Let h = {x, z, r, ζ,φ,µ, π, α, γ} denote the set of all un-observed variables. In the framework of variational infer-ence, we approximate the model posterior on h given theobserved data p(h|y) by a distribution q(h). The approxi-mation is performed through the minimization of the Gibbsfree energy function F [q] = E[log q(h)] − E[log p(y,h)].Here, and in the remainder of the paper, E[·] and V [·] in-dicate expected value and variance with respect to distribu-tion q. We assume a distribution q of the form:

q(h) = q(α)q(γ)q(r, ζ|z) (14)

·∏j,i

[q(µji)q(aji)q(λji)q(zji)

∏l

q(xjil)

],

where we explicitly account for the dependency of the aux-iliary variables on the system memberships. Including thisstructure maintains the quality of the approximation despitethe introduction of the auxiliary variables [22].

Minimizing the Gibbs free energy function in terms ofeach component of q, assuming all the rest as constant,we find the update rules in Table 2. We define njk =∑Nj

i=1 δ(zji, k) as the number of voxels in subject j that aremembers of system k. In addition, for the system member-ship updates, we find

q(zji = k) ∝ exp{Ez¬ji [log(θ̃rjk

+ n¬jijk )]+ (15)∑l

(q(xjil = 1)E[log φkl] + q(xjil = 0)E[log(1− φkl)]

)},

where Ψ(x) = ddx log Γ(x), and n¬jijk and z¬ji indicate the

exclusion of voxel i in subject j. Moreover, the posteriorover activation variables can be described as

q(xjil = 1) ∝ exp{∑

l

q(zji = k)E[log φkl] (16)

− E[λji]2

[(yjil − E[µji]− E[aji]

)2

+ V [aji]]},

q(xjil = 0) ∝ exp{∑

l

q(zji = k)E[log(1− φkl)] (17)

− E[λji]2

(yjil − E[µji]

)2}.

Note that under q(z), each variable njk is the sum ofNj independent binary variables. Therefore, we can usethe Central Limit Theorem to approximate terms of theform Ez[f(njk)] by assuming a Gaussian distribution fornjk [21].

2.4. Initialization

By iterative application of the update rules in Sec. 2.3,we can find a local minimum of the Gibbs free energy. Sincevariational solutions are known to be biased toward theirinitial configurations, the initialization phase becomes criti-cal to the quality of the results. We use the following ap-proach in order to provide reasonable starting points forthe algorithm.We first cluster the L values of yjil corre-sponding to each voxel into two clusters, active and non-active, to form initial values for the activation x and voxelresponseµ. Then, for initializing q(z), we sample the voxelmemberships by introducing the voxels one by one and inrandom order to the collapsed Gibbs sampling scheme [20]constructed for our HDP.

3. Results and DiscussionWe applied our method to data from an event-related

experiment where subjects view 69 distinct images overthe course of two 2-hour scanning sessions. Fig. 4 showsthese images; they include 8 images from each categoryof animals, bodies, cars, faces, shoes, scenes, tools, trees,along with 5 vase images. The study includes 10 subjects.

Animals

1 2 3 4

5 6 7 8

Bodies

1 2 3 4

5 6 7 8

Cars

1 2 3 4

5 6 7 8

Faces

1 2 3 4

5 6 7 8

Scenes

1 2 3 4

5 6 7 8

Shoes

1 2 3 4

5 6 7 8

Tools

1 2 3 4

5 6 7 8

Trees

1 2 3 4

5 6 7 8

Vases

1 2 3 4

5

Figure 4. The set of the 69 images used as stimuli in our experi-ment.

The data was first motion-corrected separately for the twosessions[4], and then spatially smoothed with a Gaussiankernel of 3mm width. The BOLD time course data includesabout 3,000 volumes per subject. We applied the standardGeneral Linear Model [7] to estimate the response of eachvoxel to each image. We then registered the data from thetwo sessions to the subject’s native anatomical space [10].We applied our algorithm, the finite mixture-model cluster-ing [14], and the tensorial ICA algorithm [3] to the sameset of estimated image responses for all subjects. We usedFSL/Melodic implementation of tensorial ICA.1

For our algorithm and the finite mixture-modeling, weremoved noisy voxels from the analysis. To this end, weperformed an ANOVA for all stimulus regressors and in-cluded only voxels with the F -test significance value be-low p = 10−4 (uncorrected). This procedure yielded about64,000 voxels across all subjects. We then applied the anal-ysis in each subject’s own native space. For the visualiza-tion of the resulting spatial maps, we aligned all the brainswith the Montreal Neurological Institute (MNI) coordinatespace [19]. Applying group ICA requires spatial normal-ization of all subjects’ brains prior to the analysis. Sincethe masks generated by the ANOVA tests do not necessar-ily overlap after normalization, we only excluded the voxelsoutside the brain for this analysis. To remove any stimulus-specific pattern in the responses in all the above analyses,we subtracted from the estimated voxel response to eachstimulus the average response across all voxels in that sub-

1http://www.fmrib.ox.ac.uk/fsl/melodic/index.html

0

0.5

1

Sys

1

0

0.5

1

Sys

2

0

0.5

1

Sys

3

0

0.5

1

Sys

4

0

0.5

1

Sys

5

0

0.5

1

Sys

6

0

0.5

1

Sys

7

0

0.5

1

Sys

8

0

0.5

1

Sys

9

0

0.5

1

Sys

10

−2

0.2

2.4

IC 1

−2

1

4

IC 2

−2

0.2

IC 3

−2

0.2

IC 4

−2

0.4

2.8

IC 5

−2

−0.1

1.8

IC 6

−3

−0.4

2.2

IC 7

−3

−0.5

2

IC 8

−3

−0.3

2.4

IC 9

−2

0.3

2.6

IC 1

0

0

0.2

Sys 1

0

0.2

Sys 2

0

0.3

Sys 3

0

0.3

Sys 4

0

0.3

Sys 5

0

0.3

Sys 6

0

Sys 7

0

Sys 8

0

Sys 9

0

0.2

Sys 1

0

Animal Body Car Face Scene Shoe Tool Tree Vase Animal Body Car Face Scene Shoe Tool Tree Vase Animal Body Car Face Scene Shoe Tool Tree Vase

Our model Tensorial ICA Finite-mixture model

Figure 5. The profiles for the 10 most consistent discovered systems and components found by our nonparametric method, tensorial ICA,and the finite mixture model. The profiles of our systems represent the activation probabilities E[φkl] of different systems k.

ject.We also generated the conventional significance maps

for the three well-known types of category selectivity, i.e.,selectivity for bodies, faces, and scenes. We formed con-trasts by subtracting from the average response to each ofthese categories the average response to all cars, shoes,tools, and vase stimuli. We then thresholded the maps atp = 10−4 (uncorrected).

In our method, we set the hyperparameter values(θγ,1, θγ,2) = (1, 1), (θµ,1, θµ,2) = (0, 0.1), (θa,1, θa,2) =(0, 0.1), and (θλ,1, θλ,2) = (1, 0.1). We also choose(θτ,1, θτ,2) = (1, 3) to encourage sparsity in the results bybiasing the activation prior towards being nonactive. Fur-thermore, we select (θα,1, θα,2) = (0.01, 0.0001) to in-crease both the expected value and the variance of the scaleparameter α in order to put emphasis on finding systemsthat are representative of the entire group (see Eq. (9)). Werun the algorithm 30 times with different random initializa-tions as described in Sec. 2.4 and select the solution withthe lowest Gibbs free energy.

Our algorithm finds 22 systems; applying the automaticmodel selection algorithm [17] along with tensorial ICAyields 27 components. Tensorial ICA results include a setof variables that describe the contribution of each compo-nent to the results in each subject. We can create a similarmeasure for the consistency of each system in clustering

models by first computing the ratio of all voxels assigned tothat system that belongs to each subject and then computingthe standard deviation of this ratio across subjects. For bothmethods, there are some systems or components that mainlycontribute to the results of one or very few subjects and pos-sibly reflect idiosyncratic characteristics of noise in thosesubjects. Since we are interested only in the most consis-tent systems, we rank the resulting systems and componentsbased on the computed measures of consistency across sub-jects. We apply the same procedure to the results from thefinite mixture modeling with 30 systems.

Fig. 5 shows 10 profiles corresponding to the most con-sistent (as defined above) systems or components found byour method, ICA, and finite mixture-modeling. Our acti-vation profiles correspond to the probabilities E[φkl] of thesystems. We observe that the category information is moresalient in the HDP system profiles of activation, especiallywhen compared to the ICA results. Most of our systemsdemonstrate similar probabilities of activation for imagesthat belong to the same category. This suggests that thediscovered systems in fact describe coherent patterns of re-sponse relevant to the nature of the intrinsic object repre-sentation in the visual system.

More specifically, systems 1, 3, and 8 appear mostly se-lective for bodies, faces, and scenes, respectively. Amongthe ICA results, we only find two components with rela-

Subj

ect1

Subj

ect2

Subj

ect3

Subj

ect4

system 1 bodies – objects system 3 faces – objects system 8 scenes – objects

Figure 6. The body (left column set), face (middle), and scene (right) selective areas defined by our data-driven method in Fig. 5 (the rightmap within each set), compared to the conventional contrasts (the right map within each set) in four different subjects. Our maps presentmembership probabilities while the contrast maps show significance values − log p thresholded at 4. Each pair shows the maps on thesame slice from one subject. We normalized all the subjects in the Talairach space so that all four slices in the same column set are aligned.Our hierarchical model is validated by the conventional contrasts: wherever a particular system is found, the conventional contrasts aresignificant. Moreover, our results show more consistency across subjects.

tively larger responses to bodies and scenes (components1 and 2), while the consistency of the finite-mixture mod-eling systems selective for the same categories is lowerthan the results of our model. We emphasize the advan-tage of our activation profiles over the selectivity profilesof finite mixture modeling in terms of interpretability. Theelements of a system activation profile in our model rep-resent the probabilities that different stimuli activate thatsystem. Therefore, the brain response to a stimulus can besummarized based on our results in a vector of activations[E[φ1l], · · · , E[φKl] ]T that it induces over the set of allfunctional systems. Such a representation cannot be madefrom the clustering profiles in Fig. 5 since their elements donot have any clear interpretation. Inspecting the activationprofiles of different systems more closely, we find other in-teresting patterns. For instance, we note that most imagesof animals can naturally activate body-selective systems aswell. Even more interestingly, the two non-face images thatshow some probability of activating the face selective sys-tem 6, namely, animals 5 and 7 (Fig. 4) correspond to thetwo animals that have large faces. Another interesting case

IC 1 bodies–objects System 1

IC 2 scenes–objects System 8

Figure 7. Comparison between the ICA maps for components 1and 2 in Fig. 5 (left) and the group averages of contrast (middle)and HDP maps (right).

is system 4, which seems to be activated by stimuli that arelarger in terms of non-background pixels (notice how onlyshoe 2, the largest shoe, activates this system).

Our approach also yields probabilistic spatial maps foreach discovered system. Fig. 6 compares the maps of the

three systems 1, 3, and 8 with the conventional significancemaps for the corresponding selective areas. The map forsystem k in subject j is defined by the vector of posteriorprobabilities [q(zj1 = k), · · · q(zjNj

= k)]T . We have nor-malized the maps after the analysis and show the results onthe same slice for all subjects. To choose the slice to displayfor any of the systems, we considered the number of voxelsin each slice that were assigned to that system in more thanhalf the subjects, and then chose the slice with the greatestsuch number. For the most part, voxels that are assigned toour systems also demonstrate high significance values basedon the conventional statistical tests. Moreover, our resultsshow more consistency across the subjects.We note that thecharacterization of the conventionally detected selective ar-eas is less restrictive than that of our systems. While systemactivation probabilities completely describe the response ofa system to all stimuli, conventional selectivity is defined interms of the difference of response between two groups ofstimuli. That explains the fact that, for instance, the scenecontrast for the two bottom subjects include early visual ar-eas while those voxels do not appear in our results.

Fig. 7 shows the group probability maps for the body-and scene-selective ICA components on the same slices asin Fig. 6. These maps have been computed fitting a mixturemodel to the z-scores from the original component mapsand then thresholding at 0.5 [2]. The images clearly il-lustrate how the voxel-wise alignment before the analysishas blurred the results: none of the subject-specific areas inFig.6 is as large as the maps in Fig.7. Rather, the two ICAmaps seem to be driven by the variability in the anatomicallocations of the areas across subjects. This becomes evi-dent when we compare these maps in the same figure withthe group sum of the thresholded contrast maps (p = 10−4

uncorrected) and our probability maps for the correspond-ing forms of selectivity (both maps take values from 0 toJ = 10). The two maps appear at the same approximatelocations but with very small overlapping areas.

4. ConclusionWe presented a method that combines fMRI data from

several subjects in order to identify functional systems inthe brain. We defined systems as group-wide collections ofvoxels that demonstrate coherent patterns of response. Weemployed a nonparametric hierarchical Bayesian model todevelop a generative model that captures shared structuresin the response of a group of subjects. We further derivedand implemented a fast variational algorithm for inferencebased on this model. We applied our method to an fMRIstudy of category selectivity in the visual cortex present-ing a variety of distinct images. Our results demonstratethat the systems learned by the model correspond to thecategory-selective areas previously identified in numeroushypothesis-driven studies.

Acknowledgements This research was supported in partby the NSF IIS/CRCNS 0904625 grant, the NSF CAREERgrant 0642971, the MIT McGovern Institute Neurotechnol-ogy Program, and the NIH NIBIB NAMIC U54-EB005149and NCRR NAC P41-RR13218 grants.

References[1] R. Baumgartner, G. Scarth, C. Teichtmeister, R. Somorjai, and E. Moser. Fuzzy

clustering of gradient-echo functional MRI in the human visual cortex. Part I:reproducibility. J Magn Reson Imaging, 7(6):1094–1108, 1997. 1

[2] C. Beckmann and S. Smith. Probabilistic independent component analysis forfunctional magnetic resonance imaging. TMI, 23(2):137–152, 2004. 1, 8

[3] C. Beckmann and S. Smith. Tensorial extensions of independent componentanalysis for multisubject FMRI analysis. NeuroImage, 25(1):294–311, 2005.2, 5

[4] R. Cox and A. Jesmanowicz. Real-time 3D image registration for functionalMRI. Magnetic Resonance in Medicine, 42(6):1014–1018, 1999. 5

[5] P. Downing, A.-Y. Chan, M. Peelen, C. Dodds, and N. Kanwisher. Domainspecificity in visual cortex. Cerebral Cortex, 16(10):1453–1461, 2006. 1

[6] S. Forman, J. Cohen, M. Fitzgerald, W. Eddy, M. Mintun, and D. Noll. Im-proved assessment of significant activation in functional magnetic resonanceimaging (fMRI): use of a cluster-size threshold. Magnetic Resonance inMedicine, 33(5):636–647, 1995. 2

[7] K. Friston, J. Ashburner, S. Kiebel, T. Nichols, and W. Penny, editors. Statisti-cal Parametric Mapping: the Analysis of Functional Brain Images. AcademicPress, Elsevier, 2007. 2, 5

[8] P. Golland, Y. Golland, and R. Malach. Detection of spatial activation patternsas unsupervised segmentation of fMRI data. In MICCAI, 2007. 1

[9] C. Goutte, P. Toft, E. Rostrup, F. Nielsen, and L. Hansen. On clustering fMRItime series. NeuroImage, 9(3):298–310, 1999. 1

[10] D. Greve and B. Fischl. Accurate and robust brain image alignment usingboundary-based registration. NeuroImage, 48(1):63–72, 2009. 5

[11] S. Jbabdi, M. Woolrich, and T. Behrens. Multiple-subjects connectivity-basedparcellation using hierarchical dirichlet process mixture models. NeuroImage,44(2):373–384, 2009. 2

[12] S. Kim and P. Smyth. Hierarchical Dirichlet processes with random effects. InNIPS, 2007. 2

[13] D. Lashkari and P. Golland. Exploratory fMRI analysis without spatial normal-ization. In IPMI, 2009. 1

[14] D. Lashkari, E. Vul, N. Kanwisher, and P. Golland. Discovering structure in thespace of fMRI selectivity profiles. NeuroImage, 50(3):1085–1098, 2010. 1, 2,5

[15] S. Makni, P. Ciuciu, J. Idier, and J.-B. Poline. Joint detection-estimation ofbrain activity in functional MRI: a multichannel deconvolution solution. TSP,53(9):3488–3502, 2005. 3

[16] M. McKeown, S. Makeig, G. Brown, T. Jung, S. Kindermann, A. Bell, andT. Sejnowski. Analysis of fMRI data by blind separation into independent spa-tial components. Hum Brain Mapp, 6(3):160–188, 1998. 1

[17] T. Minka. Automatic choice of dimensionality for PCA. NIPS, 2001. 6[18] J. Pitman. Poisson–Dirichlet and GEM invariant distributions for split-and-

merge transformations of an interval partition. Combinatorics, Prob, Comput,11(5):501–514, 2002. 4

[19] J. Talairach and P. Tournoux. Co-planar Stereotaxic Atlas of the Human Brain.Thieme, New York, 1988. 5

[20] Y. Teh, M. Jordan, M. Beal, and D. Blei. Hierarchical dirichlet processes. JASA,101(476):1566–1581, 2006. 2, 3, 4, 5

[21] Y. Teh, K. Kurihara, and M. Welling. Collapsed variational inference for HDP.In NIPS, 2008. 2, 4, 5

[22] Y. Teh, D. Newman, and M. Welling. A collapsed variational bayesian inferencealgorithm for latent dirichlet allocation. In NIPS, 2007. 4

[23] B. Thirion and O. Faugeras. Feature characterization in fMRI data: the In-formation Bottleneck approach. Medical Image Analysis, 8(4):403–419, 2004.1

[24] B. Thirion, G. Flandin, P. Pinel, A. Roche, P. Ciuciu, and J. Poline. Dealingwith the shortcomings of spatial normalization: Multi-subject parcellation offMRI datasets. Hum Brain Mapp, 27(8):678–693, 2006. 1

[25] B. Thirion, A. Tucholka, M. Keller, P. Pinel, A. Roche, J. Mangin, and J. Poline.High level group analysis of FMRI data based on Dirichlet process mixturemodels. In IPMI, 2007. 2