CONVOLUTIONAL NEURAL NETWORKS FOR MESH-BASED PARCELLATION OF THE CEREBRAL CORTEX

Guillem Cucurull, Konrad Wagstyl, Arantxa Casanova, Petar Veličković, Estrid Jakobsen, Michal Drozdzal, Adriana Romero, Alan Evans, Yoshua Bengio

�1

CONVOLUTIONAL NEURAL NETWORKS FOR MESH-BASED PARCELLATION OF THE CEREBRAL CORTEX

CORTEX PARCELLATION

▸ Different areas of the cerebral cortex are involved in different cognitive processes:

• Visual processing, language comprehension

▸ Mapping these areas helps us understand how the cortex is organized

Insert Cortex Parcellation Example Image

�2

Cortex Parcellation. From Glasser et al. 2016

CONVOLUTIONAL NEURAL NETWORKS FOR MESH-BASED PARCELLATION OF THE CEREBRAL CORTEX

WHAT IS A CORTICAL MESH

▸ Common coordinate system

▸ Can represent multiple imaging modalities and features

▸ Can be used to coregister cortical surfaces between different individuals

Insert Cortical Mesh Example Image

�3

Zoom of a cortical mesh

CONVOLUTIONAL NEURAL NETWORKS FOR MESH-BASED PARCELLATION OF THE CEREBRAL CORTEX

PREVIOUS WORK

▸ Glasser et al. 2016

Used an MLP classifier to recognize each cortical area, trained on semi-automatic labels of cortex parcellation.

▸ Jakobsen et al. 2016

Compared connectivity templates of each vertex of the cortical surface to parcellate Broca’s region.

�4

CONVOLUTIONAL NEURAL NETWORKS FOR MESH-BASED PARCELLATION OF THE CEREBRAL CORTEX

WHY CONVOLUTIONS

▸ Convolutional models achieve state of the art on image segmentation, which is analogous to this task.

▸ Important properties:

▸ Parameter sharing

▸ Exploits data structure

�5

Image from http://petar-v.com/GAT/

CONVOLUTIONAL NEURAL NETWORKS FOR MESH-BASED PARCELLATION OF THE CEREBRAL CORTEX

GRAPH CONVOLUTIONS

▸ Graph Convolutional Networks [Kipf 2017], ChebNets [Defferrard 2016]:

▸ Graph Attention Networks [Veličković 2018]:

�6

h′�i = σ( ∑j∈𝒩i

1cij

Whj)

h′�i = σ( ∑j∈𝒩i

αijWhj)

From the normalized

adjacency matrix

Attention coefficient

CONVOLUTIONAL NEURAL NETWORKS FOR MESH-BASED PARCELLATION OF THE CEREBRAL CORTEX

BASELINES▸ NodeMLP

• Independently classify each node

▸ NodeAVG

• Predict the most frequent label at each node, computed in the training set

▸ MeshMLP

• Process the full mesh at once, concatenating all nodes together

�7

CONVOLUTIONAL NEURAL NETWORKS FOR MESH-BASED PARCELLATION OF THE CEREBRAL CORTEX

DIFFERENCES BETWEEN MODELS

�8

Method

Properties NodeAVG NodeMLP MeshMLP GraphConv

Neigh Info

Global Info

Features

CONVOLUTIONAL NEURAL NETWORKS FOR MESH-BASED PARCELLATION OF THE CEREBRAL CORTEX

BROCA’S REGION PARCELLATION

▸ Broca’s region has an important role in language processing

It can be subdivided into two adjacent cytoarchitectonic areas: 44 and 45.

�9

▸ Dataset

▸ Manually labeled parcellation of Broca’s area from 100 different subjects.

▸ Each mesh has 1195 nodes, with 9 features per node, and 3 possible labels.

▸ Augment the features with x,y,z absolute position of the vertex

From Geranmayeh, 2014

CONVOLUTIONAL NEURAL NETWORKS FOR MESH-BASED PARCELLATION OF THE CEREBRAL CORTEX

EXPERIMENTAL SETUP

▸ Given the limited size of the dataset we use 10-fold cross validation to report results:

- 8 folds for training, 1 for validation, 1 for test

- The models are trained using Adam optimizer

- The optimized loss function is either weighted cross entropy or Dice loss

�10

CONVOLUTIONAL NEURAL NETWORKS FOR MESH-BASED PARCELLATION OF THE CEREBRAL CORTEX

RESULTS

�11

CONVOLUTIONAL NEURAL NETWORKS FOR MESH-BASED PARCELLATION OF THE CEREBRAL CORTEX

RESULTS

�12

CONVOLUTIONAL NEURAL NETWORKS FOR MESH-BASED PARCELLATION OF THE CEREBRAL CORTEX

RESULTS

�13

CONVOLUTIONAL NEURAL NETWORKS FOR MESH-BASED PARCELLATION OF THE CEREBRAL CORTEX

RESULTS

�14

CONVOLUTIONAL NEURAL NETWORKS FOR MESH-BASED PARCELLATION OF THE CEREBRAL CORTEX

RESULTS

�15

Using node

position

as features

CONVOLUTIONAL NEURAL NETWORKS FOR MESH-BASED PARCELLATION OF THE CEREBRAL CORTEX

RESULTS

�16

Subject 1

Subject 2

Subject 1

Subject 2

CONVOLUTIONAL NEURAL NETWORKS FOR MESH-BASED PARCELLATION OF THE CEREBRAL CORTEX

SUMMARY

▸ Investigated the use of graph CNNs for cortical mesh parcellation

▸ Evaluated on Broca’s area segmentation

▸ Graph CNNs models outperform the baselines

�17

CONVOLUTIONAL NEURAL NETWORKS FOR MESH-BASED PARCELLATION OF THE CEREBRAL CORTEX

REFERENCES‣ Glasser, Matthew F., Timothy S. Coalson, Emma C. Robinson, Carl D. Hacker, John Harwell, Essa Yacoub, Kamil Ugurbil et

al. "A multi-modal parcellation of human cerebral cortex." Nature 536, no. 7615 (2016): 171-178. ‣ Jakobsen, Estrid, Franziskus Liem, Manousos A. Klados, Şeyma Bayrak, Michael Petrides, and Daniel S. Margulies.

"Automated individual-level parcellation of Broca's region based on functional connectivity." Neuroimage (2016). ‣ Defferrard, Michaël, Xavier Bresson, and Pierre Vandergheynst. "Convolutional neural networks on graphs with fast

localized spectral filtering." In Advances in Neural Information Processing Systems, pp. 3844-3852. 2016. ‣ Kipf, Thomas N., and Max Welling. "Semi-supervised classification with graph convolutional networks." International

Conference on Learning Representations, ICLR, (2017). ‣ Veličković, Petar, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. "Graph Attention

Networks.” International Conference on Learning Representations, ICLR, (2018). ‣ Geranmayeh, Fatemeh, Sonia LE Brownsett, and Richard JS Wise. "Task-induced brain activity in aphasic stroke patients:

what is driving recovery?." Brain 137, no. 10 (2014): 2632-2648.

�18

Guillem Cucurull, Konrad Wagstyl, Arantxa Casanova, Petar Veličković, Estrid Jakobsen, Michal Drozdzal, Adriana Romero, Alan Evans, Yoshua Bengio

�19

THANKS!CONVOLUTIONAL NEURAL NETWORKS FOR MESH-BASED PARCELLATION OF THE CEREBRAL CORTEXhttps://openreview.net/forum?id=rkKvBAiiz