neuroimage analysis for automated brain disease...
TRANSCRIPT
![Page 1: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/1.jpg)
Neuroimage Analysis
for Automated Brain Disease Diagnosis
University of North Carolina at Chapel Hill
http://mingxia.web.unc.edu/
07-17-2019
Mingxia Liu
![Page 2: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/2.jpg)
Healthy Brain Alzheimer’s
Ventricle
Sulcus
Gyrus
Sulcus
Gyrus
Healthy Brain Alzheimer’s
Healthy Brain vs. Alzheimer’s
Background
![Page 3: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/3.jpg)
Background
• Alzheimer’s Disease (AD)
– A progressive disease
Normal
Control
Mild Cognitive
Impairment (MCI)Alzheimer’s
Disease
Bra
inH
ealth
Time
Stable MCI (sMCI)Progressive MCI (pMCI)
• Calling Need
– Developing computer-aided methods for MCI/AD diagnosis
![Page 4: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/4.jpg)
• Structural Magnetic Resonance Imaging (MRI)
• FDG-Positron Emission Tomography (PET)
• Cerebrospinal Fluid (CSF) ‐‐‐ Aβ42, t‐tau and p‐tau
Biomarkers for early diagnosis of AD and MCI
MRI PET CSF
CSF
Brain
Dura
Multi-modal data
Background
![Page 5: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/5.jpg)
M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease Diagnosis, Medical Image Analysis, 2018.
M. Liu, etc. Joint Classification and Regression via Deep Multi-task Multi-channel Learning for Alzheimer’s Disease Diagnosis. IEEE
Trans. on Biomedical Engineering, 2018.
fMRIPETsMRI
Neuroimaging
Data
Brain Disease Diagnosis – Typical Pipeline
Image
Preprocessing
Feature
Extraction/Selection
Classifier
Learning
Machine Learning & Deep Learning
![Page 6: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/6.jpg)
Challenges in Computer-aided Disease Diagnosis
• Effective feature representation of neuroimages
• Missing multi-modal data
• Heterogeneous data at different imaging sites
![Page 7: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/7.jpg)
Outline
• Missing Data
• Multi-modal Data Fusion
• Domain Adaptation
Multi-modal Neuroimage
CSFPETsMRI
Single-modal Neuroimage
• Structural MRI (sMRI)
sMRI
![Page 8: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/8.jpg)
Part I. Single-modal Neuroimage Analysis
• Structural MRI based Brain Disease Diagnosis
![Page 9: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/9.jpg)
Anatomical Landmarks for Structural MRI
• Landmark-based Deep Representation of sMRI
C. Lian, M. Liu, J. Zhang, and D. Shen. IEEE Trans. on Pattern Analysis and Machine Intelligence, 2019.
M. Liu, J. Zhang, C. Lian, and D. Shen. IEEE Trans. on Cybernetics, 2019.
M. Liu, J. Zhang, E. Adeli, and D. Shen. IEEE Trans. on Biomedical Engineering, 2019.
M. Liu, J. Zhang, E. Adeli, and D. Shen. Medical Image Analysis, 2018.
M. Liu, J. Zhang, D. Nie, P.T. Yap, and D. Shen. IEEE Journal of Biomedical and Health Informatics, 2018.
J. Zhang, M. Liu, and D. Shen. IEEE Trans. on Image Processing, 2017.
![Page 10: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/10.jpg)
...
Training Data
Patch-Level Feature Learning
Convolutional Neural
Network 1
Landmark Discovery
Disease
ClassificationTesting Data
Landmark-based Patch Extraction
Convolutional Neural
Network P
...
Image Retrieval
...Deep Feature
Representation 1
Deep Feature
Representation P
Patch-based
Features
MR Image Pre-processing
...
Landmark Detection
Landmark
Definition
...
Patch Extraction
...
Training Data
Patch-Level Feature Learning
Convolutional Neural
Network 1
Landmark Discovery
Disease
ClassificationTesting Data
Landmark-based Patch Extraction
Convolutional Neural
Network P
...
Image Retrieval
...Deep Feature
Representation 1
Deep Feature
Representation P
Patch-based
Features
MR Image Pre-processing
...
Landmark Detection
Landmark
Definition
...
...
Training Data
Patch-Level Feature Learning
Convolutional Neural
Network 1
Landmark Discovery
Disease
ClassificationTesting Data
Landmark-based Patch Extraction
Convolutional Neural
Network P
...
Image Retrieval
...Deep Feature
Representation 1
Deep Feature
Representation P
Patch-based
Features
MR Image Pre-processing
...
Landmark Detection
Landmark
Definition
...
Landmark Discovery
...
Training Data
Patch-Level Feature Learning
Convolutional Neural
Network 1
Landmark Discovery
Disease
ClassificationTesting Data
Landmark-based Patch Extraction
Convolutional Neural
Network P
...
Image Retrieval
...Deep Feature
Representation 1
Deep Feature
Representation P
Patch-based
Features
MR Image Pre-processing
...
Landmark Detection
Landmark
Definition
...
Test MRI
...
Training Data
Patch-Level Feature Learning
Convolutional Neural
Network 1
Landmark Discovery
Disease
ClassificationTesting Data
Landmark-based Patch Extraction
Convolutional Neural
Network P
...
Image Retrieval
...Deep Feature
Representation 1
Deep Feature
Representation P
Patch-based
Features
MR Image Pre-processing
...
Landmark Detection
Landmark
Definition
...
Landmark Detection
...
Training Data
Patch-Level Feature Learning
Convolutional Neural
Network 1
Landmark Discovery
Disease
ClassificationTesting Data
Landmark-based Patch Extraction
Convolutional Neural
Network P
...
Image Retrieval
...Deep Feature
Representation 1
Deep Feature
Representation P
Patch-based
Features
MR Image Pre-processing
...
Landmark Detection
Landmark
Definition
...
Training MRIs
Pre-processed
MR Images
…
…
CNN
Deep Multi-channel Convolutional
Neural Network (CNN)
DiseaseClassification
Clinical ScorePrediction
* Featured Article of IEEE Journal of Biomedical and Health Informatics, 2018
Anatomical Landmark-based Deep Network
![Page 11: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/11.jpg)
00.01 p-values
1,740 landmarks via group
comparison between AD and NCTop 50 landmarks
Anatomical Landmark-based Deep Network
M. Liu, J. Zhang, E. Adeli, and D. Shen. Medical Image Analysis, 2018.
M. Liu, J. Zhang, D. Nie, P.T. Yap, and D. Shen. IEEE Journal of Biomedical and Health Informatics, 2018.
![Page 12: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/12.jpg)
Concatenated
sMRI
FC8-32 FC8-32FC8-32 …
FC9-32L
Conv1-
32@3×3×3
Conv2-
32@3×3×3
Max-pooling
Conv3-
64@2×2×2
Conv4-
64@2×2×2
Max-pooling
Conv5-
128@2×2×2
Conv6-
128@2×2×2
Max-pooling
Conv1-
32@3×3×3
Conv2-
32@3×3×3
Max-pooling
Conv3-
64@2×2×2
Conv4-
64@2×2×2
Max-pooling
Conv5-
128@2×2×2
Conv6-
128@2×2×2
Max-pooling
FC7-128
…
FC7-128
Conv1-
32@3×3×3
Conv2-
32@3×3×3
Max-pooling
Conv3-
64@2×2×2
Conv4-
64@2×2×2
Max-pooling
Conv5-
128@2×2×2
Conv6-
128@2×2×2
Max-pooling
Patch 1 Patch 2 … Patch L
Landmark-based Deep Network
FC7-128
FC10-8L
FC11-2
Class Label
Soft-max
Global
Image-level
Representation
Local
Patch-level
Representation
Global
Representation?
M. Liu, J. Zhang, E. Adeli, and D. Shen. Medical Image Analysis, 2018.
M. Liu, J. Zhang, D. Nie, P.T. Yap, and D. Shen. IEEE Journal of Biomedical and Health Informatics, 2018.
![Page 13: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/13.jpg)
0.431
Ove
rall
Ac
cu
rac
y
0.404
0.46
7
0.486 0.487
0.518
0.325
Co
rre
lati
on
Co
eff
icie
nt
0.289
0.468
0.492
0.538
0.567
Classification Results for AD vs. sMCI vs. pMCI vs. NC Regression Results for MMSE
Results of Classification and Regression
M. Liu, J. Zhang, E. Adeli, and D. Shen. Medical Image Analysis, 2018.
M. Liu, J. Zhang, D. Nie, P.T. Yap, and D. Shen. IEEE Journal of Biomedical and Health Informatics, 2018.
![Page 14: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/14.jpg)
End-to-end Disease Diagnosis with sMRI
• Hierarchical Fully Convolutional network
C. Lian, M. Liu, J. Zhang, and D. Shen. IEEE Trans. on Pattern Analysis and Machine Intelligence, 2019.
M. Liu, J. Zhang, C. Lian, and D. Shen. IEEE Trans. on Cybernetics, 2019.
M. Liu, J. Zhang, E. Adeli, and D. Shen. IEEE Trans. on Biomedical Engineering, 2019.
M. Liu, J. Zhang, E. Adeli, and D. Shen. Medical Image Analysis, 2018.
M. Liu, J. Zhang, D. Nie, P.T. Yap, and D. Shen. IEEE Journal of Biomedical and Health Informatics, 2018.
J. Zhang, M. Liu, and D. Shen. IEEE Trans. on Image Processing, 2017.
![Page 15: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/15.jpg)
15
Hierarchical Network for ROI Identification
• Hierarchical Fully Convolutional Network (H-FCN)
– Automatically and identify disease-related ROIs in the whole sMR image
– Jointly learn multi-scale features and construct a classification model
C. Lian, M. Liu, J. Zhang, and D. Shen. IEEE Trans. on Pattern Analysis and Machine Intelligence, 2019.
C. Lian, M. Liu*, L. Wang, and D. Shen. MICCAI 2019.
![Page 16: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/16.jpg)
Hierarchical Network for ROI Identification
Input:
sMRI
C. Lian, M. Liu, J. Zhang, and D. Shen. IEEE Trans. on Pattern Analysis and Machine Intelligence, 2019.
C. Lian, M. Liu*, L. Wang, and D. Shen. MICCAI 2019.
![Page 17: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/17.jpg)
Input:
sMRI
1) Location
proposals
Hierarchical Network for ROI Identification
C. Lian, M. Liu, J. Zhang, and D. Shen. IEEE Trans. on Pattern Analysis and Machine Intelligence, 2019.
C. Lian, M. Liu*, L. Wang, and D. Shen. MICCAI 2019.
![Page 18: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/18.jpg)
1) Location
proposals
Input:
sMRI
2) Patch-level sub-
networks (PSN) (shared
weights)
Hierarchical Network for ROI Identification
PSN
PSN
Class_P
64 64 64128 12832
C. Lian, M. Liu, J. Zhang, and D. Shen. IEEE Trans. on Pattern Analysis and Machine Intelligence, 2019.
C. Lian, M. Liu*, L. Wang, and D. Shen. MICCAI 2019.
![Page 19: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/19.jpg)
PSN
1) Location
proposals
… …
PSN
PSN
PSN
PSN
PSN
PSN
PSN
Input:
sMRI
2) Patch-level sub-
networks (PSN) (shared
weights)
Hierarchical Network for ROI Identification
PSN
Class_P
64 64 64128 12832
C. Lian, M. Liu, J. Zhang, and D. Shen. IEEE Trans. on Pattern Analysis and Machine Intelligence, 2019.
C. Lian, M. Liu*, L. Wang, and D. Shen. MICCAI 2019.
![Page 20: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/20.jpg)
…
64
Conv_R Class_R
…
PSN
PSN
PSN
PSN
PSN
PSN
PSN
PSN
2) Patch-level sub-
networks (PSN) (shared
weights)
3) Region-level
sub-networks
1) Location
proposals
Input:
sMRI
Hierarchical Network for ROI Identification
PSN
Class_P
64 64 64128 12832
C. Lian, M. Liu, J. Zhang, and D. Shen. IEEE Trans. on Pattern Analysis and Machine Intelligence, 2019.
C. Lian, M. Liu*, L. Wang, and D. Shen. MICCAI 2019.
![Page 21: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/21.jpg)
…
64
Conv_R Class_R
…
64
Conv_R Class_R
…
PSN
PSN
PSN
PSN
PSN
PSN
PSN
PSN
2) Patch-level sub-
networks (PSN) (shared
weights)
3) Region-level
sub-networks
1) Location
proposals
Input:
sMRI
Hierarchical Network for ROI Identification
PSN
Class_P
64 64 64128 12832
C. Lian, M. Liu, J. Zhang, and D. Shen. IEEE Trans. on Pattern Analysis and Machine Intelligence, 2019.
C. Lian, M. Liu*, L. Wang, and D. Shen. MICCAI 2019.
![Page 22: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/22.jpg)
…
64
Conv_S
Class_S
64
Conv_R Class_R
64
Conv_R Class_R
… …
Output:
Class label
PSN
PSN
PSN
PSN
PSN
PSN
PSN
PSN
3) Region-level
sub-networks
4) Subject-level
sub-network
1) Location
proposals
Input:
sMRI
2) Patch-level sub-
networks (PSN) (shared
weights)
Classification (1 × 1 × 1 Conv)
4 × 4 × 4 Conv
1 × 1 × 1 Conv
Channel concatenation
Region-level Conv
2 × 2 × 2 max pooling
3 × 3 × 3 Conv
Spatial concatenation
Subject-level Conv
Potentially pruned sub-networks
Skipped connection Conv: Convolution
PSN
Class_P
64 64 64128 12832
Hierarchical Network for ROI Identification
C. Lian, M. Liu, J. Zhang, and D. Shen. IEEE Trans. on Pattern Analysis and Machine Intelligence, 2019.
C. Lian, M. Liu*, L. Wang, and D. Shen. MICCAI 2019.
Network Pruning: To remove less informative regions
![Page 23: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/23.jpg)
Identified Voxel-level Discriminative Locations
1
2
3
1 2 3
1 2 3
1
2
3 4
3 4
3 4
1 2
1 2
1 2 3
1 2 3
1 2 3
1 2 3
1
2 3
1 2
1 2
12
12
3
1 2 3
1 2 3
12
3
(a) AD Subject #1 (b) AD Subject #2 (c) AD Subject #3
(d) AD Subject #4 (e) AD Subject #5 (f) AD Subject #6
C. Lian, M. Liu, J. Zhang, and D. Shen. IEEE Trans. on Pattern Analysis and Machine Intelligence, 2019.
C. Lian, M. Liu*, L. Wang, and D. Shen. MICCAI 2019.
![Page 24: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/24.jpg)
Identified Patch-level Discriminative Locations
Sagittal View Axial View Coronal View 3D View
C. Lian, M. Liu, J. Zhang, and D. Shen. IEEE Trans. on Pattern Analysis and Machine Intelligence, 2019.
C. Lian, M. Liu*, L. Wang, and D. Shen. MICCAI 2019.
![Page 25: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/25.jpg)
Identified Region-level Discriminative Locations
Sagittal View Axial View Coronal View 3D View
C. Lian, M. Liu, J. Zhang, and D. Shen. IEEE Trans. on Pattern Analysis and Machine Intelligence, 2019.
C. Lian, M. Liu*, L. Wang, and D. Shen. MICCAI 2019.
![Page 26: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/26.jpg)
0.5
0.6
0.7
0.8
0.9
1
Patch Region Subject
AC
C
0.5
0.6
0.7
0.8
0.9
1
Patch Region Subject
AU
C
H-FCN before network pruning H-FCN after network pruning
Results of AD vs. NC classification obtained by patch-, region-, and subject-level sub-
networks in H-FCN without /with network pruning
Classification Results
C. Lian, M. Liu, J. Zhang, and D. Shen. IEEE Trans. on Pattern Analysis and Machine Intelligence, 2019.
C. Lian, M. Liu*, L. Wang, and D. Shen. MICCAI 2019.
1) Global subject-level representation is more useful
2) Network pruning promotes the classification performance
![Page 27: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/27.jpg)
Outline
• Missing Data
• Multi-modal Data Fusion
• Domain Adaptation
Multi-modal Neuroimage
CSFPETsMRI
Single-modal Neuroimage
• Structural MRI (sMRI)
sMRI
![Page 28: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/28.jpg)
Part II. Multi-modal Neuroimage Analysis
• Multi-modality Fusion for Disease Diagnosis
• Imaging Synthesis for Missing Modalities
• Domain Adaptation for Multi-site Data
![Page 29: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/29.jpg)
M. Liu, J. Zhang, P.-T. Yap, and D. Shen. MICCAI, 2016.
M. Liu, J. Zhang, P.-T. Yap, and D. Shen. Medical Image Analysis, 2017.
M. Liu, Y. Gao, P.-T. Yap, and D. Shen. IEEE Journal of Biomedical and Health Informatics, 2018.
Multi-modality Fusion for Disease Diagnosis
• Hypergraph Learning with Missing Data
![Page 30: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/30.jpg)
30
Multi-View Data Grouping
• Given 3 modalities (i.e., MRI, PET and CSF), 6 views are constructed
according to the availability of different modalities
Multi-Modality Data
CSFPET MRI
Multi-ViewData Grouping
View 1
View 2
View 3
View 4
View 5
View 6
PET
CSF
MRI
Missing Data
* MICCAI Young Scientist Award Nomination, 2016
* MICCAI Travel Award, 2016
![Page 31: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/31.jpg)
31
Multi-View Data Grouping
• Given 3 modalities (i.e., MRI, PET and CSF), 6 views are constructed
according to the availability of different modalities
Multi-Modality Data
CSFPET MRI
Multi-ViewData Grouping
View 1
View 2
View 3
View 4
View 5
View 6
PET
CSF
MRI
Missing Data
* MICCAI Young Scientist Award Nomination, 2016
* MICCAI Travel Award, 2016
![Page 32: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/32.jpg)
• Construct multiple hypergraphs, with each hypergraph corresponding to
a specific view
Multi-Modality Data
CSFPET MRI
Multi-ViewData Grouping
View 1
View 2
View 3
View 4
View 5
View 6
Sparse
Representation
Sparse
Representation
Sparse Representation based Hypergraph Construction
Sparse
Representation
Sparse
Representation
Sparse
Representation
Sparse
Representation
PET
CSF
MRI
Missing Data
M. Liu, J. Zhang, P.-T. Yap, and D. Shen. MICCAI, 2016.
M. Liu, J. Zhang, P.-T. Yap, and D. Shen. Medical Image Analysis, 2017.
M. Liu, Y. Gao, P.-T. Yap, and D. Shen. IEEE Journal of Biomedical and Health Informatics, 2018.
Hypergraph Construction
![Page 33: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/33.jpg)
• A view-aligned hypergraph classification model to capture the coherence
among different views
View-Aligned
Hypergraph Classification
View-Aligned Hypergraph Classification
Multi-Modality Data
CSFPET MRI
Multi-ViewData Grouping
View 1
View 2
View 3
View 4
View 5
View 6
Sparse
Representation
Sparse
Representation
Sparse Representation based Hypergraph Construction
Sparse
Representation
Sparse
Representation
Sparse
Representation
Sparse
Representation
PET
CSF
MRI
Missing Data
M. Liu, J. Zhang, P.-T. Yap, and D. Shen. MICCAI, 2016.
M. Liu, J. Zhang, P.-T. Yap, and D. Shen. Medical Image Analysis, 2017.
M. Liu, Y. Gao, P.-T. Yap, and D. Shen. IEEE Journal of Biomedical and Health Informatics, 2018.
![Page 34: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/34.jpg)
• Multi-view label fusion strategy (weighted voting)
Multi-View
Label Fusion
Multi-Modality Data
CSFPET MRI
Multi-ViewData Grouping
View 1
View 2
View 3
View 4
View 5
View 6
Sparse
Representation
Sparse
Representation
Sparse Representation based Hypergraph Construction
Sparse
Representation
Sparse
Representation
Sparse
Representation
Sparse
Representation
View-Aligned
Hypergraph Classification
PET
CSF
MRI
Missing Data
M. Liu, J. Zhang, P.-T. Yap, and D. Shen. MICCAI, 2016.
M. Liu, J. Zhang, P.-T. Yap, and D. Shen. Medical Image Analysis, 2017.
M. Liu, Y. Gao, P.-T. Yap, and D. Shen. IEEE Journal of Biomedical and Health Informatics, 2018.
Multi-view Label Fusion
![Page 35: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/35.jpg)
Label Space
𝐱𝑛1MRI
𝐱𝑛1PET 𝐱𝑛1
CSF
𝑓𝑛1MRI
𝑓𝑛1PET
𝑓𝑛1CSF
𝑓𝑛2MRI
𝑓𝑛2PET
𝐱𝑛2MRI𝐱𝑛2
PET
View-Aligned Constraint
Coherence among views
View-Aligned Regularizer
M. Liu, J. Zhang, P.-T. Yap, and D. Shen. MICCAI, 2016.
M. Liu, J. Zhang, P.-T. Yap, and D. Shen. Medical Image Analysis, 2017.
M. Liu, Y. Gao, P.-T. Yap, and D. Shen. IEEE Journal of Biomedical and Health Informatics, 2018.
Missing CSF
MRI
PET
Subject #2
CSF
MRI
PET
Subject #1
![Page 36: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/36.jpg)
36
𝑚𝑖𝑛𝐅, 𝛂, {𝐖𝑚}𝑚=1
𝑀 𝑚=1𝑀 𝛀𝑚 (𝐟𝑚 − 𝐲) 2
2
+𝜆 𝑚=1𝑀 𝐖𝑚
𝐹2
+𝜇 𝑛=1𝑁 𝑚=1
𝑀 𝑝=1𝑀 𝛺𝑛,𝑛
𝑚 𝛺𝑛,𝑛𝑝
𝑓𝑛𝑚 − 𝑓𝑛
𝑝 2
+ 𝑚=1𝑀 𝛼𝑚 2 𝐟𝑚 T 𝐋𝑚𝐟𝑚
𝑠. 𝑡. 𝑚=1𝑀 𝛼𝑚 = 1, ∀𝛼𝑚 ≥ 0;
𝑖=1𝑁𝑒𝑚
𝑊𝑖,𝑖𝑚 = 1, ∀𝑊𝑖,𝑖
𝑚 ≥ 0.
Step 3: View-Aligned Hypergraph Classification
Formulation of view-aligned hypergraph classification (VAHC)
View-aligned Regularizer
Hypergraph Laplacian matrix , and .
M. Liu, J. Zhang, P.-T. Yap, and D. Shen. MICCAI, 2016.
M. Liu, J. Zhang, P.-T. Yap, and D. Shen. Medical Image Analysis, 2017.
M. Liu, Y. Gao, P.-T. Yap, and D. Shen. IEEE Journal of Biomedical and Health Informatics, 2018.
![Page 37: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/37.jpg)
Learned Weights for Views
Learned weights for views in four classification tasks
AD vs. NC MCI vs. NC pMCI vs. sMCI pMCI vs. NC
0.0
0.1
0.2
0.3
0.4
0.5
0.6
We
igh
ts
MRI PET CSF PET+MRI
MRI+CSF PET+MRI+CSF
M. Liu, J. Zhang, P.-T. Yap, and D. Shen. MICCAI, 2016.
M. Liu, J. Zhang, P.-T. Yap, and D. Shen. Medical Image Analysis, 2017.
M. Liu, Y. Gao, P.-T. Yap, and D. Shen. IEEE Journal of Biomedical and Health Informatics, 2018.
Using all 3 modalities (MRI+PET+CSF) achieves the best results
![Page 38: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/38.jpg)
Experimental Results
pMCI vs. sMCI classification
ACC SEN SPE BAC PPV NPV AUC
50
60
70
80
90
100
Re
su
lts (
%)
(c) pMCI vs. sMCI classification
Zero KNN EM SVD VAHL
M. Liu, J. Zhang, P.-T. Yap, and D. Shen. MICCAI, 2016.
M. Liu, J. Zhang, P.-T. Yap, and D. Shen. Medical Image Analysis, 2017.
M. Liu, Y. Gao, P.-T. Yap, and D. Shen. IEEE Journal of Biomedical and Health Informatics, 2018.
![Page 39: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/39.jpg)
Imaging Synthesis for Missing Modalities
• Deep Learning based Automatic PET Synthesis from sMRI
Y. Pan, M. Liu*, C. Lian, Y. Xia, and D. Shen. MICCAI, 2019.
Y. Pan, M. Liu, C. Lian, T. Zhou, Y. Xia, and D. Shen. MICCAI, 2018.
Y. Pan, M. Liu*, C. Lian, L. Yue, S. Xiao, Y. Xia and D. Shen. ISMRM, 2019
![Page 40: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/40.jpg)
Pre-processed
MR and PET
Images
Stage 2:
Brain Disease Classification
Stage 1:
PET Image Synthesis
based on sMRI
Synthesizing Missing PET based on MRI for Brain Disease Diagnosis
Hybrid Cycle-GAN for Missing PET Synthesis
Y. Pan, M. Liu, C. Lian, T. Zhou, Y. Xia, and D. Shen. MICCAI, 2018
Y. Pan, M. Liu*, C. Lian, L. Yue, S. Xiao, Y. Xia and D. Shen. ISMRM, 2019
• Among 800+ subjects in ADNI, all subjects have sMRI, while
only half of them have FDG-PET scans.
MRI PET
![Page 41: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/41.jpg)
Hybrid cycle-consistent generative adversarial network (HGAN) to impute missing PET scans based on sMRI
Synthetic PET
Synthetic MRI3232
Residual Net
Block (RNB)
Real PET
1 128 64 32 16
𝑫𝑷
112864
3216
𝑫𝑴
1 16 32 32 16 8
𝑮𝑴
⋯
RN
B
RN
B
8 16 32
𝑮𝑷
⋯
RN
B
RN
B
32 16 1
3×3×3 Convolution
7×7×7 Convolution
3×3×3 Deconvolution
4×4×4 Convolution
Addition
Real MRI
6
6
1/01/0
𝐗𝑀
𝐗𝑃
𝐺1 𝐗𝑀
𝐺2 𝐗𝑃
Hybrid Cycle-GAN for Missing PET Synthesis
Y. Pan, M. Liu, C. Lian, T. Zhou, Y. Xia, and D. Shen. MICCAI, 2018
Y. Pan, M. Liu*, C. Lian, L. Yue, S. Xiao, Y. Xia and D. Shen. ISMRM, 2019
![Page 42: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/42.jpg)
Synthetic PET ImagesP
ET
(RID
:5016)
Y. Pan, M. Liu, C. Lian, T. Zhou, Y. Xia, and D. Shen. MICCAI, 2018
Y. Pan, M. Liu*, C. Lian, L. Yue, S. Xiao, Y. Xia and D. Shen. ISMRM, 2019
(a) GAN (b) CGAN (c) VGAN (d) HGAN (Ours) (e) Ground Truth
PE
T(R
ID:
4352)
![Page 43: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/43.jpg)
Synthetic MRI ScansM
RI
(RID
:5016)
Y. Pan, M. Liu, C. Lian, T. Zhou, Y. Xia, and D. Shen. MICCAI, 2018
Y. Pan, M. Liu*, C. Lian, L. Yue, S. Xiao, Y. Xia and D. Shen. ISMRM, 2019
(a) GAN (b) CGAN (c) VGAN (d) HGAN (Ours) (e) Ground Truth
MR
I(R
ID:
4352)
![Page 44: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/44.jpg)
PET
(Real + Synthetic)
MRI (Real)
24
2424
24
24
24
24
2424
24
2424
…
3232 32 6464 64
Sub-network 1
DCM DCM DCM
1616 16
Sub-network 2
3232 32 6464 64
DCM DCM DCM
1616 16
Concatenation
32
8K …
Clinical scores at
four time-points
8
32
8
128
32
BL
M06
M24
M12
Fully-connectedDown-sampling Copy 3×3×3 Convolution 2×2×2 Max-poolingChannel concatenation
Hybrid Cycle-GAN for Missing PET Synthesis
Landmark-based Deep Network for Brain Disease Classification using MRI and PET (Real+Synthetic)
Y. Pan, M. Liu, C. Lian, T. Zhou, Y. Xia, and D. Shen. MICCAI, 2018
Y. Pan, M. Liu*, C. Lian, L. Yue, S. Xiao, Y. Xia and D. Shen. ISMRM, 2019
PET
(Real + Synthetic)
![Page 45: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/45.jpg)
MCI conversion prediction with complete MRI
and complete (after imputation) PET
Hybrid Cycle-GAN for Missing PET Synthesis
Y. Pan, M. Liu, C. Lian, T. Zhou, Y. Xia, and D. Shen. MICCAI, 2018
Y. Pan, M. Liu*, C. Lian, L. Yue, S. Xiao, Y. Xia and D. Shen. ISMRM, 2019
![Page 46: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/46.jpg)
How to generating classification-oriented
PET/MRI scans for diagnosis?
Feature-consistent GAN for Joint PET Synthesis and Classification
1128643216
𝐷
3×3×3 Conv
7×7×7 Conv
3×3×3 Deconv
4×4×4 Conv
Addition
Difference
32 32
RNB
1163232168
𝐺
RN
B
RN
B ⋯
6
𝐺𝑀
𝐺𝑃
𝐷𝑃
𝐷𝑀 𝔏𝑔
𝔏𝑔
𝐹𝑃
𝐹𝑀 𝔏c
𝔏c
FG
AN
Synthetic
MRI
Real
PET
Synthetic
PET
Real
MRI
Synthetic
PET
Real
PET
1 2
K
1
2
K
⋮1/0
𝑙2
64643216 64
64643216
𝔏c
64
Feature-consistent
Component 𝐹𝑃
Cosin
e K
ern
el
Y. Pan, M. Liu*, C. Lian, Y. Xia, and D. Shen. MICCAI, 2019
Feature-consistent
Component 𝐹𝑃
![Page 47: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/47.jpg)
30
50
70
90
AUC ACC SEN SEP
ROI LMF LDMIL DSNN (Ours)
MCI conversion prediction with complete MRI
and complete (after imputation) PET
Feature-consistent GAN for Joint PET Synthesis and Classification
Y. Pan, M. Liu*, C. Lian, Y. Xia, and D. Shen. MICCAI, 2019
Generating task-oriented PET scans helps boost performance
![Page 48: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/48.jpg)
Domain Adaptation for Multi-site Data
• Low-rank Representation for Multi-site Data Adaptation
M. Wang, D. Zhang, J. Huang, D. Shen, and M. Liu*, and D. Shen. MICCAI, 2018
![Page 49: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/49.jpg)
Low-rank Representation for Domain Adaptation
ABIDE: 17 imaging sites with resting-state fMRI data
Scanners
ScanningParameters
PopulationNoise Level
ImageContrast
M. Wang, D. Zhang, J. Huang, D. Shen, and M. Liu*, and D. Shen. MICCAI, 2018
*Best Poster Award, MICS, 2019
![Page 50: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/50.jpg)
50
…
P2
P1
PS
P
New Representation
Latent Representation Space
Linear
Representation
Site T
Site S
Site 1
Site 2
Low-rank Representation for Domain Adaptation
M. Wang, D. Zhang, J. Huang, D. Shen, and M. Liu*, and D. Shen. MICCAI, 2018
Mapping to a
common latent spaceRepresenting source
data using target data
Source Domains
Target Domain
*Best Poster Award, MICS, 2019
![Page 51: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/51.jpg)
51
Low-rank Representation for Domain Adaptation
Formulation of multi-center low-rank representation (maLRR)
min𝐉,𝐏,𝐏𝑖,𝐙𝑖,𝐄𝑆𝑖 ,𝐄𝑃𝑖 ,𝐅𝑖𝐉 ∗ + 𝑖=1
𝐾 𝐅𝑖 ∗ + 𝛼 𝐄𝑆𝑖 1+ 𝛽 𝐄𝑃𝑖 1
s. t. 𝐏𝑖𝐗𝑆𝑖 = 𝐏𝐗𝑇𝐙𝑖 + 𝐄𝑆𝑖 ,
𝐏𝑖= 𝐏 + 𝐄𝑃𝑖 , 𝑖 = 1,… , 𝐾
𝐏 = 𝐉, 𝐙𝑖 = 𝐅𝑖 , 𝐏𝐏𝑇 = 𝐈1) Mapping to a latent space
2) Representing source data
using target data
M. Wang, D. Zhang, J. Huang, D. Shen, and M. Liu*, and D. Shen. MICCAI, 2018
*Best Poster Award, MICS, 2019
![Page 52: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/52.jpg)
52
Low-rank Representation for Domain Adaptation
Performance of different methods in the task of Autism vs. NC classification, with
NYU as the target domain and the other four sites as the source domains
ABIDE: 5 imaging sites with resting-state fMRI data
M. Wang, D. Zhang, J. Huang, D. Shen, and M. Liu*, and D. Shen. MICCAI, 2018
*Best Poster Award, MICS, 2019
![Page 53: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/53.jpg)
Outline
• Missing Data
• Multi-modal Data Fusion
• Domain Adaptation
Multi-modal Neuroimage
CSFPETsMRI
Single-modal Neuroimage
• Structural MRI (sMRI)
sMRI
![Page 54: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/54.jpg)
Acknowledge
Dr. Dinggang Shen
Dr. Daoqiang Zhang
Dr. Pew-Thian Yap
Dr. Jun Zhang
Dr. Chunfeng Lian
Dr. Ling Yue
Dr. Jing Zhang
Dr. Aimei Dong
Dr. Bo Wang
Collaborators
Dr. Ehsan Adeli
Dr. Yue Gao
Dr. Biao Jie
Dr. Tao Zhou
Dr. Yong Xia
Visiting Scholars and Students
Mr. Mingliang Wang
Mr. Yongsheng Pan
Mr. Dongren Yao
Mr. Jiashuang Huang
![Page 55: Neuroimage Analysis for Automated Brain Disease Diagnosisvalser.org/webinar/slide/slides/20190717/... · M. Liu, etc., Landmark-based Deep Multi-Instance Learning for Brain Disease](https://reader033.vdocuments.net/reader033/viewer/2022060214/5f0569617e708231d412d51b/html5/thumbnails/55.jpg)
Thanks for Your Attention!
http://mingxia.web.unc.edu/
Mingxia Liu