Large Analysis of microvascular lesions in the brain and retina using MRI and
colourfundus imaging for early detection of
CVD
Prof. Ramamohanarao Kotagiri
Department of Computing and Information Systems
The University of Melbourne, Australia 3010
Email: [email protected]
1
Presentation Outline Motivation Brief Background
(Magnetic Resonance Imaging) MRI Retinal Imaging Brain microvascular lesions
White matter lesions (WMLs) Brain infarcts
Retinal microvascular lesions Arteriovenus nicking (AV nicking) Focal arteriolar narrowing (FAN)
Objective Proposed Method
Segmentation and quantification of WMLs Detection and quantification of AV nicking Detection and quantification of FAN
Summary
2
Vessel Calibre: A New Biomarker
• Only place blood vessels can be viewed & monitored non-invasively
• Damage to the retinal circulation may reflect impact of both recognised & unrecognised risk factors, and susceptibility
• Indicator of structural vascular damage
Retinal Circulation = ‘Window’ to Brain Circulation
Human Retina
Retinal Imaging
• A retinal camera is used to capture image• A picture is taken showing the optic nerve (i.e., disc), fovea,
surrounding vessels, and the retinal layer
A retinal camera (left) retinal fundus image (right)
Significance of Retinal Imaging
Retinal vascular signs may predict Cardiovascular Diseases and Diabetes!
• CVD (Cardiovascular Disease) (heart disease & stroke) is the most common cause of death in the developed world
• 36% all deaths in Australia (2004)• Kills one Australian every 10
minutes.
Condition Australia of about 23 Million Population
Diabetes 1.7M(0.9M diagnosed)
Pre-diabetes 1.5M
Hypertension 3.7M
Coronary Heart Disease (CHD)
26,000 Deaths/Year400,000 Deaths in
the USA in 2010
Stroke 12,500 Deaths/Year
All CVD 3.7M Cases50,000 Deaths/Year
Diagnostic Tools
• Currently available prediction tools based on assessment of traditional risk factors (e.g., blood pressure, cholesterol, smoking history, MRI after the symptoms of a stroke)
• Account for only 50% of CVD cases
Need for improved diagnostic tools
New Diagnostic Tools
Traditional CVD Risk Factors
e.g. blood pressure
Unknown CVD Risk Factors e.g. genetic
factors
Sub clinical vascular damage
CARDIOVASCULAR DISEASE
AND DEATH
Retinal Vascular Signs
Retinal Vascular Sign
Observations Associations
Retinal vascular calibre
Arteriolar narrowing
2.6 fold increased risk of incident hypertension2 fold increased risk of incident CHD
Venular dilatation
2 fold increased risk of incident stroke3 fold increased risk of incident CHD
Retinal Signs Predict CVD
COMPUTER-BASED RETINAL IMAGING PROGRAM FOR
IDENTIFICATION OF CARDIOVASCULAR DISEASE RISK
Patient’s retina photographed
Photos transmitted to reading centre
Retinal grading
Retinal vascular scan report generated
Retinal Blood VesselAnalysisBlood Vessels• Artery (Red)• Vein (Blue)
Arterial Vane (AV) Nicking
AV nicking
Focal Arterial Narrowing (FAN)
Cheap Way to image Retina Using smartphone –lens adaptor
cost$5.00
Magnetic Resonance Imaging (MRI)
MRI is used for non invasively visualizing internal structures of the body .
Does not have any side effects of radiation .
Property of nuclear magnetic resonance (NMR) is used to image nuclei of atoms inside the body.
Provides higher details about soft-tissues
15
Magnetic Resonance Imaging (MRI)
Different modalities are used to characterize and discriminate among tissues
T1 weighted MRI is effective for visualizing various anatomical structures such as White matter, Gray matter and CSF very useful for identifying the location of tissues
T2 weighted MRI is effective for visualizing pathologies such as lesions and tumors
Fluid attenuated inversion recovery (FLAIR) is effective in suppressing CSF (Cerebral Spinal Fluid) and enhancing lesions.
16
T1 MRI
White Matter
WMLs
CSF(Cerebral Spinal Fluid)
Gray Matter
Anatomical Structures are clearly identifiable.
17
T2 MRI
WMLs
Pathology (WMLs) looks more clearer.
18
Flair MRI
WMLs
Intensity of CSF is suppressed
19
Relationship between WMLs and Risk of Stroke1
[1] Vermeer, Sarah E., et al. "Silent brain infarcts and white matter lesions increase stroke risk in the general population The Rotterdam Scan Study." Stroke 34.5 (2003): 1126-1129.
Motivation
20
Correlation between severity of Sub-Cortical WMLs and AV nicking2
Sub-cortical WMLs Load• Q# is the severity scale of sub-cortical WMLs. • # Reference group; * p<0.05; § p<0.01.[2] Qiu, Chengxuan, et al. "Retinal and cerebral microvascular signs and diabetes the age, gene/environment susceptibility-reykjavi
study." Diabetes 57.6 (2008): 1645-1650.
Motivation
21
Correlation between severity of Periventricular WMLs and AV nicking2
Periventricular WMLs Load • T# is the severity scale of peri-ventricular WMLs. • # Reference group; * p<0.05; § p<0.01.[2] Qiu, Chengxuan, et al. "Retinal and cerebral microvascular signs and diabetes the age, gene/environment susceptibility-reykjavi
study." Diabetes 57.6 (2008): 1645-1650.
Motivation
22
Correlation between severity of Sub-Cortical WMLs WMLs and FAN2
Sub-cortical WMLs Load
• Q# is the severity scale of sub-cortical WMLs. • # Reference group; * p<0.05; § p<0.01. [2 ] Qiu, Chengxuan, et al. "Retinal and cerebral microvascular signs and diabetes the age,
gene/environment susceptibility-reykjavi study." Diabetes 57.6 (2008): 1645-1650.
Motivation
23
Correlation between severity of Sub-Cortical WMLs WMLs and FAN2
• T# is the severity scale peri-ventricular WMLs. • # Reference group; * p<0.05; § p<0.01. [2 ]Qiu, Chengxuan, et al. "Retinal and cerebral microvascular signs and diabetes the age,
gene/environment susceptibility-reykjavi study." Diabetes 57.6 (2008): 1645-1650.
Motivation
24
Periventricular WMLs Load
MotivationRetinal microvascular lesions (AV nicking and FAN) can be an important bio-marker to predict the severity of WMLs load.
Current approach of quantification and correlation analysis is Manual
Limitations Highly subjective Expensive Time consuming High intra and inter-grader variability
25
Objective
Automatic segmentation and quantification of
WMLs
Automatic detection and quantification of AV
nicking
Automatic detection and quantification of FAN
Quantify the correlation between retinal and brain
microvascular lesions
Develop a retinal image based brain
microvascular lesions prediction model
26
Automated WMLs segmentation method
T1 MRIFlair MRI
Pre-processing
Feature extraction
Classification of WMLs
Post-processing using MRF
27
Automated WMLs segmentation method
Pre-processing for noise reduction and spatial normalization Co-registration of T1 and Flair MRI using Statistical
parametric mapping (SPM8)3
Brain skull removal using Brain Extraction Tool (BET) 4
Intensity normalization using dynamic maximum boundary5
Feature extraction Multimodal MRI (T1 and Flair) intensity Tissue probability mask (PWM, PGM, PCSF) Normalized spatial coordinate (X, Y, Z) Global neighbourhood based contrast
[3] J. Ashburner and K. J. Friston, “Unified segmentation,” Neuroimage, vol. 26, no. 3, pp. 839–851, 2005.[4] V. Popescu, M. Battaglini, W. Hoogstrate, S. Verfaillie, I. Sluimer, R. Van Schijndel, B. van Dijk, K. Cover, D. Knol, M. Jenkinson et al., “Optimizing parameter choice for fsl-brain extraction tool (bet) on 3d t1 images in multiple sclerosis,” Neuroimage, vol. 61, no. 4, pp. 1484– 1494, 2012.[5] Liang, Xi, Kotagiri Ramamohanarao et al. "Nat. ICT Australia (NICTA), Eveleigh, SA, Australia." Digital Image Computing Techniques and Applications (DICTA), 2012 International Conference on. IEEE, 2012.
28
Proposed automated WMLs segmentation method
Multimodal MRI (T1 and Flair) intensity
T1 MRI Flair MRI 29
Automated WMLs segmentation method Tissue probability mask
Multiple atlas construction from healthy T1 MRI using FAST6 Non-linear registration of atlases with input T1 MRI Tissue mask construction based on multi atlas voting
PWMPGM PCSF
[6] Y. Zhang, M. Brady, and S. Smith, “Segmentation of brain mr images through a hidden markov random field model and the expectationmaximization algorithm,” Medical Imaging, IEEE Transactions on, vol. 20, no. 1, pp. 45–57, 2001.
30
Automated WMLs segmentation method• Normalized spatial coordinate
• Subject patient’s T1 MRI is linearly registered with Montreal Neurological Institute (MNI ) space
• The voxel wise corresponding Montreal Neurological Institute (MNI) space is warped back to the experimental subject space.
• This results in normalized X, Y and Z space comparable between patients.
X Y Z 31
Automated WMLs segmentation method• Global neighbourhood based
contrast • Create global neighbours
for each pixel• Mask neighbours of
candidate pixel using PWM.• A box filter of size m is
applied on candidate and its neighbour pixels
• Global neighbourhood based contrast (GNC) is computed using
m
mk
m
mlljkiji x
mmX ,, *
1
N
n
n
X
XX
Nc
1
1
Here, X represent candidate pixel and Xn represents its neighbours.
32
Automated WMLs segmentation method
Classification of Lesion using Random Forest
33
Automated WMLs segmentation methodPost processing using Markov random field (MRF)
)1..().........,,,(),(
1maxargˆ jj
i iNjiiii
yxyxyxy
Zy
represents lesion probability computed by RF classifier represents edge potential computed by Eq. 2. xi and yi represent ith pixel intensity and RF given class label. is the penalty constant. Loopy belief propagation (LBP) 6 is used to infer the outcome
of MRF
(.)(.)
Neighbourhood structure
[6] K. P. Murphy, Y. Weiss, and M. I. Jordan, “Loopy belief propagation for approximate inference: An empirical study,” in Proceedings of the Fifteenth conference on Uncertainty in artificial intelligence. Morgan Kaufmann Publishers Inc., 1999, pp. 467–475. 34
Automated WMLs segmentation method
Segmentation outcome after applying MRF
Initial Segmentation MRF given Segmentation Ground Truth Segmentation
35
Automated WMLs segmentation method
Classification Parameters Classifier : Random Forest Number of trees in RF : 200
Data Number of Subject : 24 Source of Data: ENVISion study7 Size : 256 × 256 × 36 Resolution : 0.94 × 0.94 × 4 mm3
Training and Testing procedures 4 Fold cross-validation (4 times)
2 fold for training 1 fold for parameter selection 1 Fold for testing
• [7] C. M. Reid, E. Storey, T. Y.Wong, R. Woods, A. Tonkin, J. J. Wang, A. Kam, A. Janke, R. Essex,W. P. Abhayaratna et al., “Aspirin for the prevention of cognitive decline in the elderly: rationale and design of a neuro-vascular imaging study (envis-ion),” BMC neurology, vol. 12, no. 1, p. 3, 2012. 36
Training of Random Forest Classifier
• Number of subjects : • Resolution of each Subject• Number of data points (voxels) after skull stripping
• Number of features for each data point: 26• The number of training sample is very large
= 810,00*12 = 9.7 million voxels• To reduce the training time we randomly sample
20,000 voxels to build a random forest tree and build about 200 random forest trees. This method provides adequate accuracy!
36256256 24
Features Description
Local Features T1 , FLAIR intensityTissue probability map of White matter, Gray matter and Cerebral spinal fluid Normalized spatial coordinates
Global neighbourhood based contrast feature (GNCF)
Intensity of the lesion is enhanced and normalized based on the contrast of global neighbourhood points
Histogram based features MeanVarianceSkewnessKurtosis
Gray level co-occurrence matrix (GLCM) [1]
ContrastCorrelation EnergyHomogeneity
Run length matrix (RLM) [2] Short run emphasis inverse momentsLong run emphasis momentsGray level non uniformity Run length non uniformity
Isotropic undecimated wavelet transform (IUWT) [3]
Wavelet coefficients for five different scales
[2] M. M. Galloway, “Texture analysis using gray level run lengths,” Computer graphics and image processing, vol. 4, no. 2, pp. 172–179, 1975 .[3] P. Bankhead, C. N. Scholfield, J. G. McGeown, and T. M. Curtis, “Fast retinal vessel detection and measurement using wavelets and edge location refinement,” PloS one, vol. 7, no. 3, p. e32435, 2012.
[1] A. Baraldi and F. Parmiggiani, “An investigation of the textural characteristics associated with gray level cooccurrence matrix statistical parameters,” Geoscience and Remote Sensing, IEEE Transactions on, vol. 33, no. 2, pp. 293–304, 1995.
Automated WMLs segmentation method
Evaluation metrics Sensitivity (SEN)
Positive predictive value (PPV)
Dice Similarity Index (SI)
FNTP
TPSEN
FPTP
TPPPV
FNFPTP
TPSI
2
2
39
Automated WMLs segmentation method
WMLs Load Category
High Lesion Load (HLL [>10ml])
Medium Lesion Load (MLL [between 6 ml to 10 ml])
Low Lesion Load (LLL [between 1 ml to 5 ml ])
40
Proposed automated WMLs segmentation method
Results
41
Proposed automated WMLs segmentation method
Comparison with state of the art methods
For both methods p-value< 0.05.
[8] M. D. Steenwijk, P. J. Pouwels, M. Daams, J. W. van Dalen, M. W. Caan, E. Richard, F. Barkhof, and H. Vrenken, “Accurate white matter lesion segmentation by k nearest neighbor classification with tissue type priors (knn-ttps),” NeuroImage: Clinical, vol. 3, pp. 462–469, 2013.[9] P. Schmidt, C. Gaser, M. Arsic, D. Buck, A. F¨orschler, A. Berthele, M. Hoshi, R. Ilg, V. J. Schmid, C. Zimmer et al., “An automated tool for detection of flair-hyperintense white-matter lesions in multiple sclerosis,” Neuroimage, vol. 59, no. 4, pp. 3774–3783, 2012.
8
9
42
Automatic quantification of AVN and FAN
Retinal Image Pre-processing Feature extraction
Quantification of AVN and FAN
43
Automatic quantification of AVN and FAN
Pre-processing
Vessel Segmentation
Vessel Cross-over point detection
Candidate vessel region selection
Vessel width measurement
44
First to Analyze the Vessel: Segmentation
Although many methods have been proposed, significantimprovement is still a necessity due to the limitations in state of the-
art methods, which include:
Poor segmentation in the presence of central reflex (i.e., bright strip along the centre of a vessel).
Poor segmentation at bifurcation and crossover points.
The merging of close vessels.
The missing of small vessels.
False vessel detection at the optic disk and pathological regions.
45
Some Limitations of Existing Methods(a) a portion of a retinal
image showing the
presence of central reflex
vessels (white solid
arrows), close vessels
(white dashed arrows),
artery-vein crossing
regions (black solid
arrows), and small vessels
(black dashed arrows) and
segmentations obtained
by
(b) Staal et al. method;
(c) Soares et al. method;
(d) Ricci-line method ;
(e) Ricci-svm method
(f) Our proposed
method.
Uyen et. all (2013), Pattern Recognition
46
Our Proposed Method
A linear combination of line detectors at different scales to produce the vessel segmentation for each retinal image.
A basic line detector uses a set of approximated rotated straight lines to detect the vessels at different angles.
The difference between the average gray level of the winning line (the line with maximum average gray level) and the average gray level of the surrounding window provides a measure of ‘vesselness’ of each image pixel.
47
Vessel Segmentation Accuracy
Segmentation results on some selective regions showing theimprovements of the proposed method over existing methods in terms of segmentation quality: (a) original image; segmentations of (b) Staal et al. method; (c) Soares et al. method; (d) Ricci-line method; and (e) proposed method. 48
Vessel Segmentation Accuracy
49
Vessel Segmentation Accuracy (Cont.)
50
Vessel Crossover Point Detection
- Artery-vein crossover points in a retinal image are the locations where retinal artery and vein cross each other- Necessary for Individual Vessel segment Identification
Crossover points (left) often complicated to detect due to intensity variation (right)
51
Our Proposed Crossover Point Detection
Framework for Crossover Point Detection52
Crossover Point Detection
Analyzing branch and crossover points to detect crossover points
53
Crossover Point Detection
A retinal image with all crossover points detected by the proposed method marked as white spots.
54
Accuracy on Crossover Point Detection
55
Vessel Width Measurement
To Find mean Arterial and Venular Width in the Image and Find Association with CVD or Hypertension
Analyze the Vessel Width with Tortuosity or other Features
56
Vessel Width Measurement
Method for identifying vessel edge points representing vessel width at a centre point.
57
Width Measurement Accuracy
58
Feature for AVN severity grading
MW
CrMWF nn
Multi-scale measurement of narrowing Fn near the cross over point is used to classify AVN.
where,
c is the middle point of the vessel centreline and Wi represent the vein widths.
In this study, we have used 3 scales for Fn with n = 10,20,30.
n
iin W
nCr
1
1
59
Accuracy AVN severity classification
Severity Level Prediction Accuracy
Proposed Method Uyen et al.
0 78.43 % 70.58 %
1 29.41 % 17.64 %
2 42.85 % 14.28 %
3 81.81 % 45.45 %
Uyen T. V. Nguyen, Alauddin Bhuiyan, Laurence A.F. Park, Ryo Kawasaki, Tien Y Wong, Jie Jin Wang, Paul Mitchell and Kotagiri Ramamohanarao, An Automated Method for Retinal Arteriovenous Nicking Quantification From Color Fundus Images, IEEE Transactions on Biomedical Engineering, vol. 60, number.11, pp. 3194–3203, 2013.
60
Feature for FAN severity grading
Lmin and Lmax represents the set of local minima and local maxima for the vessel widths D ,
For each , the closest right and left local maxima and is computed using
is used as a feature to classify FAN
11minmin :LLi iiii DDDD
1maxLi
11minmax :LLi iiii DDDD
iminL
1maxLi
2
)()(V
minmaxminmaxi
11 iiiiLLLL
)max(Vi
61
Accuracy FAN severity classification
Severity Level Prediction Accuracy
0 87%
1 78%
62
A novel WMLs segmentation method. A novel AV nicking quantification method A novel FAN quantification method In future: Proposed method will be applied on a large dataset
to quantify the correlation between brain and retinal micro-vascular lesions.
Quantified correlation will be used to develop a retinal image based brain microvascular lesion severity prediction model.
Summary
63
Thank you!
64