color-based diagnosis: clinical images research project funded in part by nih yue (iris) cheng, dr....
TRANSCRIPT
Color-based Diagnosis: Clinical Images
Research Project Funded In Part by NIH
Yue (Iris) Cheng, Dr. Scott E Umbaugh@
Computer Vision and Image Processing Research Lab
Electrical and Computer Engineering Department
Southern Illinois University EdwardsvilleE-mail: [email protected]
https://www.ee.siue.edu/CVIPtools
Yue (Iris) Cheng, Dr. Scott E Umbaugh@
Computer Vision and Image Processing Research Lab
Electrical and Computer Engineering Department
Southern Illinois University EdwardsvilleE-mail: [email protected]
https://www.ee.siue.edu/CVIPtools
07/13/2005 Computer Vision and Image Processing Research Lab @ ECE Dept., SIUE
Overview Skin tumors can be either malignant or benign
Clinically difficult to differentiate the early stage of malignant melanoma and benign tumors due to the similarity in appearance
Proper identification and classification of malignant melanoma is considered as the top priority because of cost function
Classification of skin tumors using computer imaging and pattern recognition
Previous texture feature algorithm successfully differentiate the deadly melanoma and benign tumor seborrhea kurtosis
Relative color feature algorithm is explored in this research for differentiate melanoma and benign tumors, dysplastic nevi and nevus
Successfully classify 86% of malignant melanoma using relative color features, compared to the clinical accuracy by dermatologists in detection of melanoma of approximately 75%
07/13/2005 Computer Vision and Image Processing Research Lab @ ECE Dept., SIUE
Materials and Tools Image database
Original tumor images 512x512 24-bit color images digitized from 35mm color
photographic slides and photographs 160 melanoma, 42 dysplastic, and 80 nevus skin tumor
images Border images
Binary images drawn manually and reviewed by the dermatologist for accuracy
Software CVIPtools
Computer vision and image processing tools developed at our research lab
Partek Statistical analysis tools
07/13/2005 Computer Vision and Image Processing Research Lab @ ECE Dept., SIUE
CVIPtools
07/13/2005 Computer Vision and Image Processing Research Lab @ ECE Dept., SIUE
Method Design
Creation of relative color images
Segmentation and morphological filtering
Relative color feature extraction
Design of tumor feature space and object feature space
Establishing statistical models from relative color features
07/13/2005 Computer Vision and Image Processing Research Lab @ ECE Dept., SIUE
Create Relative Color Skin Tumor Images
Purpose to equalize any variations caused by
lighting, photography/printing or digitization process
to equalize variations in normal skin color between individuals
the human visual system works on a relative color system
Algorithm Mask out non-skin part in the image to
calculate the normal skin color Separate tumor from the image Remove the skin color from the tumor to
get a relative color skin tumor image CVIPtools functions were used to create
relative color skin tumor images
07/13/2005 Computer Vision and Image Processing Research Lab @ ECE Dept., SIUE
Calculate Skin Color
Original Noisy Skin
Tumor Image
Non-skin Algorithm
Calculate
Mask out tumor
Skin Tumor Image W/O
Noise
Average R, G, B Value
of Skin
Skin-OnlyImage
07/13/2005 Computer Vision and Image Processing Research Lab @ ECE Dept., SIUE
Tumor Image
AND
Original Noisy Skin
Tumor Image
Border Image
Tumor Image
07/13/2005 Computer Vision and Image Processing Research Lab @ ECE Dept., SIUE
Relative Color Tumor Image
SUBTRACT
TumorImage
Average R, G, B Value
of Skin
Relative Color Image of the Tumor
07/13/2005 Computer Vision and Image Processing Research Lab @ ECE Dept., SIUE
Segmentation and Morphological Filtering
Image segmentation was used to find regions that represent objects or meaningful parts of objects
Morphological filtering was used to reduce the number of objects in the segmented image
Easy to use CVIPtools for experimenting and analysis
07/13/2005 Computer Vision and Image Processing Research Lab @ ECE Dept., SIUE
Relative Color Feature Extraction Necessary to simplify the raw image data into higher
level, meaningful information Feature vectors are a standard technique for classifying
objects, where each object is defined by a set of attributes in a feature space.
Totally 17 color features and binary features were extracted using CVIPtools
The three largest objects, based on the binary feature ‘area’, were used in feature extraction
Histogram features, that is, color features, were extracted in each color band from relative color image objects
07/13/2005 Computer Vision and Image Processing Research Lab @ ECE Dept., SIUE
17 Features Binary features
Area
Thinness
Histogram features in R, G, B bands
Mean
Standard deviation
Skewness
Energy
Entropy
r c
crIArea ),(
24
Perimeter
AreaThinness
r c M
crIMean
),(
1
0
2 )()(L
gg gPgg
)()(1 1
0
33
gPggSkewnessL
gg
1
0
2)(L
g
gPEnergy
1
02 )(log)(
L
g
gPgPEntropy
07/13/2005 Computer Vision and Image Processing Research Lab @ ECE Dept., SIUE
17 Features (Cont.)
07/13/2005 Computer Vision and Image Processing Research Lab @ ECE Dept., SIUE
Design Two Feature Spaces
Tumor feature space consists of 277 feature vectors correspond to 277
skin tumor images. each feature vector has 51 feature elements, which
are the total of 17 features of each three largest objects within the same tumor.
Object feature space had 842 feature vectors corresponding to 842 image
objects each feature vector has 17 feature elements, which
were the binary features and color features stated as above
07/13/2005 Computer Vision and Image Processing Research Lab @ ECE Dept., SIUE
Establishing Statistical Models Two feature spaces serve as two data models in order to
maximize the possibility of success
Two classification models, Discriminant Analysis and Multi-layer Perceptron, were developed for both data models
The training and test paradigm is used in statistical analysis to report unbiased results of a particular algorithm
due to small size of data set, 282 images, we used the leave x out method, with both one and ten for x
Partek software was used to analyze the data representing the features to develop a model or rules for classifying the tumors
07/13/2005 Computer Vision and Image Processing Research Lab @ ECE Dept., SIUE
Quadratic Discriminant Analysis A statistical pattern recognition technique based on
Bayesian theory, which classifies data based on the distribution of measurement data into predefined classes
Normalization the feature data as preprocessing performed to maximize the potential of the features to
separate classes and satisfy the requirement of the modeling tool such as Quadratic discriminant analysis for a Bayesian distribution of the input data
Variable selection was used to choose dominant features.
07/13/2005 Computer Vision and Image Processing Research Lab @ ECE Dept., SIUE
Multi-Layer Perceptron A feed forward neural network
neural networks modeled after the nervous system in biological systems, based on the processing element the neuron
widely used for pattern classification, since they learn how to transform a given data into a desired output.
Principal Component Analysis (PCA) as preprocessing a popular multivariate technique, is to reduce
dimensionality by extracting the smallest number components that account for most of the variation in the original multivariate data and to summarize the data with little loss of information
the dispersion matrix selected for PCA in this project is correlation
07/13/2005 Computer Vision and Image Processing Research Lab @ ECE Dept., SIUE
Multi-Layer Perceptron (Cont.) Creation, training and testing of neural networks
Creation a neural network involves selection of hidden and output neuron types and a random number generation.
Four output neuron types – Softmax, Gaussian, Linear and sigmoid Three hidden neuron types – Sigmoid, Gaussian and Linear
Scaled Conjugate Gradient algorithm is used for learning in this project.
Automated and independent of user parameters Avoids time consuming Stopping criteria, sum-squared error, is selected to determine after
how many iterations the training should be stopped The trained data is then tested on itself first to examine how
far the neural network is able to classify the objects correctly. Leave x partition out method is used for testing the algorithm
07/13/2005 Computer Vision and Image Processing Research Lab @ ECE Dept., SIUE
Experiments and Analysis in Tumor Feature Space
Discriminant Analysis 24 features selected for leave ten out method
HistogramFeatures
Mean STD Skewness Energy Entropy
R G B R G B R G B R G B R G B
Object 1 X X X X X X X X
Object 2 X X X X X X X
Object 3 X X X X X X X X 10 features selected for leave one out method
HistogramFeatures
Mean STD Skewness Energy Entropy
R G B R G B R G B R G B R G B
Object1 X X X
Object 2 X X X X
Object 3 X X X
07/13/2005 Computer Vision and Image Processing Research Lab @ ECE Dept., SIUE
Experiments and Analysis in Tumor Feature Space (Cont.)
0
10
20
30
40
50
60
70
80
90
100
DA on data with 24features usingleave 10 out
DA on data with 10features usingleave 10 out
DA on data with 24features using
leave 1 out
DA on data with 10features using
leave 1 out
Su
cces
s P
erce
nta
ge
Dys %
Mel %
Nev %
Discriminant Analysis (Cont.)
07/13/2005 Computer Vision and Image Processing Research Lab @ ECE Dept., SIUE
Experiments and Analysis in Tumor Feature Space (Cont.)
Multi-layer Perceptron
0
10
20
30
40
50
60
70
80
90
1000 iterationsoutput layer
softmax,hidden layer
sigmod
700 iterationsouter layersoftmax,
hidden layersigmod,
700 iterationsouter layer
softmax, 17hidden layers
sigmod,
100 iterationsoutput layer
sigmod,hidden layer
sigmod
800 iterations ,output_layer
softmax,hidden layer
gauss
Dys correct%
Mel correct %
Nev correct %
Best features, being in the first three components of the PCA projection data, were used
Success percentages of melanoma as high as 77% and nevus is as high as 68%
07/13/2005 Computer Vision and Image Processing Research Lab @ ECE Dept., SIUE
Experiments and Analysis in Object Feature Space Discriminant Analysis
8, 9, 11 and 12 significant features were selected respectively for leave one out method
Number of HistogramFeatures
Area
Mean STD Skewness Energy Entropy
R G B R G B R G B R G B R G B
8 X X X X X X X X
9 X X X X X X X X X
11 X X X X X X X X X X X
12 X X X X X X X X X X X X
07/13/2005 Computer Vision and Image Processing Research Lab @ ECE Dept., SIUE
Experiments and Analysis in Object Feature Space (Cont.)
Discriminant Analysis (Cont.)
0
10
20
30
40
50
60
70
80
90
12 Features 11 Features 9 Features 8 Features
Number of Features
Su
ce
ss
Pe
ce
nta
ge
Dys %
Mel %
Nev %
Yield consistent results in classifying melanoma from other skin tumor with above 80% success rate
07/13/2005 Computer Vision and Image Processing Research Lab @ ECE Dept., SIUE
Experiments and Analysis inObject Feature Space (Cont.)
Multi-layer Perceptron (MLP)
0
10
20
30
40
50
60
70
80
90
100
130 iterationsoutput layer
sigmoidhidden layer
sigmod
425 iterationsouter layergaussian
hidden layergaussian
255 iterationsouter layer
linear, hiddenlayers
gaussian
700 iterationsoutput layer
softmaxhidden layer
gaussian
130 iterations ,output_layer
softmax,hidden layer
sigmoid
Dys correct%
Mel correct %
Nev correct %
5 out of 12 hidden-output layer neuron combinations gave better classification results
Leave one out method Yield success
percentage as high as 86% for classifying melanoma.
MLP is more consistent in classifying melanoma as well as nevus
07/13/2005 Computer Vision and Image Processing Research Lab @ ECE Dept., SIUE
Conclusion Multi-Layer perceptron (MLP) with feature data
preprocessed by Principal Component Analysis (PCA) gave better classification results for melonoma than Discriminant Analysis (DA)
The best overall successful rate of 78%, of which percentage correct of melanoma is 86%, nevus is 62% and dysplastic is 56%.
The best classification results are achieved with sigmoid used as the hidden and output layer neuron
type for the MLP with PCA on Object Feature Space. The three largest tumor objects are representative
for the whole skin tumor.
07/13/2005 Computer Vision and Image Processing Research Lab @ ECE Dept., SIUE
Conclusion (Cont.) However the small percentage of melanoma
misclassification as well as the relatively low success rate for nevus and dysplastic nevi suggests that we may not have the complete data set for the experiments.
In order to achieve better classification results, future experiments
Needs more complete skin tumor image database. Should combine texture and color methods to get better
results Will include dermoscopy images
07/13/2005 Computer Vision and Image Processing Research Lab @ ECE Dept., SIUE
Acknowledgement
Dr. Scott E Umbaugh, SIUE Mr. Ragavendar Swamisai Ms. Subhashini K. Srinivasan Ms. Saritha Teegala Dr. William V. Stoecker,
Dermatologist, UMR
07/13/2005 Computer Vision and Image Processing Research Lab @ ECE Dept., SIUE
Thank You!
Yue (Iris) ChengGraduate Student
@Computer Vision and Image Processing Research
LabElectrical and Computer Engineering Department
Southern Illinois University EdwardsvilleE-mail: [email protected]
https://www.ee.siue.edu/CVIPtools
Yue (Iris) ChengGraduate Student
@Computer Vision and Image Processing Research
LabElectrical and Computer Engineering Department
Southern Illinois University EdwardsvilleE-mail: [email protected]
https://www.ee.siue.edu/CVIPtools