Acknowledgements:

Barry Kalman Stan Kwasny

Koong-Nah Chung John Heuser

Detection of Intracellular Organelles in Digitized Electronmicroscopic Images Using Wavelets and Neural Networks

Final Report for CS513John Olsen and Wei Yan

The Experiment

• We used Kalman & Kwasny’s neural network tool to create a set of trained feedforward neural networks (FFNNs) that detect particular intracellular organelles, called caveolae, in scanning EM images.

Overview of Experiment

Training Set

Test Set

32 N 32 P

99 N100 P

WaveletTransform

WaveletTransform

Unsupervised,Recurrent Net:LOSRAAM

Supervised,Feed-ForwardNeural Net

Unsupervised,Recurrent Net:LOSRAAM

5 TrainedFFNNs

Final DecisionBy Vote

Performance is 84% correct

0

5

10

15

20

25

30

35

Test1 Test2 Test3 Test4 Total Perfect

Negatives

Positives

Images

Components: Vectors of waveletcoeffs(resolution, dimension)

Vectors ofLOSRAAM StateTransitions

K Centers ofAttraction

K x K FuzzyTransitionMatrix

Hold-outSet

FuzzyFeatureVectors

Reduced,CombinedFFV

PACSet

TestSet

TrainedFFNNs

FinalDecision

Data Flow Diagram

Wavelet Transform

• Equivalent to the 1st order derivative of a smoothing function.

• Analogous to the Fourier Transform.

• Multiple levels of Resolution.

• 2-Dimensions

• Image component: The set of wavelet coefficients for a particular dimension at a particular resolution.

• Image components: Multiple representations of the entire image at different resolutions.

Image L1 L2 L3 L4

320 x 240 160 x 120 80 x 60 40 x 30 20 x 15

1 4 16 64 256

76,800 19,200 4,800 1200 300

- 2 2 2 3

- 38,400 9,600 2,400 900

Level

Image size(grains)

Grain size(pixels)

Total Grains

No. of Comps (x, y, r)

No. of Coeffs

Four-level, 2-D Wavelet Hierarchy

• Maximum no. of coeffs at any one level, direction = no of grains in image. For our data, the max no. of coefficients/image = 51, 300.

• A Component consists of the set of wavelet coefficients for a particular direction and level of resolution.

• Total of 9 components: 2 for each level of decomposition, 1 for undecomposed components at the last level (DC, very low frequency).

Why Wavelets ?

• Enhance contrast edges.

• Multi-Resolution emphasizes features of different sizes.

• Data reduction from thresholding.

Wavelet Coefficient Thresholding

• For all images, dropped coefficients with magnitudes < 0.4

• 16 million coefficients reduced to 1.6 million.

• Threshold determined by experience and experiment: Set of coefficients after thresholding reverse transformed to image, and this image is visually compared to the original.

LOSRAAM Training Set

• Each component is made a separate vector: 9 components per image * 199 images = 1791 components.

• 1.6 M coefficients distributed over 1791 components.

• Each coefficient represented by a doublet: [magnitude, normalized index into component array].

• Coefficients fed into LOSRAAM Neural Network one by one.

The LOSRAAM ANN

• Linear Output Recurrent Recursive Auto-Associative Memory

• AAM: Targeting the outputs to be the same as the inputs.

• Recurrence: a portion of the input is the activation pattern from the hidden layer of the previous iteration.

• Unsupervised Learning: A criterion for judging outputs is determined. The ANN learns a mapping of inputs to outputs that fits the criterion.

Output Units

Linear State (“Context”)

Input Units

HiddenUnits

Coefficient Index

6 input units 4 hidden units, determined by experiment 6 output units 10 bias units 74 adjustable weights, initially all 0. Recurrent connections from hidden units to

input state units

6-4-6 LOSRAAM Neural Network

LOSRAAM Training Statistics

• Running time 55 hours.• Sun Sparc, 4 parallel processors.• Error function: initial value = 1,800,000

final value = 401.

Clustering LOSRAAM State Vectors

• Image structure produces a sequence of activation patterns of the 4 unit hidden layer.

• Activation patterns represented by a sequence of vectors in 4-D space.

• The trajectory of these vectors is represented by a series of point values in 4-D space.

• These points are clustered into “centers of attraction”.

• Clustering is a “fuzzy” process.

• Our LOSRAAM data yielded 4 centers of attraction.

Fuzzy Transition Matricesand Fuzzy Feature Vectors

• An FTM is a 4 x 4 matrix of transitions between centers of attraction.

• FTMs were computed for each component, for each image.

• Thus, each image is represented by 9 FTMs, each containing 16 elements, 144 elements total.

• Linearization of FTMs gives 144 element FFVs, one per image.

Singular Valued Decomposition of FFVs

• SVD is a data reduction and conditioning technique.• Combine FFVs for all images into a 144 x 199 matrix.• Using SVD, identified 9 of the 144 columns that accounted

for > 99% of the variance in the entire training data set.• These 1791 values used to train the Feed Forward ANNs.

144 elements

199 FFVs 199 reduced FFVs

9 elements

Feed-Forward Neural Network

• Supervised Learning: Input and output are provided. The FFNN learns mapping by example.

• We used a 9-1-1 FFNN.

Image’s 9 element

FFV

Input Units

9-1-1 Feed-Forward Neural Network

Output Unit

Hidden UnitFully skip-connected

FFNN Training

• Input 199 training patterns (9 values each), one at a time.• Periodically check performance with PAC set. (A set of 16

images, 8 positive, 8 negative).• Harvest weights if performance on training set and on PAC are

both > 85%.• If PAC test failed, weights are discarded.• Trained 5 FFNNs to criterion performance on training set and

PAC.

FFNN Training Issues

• Overtraining. First time training with a two-hidden-unit FFNN led to 100% performance for both training and PAC, but only 65% performance on the test set.

• Attempt to raise harvest criterion to 88% for one-hidden-unit FFNN failed. No trained networks were produced.

Images

Components: Vectors of waveletcoeffs(resolution, dimension)

Vectors ofLOSRAAM StateTransitions

K Centers ofAttraction

K x K FuzzyTransitionMatrix

Hold-outSet

FuzzyFeatureVectors

Reduced,CombinedFFV

PACSet

TestSet

TrainedFFNNs

FinalDecision

Data Flow Diagram

Can performance be improved?

• Human performance ~ 100%.

• ANN must handle several different sources of variation: position, view angle, size, number, flatness.

• Increase the size and depth of the training set, e.g., 500 images.

• Add positional ‘hint’ units, number hint units, etc.

Significance of Results to Kalman and Kwasny’s ANN

• Scanning EM images are rich in detail, perhaps more so than mammograms.

• Absolute performance on EM images is better than performance on mammograms, particularly with respect to specificity.

• The results suggest that the ANN is capable of using the rich detail found in EM images to achieve a low false positive rate.

Mammograms EMs

False Pos. 44% 14%

False Neg. 25% 14%

Potential Applications

• Automated scanning of EM images.• Large scale screening of histological slides for

abnormal cell morphologies.