a microscopic telepathology system for multiresolution

A Microscopic Telepathology System forMultiresolution Computer-Aided Diagnostics

Grigory Begelman, Michael Pechuk, Ehud RivlinDepartment of Computer Science, Technion – Israel Institute of Technology, Haifa, Israel

{gbeg,mpechuk,ehudr}@cs.technion.ac.ilEdmond Sabo, M.D.

Department of Pathology, Rhode Island Hospital and Brown School of Medicine, Providence, Rhode [email protected]

Abstract— The aim of the presented system is simplificationand speedup of the daily pathological examination routine.The system combines telepathology with computer-aideddiagnostics algorithms. To the best of our knowledge, this isthe first approach proposing such a comprehensive method.Our system is designed to accumulate knowledge througha learning process during diagnostics. Our system targetsimage acquisition and interpretation stages. The imageacquisition subsystem solves various problems related tomicroscopical slide digitization such as biomedical imageregistration, data representation, and processing. The in-terpretation subsystem is based on Gabor filter texturefeatures as well as on color features. A support vectormachine classifier together with a feature selection is usedfor computer-aided diagnostics. The system design allowseasy adaptation to a wide range of microscopical pathologyexaminations. The system is easily deployed and scaled. Ithas a low support cost and can aggregate a wide range ofexisting hardware. The experimental validation of the systemis based on a database of more than three thousand samples.During the experimental evaluation, the system exhibitedsuccessful interaction with a pathologist.

Index Terms— computer-aided diagnostics, microscopicaltelepatology, multiresolution analysis

I. INTRODUCTION

Many different pathologies can be detected by micro-scopic examination of histological tissues. For example,prostate carcinoma, breast cancer, etc are be identifiedthrough such procedures. In this paper we describe asystem aimed at facilitating the diagnosis of such diseases.We validate the system design and efficiency in thediagnosis of prostate carcinoma.

Prostate carcinoma is the second most common causeof cancer deaths among men in the United States. Thediagnostics of prostate carcinoma involves pathologyanalysis of prostate tissue. Gleason grading [1] is themost widely used grading system for estimating the levelof aggressiveness of the prostate carcinoma. The gradinghas five levels of severity, corresponding to five differentarchitectural patterns formed by the prostate glands (seeFig. 1). The routine for pathology analysis of prostate tis-sue, widely accepted in hospitals, includes the following:

1) Obtaining a specimen from a patient.2) Preparing the specimen (staining, sectioning, and

mounting on glass slides).

3) Microscopic examination of the prepared specimenby a screener. At this stage, the screener locates andmarks suspicious regions on the slides.

4) Microscopic examination of the marked regions bya pathologist. At this stage, the pathologist gives thefinal diagnosis.

The classical routine of pathology analysis of theprostate tissue has a number of shortcomings:

• The human factor may cause diagnostic inaccuracy.• The heavy load on pathologists. A pathologist in a

typical laboratory analyzes many slides per day.• Interobserver disagreement [2], [3]. The Gleason

grading system is subject to interobserver disagree-ment.The final diagnosis for the same specimengiven by different pathologists may differ.

• Second opinion problem. For difficult cases, a secondopinion is often necessary, requiring physical transferof the slide. As a result, patients wait more time toget a conclusive diagnosis.

Systems with computer-aided diagnostics and tele-pathology capability will help to eliminate the shortcom-ings of the classical pathology routine. Some diagnostictasks can already be reliably performed by computer.For example, a system for automated Pap smear screen-ing [4] achieves similar or higher accuracy compared tothe classical methods of examination. Several researcheshave investigated computer-aided diagnostics of prostatecarcinoma, e.g., [5]–[7]. At the same time, advancesin computer networking now provide the possibility oftelepathology: distant diagnosis of specimens (e.g., [8]–[15]).

We present a system combining the advantages ofcomputer-aided diagnostics, active learning algorithms,and telepathology. Our system addresses the problems ofthe classical routine. It automates detection of suspiciousregions on the microscopic slides, assists pathologistsin diagnostics, and, finally, provides an environment fordistant collaborative diagnostics.

The paper is organized as follows. In Section II, thestructure of the proposed system is explained. In SectionIII, the experimental result are presented. Finally, weconclude in Section IV.

40 JOURNAL OF MULTIMEDIA, VOL. 1, NO. 7, NOVEMBER/DECEMBER 2006

© 2006 ACADEMY PUBLISHER

Prostate Glands

(a) (b)

Figure 1. Gleason grading of prostate carcinoma. Prostate glands (a) form patterns used for Gleason grading. Five Gleason grades of prostate cancercorrespond to five patterns formed by prostate glands (b).

II. SYSTEM STRUCTURE

The structure of our system for computer-aided mi-croscopic pathology diagnostics is presented in Fig. 2(a).Our system combines image acquisition and interpretationphases(see Fig. 2(b) and 2(c)). It targets the last twosteps of the classical routine: microscopic examinationof the prepared specimen by a screener and microscopicexamination of the marked regions by a pathologist.

The system comprises the following components: acomputer-aided diagnostics component, a slide scanner,and a telepathology interface. Below we describe thesecomponents in detail.

A. Computer-aided Diagnostics Component

This component analyzes the scanned slide and gener-ates hypotheses about optimal magnification for analysisand diagnosis of each slide region. The component usesan active learning algorithm for knowledge accumulationduring primary learning. Feature selection algorithms areused to eliminate redundant and irrelevant features. Dur-ing diagnostics the component refines obtained knowledgeusing the pathologists’ feedback. In the following sectionwe describe the implementation details of the component.

Multiresolution Analysis Traditionally, pathologyanalysis of tissues is performed under several magnifi-cation levels. Different characteristics of the examinedtissues are revealed under various magnifications. Forexample, in Fig. 3(a) the morphology of prostate glands isobserved under ×40 magnification, while in Fig. 3(b) thestructure of cells’ nuclei is observed under ×100 magnifi-cation. Our system allows multiresolution analysis via thetelepathology interface (see section II-C). In addition, thesystem is designed to determine the optimal magnificationlevels for the slide regions. This is done by classifying theslide tiles into one of two categories: “suitable for analysisunder ×40 magnification” and “need better magnifica-tion”. The classifications for initial training were providedby the expert pathologist. The following sections providemore details on feature extraction for the classifier andbuilding the classifier.

Feature Computation The Gleason system is basedon the architectural pattern of the glands of the prostatetumor. According to [7], [16], the architectural pattern

(a)

(b)

Figure 3. The characteristic objects of the prostate tissue captured undervarious magnifications. The characteristic objects are accented by whitearrows. Prostate glands are observed under magnification ×40 (a). Cellnuclei are observed under magnification ×100 (b).

of the prostate glands is well modeled by texture de-scriptors. Gabor filters are widely used in texture analysisapplications [17], including texture classification, texturesegmentation and image retrieval. The family of two-dimensional Gabor functions is defined as

G(x, y) = e− x′2

2σ2x e− y′2

2σ2y cos(

2πx′2

λ+ φ),

JOURNAL OF MULTIMEDIA, VOL. 1, NO. 7, NOVEMBER/DECEMBER 2006 41


slidevirtual

Pathologist

diagnosiscorrect

Microscopical Slide

Image acquisition Image interpretation

Slide Scanner

dataslide

learning

Computer−aided

Diagnostics InterfaceTelepathology

scannedimage

active

(a)

Registration

Tiling Computation

Feature

Classification

Feature Selection

DatabaseFeature

Image and

Scanner

Slide

Automatic tilessubimagesscanned

diagnosisfeaturestiles

featurestiles

(b)

diagnosisfinal

Classification

Feature Selection,

Image andFeature

DatabaseInterface

Telepathology

Virtual Microscopevirtuallabeled

slide

Pathologist

active

slide

data

diagnosis,

new classifierlearning

(c)

Figure 2. The main system components and their interactions. The arrows denote information and control flow. High-level system overview (a).Image acquisition scenario (b). Image interpretation scenario (c).

where{x′ = (x−mx)cos(γ)− (y −my)sin(γ)y′ = (x−mx)sin(γ) + (y −my)cos(γ)

• γ specifies the orientation• φ specifies the phase offset• (mx,my) specify the center of the receptive field in

image coordinates• (σx, σy) determines the size of the receptive field• 1

λ is the preferred spatial frequency of G

In our experiments we used a bank of Gabor filters withsix equidistant orientations and four spatial frequencies.The bank of Gabor filters is presented in Fig. 4. Mean andstandard deviation of the response energies for each ori-entation and scale of a Gabor filter in the bank constitutetexture features.

Prostate tissue subject to microscopic examination isstained by the histochemical dyes hematoxylin and eosin.Nuclei of cells are stained in various blue hues (ba-sophilic stain) and surrounded by a pinkish background(eosinophilic stain). We used values from the 10-bin huehistogram as color features.

Classification For classification we used a SupportVector Machine (SVM) algorithm. Support vector ma-chines, a supervised machine learning technique, havesolid theoretical justification [18] and perform well inmultiple areas of biological analysis (see the survey ofSVM applications to biological problems in [19]). We

(a)

(b)

Figure 4. Bank of Gabor filters (six equidistant orientations and fourspatial frequencies). Odd filters in the spatial domain (a) and filter bankin the frequency domain (b).

built the SVM classifier, determined its optimal param-eters and estimated the classifier performance using themethod described in the next section.

Training-Validation Protocol A frequently employedmethod to estimate the classifier’s performance is to splitthe data into a training set and testing set. The training setis used to construct the classifier. The testing set is usedto estimate the performance of the classifier on unseen



samples. There are several popular methods for dividingthe data into training and validation sets. We choose thecross-validation method [20], which provides an accurateestimation for real-world problems [21].

The training protocol (see Fig. 5) delivers, given a train-ing dataset, an estimate of L̂A for the best achievable errorof learning algorithm A. To this end, the training datasetS is randomly permuted and partitioned into Nfolds sub-sets of equal size S1, . . . , SNfolds

. Nfolds training/testingdatasets are constructed as follows: Si

test = Si constitutesa testing partition and Si

train = S\Si constitutes atraining partition (i = 1 . . . Nfolds). Then the classifier isconstructed for the training partition, and its performanceis evaluated on the testing partition. As the SVM classifierwith an RBF kernel is a parametric classifier, the processof classifier construction for each testing partition involvesoptimization of the RBF kernel parameters to maximizethe classifier performance. To achieve this, the testingpartition is randomly divided into Nval subsets of equalsize, and Nval training/validation partitions V j

train, V jtest

are constructed in a similar manner (j = 1 . . . Nval).Then, for each feasible hyper-parameter vectors θ ∈Θ, a classifier is constructed on the training partitionV j

train and the performance L̂A(i, j, θ) is evaluated onthe corresponding testing partition V j

test. Then the errorL̂med

A (i, θ) is computed as the median of L̂A(i, j, θ)and, finally, the best hyper-parameters are selected θ∗i =argminθL̂

medA (i, θ). The hyper-parameters θ∗ are used

to construct the classifier for the training set Sitrain.

Thereafter, the classification error L̂A(i) is evaluated onSi

test. Eventually, the classification error is calculated asmean of all classification errors L̂A(i), i = 1 . . . Nfolds.

Feature Selection Some features may be irrelevantor redundant. Large amounts of irrelevant features affectlearning algorithms at three levels [22]. First, most learn-ing problems do not scale well with the growth of irrel-evant features. Second, classification accuracy degradesfor a given training set size. The third aspect concernsthe run time of the learning algorithm. Following Wolf etal. [22], we select feature subset providing the best clusterarrangement of the data points in the feature space. Theselected subset is fed to the classifier.

Primary Learning Usually, in order to train a classi-fier, a significant representative set of labeled samples isrequired. Therefore, a pathologist should label a signifi-cant number of sample regions. However, in most casesthe total area occupied by irrelevant and normal regionsis much larger compared to the area taken up by suspectregions. Thus, the probability of running across a diseasedregion is low. A pathologist may screen hundreds of im-ages without coming across an area that requires specialinvestigation. In order to boost the primary learning, weuse the active learning algorithm described in [23]. Onlyimages whose grading improve the expected performanceof the SVM classifier are provided to a pathologist. Aftereach iteration a new SVM classifier is built, according tothe protocol described in Fig. 5, and the sample selectioncontinues.

Require: 1) A labeled dataset S = {(xi, yi)}mi=1

2) A learning algorithm A(θ)3) A set Θ of feasible hyper-parameter vectors4) Nfolds = number of folds5) Nv = number of validation folds

Ensure: An estimate L̂A for the best achievable errorof the learning algorithm A trained on a set of size|S|(1− 1/Nfolds).

1: Randomly permute and partition S into Nfolds

subsets of equal size; denote these subsets byS1, . . . , SNfolds

these subsets2: Construct Nfolds training/test partitions

(Sitrain, Si

test)Nfolds

i=1 such that Sitrain = S\Si

and Sitest = Si

3: for 1 ≤ i ≤ Nfolds do4: for all θ ∈ Θ do5: Randomly permute and partition Si

train intoNval subsets of equal size; denote these subsetsby V i

1 , . . . , V iNval

these subsets.6: Construct Nval train/test partitions

(V i,jtrain, V i,j

test)Nvalj=1 such that V i,j

train =Si

train\V ij and V i,j

test = V ij .

7: for 1 ≤ j ≤ Nval do8: Compute L̂A(i, j, θ) = L̂A(V i,j

test|θ, V i,jtrain)

9: end for10: Compute the median L̂med

A (i, θ) of{L̂A(i, j, θ)}Nval

j=1

11: end for12: Let θ∗i = argminθL̂

medA (i, θ)

13: Compute L̂A(i) = L̂A(Sitest|θ∗i , Si

train).14: end for15: return 1

Nfolds

∑i L̂A(i)

Figure 5. Experiment protocol for estimation of the classification error.

B. Slide Scanner

The slide scanner component digitizes an analyzedslide. The digitization includes capturing sub-images (ad-jacent regions of slides), registering them, and, finally,slide tiling. The following section describes the imple-mentation details of the slide scanner component.

Sub-image capturing Time spent for sub-image cap-turing is the bottleneck of slide digitizing. The motionof the mechanical microscope stage is the most time con-suming part of sub-image capturing. In order to minimizethe overall time of microscope stage travel, we use theboustrophedon scanning route [24]. The scan direction ofconsecutive scan bands are alternated.

The white point of the captured images differs from thecomputer monitor’s white point. In addition, each pixelhas a different white point. This difference is introducedby the microscope lamp, the optical path, and also thedigital camera. Since the parameters of the microscopelamp, the microscope optical path, and the camera do notchange during the slide digitization, we can assume thateach pixel has an individual independent color aberrationmodel. The color models for each pixel are computedfrom the captured images of the empty slide regions



once per hardware adjustment. To this end, we take themedian of pixel values in XYZ color space and adjust thewhite point to match the D50 white point (X = 0.9642,Y = 1.000, Z = 0.8249) – the white point of a correctlycalibrated monitor (see the example of color correction inFig. 6).

(a)

(b)

Figure 6. Example of color correction for a captured image. Before colorcorrection (a): white does not correspond to the white color of a correctlycalibrated monitor. After color correction(b): white corresponds to thewhite color of a correctly calibrated monitor.

Sub-image registration The mechanical precision ofthe motorized microscope stage is limited. Therefore,image processing techniques should be used to regis-ter the adjacent sub-images. We register each image toits right and bottom neighboring sub-images. Sufficientoverlapping of the registered sub-images is the necessarycondition for registration. We use a phase correlationtechnique for image registration due to its reliable workon relatively small overlapping regions and good timeperformance.

Due to the properties of the microscopic stage movingmechanism and microscope optics, the transformationbetween two adjacent sub-images is limited to translation.We successfully verified this hypothesis experimentally. Iff2(x, y) is a translated replica of f1(x, y) with translation(x0, y0), then f2(x, y) = f1(x − x0, y − y0). Accordingto the Fourier translation property, the Fourier transformsof f1(x, y) and f2(x, y) are related by

F2(ξ, η) = e−2πi(ξx0+ηy0)F1(ξ, η).

The cross-power spectrum of the two images f1 and f2

is defined as:F1(ξ, η)F ∗2 (ξ, η)|F1(ξ, η)F2(ξ, η)| = e−2πi(ξx0+ηy0)

The inverse Fourier transform of the cross-power spec-trum results in an impulse function

F−1{e−2πi(ξx0+ηy0)} = δ(x + x0, y + y0)

Ideally, the peak value of the impulse function should beequal to 1.0. However, due to the presence of noise and

dissimilar parts in the sub-images, the delta function de-grades. Therefore, the translation parameters are estimatedby

(x0, y0) = arg max{F−1{e−2πi(ξx0+ηy0)}}.We follow the method described in Keller et al. [25] toobtain sub-pixel registration accuracy.

To achieve better time performance of the registrationwe adopt a multiresolution approach. First, a coarseregistration is performed on images scaled down by 0.2.Second, the sub-pixel registration accuracy is achieved byregistering only the intersecting parts of the images.

Require: 1) A set of captured sub-images Iij .2) Transformations Ti1j1 − Ti2j2 between adjacent

sub-images Ii1j1 and Ii2j2

Ensure: Tiles: ideally aligned non-overlapping imagescovering the whole slide area.

1: Detect connected components on the scanned slide2: for all connected components do3: Calculate global transform for all captured sub-

images relating to the origin of the central imageof the connected component.

4: Calculate the coordinates of the tiles relating tothe origin of the central image of the connectedcomponent.

5: Build the tiles from the corresponding images6: end for

Figure 7. Algorithm of tiles construction from the captured sub-images.

Slide tiling The resulting size of microscopic slidesdigitized under×100 magnification is in tens of giga-pixelrange. Such a volume of data causes technical difficultiesfor storing and accessing the images. The digitized slidesare accessed via a computer network, so to make the datatransfer efficient we divide the whole slide into relativelysmall tiles. The tiles are ideally aligned non-overlappingimages covering the whole slide area.

Due to the large volume of the resulting digitized slideit is impractical to construct the whole slide and then todivide it into tiles. Instead, we construct the tiles directlyfrom the captured sub-images and the transformationbetween the adjacent sub-images. The tiles constructionalgorithm is presented in Fig.7.

Each slide may include several disconnected tissueparts. This problem is addressed by detecting connectedcomponents using the transformations between adjacentsub-images. To accomplish this we build an undirectedgraph representing registration results. The vertices ofthe graph are the origins of the sub-images. The edgesof the graph connect vertices corresponding to adjacentsub-images, if the registration between these sub-imageswas successful. This graph may contain several connectedcomponents corresponding to disconnected fragments ofthe tissue. Once we identify the connected components,tiles are constructed for each component separately.

We take the origin of the central image of each com-ponent as a global origin. Then we calculate the global



Marked regions

Digitized slide

Magnification Control

and labelling tools

Image Markup

Figure 9. Application screen-shot. The main controls (image markup, labeling tools and magnification control), a digitized slide, and the regionsmarked on the slide are denoted by arrows.

(0,0)−(1,1)T

(0,0)−(1,0)T

T(1,0)−(1,1)

(0,1)−(1,1)T(0,0)−(0,1)T Sub−image (0, 1)

Sub−image (1, 1)Sub−image (1, 0)

Y

X

(a)

Sub−imagek

jSub−image

Rk

jR

X

(b)

Figure 8. (a) Image registration. Images are captured with intersection.Sub-image (0,0) denotes the central image. Ti−j is the transformationbetween the origins of sub-images I and J . (b) Tile construction. Theconstructed tile is denoted by the bold line. Rj and Rk are the centers ofthe sub-images. X is a tile pixel. The dashed line is the border betweenthe Voronoi cells corresponding to Rj , Rk .

transform of each sub-image of this connected componentin a breadth-first search (BFS) order starting from acentral sub-image. For some sub-images several variantsof the global transform may be calculated. For example,

the global transform for sub-image (1, 1) in Fig. 8(a))may be calculated as

T 1(0,0)−(1,1) = T(1,0)−(1,1)T(0,0)−(1,0)

T 2(0,0)−(1,1) = T(0,1)−(1,1)T(0,0)−(0,1).

In such cases, the resulting global transform for this sub-image is taken as average:

T(0,0)−(1,1) =T 1

(0,0)−(1,1) + T 2(0,0)−(1,1)

2After calculating the global transform for each sub-

image, we build tiles of the connected component. First,the tiles are positioned to cover the whole connectedcomponent, and the tile coordinates are calculated. Then,we build Voronoi diagram on the centers Ri of the sub-images and use it for efficient calculation of the nearestsub-image for the tile pixels (see Fig. 8(b)).

C. Telepathology Interface

The telepathology interface provides a convenientmulti-user environment for interaction with our system.To enable fast system adoption by pathologists, the inter-face was build to resemble the microscopic environment(see the screenshot of the telepathology interface appli-cation in Fig. 9). To this end, a pathologist may browsedigital slides using the client software by fast panning(simulates movement of the microscope stage), zoomingin/out to/from arbitrary selected regions up to a predefinedresolution (simulates magnification changes).

In addition, to provide pathologists a means of commu-nicating with the system and with each other, the interfaceallows image labeling by selecting a digital slide regionand assigning a diagnosis and an arbitrary comment.All interesting/suspicious regions detected by the CAD



TABLE I.EXPERIMENTAL DATABASE CONTENTS.

×40 magnification ×100 magnificationNormal or Non-Relevant 863 2304Gleason 3 59 329Gleason 4 10 37Gleason 5 0 139Need Better Magnification 111 NA

TABLE II.CLASSIFICATION ACCURACY ESTIMATED BY FIVE-FOLD CROSS-VALIDATION METHOD.

Classification categories Accuracy×40 magnification vs. ×100 magnification 93%“Normal or Non Relevant” vs. “Attention Required” under ×40 magnification 95%“Low Gleason” (i.e., Gleason 3) vs. “High Gleason” under ×40 magnification 98%“Low Gleason” (i.e., Gleason 3) vs. “High Gleason” (i.e., Gleason 4 and Gleason 5) under ×100 88%

component are presented at relevant magnification inorder to speedup the diagnosis process. Comments andlabeling are stored in a database along with the slideand can be viewed by all authorized users. Moreover,the final diagnosis is provided to the CAD system inorder to improve classification results. The features ofthe user interface allow convenient communication anddiscussion of difficult cases within a wide audience ofpathologists and provide the option for a second opinionin timely manner and without physically having to transferthe slide.

The telepathology interface was built using the client-server architecture. Since the Internet is a common andaffordable data transfer infrastructure, we chose it tobe the communication layer between the client and theserver. The implemented interface uses the Secure SocketLayer provided by a standard library to ensure encryptedcommunication between the client and the server. Theserver side is responsible for storing the digitized slidesdatabase. The database consists of

• the digitized slides tiles• the regions, created by pathologists using the slide

labeling tool and by the system as a result ofcomputer-aided diagnostics

• the slides’ meta-information

The server side is based on open-source free domainsoftware. We selected MySQL 5.0 for the database engine.All communication between the user interface applicationand the database is implemented over the HTTP protocol,and an Apache HTTP server handles the HTTP requests.Both MySQL and the Apache HTTP server are scalableand exist in a wide range of platforms. This makes oursystem easily deployable and scalable. The system has alow support cost and can aggregate wide range of existinghardware.

The client side application is written in Java to providehigh compatibility with a wide range of platforms. Theinterface provides the possibility of a low magnificationslide overview and a high magnification study of thepossible diagnostic details. To ensure high responsiveness

of the telepathology interface, the digital slides are com-pressed after scanning and tiling.

At the first step of slide compression algorithm, a Gaus-sian pyramid of the slide image is built for bandwidth-efficient presentation of the slide at different magnifica-tion levels. Then each level of the pyramid is dividedinto small sub-images of dimension 256 × 256 pixels insize. These sub-images are stored in the database aftercompression using a JPEG compression algorithm [26].

This method allows that only data relevant to thecurrent resolution and position to be transferred to theclient. This is an important feature of the system, becausethe requirement of storing (and potentially presenting)the whole digitized slide leads to huge data volumes.For example, the ×100 scanning magnification impliesa resolution of approximately 100000× 500000 pixels orapproximately 10Gb raw data per slide.

To utilize all available bandwidth, the client applicationis designed to transfer simultaneously a number of thesesub-images. The sub-images neighboring to currentlyviewed region are downloaded in background. The combi-nation of the image storing and caching techniques makesthe scheme low bandwidth consuming and responsive.

III. EXPERIMENTAL RESULTS

The hardware part of our system consists of a NikonEclipse E600 microscope equipped with a motorizedstage, a Point Grey CCD camera and a Xeon 2.4GHzbased computer. We experimented with histological slidesof prostate tissue samples, sectioned into five micron thickslices and stained by the histochemical dyes hematoxylinand eosin. The slides contained all types of tissue, fromnormal tissue to Gleason 5. The tile dimensions were1024×1024 pixels. To establish ground truth and to trainthe computer-aided diagnostics component, a number oftiles were graded by an expert pathologist according to theGleason method. The tiles were classified as: “Normal ornot relevant”, “Gleason 3”, “Gleason 4”, “Gleason 5”,and “Need better magnification”. A tile was classifiedas “Need better magnification” if the pathologist had



difficulties in grading the tile under ×40 magnification.The ground truth for the optimal magnification level anddiagnosis was provided by an expert pathologist for 3852tiles (see Table I).

The classifiers ensuring the lowest classification errorwere built using the protocol described above (see Fig. 5).The learning algorithm A is SVM, the set Θ of feasiblehyper-parameters is the set of radial-based function kernelparameters, sampled by the grid search method. Finally,Nfolds = 5, and Nv = 5.

The classification results are presented in Table II.The accuracy of correct magnification detection allowsthe pathologist to rely on the system for choosing thecorrect resolution for diagnosis verification. The classifi-cation accuracy between different grades of cancer andnormal tissue allows the system to assist pathologists indiagnostics.

The Receiver Operating Characteristic (ROC) curveof the classifiers are presented in Fig. 10. From theROC curve for “Normal Tissue” vs. “Attention Required”(Gleason 3, Gleason 4, Gleason 5) we see that it ispossible to achieve a high true positive rate while keepingthe false positive rate relatively low.

IV. CONCLUSIONS

In this paper we presented a system for computer-aided multiresolution microscopic pathology diagnostics.To the best of our knowledge, this is the first approachcombining the principle of telepathology with computer-aided diagnostics algorithms. The telepathology interfacemakes a second opinion available fast, without the actualneed to physically transfer the slide. The system designimplies that a diagnosis is approved by an expert patholo-gist in accordance with the hospital’s standard procedure.Therefore the system can be gradually introduced intopathology departments. Consequently, the load placed onpathologists should be noticeably reduced due to theautomatization of the routine work. This, in turn shoulddiminish the risk of errors and interobserver diagnosticdisagreement. The software implementation of the systemis based on platform-independent and scalable software.The system is easily deployed and scalable. The systemhas a low support cost and could aggregate a wide rangeof existing hardware. The system design and efficiencywas validated on diagnostics of prostate carcinoma. Thesuccessful experimental validation of the system is basedon a database of more than three thousand samples.During the experimental evaluation, the system exhibitedsuccessful interaction with a pathologist and diagnosticsspeedup.

REFERENCES

[1] D. G. Bostwick and J. N. Eble, Urologic Surgical Pathol-ogy. New York, NY: Mosby - Year Book Inc., 1997.

[2] A. Taille, A. Viellefond, N. Berger, and et al., “Evaluationof the interobserver reproducibility of gleason gradingof prostatic adenocarcinoma using tissue microarrays,”Human Pathology, vol. 34, pp. 444–449, 2003.

0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

False Positive Rate

Tru

e P

ositi

ve R

ate

Normal vs. Attention (x100)Normal vs. Attention (x40)

(a)

0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

False Positive Rate

Tru

e P

ositi

ve R

ate

Low vs. High Gleason (x100)Low vs. High Gleason (x40)

(b)

Figure 10. Receiver Operating Characteristic curves of the classifiers.(a) “Normal Tissue” vs. “Attention Required”. The area under the ROCcurve for magnification ×40 is 0.96 and for magnification ×100 is0.98. (b) “Low Gleason” vs. “High Gleason”. The area under the ROCcurve for magnification ×40 is 0.99 and for magnification ×100 is0.93.

[3] W. Allsbrook, K. Mangold, M. Johnson, R. Lane, C. Lane,and J. Epstein, “Interobserver reproducibility of gleasongrading of prostatic carcinoma: general pathologist,” Hu-man Pathology, vol. 32, pp. 81–88, 2001.

[4] C. Sturgis, C. Isoe, N. McNeal, G. Yu, and D. DeFrias,“Papnet computer-aided rescreening for detection of be-nign and malignant glandular elements in cervicovaginalsmears: A review of 61 cases,” Diagnostic Cytopathology,vol. 18, no. 4, pp. 307–311, 1998.

[5] A. Wetzel, R. Crowley, S. Kim, R. Dawson, L. Zheng,Y. Joo, Y. Yag, J. Gilbertson, C. Gadd, D. Deerfield,and M. Becich, “Evaluation of prostate tumor grades bycontent-based image retrieval.” in SPIE Workshop: Ad-vances in Computer-Assisted Recognition, vol. 3854, 1999,pp. 244–252.

[6] Y. Smith, G. Zajicek, M.Werman, and G. P. Y. Sherman,“Similarity measurement method for the classificationof architecturally differentiated images.” Computers andBiomedical Research, vol. 31, no. 1, pp. 1–12, 1999.

[7] K. Jafari-Khouzani and H. Soltanian-Zadeh, “Multiwaveletgrading of pathological images of prostate,” in IEEETransactions on Biomedical Engineering, vol. 50, 2003,pp. 697–709.

[8] G. Begelman, M. Lifshits, and E. Rivlin, “Map-basedmicroscope positioning,” in British Machine Vision Con-



ference, 2004.[9] A. Sussman, A. Demarzo, A. Afework, F. Bustamante,

H. Tsang, M. Silberman, M. D. Beynon, and R. Ferreira,“Digital dynamic telepathology – the virtual microscope,”in American Medical Informatics Association Fall Sympo-sium, 1998.

[10] U. Catalyurek, M. D. Beynon, T. C. C. Kurc, A. Sussman,and J. Saltz, “The virtual microscope,” IEEE Transactionson Information Technology in Biomedicine, vol. 7, pp. 230–248, 2003.

[11] D. Comaniciu, W. Chen, P. Meer, and D. Foran, “Multiuserworkspaces for remote microscopy in telepathology,” inProceedings of IEEE Computer-Based Medical Systems,vol. 2, 1999, pp. 150–155.

[12] S. Olsson and C. Busch, “A national telepathology trial insweden: Feasibility and assessment,” Archives d’Anatomieet de Cytologie Pathologiques, vol. 43, p. 234241, 1995.

[13] J. Z. Wang, J. Nguyen, K.-K. Lo, C. Law, and D. Reg-ula, “Multiresolution browsing of pathology images usingwavelets,” in American Medical Informatics AssociationlSymposium, 1999, p. 340344.

[14] M. Weinstein and J. I. Epstein, “Telepathology diagnosisof prostate needle biopsies,” Human Pathology, vol. 28,no. 1, pp. 22–29, 1997.

[15] R. S. Weinstein, A. Bhattacharyya, A. R. Graham, andJ. R. Davis, “Telepathology: A ten-year progress report,”Human Pathology, vol. 28, no. 1, pp. 1–7, 1997.

[16] R. Stotzka, R. Manner, P. H. Bartels, and D. Thomp-son, “A hybrid neural and statistical classifier system forhistopathologic grading of prostate lesions,” AnalyticalQuantitative Cytology and Histology, vol. 17, no. 3, pp.204–218, 1995.

[17] J. Daugman, “Complete discrete 2-d gabor transforms byneural networks for image analysis and compression,”Acoustics, Speech, and Signal Processing, IEEE Transac-tions on, vol. 36, no. 7, pp. 1169–1179, 1988.

[18] V. Vapnik, Statistical Learning Theory. John Willey andSons, Inc., 1998.

[19] W. Noble, “Support vector machine applications in com-putational biology,” Kernel Methods in ComputationalBiology, pp. 71–92, 2004.

[20] E. Alpaydin, Introduction to Machine Learning. The MITPress, 2004.

[21] R. Kohavi, “A study of cross-validation and bootstrap foraccuracy estimation and model selection,” in IJCAI, 1995,pp. 1137–1145.

[22] L. Wolf and A. Shashua, “Feature selection for unsuper-vised and supervised inference: the emergence of sparsityin a weighted-based approach.” in International Confer-ence on Computer Vision, 2003, pp. 378–384.

[23] S. Tong and E. Chang, “Support vector machine activelearning for image retrieval,” in ACM International Con-ference on Multimedia, 2001, pp. 107–118.

[24] D. Mark, “Neighbor-based properties of some orderings oftwo-dimensional space.” Geographical Analysis, vol. 22,pp. 145–147, 1990.

[25] Y. Keller and A. Averbuch, “Fft based image registration,”in IEEE International Conference on Acoustics, Speech,and Signal Processing, 2002.

[26] W. B. Pennebaker and J. L. Mitchell, JPEG Still ImageData Compression Standard. Norwell, MA, USA: KluwerAcademic Publishers, 1992.

Grigory Begelman received the B.Sc and M.Sc degrees inmathematics and physics from the Moscow Institute of Physicsand Technology, Russia. Currently, he is a Ph.D student in theComputer Science Department at the Technion, Israel Institute ofTechnology. His current research interests are in computer-aideddiagnostics, hyper-spectral microscopy, blind source separation,processing and analysis of microscopic images.

Michael Pechuk received the B.Sc degree in computer sciencefrom Haifa University, Israel. Currently he is an M.Sc studentin the Computer Science Department at the Technion, IsraelInstitute of Technology. His current research interests are inmachine vision and function-based recognition.

Prof. Ehud Rivlin received the B.Sc and M.Sc degrees incomputer science and the MBA degree from the Hebrew Uni-versity in Jerusalem, Israel and the Ph.D from the University ofMaryland, USA. Currently, he is an associate professor in theComputer Science Department at the Technion, Israel Instituteof Technology. His current research interests are in machinevision and robot navigation.

Dr. Edmond Sabo is a Research Associate at Rhode IslandHospital/Lifespan, in the Department of Pathology as well as anAssistant Professor of Pathology at Brown Medical School. Hecame to Rhode Island in 2004 from Israel where he worked asDirector of Surgical Pathology Unit at Carmel Medical Centerin Haifa. Dr. Sabo received his M.D. degree from Universityof Medicine and Pharmacy in Iasi, Romania and completedhis residency in Pathology in Israel. His research subspecialtyincludes analysis of microscopical patterns in tumor tissuesusing chaos and fractal theories, computerized morphometry,Biostatistics and Artificial Intelligence.



a microscopic telepathology system for multiresolution

Documents