Automatic Liver Tissue Section Image Characterization

Fei (Frank) Yang1 and Ayooluwakunmi Jeje2

Abstract Analyzing liver cross-section images is a manualand laborious task, slowing down critical research towardfinding alternative cures for patients with end-stage liverdisease. In this paper, we present methods to automaticallycount hepatocytes cells, count nuclei and classify liver vesseltypes, using input images of cell boundary and cell nuclei basedon a dataset of 21 images. Compared to a trained researcher,the methods are able count cells, segment overlapping nuclei,and classify vessel types, including portal vein, central vein,and bile duct with reasonable precision and accuracy. Futurework includes improving classification accuracy and detectingother cell types.

Index Terms microscopy image processing, cell counting,nuclei segmentation, vessel classification, SVM, Canny edgedetection, Hough transform

I. INTRODUCTIONLiver transplantation is currently the only cure for patients

with end-stage liver disease, yet currently over 14,000 pa-tients are on the liver transplant waitlist awaiting a donor[1]. Alternative treatment options are critically needed forthese patients.

Physicians and scientists contemplating new treatmentmethods face the same major obstacle: understanding themolecular mechanisms by which the proliferation of animportant type of liver cell called hepatocytes is regulated.The Nusse Lab of the Stanford Institute of Stem Cell Biologyand Regenerative Medicine studies this class of specializedcells and recently published work in identifying a populationof hepatocyte stem cells in the liver [2]. A major obstaclein scaling the research is the laborious task of manual cellcounting from tissue section images.

This project leverages image processing and machinelearning techniques to tackle the unique challenges presentedby tissue section images. First, compared to tissue cultureimages, cells in a tissue section image is highly heterogenouswith complex structures and ambiguous boundaries, makingcell counting a more difficult task. Second, tissue sectionimages may contain different types of cells, including hepato-cytes, bile duct epithelial cells, endothelial cells, and immunecells. In this project, we focus exclusively on the hepatocytescells which is the focus of the research. Third, in contrast tomost other cells in our body, hepatocytes can have more thanone nucleus. A final challenge is that in addition to cells, theliver section images also show important features of a liver,including central veins, portal veins, and bile ducts.

There are three distinct goals in the project: accuratelycount the number of hepatocyte cells irrespective to theirstain, accurately segmented clustered nuclei, and accurately

1F. Yang: Department of Electrical Engineering, Stanford University2A. Jeje: Department of Computer Science, Stanford University

classify liver features, including central vein, portal vein andbile duct. This work focuses on developing algorithms toaccomplish each goal and analyzing their performances. Thealgorithm is applied to a dataset provided by the NusseLab, consisting of 21 liver tissue section images, which aremanually labeled by a trained researcher.

This paper is structured as the following: first we willpresent the implementation pipelines for cell boundary seg-mentation, cell nuclei segmentation, and liver vessel typeclassification. Next, the performance of each task usingvarious techniques is analyzed and compared.

II. BACKGROUND

Microscopy has been the tool used by biomedical scientiststo advance our fundamental understanding of life at thecellular level. Advances in modern microscopic techniques,such as the use of fluorescent markers and microscopicsystems with sub-light diffraction limits, lowered the cost ofacquiring images and enhanced the level of details of theseimages [3]. On the flip side, the abundance, heterogeneity,dimensionality, and complexity makes manual image analy-sis a laborious and expensive task. Consequently, automatingthe analysis of these images can help accelerate the progressin cell biology and other related biomedical fields.

There are numerous benefits associated with potentiallyautomating these tasks, based on a survey of existing liter-ature. Automating these tasks using image processing andmachine learning may result in significant time savings aswell as reduce measurement variability due to operator-dependent and parameter-sensitive conditions [3]. Addition-ally, automation has the potential of quantifying numerouscell topology characteristics that are difficult or expensive todo so manually [4].

There have been many papers published on developingautomated methods for segmenting and counting cells inmicroscopy images. Common approaches range from classi-cal image processing techniques, such intensity thresholdingand morphological operations, to modern approaches such astensor voting schemes, neural networks, and Markov randomfields [3]. Due to the astounding variety of the differenttypes of cell analysis problems, there are just as many cellsegmentation methods that combines existing techniques andapply it to a new problem, and our project is no exception.

III. METHOD

The high-level image processing pipeline is illustrated infigure 1. Each step in the pipeline is described in detail inthe following sections.

Fig. 1: Image processing pipeline for cell boundary segmentation, cell nuclei segmentation, and liver features classifications.

A. Cell Boundary Segmentation

a) Edge Detection: The cell boundaries of a gray-scalecell boundary input image are detected using the Canny edgegradient. The Canny edge detector is chosen over other edgedetection methods such as Prewitt and Sobel edge detectionbecause the Canny edge detector calculates gradients basedon derivative of a Gaussian filter and therefore is lesssusceptible to errors due to ambiguous cell boundaries. Asample image resulting from Canny Edge detection is shownin figure 2 .

Fig. 2: A sample cell boundary output image using Cannyedge detection.

b) Morphological Operations: A few morphologicaloperations are performed to the resulting edge image toenhance the salient cell boundaries so that distinct cellregions can be identified. First, the image is dilated usinga line structure of 2 pixel length in both the horizontal andvertical direction. Second, the image is negated to so thatonly the cell boundary information is present. Next, smallregions are flood filled to reduce noise. Finally, the cellregions are smoothed using opening and closing operationswith a 1 pixel diamond structuring element. Sample output

images from each of the morphological operations is shownin figure 4.

c) Region Segmentation: Following edge detection andmorphological operation that enhance the boundary of cellstructures, regions are segmented by getting 8-connectedneighborhoods. While there are many techniques in the tech-nical literature for extracting regions in microscopy images,including the principal component analysis approach [5],[6] or the graph mining process [7], segmenting regionsbased on connectivity matches the identification pattern ofa researcher thereby providing satisfactory results. A sampleimage of segmented regions, including potential cells, non-cellular regions, and veins, are distinctively colored in agradient in figure 3.

Fig. 3: A sample image of segmented regions, includingpotential cells, non-cellular regions, and veins, each distinc-tively colored in a gradient.

B. Cell Nuclei Segmentation

a) Pre-processing: An input gray-scale image of thenuclei is preprocessed using global image thresholding viaOtsu's method that is adjusted by a small tuning parameterto reject ambiguous regions in the image.

Fig. 4: A sample image illustrating the following morphological operations: a) dilated gradient mask, b) binary image withsmall holes filled in, c) smoothed cell boundaries using opening and closing.

b) Edge Detection: Following the binarization of theimage, Canny edge detection is used to detect nuclei edgesin the image. Canny edge detection also works well for nucleisegmentation due to its robustness to noise.

c) Morphological Operation: Following edge detec-tion, the nuclei boundaries are enhanced by dilating theimage using a line structuring element of 3 pixel length inthe horizontal and vertical directions.

d) Hough Transform: To detect the circular shapedhepatocyte cell nuclei, circular Hough Transforms emergedas the best option among other techniques attempted whichinclude SIFT keypoint, SURF keypoint, and morphologicaloperation. The performances of each operation are comparedin the results section. MATLAB implements the circularHough Transform via the function imfindcircles, which takesin the following parameters:

Radius: Circle radius is the approximate radius of thecircular objects to detect, specified as a scalar of anynumeric type.

Sensitivity: Sensitivity factor is the sensitivity for thecircular Hough transform accumulator array. Increasingthe sensitivity factor leads to detection more circularobjects and a potential increase in false detections.

EdgeThreshold: Edge gradient threshold sets the gradi-ent threshold for determining edge pixels in the image.Decreasing the threshold leads to detection of morecircular objects with both weak and strong edges.

Based on testing with training images the circular HoughTransform is robust to the EdgeThreshold parameters, be-cause the Hough Transform is applied on an edge mapinstead of a raw input image. Also, there is little variancein the optimal sensitivity parameter among images, since themicroscopy images are taken in uniform and controlled light-ing conditions. However, the algorithm is highly sensitive tothe radius ran