castle: cell adhesion with supervised training and learning … · 2020. 6. 2. · castle: cell...

CASTLE: Cell Adhesion with Supervised Training and LearningEnvironment

S G Gilbert1,2,†, F Krautter3, D Cooper4, M Chimen5, A J Iqbal3 and F Spill1,∗

1 School of Mathematics, University of Birmingham, Edgbaston, Birmingham, B15 2TT, UK2 EPRSC Centre for Doctoral Training in Topological Design, School of Physics, University ofBirmingham, Edgbaston, Birmingham, B15 2TT, UK3 Institute for Cardiovascular Sciences, College of Medical and Dental Sciences, University of Birmingham,Edgbaston, Birmingham, B15 2TT, UK4 Centre for Biochemical Pharmacology, William Harvey Research Institute, Bart’s and the London Schoolof Medicine, Charterhouse Square, London, EC1M 6BQ, UK5 Institute of Inflammation and Ageing, Birmingham University Research Labs, Queen Elizabeth Hospital,Mindelsohn Way, Birmingham, B15 2WB, UK

E-mail: †[email protected], ∗[email protected]

Abstract. Different types of microscopy are used to uncover signatures of cell adhesion and mechanics.Automating the identification and analysis often involve sacrificial routines of cell manipulation such as invitro staining. Phase-contrast microscopy (PCM) is rarely used in automation due to the difficulties withpoor quality images. However, it is the least intrusive method to provide insights into the dynamics ofcells, where other types of microscopy are too destructive to monitor. In this study, we propose an efficientworkflow to automate cell counting and morphology in PCM images. We introduce Cell Adhesion withSupervised Training and Learning Environment (CASTLE), available as a series of additional plugins toImageJ. CASTLE combines effective techniques for phase-contrast image processing with statistical analysisand machine learning algorithms to interpret the results. The proposed workflow was validated by comparingthe results to a manual count and manual segmentation of cells in images investigating different adherent celltypes, including monocytes, neutrophils and platelets. In addition, the effect of different molecules on celladhesion was characterised using CASTLE. For example, we demonstate that Galectin-9 leads to differencesin adhesion of leukocytes. CASTLE also provides information using machine learning techniques, namelyprincipal component analysis and k-means clustering, to distinguish morphology currently inaccessible withmanual methods. All scripts and documentation is open-source and available at the corresponding GitLabproject.

Keywords : machine learning, image analysis, cell adhesion, cell morphology, galectins, ImageJ, Ilastik.

Submitted to: J. Phys. D: Appl. Phys.

.CC-BY 4.0 International licenseavailable under awas not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made

The copyright holder for this preprint (whichthis version posted June 3, 2020. ; https://doi.org/10.1101/2020.06.02.130708doi: bioRxiv preprint

https://doi.org/10.1101/2020.06.02.130708http://creativecommons.org/licenses/by/4.0/

Automated Image Analysis of Cell Adhesion 2

1. Introduction

The function of almost all cell types, includingleukocytes, platelets, and cancer cells, criticallyinvolves a number of mechanical processes[1]; theseinclude the regulation of cell-cell and cell-matrixadhesion, flow sensing and changes in morphologyduring cell migration[2]. A particularly important case,where adhesions need to dynamically change on shorttime scales, involve the adherence of blood cells. Forexample, leukocytes adhere to the endothelium andtransmigrate out into inflamed tissues, and plateletsactivate and adhere to a substrate or to each otherduring wound closure. To investigate the effect ofspecific adhesion molecules on leukocyte behaviour,e.g. those appearing on the endothelium, biologistsoften employ flow based adhesion assays wherecells adhere to coated substrates, or to endothelialmonolayers. Subsequently, cells may migrate on thesubstrate or monolayer, or form aggregates. Toidentify adhesive states or morphological changesduring migration, experimentalists frequently analysea large number of cells under different conditions.

Humans are highly perceptive in identifyingpatterns in complex biological images. However,manual counting of the same object over hundreds ofimages is exhausting and time-consuming, increasingerrors in the image analysis. Developments incomputer power and algorithms therefore have greatpotential to allow more efficient routes to automaticallyprocess and analyse masses of data with limited humanintervention if a suitable methodology can be put intoplace. Moreover, such automation can help to extractnew quantitative features from the images.

High-throughput microscopy is the acquisitionand processing of large volumes of data by automatingsample preparation and data analysis techniques[3]. The methodology for these types of study arereferred to as workflows. Workflows utilise the bestcombination of image analysis techniques, specific toa researcher’s problem, and follow the steps to ensurethe method is repeated exactly for all samples. Theprimary objective of the methods chosen is in theiraccuracy for which features of an image are convertedinto meaningful quantitative data [4]. Once validatedas accurate, designing a workflow with flexibility forsimilar research enables a larger impact. To beconsidered more widely usable by the community ofbio-imaging researchers they need to excel in certainattributes. Carpenter et al. [5] list the criteria thatsuch research software should meet. The criteriacan be summarised into five key attributes: User-friendly, Modular, Developer-friendly, Validated andInteroperable. All of these must be considered for a

successfully designed workflow.

In our study, we are considering the imagesacquired by PCM. Since most thin biological samplesare optically transparent in visible light, amplitudeinformation does not provide good contrast forimaging. However, even these transparent samplesprovide a significant optical phase delay. PCMutilises both types of information in the transmittedlight, whereas conventional bright-field imaging onlymeasures the amplitude. Full details of quantitativephase imaging techniques can be found elsewhere [6].

Once the images are uploaded to a digital form,the automated workflow processes the image into itsconstituent parts. There are many techniques inimage processing for pixel classification [7, 8]. Inaddition, a methodology to interpret the vast amountsof statistics produced is necessary to prevent a backlogof valuable information going unused or slowing downthe research. Again, techniques in machine learningcan help automate the process with little humanintervention to increase both speed and objectivity[9, 10].

In the present study, we test our workflow ondifferent types of recently acquired data sets: wequantify adhesion and morphology of peripheral bloodmononuclear cells (PBMCs) and polymorphonuclearleukocytes (PMNs) in dependence on galectins, andplatelet adhesion depending on von Willebrand factor(vWF) with stimulation by a platelet agonist,Adenosine Diphosphate (ADP). Galectins are a familyof ß-galactoside binding proteins which have a rangeof immunomodulatory functions [11]. More recently,Galectin-9 (Gal-9), a tandem repeat type galectin, hasbeen proposed to play a role in modulating leukocytetrafficking [12]. However, its precise role in thisprocess, in particular the way it regulates adhesion,remains elusive. Similarly, platelets need to adhereto substrates and to other platelets to perform theirfunction. ADP is a known activator of plateletsthat switches on their GPIIb/IIIa receptors [13, 14],allowing them to bind to vWF that is present in theendothelium and subendothelial tissue [15]. Studyingthe response of adherent cells to stimuli such as ADP,or proteins such as vWF or Gal-9 that are involvedin cell adhesions, is therefore critical to understandingcell adhesion.

This paper introduces a novel workflow to analyseimages of adherent cells for images that are commonlyobtained in experimental setups modelling bloodcell recruitment within the vasculature towards sitesof infections, chronic inflammation or wounds andcoagulation. We demonstrate that our workflow iscapable of identifying changes in cell adhesion as well





as morphology depending on different experimentalconditions; however, these adhesive and morphologicalchanges are very common to many cell types involvedin recruitment and migration assays and will thereforebenefit a wide range of researchers in these areas.

In the following we will first describe the algo-rithms and techniques in image processing that areimplemented by CASTLE, leading to the subsequentImageJ plugins that implements the workflow devel-oped in this paper. Then, we explain the methodsused in evaluating the results. Finally, we gauge theperformance of the designed workflow and analyse theresults produced in relation to our biological applica-tion. Specifically, we evaluate the precision and re-call of our automatic segmentation, and demonstratehow our high throughput analysis can be used to ex-tract new information from data through the machinelearning techniques: principal component analysis andk-means clustering.

2. Methods and Materials

In this section, the methodology of the investigation isexplained. This includes details of the final automatedworkflow, image acquisition, and the validation processused to gauge the performance.

Images are commonly stored as a matrix withentries for each pixel. An entry’s value represent thebrightness intensity. In our study we consider 8-bitgrayscale images. This means each pixel varies inshade from 0 to 255, with 0 representing absoluteblack and 255 absolute white. The objective of imageprocessing is to prepare an image for analysis, inour case by obtaining a binary image that identifiesexactly the regions of interest (ROI). A binary imagehere is one where we assign pixels part of the ROIwith a value of 1, and other pixels with a value of0. For images with more than one type of ROI,multiple binary images are created and organised intochannels for the respective ROI. To convert our imageinto the desired form, we must first pass our matrixunder a series of transformations to reduce noise andto accurately identify the pixels truly part of a ROI.These algorithms of transformation broadly fit underthree classes: pre-processing, segmentation and post-processing. For each we will identify the obstacles inthe acquired data; then explain the techniques used toresolve them.

Figure 1 shows a typical image used in thisinvestigation. This image is obtained from a co-culturesystem where peripheral blood mononuclear cells(PBMCs) and platelets flow over an adhesive substrate.Note that the PBMCs which are initially adheredappear almost perfectly spherical and very bright,while the PBMCs in firm adhesion are comparatively

dark with highly heterogeneous shape. Because ofthese differences, the stages of the cell are often referredto as phase bright and phase dark for initial adhesionand firm adhesion, respectively.

2.1. Pre-processing

Pre-processing uses routines to convert raw, heteroge-neous images into a set of standardised images withas few unwanted artefacts and accentuate features forseparation in segmentation. The techniques of PCMdo introduce certain artefacts in an image which couldbe incorrectly identified as a ROI. During this stage weconsider techniques that are used to normalise image-to-image brightness and account uneven illumination,which images like Figure 1 and Figure 2 contain.

Image-to-image brightness heterogeneity: To begineradicating the various artefacts in the image, we firstnormalise the pixel values so later processing tech-niques are repeatable across images captured withvarying illumination. The average pixel intensityacross a set of images, in our case sampled from thetraining set, and the individual difference in meanpixel intensity in each subsequent image is corrected byadding the difference to every pixel in an image so thatthe mean becomes that of the set. This works well forimages that are considered reasonably similar. How-ever, an image with a high proportion of ROIs whichare either really bright or really dark will skew theset’s mean. This drawback could reduce the range ofthe pixel intensities in other images as negative valuesare made equal to 0 and those over 255 reduced to themaximum and so previous variations in pixel intensityare lost, which is also known as contrast.

Uneven image illumination: Now, we address theuneven illumination of the image and the ways toaccount for it. We can observe in Figure 2 thatthere exists an uneven illumination of the image fromone side of the image to the other. We consider atechnique described by Russ [8] using an averagingkernel operation with an array of large dimensionsfollowed by image arithmetic.

The averaging filter is an iterative application ofa kernel operation. The kernel operation is usually asquare array of dimension (2m+ 1)× (2n+ 1), so thatthere is a central pixel, with each cell containing aninteger weight. The array is put over an initial pixelat its centre. The centre and neighbouring pixels arethen multiplied by the weight overlaying it. The resultis summed and divided by the sum of the weights to fitthe same 8-bit scale. This is then put as the value of thecentral pixel in the output image. This is repeated forevery pixel in the input image. This can be expressed





Figure 1: A cropped area of a representative image that can be analysed through our workflow. The imageshows co-cultures with peripheral blood mononuclear cells (PBMCs) and platelets on adhesive substrates, wherethe PBMCs and platelets appear in different adhesive states. The image has been labelled with a number of keyfeatures identified. First, the cells we are identifying are considered to be at two different phases of adhesion,rolling adhesion (red) and firm adhesion (blue). Moreover, cells may be in a transition state between these twophases (green). All cells produce the characteristic ‘halo’ as an emanating set of bright pixels surrounding thecell. However, sometimes the ‘halo’ effect is overcompensated and can cause a following darkness in pixels (greenarrow). The uneven illumination is not obvious in this image but is normally more prominent, such as in Figure2. Irregular objects in the image are also pointed out with arrows. The orange arrows identify platelets adheredto the surface. The magenta arrow identifies a disruption in the coating of the protein causing a significant phasedelay similar to a halo.

for an input image f , transformed to g by the kernel ωof dimension a× b as:

g(x, y) = ω ∗ f(x, y) (1)

=

∑as=−a

∑bt=−b ω(s, t)f(x− s, y − t)∑s=a−a

∑bt=−b ω(s, t)

(2)

for all pixels (x, y) in f .

For pixels close to the boundary the array willextend outside the perimeter of the image. To accountfor this a simple method is to mirror the pixels from the

boundary up to the 2m pixel in the horizontal and the2n pixel in the vertical as padding around the image.

The techniques described by Russ is to take theaverage of a large area of the image, so that extremeintensities from individual ROIs do not skew the kerneloperation, this is known as the Mean kernel. However,the area cannot be too large so that the overalldifferences in illumination are not lost in the output.The array of weights in ω need all pixels in the arrayto have an equal contribution, so for a 3 × 3 region we





Figure 2: A cropped area taken from another representative image with the values of the pixels plotted againstthe path across the width of the image (yellow). Here we can see an uneven illumination from a light source,where the background illumination decreases in brightness the further left we go.

have:

ω =

1 1 11 1 11 1 1

(3)By applying a similar array with larger dimensions,fitting the properties as mentioned before, results inan output showing the changes in the backgroundillumination. Then, the original image is divided bythis background to account for the illumination acrossthe image.

2.2. Segmentation

Segmentation is the identification of pixels as eitherassociated with a ROI or not. The segmentationprocess works most efficiently on images which havehad noise and other imperfections removed to allowfor the most accurate separation of pixels to becategorised. Techniques for this stage may differ inthree key areas: detection efficiency, user-operabilityand computational cost. The first of these relates tothe success in correctly identifying the ROI. The secondrelates to the amount of user knowledge needed inbeing able to use the routine. The last analyses the

time taken to execute once the program is running.These were the properties considered when designingthis automated workflow. We first describe thealternative techniques considered for this methodologybefore describing the active learning segmentation usedin the final methodology.

Thresholding: A key component of segmentation isthresholding. Thresholding involves an input ofminimum or maximum pixel intensities which a ROIhas been identified to have. If the information forsetting such limits is known, then the process issimple: pixel-by-pixel, an if-statement transforms an8-bit image to a binary image with the pixels partof the ROI as 1 and the rest 0. However, in manycases the best thresholds to choose are unknown, orthey are required to be automated. There are manypublished algorithms that try to resolve this problem asthey depend on the type of image histograms producedafter pre-processing [16, 17, 18]. These have thenbeen integrated into workflows to form some of thecurrent techniques used for segmentation, known hereas automated thresholding workflows.





Automated thresholding workflows: Usually a singleor multiple threshold value is unable to differentiatebetween ROI and other artefacts that may have pixelsof similar intensity. Therefore, other techniques areused to utilise common features in a set of images toaid the classification of a true ROI.

Two very common effects in phase contrast imagesare the characteristic ‘halo’ corresponding to therefractive index and thickness values between the celland the surrounding medium.

The halo effect is exploited in the work ofSelinummi et al. [19] and Flight et al. [20] in densecolonies of epithelial cells. The intuition behind bothtechniques is by applying a Gaussian blur or Meankernel operation for a variety of different dimensionsand then monitor the variation in pixel intensities.When the blur is applied, the halos are already closeto maximum pixel intensity so do not vary but thedarker region of the epithelial cell quickly becomebrighter as the surrounding halo encroaches on theROI. Therefore, the image is transformed to one withhigher pixel intensity representing higher variations.A subsequent application of Otsu thresholding [16]determines the threshold value to separate the halo,and thus background, from the ROI. This wascombined with fluorescent dyes identifying constituentparts of the ROI, such as the nucleus, and DiscreteMereotopology to validate the segmented area as aROI [21]. However, the dependency on densely packedcell colonies made these techniques unsuitable for theimages captured here.

Active learning segmentation: Active learning is atype of machine learning method where a useriteratively adds training data to a supervised learningalgorithm until the desired classification is achieved.In image analysis, a classifier is a set of rules basedon active learning methods to identify the constituentparts of images not originally trained upon. Theconstituent parts are the different types of ROImentioned before, and are referred to as classes inclassification. The process can be applied in ourworkflow by creating a classifier from a random sampleof the data and then using it during our segmentationstep to identify the different classes of ROIs in the restof the images.

There are many active learning software toolsfor segmentation already available in the bio-imagingcommunity [3, 4]. The majority of these use analgorithm called a random decision forest as thefoundation for the classifier [22]. This is wherebya multitude of decision trees are constructed duringtraining. Then, when applied to a new object toclassify, the output class is the modal class from allof the decision trees.

Ilastik is one such interactive learning andsegmentation toolkit [23]. The Pixel Classificationmodule calculates 35 classification features from animage matrix to interpret information about intensity,edge, texture and orientation. These help producea classifier to be applied to other images to providea prediction of the designated class for every pixel.Adjacent pixels of the same class then form ourROIs. So, unlike before, we are able to train theclassifier to the kinds of images specifically collectedwith our research. In addition, the user-friendlygraphical user interface (GUI) is well-suited for easyoperation and limited user knowledge. Finally, thereis some additional computational cost with the addedcomplexity of the algorithm although this was stillsignificantly faster than the current manual approach,mentioned later in the results.

2.3. Post-processing

Post-processing describes the type of techniquesused to sieve through a binary image produced bysegmentation to remove ROI that are in fact irregularparticles. An example are the platelets pointedout in Figure 1. These cells could be detected assmaller versions of cells as they undergo respectivestages of adhesion albeit on a smaller scale. Suchtechniques range from counting only those ROIsthat are above a minimum and/or maximum numberof pixels to machine learning algorithms separatingunusual artefacts by measuring multiple morphologicalfeatures. In our final workflow, a minimum and/ormaximum threshold for the number of pixels in aROI is set as the method used in post-processing.This is because the cells being identified follow fairlyhomogeneous measurements in size for the adheredarea or are significantly distinct from particles otherthan that being considered.

2.4. Analysis

We now demonstrate the effectiveness of our approachby analysing images with adherent cells. We firstquantify the number of cells depending on theiradhesion status. This is done for different experimentalconditions; e.g. by varying concentrations of moleculesthat are suspected to affect cell adhesion. Afterwards,we utilise a by-product of the automated workflow andbegin to quantify the morphology of the cells at thedifferent experimental conditions.

As the images are segmented pixel-by-pixel, towhether a pixel belongs to the ROI or not, we havea representation of the adhered surface area of eachcell. As seen in Figure 1, during initial adhesion thecell remains in a highly circular form, however duringfirm adhesion the cell migrates on the surface looking





to pass through. The shape of the cell can also beaffected by surface proteins and can therefore provideuseful information about the impact of molecules onthe cells. Thus, an interest is to categorise themorphology of the firm adhesion cells to investigatethe effect of different levels of molecules in this process.We describe the machine learning techniques used toanalyse the morphology of phase dark cells.

Principal Component Analysis: Principal ComponentAnalysis (PCA) is a methodology to reduce thedimensionality of a data set in which there are a largenumber of interrelated variables. The methodology isexplained in more detail by Jolliffe [24] or Goodfellowet al. [25]. Here each ROI identified has 10 valuescalculated from shape descriptor formulae, refer to A.1for exact formulae. The result is a representationwhose elements have no linear correlation with eachother. The order depends on the variance in thatdimension. The representations, called principalcomponents, provide a summary ordering the mostinfluential factors in describing morphology. Here itwas reasonable to not reduce the dimensionality of thedata, however the technique is useful to to display themost significant contributions to the variability for auser to interpret. The relationship between varianceand information is that, the larger the variance carriedalong the axis, the larger the dispersion of the datapoints along it. Therefore, if the dispersion in thatdimension is larger, the particles are more likelydistinguishable.

k-means Clustering: The k-means clustering algo-rithm divides the data set into k different clusters ofdata-points that are considered close to each other.First, each ROI is represented by a vector with eachentry corresponding to the value of a feature. Thealgorithm works by initialising k different centroids tosome ROI. Afterwards, the algorithm assigns each ROIto the cluster with minimal Euclidean distance in thefeature space to the centroid of that cluster. Then,each centroid µi is updated to the ROI closest to themean of all the training examples assigned to cluster i.The algorithm alternates between the two steps untilconvergence or the maximum number of iterations isachieved. The methodology is explained in more detailby Lloyd [26].

Statistical testing and inference: In our investigation,we are looking to classify the morphology of the cellsof firmly adhered cells depending on the differentconcentrations of molecules. Therefore, using thenull hypothesis that groupings are independent ofthe molecule we are investigating, we can test thestatistical significance of the outcome in the analysis.

If proven significantly different, then the shapedescriptors contributing the most to the principalcomponents from the PCA indicate the features mosteffected by these differences.

2.5. Validation

The success of classification was measured by theworkflow’s precision, recall and combined score of thetwo as described by Powers [27]. This is similar to thevalidation process used by many publications of thearea ([19], [28], [29]) and hence gives a comparativestatistic between different automated workflows. Thestatistics necessary for these calculations describeall the possible outcomes for a classifier. Thesepossibilities are:

• true positive (TP) – correctly identifies theadhered cell of interest as a ROI.

• true negative (TN) – correctly does not identifynoise as a ROI.

• false positive (FP) – incorrectly identifies noise asan ROI.

• false negative (FN) – incorrectly identifies adheredcell as not a ROI.

Classification precision (p), or positive predictivevalue, indicates the fraction of objects correctlyclassified as a ROI out of the total number detected,and is calculated as:

p =TP

TP + FP. (4)

Classification recall (r), or sensitivity, indicatesthe fraction of objects correctly classified as a ROI outof all adhered cells, and is calculated as:

r =TP

TP + FN. (5)

The proportions for r and p are expressed aspercentages.

The F1 score is the harmonic mean of precisionand recall:

F1 = 2

(pr

p+ r

)(6)

An F1 score of 0 indicates no agreement for theautomated and manual analysis, while 1 indicatescomplete agreement. The final F1 score, precision andrecall is calculated as the average of the scores fromboth phases.

Next we investigate where the automated work-flow can be improved with image-by-image compar-isons of the manual and automated number of ROI.We can categorise the incorrect detection of adheredcells into four categories: missing, noise, merged andsplit. Missing describes those cells that are not de-tected by CASTLE but counted by a human expert.





Noise are the particles identified as a ROI by the au-tomated workflow but are not counted by the expert.Merged detections are the number of adhered cells thatare counted as being part of another detected ROI. Fi-nally, a split observation is a single adhered cell that isdetected as two distinct ROIs.

2.6. Image acquisition

Here we describe briefly the specific experimentsperformed to acquire the images used to validate ourworkflow.

Blood Sample Collection: Blood samples were ob-tained from donations of volunteers who have givenwritten informed consent. Approval was obtained fromthe University of Birmingham or Queen Mary Univer-sity of London Local Ethics Review Committee.

Gal-9 experiments: A flow chamber assay was usedto investigate the adhesion behaviour of PBMCs andpurified PMNs for different amounts of Gal-9 underphysiological flow conditions.

Channels in an Ibidi chamber were coated witheither 20, 50 or 100 µg/ml Gal-9 or 1.5 % bovine serumalbumin (BSA) in Phosphate-buffered saline (PBS)as the control. During the experiment, the isolatedPBMCs or PMNs were perfused through the channelsat a concentration of 1 × 106 in PBS containing Ca2+and Mg2+ for 4 min before a wash out period of 1 min.A shear wall stress of 0.1 Pa was applied throughoutthe perfusion to mimic physiological conditions.

After the wash out period, a series of images wastaken under continuous flow across the channel. Thecell number adhered per mm2 and per 1×106 cells wascalculated from the series of images.

ICAM-1 experiments: A flow chamber assay was usedto investigate the adhesion behaviour of purified PMNson ICAM-1 under physiological flow conditions.

Channels in an Ibidi chamber were coated with10 µg/ml ICAM-1-Fc. During the experiment, theisolated PMNs were perfused through the channels ata concentration of 1×106 in PBS containing Ca2+ andMg2+. PMNs were allowed to enter the chamber andthen flow was suspended for 5 mins to allow adhesion ofcells to ICAM-1-Fc. Flow was then restarted at a wallshear stress of 0.1 Pa, which was applied throughoutthe perfusion to mimic physiological conditions.

After the wash out period, a series of images wastaken under continuous flow across the channel. Thetotal cell number adhered was calculated from theseries of images.

vWF experiments: A flow chamber assay was used tomeasure adhesion of platelets to von Willebrand Factor(vWF) under physiological flow conditions.

Glass microslides were coated with 0.1 mg/mlvWF and blocked using PBS containing 2% BSA.During the experiment, a flow rate of 0.8 ml/min wasmaintained to give the desired wall shear stress of 0.1Pa. Anti-coagulated human whole blood was perfusedfor 2 minutes over vWF. The microslides were thenwashed with PBS 0.1% BSA with or without 30 µmol/lAdenosine Diphosphate (ADP), a platelet activatingagent.

After the wash out period, a series of images andvideos were taken under continuous flow across themicroslide. Platelet coverage was calculated as thenumber of cells adhered. Phase bright platelets wereclassed as initially adhered platelets and phase darkplatelets were classed as activated platelets.

2.7. Image processing and analysis using CASTLE

The automated workflow is visualised in Figure S1.The batch processing was designed as a script in theImageJ Macro language and implemented as a pluginin Fiji is Just ImageJ 2.0.0 [30]. The Mean and ImageCalculator plugins were used in Pre-processing [31].Training of the classifier and Segmentation was carriedout using Ilastik 1.3.2post2 [23]. Post-processing anddata collection for analysis were completed using theAnalyse Particles... plugin [31]. Analysis was carriedout in R 3.6.1. [32]. The final automated workflow isavailable as a collection of open-source plugins on theCell Adhesion with Supervised Training and LearningEnvironment (CASTLE) GitLab project, refer toSection 5. The computer used in the implementationof the CASTLE plugins had an Intel(R) Core(TM) i7-9850H CPU at 2.60GHz with 16GB RAM.

3. Results

We now apply our workflow to several data sets.We focus most of our discussion to the analysis ofPMNs and PBMCs depending on concentrations ofGal-9, since these data sets are the most challengingto the algorithms and involve different cell types withquantitatively varying concentrations of molecules. Wethen demonstrate that our workflow can similarly beapplied to the same cell types stimulated with othermolecules (here, ICAM-1), or very different cell types(here, platelets).

3.1. Validation of workflow

Validation of cell count: The automated workflow,CASTLE, was applied to the images from an ongoingstudy of the role of the protein Gal-9 on leukocyte





(a) CASTLE validation with no Transition class duringtraining.

(b) CASTLE validation with a Transition class duringtraining.

Figure 3: Validation of the automated workflow against manually counted and verified results from the studyinto the role of the protein Gal-9 on leukocyte adhesion. (a) CASTLE validation with no Transition class duringtraining. (b) CASTLE validation with a Transition class during training. One image from both analyses wasexcluded after being identified as anomalous. The stacked bar chart shows, of all the manually counted cells,how the automated workflow performed, including how mis-identified cells of the automated workflow effectedperformance as a percentage of the true number of cells.

(a) CASTLE validation with no Transi-tion class during training.

(b) CASTLE validation with a Transi-tion class during training.

(c) CASTLE validation with Transition class duringtraining and the anomalous image removed fromanalysis.

Figure 4: Comparison of automatic to manual detection of adherent phase bright cells for each image in thestudy into the role of the protein Gal-9 on leukocyte adhesion. (a) CASTLE validation with no Transition classduring training. (b) CASTLE validation with a Transition class during training. (c) CASTLE validation withTransition class during training and the anomalous image removed from analysis. The regression line (blue,dashed) shows a prediction for the detected counts across the range of cells in any one image. The line of perfectagreement (black, solid) is the desired outcome with the manual count equal to the automatic count.





(a) CASTLE validation with no Transi-tion class during training.

(b) CASTLE validation with a Transi-tion class during training.

(c) CASTLE validation with Transition class duringtraining and the series of PBMC BSA imagesremoved from analysis.

Figure 5: Comparison of automatic to manual detection of adherent phase dark cells for each image in thestudy into the role of the protein Gal-9 on leukocyte adhesion. (a) CASTLE validation with no Transition classduring training. (b) CASTLE validation with a Transition class during training. (c) CASTLE validation withTransition class during training and the series of PBMC BSA images removed from analysis. The regression line(blue, dashed) shows a prediction for the detected counts across the range of cells in any one image. The line ofperfect agreement (black, solid) is the desired outcome with the manual count equal to the automatic count.

adhesion. For each of the 5 different levels of proteinexposure, we obtained a sample of 7 different images.One image from each protein exposure was chosenfor the classifier to be trained upon. After theautomated workflow was applied, the plugin tookunder 11 minutes for all 35 images to be processed(see Table S1). One image was excluded from thevalidation after statistical analysis identified the imageas an anomaly compared to similar images of the sameprotein exposure (see Figure S2). The final F1 scorewas 87.0%, with a 81.2% classification precision andan 93.7% classification recall.

The final validation scores were the result ofintroducing a third ROI for classification. This wasidentified in Figure 1 as a cell transitioning from initialadhesion to firm adhesion, which in manual countswere associated with the latter but were consideredstill too bright in the automated workflow. Before thenew class was introduced, the F1 score was 86.2% witha classification precision of 89.9% and a classificationrecall of 82.8%.

Besides the increase in F1 score for the count,the introduction of the new class was preferred asthe following analysis into morphology depended oncorrectly identifying all of the cell that forms itsshape. Using the same classifiers, a validation onall the classified pixels was carried out on a sampleof 5 images, one from each of the different cell typeand protein exposures. The 5 images were segmentedmanually by an expert to identify the phase dark cells.The same 5 images were processed by CASTLE. The

images were then compared pixel-by-pixel. Before theintroduction of the Transition class during training, theF1 score was 69.6% with a classification precision of85.0% and a classification recall of 59.0%. The F1score remained similar at 69.3% but with a higherclassification precision of 89.2%, the outcome preferred,and a classification recall of 56.7% .

Figure 3 shows the analysis carried out on theperformance of the automated workflow compared tothat counted by an expert. A perfect identificationof the number of cells would result in the percentageof correctly identified cells in the respective phasesequal to 100% of the true count. We see acrossthe phases of cells identified that the majority ofincorrect counts are due to noise and missed cells.Noise adds to the total count by leading to falsepositives, while missed cells reduce the final count.Figure 3b confirms that the introduction of a class fortransitioning cells significantly reduces the number ofincorrectly classified cells (i.e. improving the precision)and the number of cells missed (i.e improving therecall) compared to the classification based on twoclasses in Figure 3a.

Similar analysis for the validation of CASTLE wascarried out on the other studies, one on leukocytesexposed to the protein ICAM-1 and another onplatelets exposed to the protein vWF. The final F1score was 87.0% and 80.7%, respectively. Similarobservations were made to that already discussed (seeTables S2, S3 and Figures S3, S4). This time theiterations of training were carried out to demonstrate





the application of CASTLE. First, we validated howCASTLE processed the images using the final trainingused in the study of the role of the protein Gal-9 onleukocyte adhesion. Then, we validated how CASTLEprocessed the images with an additional image from thenew data set, respectively. The rationale was to showhow CASTLE may be applied to new data sets: byadding to the original data set of a similar study. Theresult is a more thorough training due to more imagesidentifying the variability in cell appearance yet not atthe expense of the researcher’s time, who would haveotherwise trained the data themselves.

Comparison to other methods: To confirm that theperformance of CASTLE was an improvement to otheravailable workflows, we compared the software to acomparable plugin in ImageJ/Fiji. The plugin iscalled Trainable WEKA Segmentation (TWS) [33] andforms part of the Waikato Environment for KnowledgeAnalysis (WEKA) collection for data mining withopen-source machine learning tools [34]. Using thesame pre-processing and post-processing techniques asCASTLE on the same 35 images, TWS produced a finalF1 score of 63% with 48% precision and 95% recall (seeTable S1). TWS identified a similar number of correctcells as CASTLE. However, for this data set, TWSseemed to mis-identify as many objects which increasedthe final number detected by about twice as many forboth phases of cell (see Figure S5). Also, the time torun TWS to batch process all 35 images took 6 timeslonger than the comparable application by CASTLE(see Table S1). Therefore, the CASTLE workflowwould appear to be an improvement in accuracy andspeed than what is currently available.

Distinguishing cells types and experimental conditions:In addition to the total count across all of the images,an investigation into the performance of the programfor the full range of cells in a particular image ispresented in Figures 4 and 5. These plots displaythe number of cells counted manually against thosedetected by CASTLE. A line of least square regressionis drawn to predict the program’s performance acrossthe range of cell numbers in comparison to another linefor what would be considered a perfect count, wherethe detected number of cells equal those identified bythe expert.

Figure 4 is the series of least regression plotswhen comparing the performance of CASTLE to amanual analysis when confronted with the images ofour study for phase bright leukocytes. Figure 4ais the phase bright analysis for all of the imageswhen CASTLE is trained without a Transition class.However, the performance of the program was seen tostill be detecting more cells which would be considered

as transitioning from phase bright to phase dark, whichan expert would consider as phase dark in their counts.After introducing a Transition class in the training,we see in Figure 4b an improvement in the accuracyreflected by the closer alignment of the regression lineto the line representing a perfect count. Nonetheless,we can see that an image with PBMCs exposed to thecontrol protein BSA has a significantly larger numberof cells detected in comparison to the other 6 imagesof the same experiment. On investigation of this groupof images, an image appeared to have many irregularparticles seemingly trapped under the surface of theprotein which induced enough of a phase difference tocreate the ‘halo’ effect of adhered cells (see Figure S2).After excluding the anomalous images from the inputdata we produce the results in Figure 4c. Therefore,with the introduction of a new training class and theexclusion of the anomalous image the final performanceacross the different cell types and their respectiveprotein exposures produced a significant improvementin the correct number of detected cells.

Figure 5 is the series of least regression plotfor comparing the performance of CASTLE to amanual analysis when confronted with images fromour study for phase dark leukocytes. Figure 5a showsthe performance when both the Transition class isyet to be introduced to the training and before theimages of PBMCs exposed to BSA were analysed foranomalies. The regression line indicates that CASTLEwas for the majority of images accurate but divergesto underestimate the counts for ever larger numbers ofcells. Then, the Transition class was introduced intothe training and the performance across the imagesis shown in Figure 5b. Here we are unable to observethe point representing the anomalous image mentionedbefore as the 521 detected exceeds the limits of theaxes, the axes chosen to focus on comparing regressionlines between each stage (for the inclusion of theanomalous image, see Figure S6). The regression linehere shows the trend of CASTLE consistently over-estimating the number of detected cells. In additionto identifying the anomalous image, after furtherinvestigation, all the images with PBMCs exposed tothe protein BSA were found to have no phase darkcells and so were subsequently excluded from furtheranalysis using phase dark cells. Using this knowledge,Figure 5c shows the final comparison after excludingthese images. We know from our earlier validationthat this final iteration is an improvement in F1 scorefrom the first. For larger numbers of adhered cellsin an image, the regression line is above rather thanbelow the line of perfect count. However, in thefinal iteration, larger values detected do not exceedmore than 14% of the manual count, unlike the 16%from before, and thus represent a lower proportion of





additions and reason for improvement from the firstiteration.

3.2. Effect of Galectin-9 on Leukocyte Adhesion

We now demonstrate how our workflow can lead to newbiological insights by further analysis of the segmentedimages from the data set investigating the effect of Gal-9 on leukocyte adhesion.

Count: The purpose of the data that we analysedwith CASTLE was to determine the effects of Gal-9 oncell adhesion. We now present the analysis performedby CASTLE. This is to demonstrate the ability ofthe program to process images into information easilyinterpreted by a researcher. The result is a comparisonbetween the protein exposures within each cell type.This was carried out in the form of a Student’s t-test,where the sample variance was assumed to be unequalfor each series of images. This was an assumptionmade in the analysis and any future use of the R scriptprovided on the GitLab project (see Section 5) allowsthe use of images assumed to have an equal variance.

Our primary goal was to investigate the roleof Gal-9 on the adhesion of PBMCs and PMNs.Table S4 displays the results from the Student’s t-test for each pairwise comparison of protein exposuresbetween similar cells. At the 99% confidence level, wecan conclude that for PBMCs there is a statisticallysignificant effect on adhesion when a surface is coatedwith either 50 or 100µg/ml of Gal-9 compared to thecontrol. There is no statistically significant differencein adhesion between 50 or 100µg/ml Gal-9 applied, ateither the 99% or 95% confidence level. For PMN, thedifference in cell adhesion between 20µg/ml of Gal-9 compared to the control is considered statisticallysignificant at the 99% confidence level. Figure 6 isa visual representation of these differences betweenprotein exposure and their effect on the number ofadhered cells per mm2 per 106 cells flown through thechamber.

A secondary outcome of the research was todiscern whether more cells were remaining in a stateof initial adhesion or whether they would form a firmadhesion in the period between perfusion and when theimages were taken.

First, we consider Table S5 which displays theresults of the Student’s t-test for cells during initialadhesion, known as phase bright cells. We are ableto conclude, at the 95% confidence level, that there isa statistically significant difference between the meannumber of phase bright cells when exposed to Gal-9compared to the control. However, the difference ateither the 99% or 95% confidence level is not consideredstatistically significant for distinct levels of Gal-9. For

PMN, the difference in phase bright occurrence for20µg/ml of Gal-9 compared to the control is also notconsidered statistically significant at the 99% or 95%confidence levels.

Second, we compare the number of firmly adheredcells, known as phase dark cells, between similar cellsthat have been exposed to the different levels of Gal-9. Table S6 displays the results of the Student’s t-test for each combination of pairwise comparisons. ForPBMCs, we can conclude at the 99% confidence levelthat there is a statistically significant difference in thenumber of adhered cells that are phase dark when Gal-9 is present on the surface at both levels. On the otherhand, the number of phase dark cells between the twoamounts of Gal-9 were considered not to be statisticallysignificant at either the 99% or 95% confidence level.For PMNs, the number of adhered cells that were phasedark when 20 µg/ml Gal-9 was applied, is consideredstatistically significant at the 99% confidence level.

Figure S7 allows us to visualise the differences inthe total number of adhered cells detected for eachprotein exposure, with the number of phase bright andphase dark cells that contribute to the total shown.Here we can see the different effects that Gal-9 appearsto have on the differing cell types. For PBMCs, theGal-9 appears to have the largest effect on the numberof phase bright cells detected in the image comparedto PMNs having an enormous increase in the numberof phase dark cells.

Morphology: The final analysis for the series of imagesis the shape of the adhered area between the cell andthe surface coated in the protein. The following arethe results from this analysis.

Cluster 1 Cluster 2 Cluster 3 Cluster 4PBMC 50 0.0280 0.0021 0.0042 0.0102PBMC 100 0.0285 0.0063 0.0080 0.0285PMN 20 0.0787 0.2127 0.2876 0.2540PMN BSA 0.0165 0.00127 0.0119 0.0208

Table 1: Results of the k-means clustering on all 2496phase dark cells detected by CASTLE. The valuesrepresent the proportion of cells compared to all phasedark cells detected. We then compare to the numberof each cell type and their exposure to the subsequentclusters that cells with similar morphological featureshave.

Table 1 displays the categorisation of the cellmorphology across all protein exposures for eachtype of cell. First, observe that the series ofimages taken for PBMC cells exposed to the controlprotein are not present. Similar to before, afterinvestigating the anomalous image it was seen that the





Figure 6: Boxplot for the total number of cells adhered per mm2×106 from the sample of seven images taken foreach protein exposure. This excludes the previously discovered anomalous image from the PBMC BSA group(see Figure S2).

corresponding series of images contained no phase darkcells. Therefore, none are in this analysis. Next, thek-means clustering was performed on 4 groups and sok = 4 in this analysis. Consider the row reflecting onthe categorisation of phase dark PMN cells exposedto 20µg/ml of Gal-9 to observe how well the k-meansclustering has been able to separate the cells dependingon their morphology. We see that out of the phasedark leukocytes detected, the majority are split evenlybetween the 2nd, 3rd and 4th cluster. This suggeststhat the k-means clustering was unable to differentiatethe morphology compared to the other cells and proteinexposures. However, the majority of the detectedPMN cells exposed to the control were also groupedinto the 4th cluster. This suggests that there may besimilarities within each of the different cell types whichare interfering with the heterogeneity between proteinexposures. Two more k-means clustering analyses werecarried out considering only the same type of phasedark cells.

Table 2 shows the analysis in morphology of thePBMC cells exposed to the different levels of Gal-9.The first observation is the relatively even distributionof cells between both clusters for each level of protein

exposure. Thus, there appears to be a difference inmorphology within the class of phase dark cells whichis more apparent than the difference caused by theadhesion to Gal-9.

Cluster 1 Cluster 2PBMC 50 0.1611 0.2234PBMC 100 0.3699 0.2454

Table 2: Results of the k-means clustering on the298 phase dark PBMCs detected by CASTLE. Thevalues represent the proportion of cells compared tothe number of phase dark PBMCs detected. We thencompare to the number of each cell type and theirexposure to the subsequent clusters that cells withsimilar morphological features have.

Table 3 is the result in the k-means clusteringfor phase dark PMNs. It appears that PMN cellsexposed to 20µg/ml of Gal-9 is still evenly distributedbetween the two clusters although those exposed toGal-9 are more similar to the second cluster than thefirst. However, no significant distinction can be madefrom this analysis.





(a) All types of phase dark cells detected in the study.

(b) PBMC type cells only. (c) PMN type cells only.

Figure 7: Morphology of adhered cells considered phase dark by CASTLE plotted in the first and second principalcomponents for each of the PCAs. (a) All types of phase dark cells detected in the study. The first two principalcomponents capture 73.27% of the total variation in cell morphology. (b) PBMC type cells only. The first twoprincipal components capture 76.39% of the total variation in PBMC morphology. (c) PMN type cells only. Thefirst two principal components capture 72.13% of the total variation in PMN morphology.





Cluster 1 Cluster 2PMN 20 0.3806 0.5486PMN BSA 0.0341 0.0365

Table 3: Results of the k-means clustering on onlythe 2112 phase dark PMNs detected by CASTLE. Thevalues represent the proportion of cells compared tothe number of phase dark PMN cells detected. Wethen compare to the number of each cell type andtheir exposure to the subsequent clusters that cells withsimilar morphological features have.

In all of the k-means clustering analyses ofmorphology we have been unable to determine aseparation of a cell’s morphology depending on theirprotein exposure. To better understand this, weshow the first two principal components in Figure 7.This shows the first two independent dimensions thatcapture the highest variance to help separate the dataand feed into the k-means clustering algorithm. Wechoose the first two components only as they can beeasily displayed. For the first analysis, shown hereas Figure 7a, the PCA is not able to separate thedata sufficiently enough to be easily distinguished intothe different groups of cells. This seems similar tothe Figures 7b and 7c for the PCA for PBMC andPMN cells, respectively. However, these plots are stillable to convey relationships between the morphologyand the protein exposure. For example, Figure 7a canbegin to identify characteristics between the two typesof cell through their morphology. Here, we are ableto see in the first principal component that PBMCsappear with a negative value in the first principalcomponent for all but a few points. This is in contrastto PMNs which the majority exposed to Gal-9 appearabove 0 and those exposed to the control appearingacross the full range. When considering the secondprincipal component, there appears to be 2 general sub-classifications forming with a cluster of points centredabout (1,-1) and the rest sparsely centred about (-4,0).

Figure 7b is a PCA on PBMCs only. We observethat although both groups of Gal-9 exposure havevalues distributed across the range of first and secondprincipal component values, that there could be sometrend. This trend is in reference to many of the PBMCswith 50 µg/ml are more apparent from just lowerthat -5 to 2.5 in the first principal component but formainly positive values in the second component. Thisis compared to the more apparent regimes for PBMCsappearing between 0 to 2.5 in the first principalcomponent and from lower than -5 to 2.5. This is aresult not obvious in Figure 7a but is more obviouswith the individual analysis, most likely due to lesscell heterogeneity.

In Figure 7c, inferences in trend can be madeabout the differences from exposure to Gal-9 tothe control for PMNs. Many of the points aredispersed closely about (0,0). However, the majority ofpoints appear for positive values in the first principalcomponent for those exposed to Gal-9 than those thatdo not. On the other hand, those exposed to thecontrol, although some begin to cluster about (-1,-1),have many values at the extremes of one or both ofthe principal components. Referring back to Figure7a, this appears to be the case too. This observationcould suggest that instead of exposure to Gal-9 causingnew or more extreme phenotypes in the cell that itencourages a specific morphology already possible bythe cells under normal adhesion. To understand whatmorphology a cluster represents, referring back to theshape descriptors that contribute the most to the firstand second components can identify which are lessvariable to exposures of Gal-9.

3.3. Application of CASTLE to other data sets

We now demonstrate the applicability of our workflowto two other studies of cell adhesion: one investigatingthe effect of ICAM-1 on PMN adhesion, and the otherinvestigating the adhesion of platelets in dependenceon vWF and ADP. The biological interpretation ofthese results are not discussed here, but the resultsfrom CASTLE are reproduced in the SupplementaryMaterials to show the applicability of CASTLE todifferent data sets. Using a similar flow assay as forthe studies into Gal-9 (see Section 2.6), we quantifythe number of adherent cells in the images from bothstudies.

In the study introducing ICAM-1 exposure onPMNs, each image was from a repeated flow assayexperiment. The results produced by CASTLE aredisplayed in Figure S8, showing the total number ofadhered cells and the proportions of which are phasebright and phase dark.

In the study on platelets, an investigation into therole of vWF in the adhesion of platelets is considered,in dependence of ADP. As in the application ofCASTLE to the first study, Figure S9 shows the totalnumber of adhered cells and the proportions of whichare phase bright and phase dark. In addition, as therewas a dependence on ADP between images, FigureS10 shows the number of adherent platelets obtainedat different time points within the same experimentwith and without ADP. We observe a higher numberof adherent cells in the presence of ADP. Moreover,Table S7 shows that the difference in total, phasebright and phase dark adhered platelets when ADPwas introduced is statistically significant at the 99%confidence level. This is expected, since ADP is known





to activate platelets, allowing them to bind to vWF[13, 14].

4. Discussion

We have introduced a new workflow for the automaticanalysis of PCM images of adherent cells. Wedemonstrated that the performance of CASTLE iscomparable to methods already established in thedetection of cells undergoing similar processes. Onesuch example are the methods used by Huh et al. [29]in the detection of mitosis in PCM images for stem cellpopulations. In their study, the method at greatestaccuracy achieved an average F1 score of 88.6% witha 91.0% precision and 86.7% recall on a population ofstem cells. Therefore a final F1 score of 87.0% with a81.2% precision and an 93.8% recall describes a systemready for use in future research, if not a benchmark toimprove upon.

The most successful property of the program is thereduction in manual interaction. For a similar analysisof the 35 images counted in the the Gal-9 study, theprocess takes approximately 1.5 hours to count andanalyse by hand. In comparison, CASTLE is recordedtaking under 12 minutes on a conventional laptopcomputer. Moreover, CASTLE provides tools forthe analysis of shape, yielding additional informationbeyond simple cell counts.

CASTLE was able to give a succinct set of re-sults for quantifying the number and proportion ofcells adhered to a surface exposed to varying levels ofmolecules compared to a control. The CASTLE work-flow determined a statistically significant difference inthe number of adhered cells for both PBMCs and PMNcells depending on Gal-9 presented on the chamber sur-face. In addition, the case study was able to referencefor which phase of cell the protein had the most impacton cell adhesion. By distinguishing between the num-ber of phase bright and phase dark adhered cells, infer-ences can be made on cell exposure to Gal-9. However,the program currently has a reduced statistical powerdue to the accuracy of cells currently detected, demon-strated by the lower classification precision. Methodshave already been considered to improve CASTLE inthis aspect. One such method would be to use a seriesof pixel classifiers and include their output as an addi-tional channel of information in a final pixel classifier.The intuition is called Auto-context and the core algo-rithm was introduced by Tu and Bai [35]. In addition,the validation process has proven that further under-standing of the problem at hand and refining the work-flow produces improved results. The final iteration ofleast square regression plots, Figure 4c and Figure 5c,are prime examples for where the line of regression be-tween each previous iteration improves with each res-

olution of the processing problems. First, a new classwas introduced in training, with the majority of spuri-ous ROI eradicated. Then, the anomalous images wereexcluded and an improvement in the final performanceof CASTLE.

The pixel-by-pixel output by CASTLE was lowerin classification precision and recall than the countingperformance. This is to be expected as detection holdsa lower threshold than the additional identificationof each pixel within that cell. This can be seen asone of the inaccuracies in the subsequent morphologyanalysis. Without a higher F1 score the statisticalpower of the k-means clustering remains lower.Likewise, the aforementioned Auto-context processwould be able to improve the identification of pixelsbelonging to a cell in addition to detecting a cell asa ROI. The morphology of the phase dark cell stillgives an insight into the effect of Gal-9 on the cell’smigration across the surface. Despite the methodsused being unable to distinguish differences betweenthe morphology, the workflow was able to efficientlyprocess each cell and the 10 features describing theirmorphology into clusters to show their similarityclearly to a user, with the first principal component inthe PCA describing the most influential factors in thatseparation. Without the use of the machine learningalgorithms both computation and interpretation of the10 dimensional data set would be too inefficient toprocess and the information forfeited. Nonetheless,limitations in the assessment of morphology areapparent.

The shape descriptors used as features could beconsidered insufficient for describing the complex andheterogeneous shape displayed by the migrating cell.While Area and Perimeter are tangible measures ofshape, the remaining features are approximations ofhow similar the appearance is to well-defined shapes.The measures can become problematic when theindicator has a result at neither extreme: eitherthe region in question is almost exactly the sameas the comparable shape or not at all. When thisoccurs a small difference in the calculation can indicatevastly different outcomes in the true shape. Examplescomparing and contrasting different dimensionlessstatistics of shape are described by Russ and Russ [36].To avoid misconceptions on shape, either the featureschosen should be verified by the user, then exposingthe analysis to human bias, or more interpretations ofshape need to be considered to lessen the weight ofother measures, thus reducing the chance of peculiarsimilarities between heterogeneous cell shapes.

An alternative approach is to rethink the currentmorphological analysis and integrate previously vali-dated machine learning techniques from similar fields.A candidate for this proposal is the work of Yin et





al. [37] where a Support Vector Machine (SVM), a su-pervised machine learning model, was used to identifydiscrete shapes from 211 geometric and texture prop-erties of each segmented Drosphilia Kc cell. Anotherexample is the work of Wu et al. [38] with their imageanalysis tool VAMPIRE. The tool reduces the morpho-logical description for an outline into similar modes ofshape using PCA before analysing their distributionin cancer metastasis. This work also introduces anal-ysis of cell-cell contact, possibly relevant to the ag-gregation of platelets. Finally, a work that does notdepend on machine learning, or even human interven-tion, is that of Sanchez et al. [39]. Here, they develop amethod called lobe contribution elliptical Fourier anal-ysis (LOCO-EFA), an extension of elliptical Fourieranalysis, whereby a contour is reconstructed from aninfinite summation of related ellipses. The summationof ellipses then infer information about the originalshape of the cell, in this case complex shaped pavementcells of Arabidopsis thaliana wild-type and speechlessleaves, and Drosophila amnioserosa cells. Future ex-tensions of the automated workflow could consider in-corporating these morphological analysis tools.

Alongside the quantification of still images, theworkflow used in CASTLE can be easily applied tovideos. Videos are characterised by computers as aseries of still images with a time dependence. Hence,the successful classification of multiple images is similarto a classification of a whole video with an ordering.Combined with other programs in cell tracking, suchas Mosaic Suite’s Particle Tracker [40], the work can beextended to observe the whole duration of adhesion fora particular cell. Thus, other features like migrationtracking can be investigated.

The methodology behind CASTLE is not limitedto cell classification. The Ilastik program allows for thepossibility of other types of images to be segmentedinto any number of classes. The utilisation of theautomated workflow is only limited by the universalityof pre- and post-processing techniques. The largerimpact of the work is yet to be tested in other areas,such as introducing a monolayer of endothelial cells tomonitor extravasation.

Here we have designed an automated workflow,from data acquisition to analysis, which can beapplied to numerous PCM images. A fundamentalunderstanding of the biological processes allowed forthe method to be refined to obtain an accurateconclusion. It was shown that the results werecomparable to the manual analysis of the same data setand produced information beyond what was previouslyavailable. In addition, these conclusions can bereached significantly faster than the alternative manualmethod with minimal expense to a researcher’s time.Potential ways to enhance the performance are already

being investigated and new lines of interest are beingconsidered for application.

While much of our discussions focused on theinvestigation of adhesion of PMNs and PBMCs independence on Gal-9, we demonstrate the applicabilityof our workflow to other cell types and otherexperimental conditions. Notably, we investigatedlecukocyte adhesion in dependence on ICAM-1, andplatelet adhesion to vWF in dependence on ADP.Strikingly, cells were well recognised even when usingthe training set from the Gal-9 studies. Recognitioncan then be improved by adding single annotatedimages to the training set, minimising the manualinput for this automatic analysis. This demonstratesthe utility of CASTLE for the analysis of a wide rangeof adherent cells under various conditions.

5. Software availability

The ImageJ macro scripts created for the purposes ofthis research, namely Cell Adhesion with SupervisedTraining and Learning Environment (CASTLE), areavailable to download from the following projecton GitLab: https://gitlab.bham.ac.uk/spillf/castle. The R script to perform the Principalcomponent analysis (PCA) and k-means clusteringalgorithms, to analyse the results from CASTLE, areavailable at the same location. Finally, the validationperformed is also available as a series of spreadsheetsat the same location.

Acknowledgments

The authors would like to thank Jeremy Pike and IainStyles for discussions and advice during the prepa-ration of this article. SGG is supported in part bythe EPSRC Centre for Doctoral Training in Topologi-cal Design, funded by the UK Engineering and Phys-ical Sciences Research Council (grant EP/S02297X/1)and the University of Birmingham. MC was sup-ported by a Royal Society Dorothy Hodgkin fellowship(DH160044). AJI is supported by the University ofBirmingham Fellowship and AMS Springboard Award(SBF003/1156). The Institute of Cardiovascular Sci-ences, University of Birmingham is recipient of a BHFAccelerator Award (AA/18/2/34218).

References

[1] RJ Seager, Cynthia Hajal, Fabian Spill, Roger D Kamm,and Muhammad H Zaman. Dynamic interplay betweentumour, stroma and immune system can drive or preventtumour progression. Convergent Science PhysicalOncology, 3(3):034002, 2017.

[2] Michael Mak, Fabian Spill, Roger D Kamm, andMuhammad H Zaman. Single-cell migration in complex





microenvironments: mechanics and signaling dynamics.Journal of biomechanical engineering, 138(2), 2016.

[3] Mojca Mattiazzi Usaj, Erin B Styles, Adrian J Verster,Helena Friesen, Charles Boone, and Brenda J Andrews.High-content screening for quantitative cell biology.Trends in cell biology, 26(8):598–611, 2016.

[4] Paul Michel Aloyse Antony, Christophe Trefois, AleksandarStojanovic, Aidos Sagatovich Baumuratov, and KarolKozak. Light microscopy applications in systems biol-ogy: opportunities and challenges. Cell Communicationand Signaling, 11(1):24, 2013.

[5] Anne E Carpenter, Lee Kamentsky, and Kevin W Eliceiri.A call for bioimaging software usability. Nature methods,9(7):666, 2012.

[6] Gabriel Popescu. Quantitative phase imaging of cells andtissues. McGraw Hill Professional, 2011.

[7] Rafael C Gonzalez and Richard E Woods. Digital imageprocessing, 2008.

[8] John C Russ. The image processing handbook, 2002.[9] Computational intelligence and its applications : evolution-

ary computation, fuzzy logic, neural network and sup-port vector machine techniques / edited by h.k. lam,steve s.h. ling, h.t. nguyen., 2012.

[10] Andrey Kan. Machine learning applications in cell imageanalysis. Immunology and cell biology, 95(6):525–530,2017.

[11] Marta A Toscano, Verónica C Mart́ınez Allo, Anabela MCutine, Gabriel A Rabinovich, and Karina V Mariño.Untangling galectin-driven regulatory circuits in autoim-mune inflammation. Trends in molecular medicine,24(4):348–363, 2018.

[12] Shuguang Bi, Patrick W Hong, Benhur Lee, and Linda GBaum. Galectin-9 binding to cell surface proteindisulfide isomerase regulates the redox environment toenhance t-cell migration and hiv entry. Proceedings ofthe National Academy of Sciences, 108(26):10650–10655,2011.

[13] David C Warltier, Peter CA Kam, and Mark K Egan.Platelet glycoprotein iib/iiia antagonistspharmacologyand clinical developments. Anesthesiology: TheJournal of the American Society of Anesthesiologists,96(5):1237–1249, 2002.

[14] Marian A Packham and Margaret L Rand. Historical per-spective on adp-induced platelet activation. Purinergicsignalling, 7(3):283, 2011.

[15] Christopher J Kuckleburg, Clara M Yates, Neena Kalia,Yan Zhao, Gerard B Nash, Steve P Watson, andGeorge Ed Rainger. Endothelial cell-borne plateletbridges selectively recruit monocytes in human andmouse models of vascular inflammation. Cardiovascularresearch, 91(1):134–141, 2011.

[16] Nobuyuki Otsu. A threshold selection method from gray-level histograms. IEEE transactions on systems, man,and cybernetics, 9(1):62–66, 1979.

[17] TW Ridler, S Calvard, et al. Picture thresholding usingan iterative selection method. IEEE transactions onSystems, Man and Cybernetics, 8(8):630–632, 1978.

[18] António dos Anjos and Hamid Reza Shahbazkia. Bi-levelimage thresholding. Biosignals, 2:70–76, 2008.

[19] Jyrki Selinummi, Pekka Ruusuvuori, Irina Podolsky, AdrianOzinsky, Elizabeth Gold, Olli Yli-Harja, Alan Aderem,and Ilya Shmulevich. Bright field microscopy as analternative to whole cell fluorescence in automatedanalysis of macrophage images. PloS one, 4(10):e7497,2009.

[20] Rachel Flight, Gabriel Landini, Iain B Styles, RichardShelton, Michael Milward, and Paul Cooper. Semi-automated cell counting in phase contrast images ofepithelial monolayers. In Medical Image Understandingand Analysis Conference (MIUA), pages 241–246, 2014.

[21] Rachel Flight, Gabriel Landini, Iain Styles, RichardShelton, Michael Milward, and Paul Cooper. Automatedoptimisation of cell segmentation parameters in phasecontrast using discrete mereotopology. In Medical ImageUnderstanding and Analysis Conference (MIUA), 2015.

[22] Tin Kam Ho. Random decision forests. In Proceedings of3rd international conference on document analysis andrecognition, volume 1, pages 278–282. IEEE, 1995.

[23] Christoph Sommer, Christoph Straehle, Ullrich Koethe,and Fred A Hamprecht. Ilastik: Interactive learningand segmentation toolkit. In 2011 IEEE internationalsymposium on biomedical imaging: From nano to macro,pages 230–233. IEEE, 2011.

[24] I T Jolliffe. Principal component analysis / i.t. jolliffe.,1986.

[25] Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deeplearning. MIT press, 2016.

[26] Stuart Lloyd. Least squares quantization in pcm. IEEEtransactions on information theory, 28(2):129–137, 1982.

[27] David Martin Powers. Evaluation: from precision, recalland f-measure to roc, informedness, markedness andcorrelation. Mach. Learn., 2011.

[28] Rachel Flight, Gabriel Landini, Iain Styles, RichardShelton, Michael Milward, and Paul Cooper. Automatednon-invasive cell counting in phase contrast microscopywith automated image analysis parameter selection.Journal of Microscopy, 2018.

[29] Seungil Huh, Ryoma Bise, Mei Chen, Takeo Kanade, et al.Automated mitosis detection of stem cell populations inphase-contrast microscopy images. IEEE transactionson medical imaging, 30(3):586–596, 2010.

[30] Johannes Schindelin, Ignacio Arganda-Carreras, ErwinFrise, Verena Kaynig, Mark Longair, Tobias Pietzsch,Stephan Preibisch, Curtis Rueden, Stephan Saalfeld,Benjamin Schmid, et al. Fiji: an open-source platformfor biological-image analysis. Nature methods, 9(7):676,2012.

[31] Tiago Ferreira and Wayne Rasb. Imagej user guide: Ij 1.46r. 2012.

[32] Ross Ihaka and Robert Gentleman. R: a language for dataanalysis and graphics. Journal of computational andgraphical statistics, 5(3):299–314, 1996.

[33] Ignacio Arganda-Carreras, Verena Kaynig, Curtis Rueden,Kevin W Eliceiri, Johannes Schindelin, Albert Cardona,and H Sebastian Seung. Trainable weka segmentation: amachine learning tool for microscopy pixel classification.Bioinformatics, 33(15):2424–2426, 2017.

[34] Mark Hall, Eibe Frank, Geoffrey Holmes, BernhardPfahringer, Peter Reutemann, and Ian H Witten. Theweka data mining software: an update. ACM SIGKDDexplorations newsletter, 11(1):10–18, 2009.

[35] Zhuowen Tu and Xiang Bai. Auto-context and itsapplication to high-level vision tasks and 3d brain imagesegmentation. IEEE transactions on pattern analysisand machine intelligence, 32(10):1744–1757, 2009.

[36] J C Russ and C Russ. Dimensionless ratios as shapedescriptors. infocus Magazine, 48:58–72, 2017.

[37] Zheng Yin, Amine Sadok, Heba Sailem, Afshan McCarthy,Xiaofeng Xia, Fuhai Li, Mar Arias Garcia, Louise Evans,Alexis R Barr, Norbert Perrimon, et al. A screen formorphological complexity identifies regulators of switch-like transitions between discrete cell shapes. Nature cellbiology, 15(7):860, 2013.

[38] Pei-Hsun Wu, Jude M Phillip, Shyam B Khatau, Wei-Chiang Chen, Jeffrey Stirman, Sophie Rosseel, KatherineTschudi, Joshua Van Patten, Michael Wong, SonalGupta, et al. Evolution of cellular morpho-phenotypesin cancer metastasis. Scientific reports, 5(1):1–10, 2015.

[39] Yara E Sánchez-Corrales, Matthew Hartley, Jop Van Rooij,Athanasius FM Marée, and Verônica A Grieneisen.





Morphometrics of complex cell shapes: lobe contribu-tion elliptic fourier analysis (loco-efa). Development,145(6):dev156778, 2018.

[40] Ivo F Sbalzarini and Petros Koumoutsakos. Feature pointtracking and trajectory analysis for video imaging in cellbiology. Journal of structural biology, 151(2):182–195,2005.





A. Supplementary Material

A.1. Shape Descriptors

The following are morphological features availableas measurements for shape analysis in the imageprocessing software ImageJ as defined in the user guide[31].

Area Number of square pixels contained in a ROI, orin calibrated square units (e.g., mm2, µm2, etc.).

Perimeter The length of the outside boundary of theROI.

Bounding rectangle The smallest width and height ofthe rectangle enclosing the ROI.

Fit ellipse Fits an ellipse to the ROI. Parameters Majoraxis and Minor axis are the primary and secondary axisof the best fitting ellipse.

Circularity 4π × Area(Perimeter)2 with a value of 1.0indicating a perfect circle. As the value approaches0.0, it indicates an increasingly elongated shape.

Aspect ratio The aspect ratio of the particle’s fittedellipse, i.e., MajoraxisMinoraxis .

Roundness 4 × Areaπ×(Majoraxis)2 .Feret’s diameter The longest distance between any twopoints along the selection boundary, also known as themaximum caliper. In addition, the minimum caliperdiameter is calculated, also known as the minimumFeret’s diameter.





Figure S1: The workflow for the application and validation of Cell Adhesion with Supervised Training andLearning Environment (CASTLE) workflow. ‘Batch Process’ and ‘Analysis’ are automated in the CASTLEworkflow, while ‘Training’ and ‘Validation’ involve manual interaction.

Validation Phase F1 score Precision Recall Time (min:sec)

ManualBright - - - 26:00Dark - - - 49:00

Gal-9 study processed by CASTLE Bright 75 67 84 05:08(no Transition class) Dark 87 96 83 05:13

Gal-9 study processed by CASTLE Bright 84 80 90 05:11(with Transition class) Dark 90 83 98 05:17

Gal-9 study processed by TWS Bright 68 54 93 27:00(with Transition class) Dark 58 41 97 40:52

Table S1: Table of the validation results on the data produced from the study investigating the role of theprotein Gal-9 on leukocyte adhesion. The images automatically processed were compared to a manual analysisby an expert, including the time it took CASTLE and TWS to analyse the images on the computer configurationdetailed in Section 2.7. For all iterations, 35 images were validated.

Validation Phase F1 score Precision Recall Time (min:sec)ICAM-1 study processed by CASTLE Bright 91 87 96 03:59

(with no additional training) Dark 73 96 59 04:04ICAM-1 study processed by CASTLE Bright 91 89 92 05:51

(with additional training) Dark 83 87 80 05:29

Table S2: Table of the validation results on the data produced from the study investigating the role of the proteinICAM-1-Fc on leukocyte adhesion. The images automatically processed were compared to a manual analysis byan expert, including the time it took CASTLE to analyse the images on the computer configuration detailed inSection 2.7 For the study investigating the protein ICAM-1, 28 images were analysed and validated. Here, ‘withno additional training’ refers to the same training in CASTLE as that in the study of the role of the proteinGal-9 on leukocyte adhesion while ‘with additional training’ refers to the addition of 1 image from the samestudy being analysed was added to the training in CASTLE.





Validation Phase F1 score Precision Recall Time (min:sec)vWF study processed by CASTLE Bright 68 99 51 08:25

(with no additional training) Dark 80 77 83 08:20vWF study processed by CASTLE Bright 89 98 81 08:21

(with additional training) Dark 71 59 90 08:29

Table S3: Table of the validation results on the data produced from the study investigating the role of theprotein vWF on platelet adhesion. The images automatically processed were compared to a manual analysis byan expert, including the time it took CASTLE to analyse the images on the computer configuration detailed inSection 2.7. For the study investigating the protein vWF on platelet adhesion, 2 images were validated beforecompleting the analysis on 30 images taken sequentially on the same assay for each condition. Here, ‘with noadditional training’ refers to the same training in CASTLE as that in the study of the role of the protein Gal-9on leukocyte adhesion while ‘with additional training’ refers to training on only 2 images from the study of therole of the protein vWF on platelet adhesion.

castle: cell adhesion with supervised training and learning … · 2020. 6. 2. · castle: cell...

Documents