Automatic Counting and Visual Multi-trackingSystem for Human Sperm in Microscopic

Video Frames

Nour Eldeen M. Khalifa1,2(&) , Mohamed Hamed N. Taha1 ,and Aboul Ella Hassanien1,2

1 Information Technology Department, Faculty of Computers and Information,Cairo University, Giza, Egypt

{nourmahmoud,mnasrtaha,aboitcairo}@cu.edu.eg2 Scientific Research Group in Egypt (SRGE), Giza, Egypt

http://www.egyptscience.net

Abstract. In this paper, a proposed system for automatic counting and visualmulti-tracking for human sperm in microscopic video frames is presented. It canbe easily turned into a commercial computer-assisted sperm analysis (CASA)system. CASA systems help in detecting infertility in human sperm according toclinical parameters. The proposed system consists of nine phases and it countssperm in every single frame of video in real time and calculates the averagesperm count through the whole video with accuracy 94.3% if it is compared tothe manual counting. Also, it tracks all identified sperm in video frames in realtime. It works with different frame rates above 15 frame/s to track visually themovements of the sperm. The dataset consists of three high-quality 1080pvideos with different frame rates and durations. Finally, the open challengingresearch points are addressed.

Keywords: Sperm counting � Sperm trackingComputer-assisted sperm analysis (CASA)

1 Introduction

Human sperm motility is of great interest to biologists studying sperm function and tomedical physician for male infertility evaluating and treating [1]. Approximately theinfertility rate in Egypt is 9% according to the Egyptian IVF registry report [2]. Most ofthe married couples seek diagnostic semen analysis to help assess the cause.

Semen examination is divided into two groups, namely examination in macroscopicand microscopic. The macroscopic examinations are general semen inspections withoutthe need for complicated tools. The inspections include the volume, semen color,viscosity, and the pH of the semen. The microscopic examinations are performed formore detail conditions of the semen where a sophisticated tool needed in the process.These include sperm mass movements, motility, sperm concentration, viability, andsperm morphology [3].

© Springer Nature Switzerland AG 2019A. E. Hassanien et al. (Eds.): AISI 2018, AISC 845, pp. 525–531, 2019.https://doi.org/10.1007/978-3-319-99010-1_48

Today, the general method for analyzing sperm at fertility clinics and researchlaboratories is strenuous and subjective [4]. Typically, lab technicians use microscopesfor manually counting the number of sperm cells and for visually evaluating the qualityof sperm movement according to standard protocols. The common parameters evalu-ated during a typical semen analysis can be divided into nine different parameters [5].In some modern clinics, more objective Computer Assisted Semen Analysis (CASA)instruments are used to trace the swimming paths of sperm automatically in time-lapsemicroscopy image sequences [4]. Digital image processing algorithms could help inperforming the semen analysis according to the previous four parameters. Katz andDavis were the first authors who proposed an automatic sperm tracking in the mid-1980s [6], they proposed a software contains over 60 commands. System commandsenable the user to capture and manipulate data. The program automatically performs asequence of data capture, reduction, analysis, and reporting steps with a minimum ofuser input.

Beresford-Smith et al. [7] applied radar tracking algorithms on sperm tracking.They adapted the probabilistic data association filter to track a single sperm in clutter.However, they reported no experimental data. Tomlinson et al. in [8] described aCASA system based on multi-target tracking algorithms. They tracked multiple spermsand graded their motility using 1-s video clips. However, the authors did not addressthe problem of tracking through collisions or over long durations.

Hidayatullah et al. in [9] survey on multi-sperm tracking for sperm motility mea-surement and illustrate the advantages and disadvantages for a group of the proposedCASA system until 2016.

The rest of the paper is organized as follows. Section 2 is a brief description of theCASA systems. Section 3 presents the proposed system for visual multi-tracking forhuman semen. Experimental results are discussed in Sect. 4. Finally, Sect. 5 concludethe paper and list the possible future work.

2 Computer Assisted Semen Analysis (CASA)

The term CASA stands for “computer-aided sperm analysis” or “computer-assistedsperm analysis”. Over approximately 30 years the Computer-assisted sperm analysis(CASA) systems have improved. This evolution happened through amelioration indevices to capture the image from a microscope, huge increases in computationalpower, new computer programming languages, and advanced software algorithms. Theevolution grants great benefits for animal and human semen analysis. However, themethods for identification of sperm and their motion patterns are little changed. Out-dated and slower systems stay in use [10].

In general, the CASA system consists of microscope that is given an additionaldigital camera. This camera will replace the function of the veterinarian or doctor eye torecord video of the sperm being analyzed. This digital video is then sent to the com-puter for processing.

The recording results are stored in the form of a digital video file. To be processed,this video must be extracted into image frames. The analysis is done on each frame to

526 N. E. M. Khalifa et al.

identify sperm. After the sperm can be identified, the comparison between sequentialframes is carried out to obtain sperm motility parameters [9].

Majority of fertility laboratories and semen processing facilities have a CASAsystem, but the extent of dependence thereon ranges exceedingly. Each system isdifferent. Modern CASA systems can automatically assess several fields in a shallowspecimen. Images of 500 to >2000 sperm can be captured, at 50 or 60 frames persecond, in clear or complex extenders, and in <2 min, store information for 30 framesand deliver summary data for each spermatozoon and the population [10].

The accurate prediction fertility, from a semen sample, cannot be calculated byCASA. However, current CASA systems provide significant information for qualityassurance of semen planned for understanding the responses of sperm responses tochanges in the microenvironment in research, and for marketing [10].

3 The Proposed Counting and Visual Multitrackingfor Human Sperm System

The proposed system consists of nine phases. The first phase is reading the video fileframe by frame, while the second phase is converting the video frame from RGBdomain to HSV domain according to Eq. (1). In phase number 3, a thresholding valuewill be applied to video frame according to specific values of Hue, Saturation andValue to segment the semen from the background according to Eq. (2).

h ¼

0�

if max ¼ min

60� � g�b

max�min þ 0�

if max ¼ r and g� b

60� � g�b

max�min þ 360�

if max ¼ r and g\b

60� � b�r

max�min þ 120�

if max ¼ g

60� � r�g

max�min þ 240�

if max ¼ b

8>>>>>><

>>>>>>:

s ¼ 0; if max ¼ 0max�min

max ¼ 1� minmax ; otherwise

(

v ¼ max

ð1Þ

ThresholdedHSVframe ¼

h ¼ 1; 0:973� h� 0:966h ¼ 0; otherwise

�

s ¼ 1; 0:000� s� 0:351s ¼ 0; otherwise

�

v ¼ 1; 0:407� v� 0:658v ¼ 0; otherwise

�

8>>>>>><

>>>>>>:

ð2Þ

A binarization of the image is required in phase number 3, while in phase number 5a pixel grouping is a necessary step for phase number 6 which accept and reject pixelgroups according to a pixel group value to remove noise and an unwanted group ofpixels as illustrated in Eq. (3).

Automatic Counting and Visual Multi-tracking System for Human Sprem 527

Thresholdpixel group ¼ 1 number of pixels in a group� 500 number of pixel in a group[ 50

�

ð3Þ

Fig. 1. Proposed system block diagram for multi-tracking and counting human sperm.

Fig. 2. The output of video frame after every phase of the proposed system

528 N. E. M. Khalifa et al.

In phase number 7, the proposed system calculated the center of the pixel groupsaccording to x–y coordinates and append them to a global x–y parameter and count allpixel groups which indicated the number of sperm. Drawing a rectangle around thepixel groups is done in phase number 8 for tracking purposes. In the final phase, thesystem draws all the old x–y coordinates saved in the global x–y parameter. Figure 2illustrates all the required phases in a block diagram for the proposed system.

One of the main functions of the proposed system is visually tracking all sperm inthe video frame. Figure 1 presents the output video frame after every phase of the 9phases the proposed system had.

As illustrated in Fig. 2, the resulted frame after phase 5, 6 contains only the actualvisual sperm without noise while the final frame after phase 9 contains the originalvideo frame with yellow rectangle tracking the sperms and blue dots for the previousx–y coordinates for all sperms spotted in the previous frames.

4 Experimental Results

The proposed system was implemented using a commercial software package(MATLAB). All experiments were performed on a server with Intel Xeon E5-2620processor and 96 GB Ram. To evaluate the proposed system, three microscopic videosfor human sperm sample were selected. The video’s description is presented in Table 1.

The proposed system is designed to perform 2 main functions. The first function isto count the pixel groups (Human Sperm) in every frame and display the count numberabove the frame in real time as illustrated in Fig. 3. The average number of sperms iscalculated at the end of the video. The average number of sperm for the whole video is94.3% accuracy with +2, −2 as the standard deviation for error if it is compared to themanual counting.

The second function of the proposed system is to track every sperm and itsmovements through the video by tracking it is previous x–y coordinates in the previousvideo frames. Figure 4 illustrated a zoomed tracking for different sperm movement invideo frames.

The proposed system analyze the video frames in different frame rates. Figure 4illustrated the movements of sperms in 25 frame/s. It can operate in 1 frame/s, but themovements can’t be noticed and that will affect the whole performance of the system.So, the preferable frame rate is above 15 frame/s to notice the movement of the spermsand track them visually.

Table 1. Videos description used by the proposed system

Video name Duration (sec) # of frames/sec Quality

Video 1 93 25 1080pVideo 2 60 27 1080pVideo 3 28 30 1080p

Automatic Counting and Visual Multi-tracking System for Human Sprem 529

The proposed system achieved its goals to count the sperms in every frame andcalculated the average the sperm on the whole video. It visually tracks every sperm inthe video highlighting its path through the video sample.

5 Conclusions and Future Work

The counting and visual multitracking for human sperm is considered one of themotivating topics of interest to researchers over the past 30 decades. The developmentof computer-assisted semen analysis (CASA) has also attracted more researchers intothis field as those systems are applied in most of the laboratories across Egypt and evenworldwide. In this paper, a proposed system was introduced to count human sperm invideo frames. It counts sperm in every single frame in real time and calculates theaverage count of sperm in the whole video with accuracy 94.3% and error +2 or −2percent if it is compared to the manual counting. Also, the proposed system tracksvisually all identified sperm across the video and visually mark the sperm with a yellow

Fig. 3. Number of sperms in every frame

Fig. 4. (a) Sperm with fast movement, (b) sperm with medium movement and (c) sperm withslow movement

530 N. E. M. Khalifa et al.

rectangle and draw blue dots for the previous x–y coordinates of sperm movementacross the previous video frames.

There is still a room for improvements as a future work for the proposed system.The following future points are under open research and they are: (1) The intersectionsof sperm movements, it will be visually hard to track; (2) Analyzing x–y coordinates todetect the quality of the sperm by its speed, and (3) Improving the system to work withlow-quality videos.

References

1. Hallak, J.: A call for more responsible use of assisted reproductive technologies (ARTs) inmale infertility: the hidden consequences of abuse, lack of andrological investigation andinaction. Transl. Androl. Urol. 6, 997–1004 (2017). https://doi.org/10.21037/tau.2017.08.03

2. Mansour, R., El-Faissal, Y., Kamal, O.: The Egyptian IVF registry report: assistedreproductive technology in Egypt 2005. Middle East Fertil. Soc. J. 19, 16–21 (2014). https://doi.org/10.1016/j.mefs.2014.01.001

3. Gómez Montoto, L., Magaña, C., Tourmente, M., Martín-Coello, J., Crespo, C., Luque-Larena, J.J., Gomendio, M., Roldan, E.R.S.: Sperm competition, sperm numbers and spermquality in muroid rodents. PLoS ONE 6, e18173 (2011). https://doi.org/10.1371/journal.pone.0018173

4. Urbano, L.F., Masson, P., VerMilyea, M., Kam, M.: Automatic tracking and motilityanalysis of human sperm in time-lapse images. IEEE Trans. Med. Imaging 36, 792–801(2017). https://doi.org/10.1109/TMI.2016.2630720

5. World Health Organization: WHO Laboratory Manual for the Examination and Processingof Human Semen. World Health Organization, Geneva (2010)

6. Katz, D.F., Davis, R.O., Delandmeter, B.A., Overstreet, J.W.: Real-time analysis of spermmotion using automatic video image digitization. Comput. Methods Programs Biomed. 21,173–182 (1985). https://doi.org/10.1016/0169-2607(85)90002-1

7. Beresford-Smith, B., Van Helden, D.F.: Applications of radar tracking algorithms to motionanalysis in biomedical images. In: Proceedings of 1st International Conference on ImageProcessing. IEEE Computer Society Press, pp. 411–415 (1994)

8. Tomlinson, M.J., Pooley, K., Simpson, T., Newton, T., Hopkisson, J., Jayaprakasan, K.,Jayaprakasan, R., Naeem, A., Pridmore, T.: Validation of a novel computer-assisted spermanalysis (CASA) system using multitarget-tracking algorithms. Fertil. Steril. 93, 1911–1920(2010). https://doi.org/10.1016/j.fertnstert.2008.12.064

9. Hidayatullah, P., Mengko, T.L.E.R., Munir, R.: A survey on multisperm tracking for spermmotility measurement. Int. J. Mach. Learn. Comput. 7, 144–151 (2017). https://doi.org/10.18178/ijmlc.2017.7.5.637

10. Amann, R.P., Waberski, D.: Computer-assisted sperm analysis (CASA): capabilities andpotential developments. Theriogenology. 81, 5.e3–17.e3 (2014). https://doi.org/10.1016/j.theriogenology.2013.09.004

Automatic Counting and Visual Multi-tracking System for Human Sprem 531