Approach for Overlapped Segmentation of Bacterial Cell Images: History
View Latest Version
Please note this is an old version of this entry, which may differ significantly from the current revision.
Contributor:
Dilanga Abeyrathna
,
Shailabh Rauniyar
,
Rajesh K. Sani
,
Pei-Chi Huang

Scanning electron microscopy (SEM) techniques have been extensively performed to image and study bacterial cells with high-resolution images. Bacterial image segmentation in SEM images is an essential task to distinguish an object of interest and its specific region.

  • overlapping cell segmentation
  • bacterial cell segmentation
  • U-Net

1. Introduction

Microscopic observation of cellular morphology is a preliminary step in microbiology research. Cell size variability in microbes is indicative of cellular response to environmental stimuli through physiological and gene expression changes [1][2]. It has also been reported that cellular housekeeping, nutrient transport, and cell reproduction are associated with cell size variations [3]. Automated cell segmentation techniques in microscopy imaging are essential for measuring cellular characteristics such as alteration in size for the evaluation of the impact of specific environmental change [4][5]. Analysis of a large number of cells requires automated cell recognition techniques to enable faster decision-making.
To gain valuable information for semantic understanding [6] of images, object detection techniques should precisely estimate the shapes and locations of objects in each image. In machine learning, semantic understanding is defined as the ability of a machine to understand the meaning and context behind real-world information [7]. Understanding an image containing a large number of objects of interest is a task of great importance for monitoring in multiple domains, such as cells of in vitro cultures and developing embryos [8], grain analysis [9], and nuclei analysis [10]. However, the complexity of the bacterial cell segmentation task is a barrier, as the intensity and texture of the foreground and background are similar, and most clustered bacterial cells in images that are partially overlapping with each other. Many practical approaches have been proposed in the literature to tackle the overlapping objects segmentation problem, such as objects in different colors [11], level sets [12], ellipse fitting [13], and the watershed algorithm [14][15]. The traditional approaches to identify cell regions in microscopy images are labor intensive and heavily dependent on the professional knowledge of researchers.
Cell segmentation and growth rate measures of bacteria in a changed growth condition are crucial for developing an instrument with different materials for bacteria cell growth, pattern recognition, and further understanding the behavior and impact on cell cycle progression. A considerable research effort is being made to develop methods for extracting cell regions from its surface and background, segmenting the cell surfaces, and counting/measuring the cell growth proliferation [12]. Direct imaging techniques, such as confocal laser microscopy, electron microscopy, light microscopy, bioluminescence imaging, and macroscale photography have been frequently used in experiments on bacteria cell detection and to extract measurements. These imaging techniques have become more convenient to experiment with, rapid, and standard diagnostic methods for analyzing morphological disruption [16]. Scanning electron microscopy (SEM) techniques have been extensively performed to image and study bacterial cells with high-resolution images [17][18]. However, distinguishing the cells in a cluster remains challenging in many SEM images because of the similar pixel intensities and touching and overlapping cell clusters, leading to significant difficulties in extracting quantitative information.

2. Traditional Approaches for Overlapping Objects Segmentation

Traditional early-stage image segmentation tasks have applied thresholding techniques where each pixel is classified as either an object or as part of the background based on a chosen threshold value. The well-known approaches include morphological operations [19][20], watershed segmentation [21], levelset methods [22][23], graph-based approaches [24][25], and their variations [26][27]. These methods mainly focus on features, such as gradient, color, and/or structural distributions. However, due to the variability of cell segmentation tasks, e.g., heterogeneous object/cell structures and shared features between foreground and background, the parameters in these methods need to be customized for application by the proper selection of the structuring elements. Similarly, traditional machine learning approaches have been used for cell segmentation [28][29], but most of the approaches were evaluated in situations with uniform cell appearance and a high contrast between the foreground and background. Furthermore, these approaches are too labor-intensive and time consuming (especially for large datasets) to have generalized segmentation performance and robustness for cell segmentation tasks [30]. Furthermore, optimization algorithms such as swarm intelligence and genetic algorithms have been extended to perform medical image segmentation applications [31][32]. However, these approaches have not been used in complex segmentation applications.

3. Contour-Based Methods

The contour-based method employs a set of steps to represent the curve evolution of segmented regions in an image, including curvature, skeleton, and polygon approximation. The method treats the instance segmentation as a regression task where a contour can be reduced to a series of discrete vertex coordinates. Contour detection has attracted extensive attention in the field of image segmentation for overlapping or touching object segmentation [33][34][35][36]. Many contour detection algorithms have been introduced in the literature to understand the nature of overlapping objects. Refs. [37][38] adopted sliding window-based approaches to extract the contours and foreground object from the background. Similarly, Ref. [39] proposed a bottleneck detector that identifies a pair of splitting points with a minimum Euclidean distance transform (EDT) while maximizing their distance over the contours for concave corners identification. However, these methods are highly prone to noise [40] and tend to retrieve false corner points, leading to low accuracy. Hence, extensive preprocessing is required to avoid noise before the ellipse fitting stage.
Besides the aforementioned corner detection approaches with manual object segmentation, the general corner identification algorithms, such as k-curvature [41][42], have been successfully used to extract corners for overlapping object instances as well. For example, Ref. [10] proposed a corner point identification that uses the Harris corner detection algorithm to generate candidate points and then extract obvious and uncertain concave points from the candidate points. However, the method is constrained by the parameter settings, which is reflected in the low degree of generalizability of the models. Furthermore, these algorithms cannot perform well when object boundaries appear blurry. Such a situation requires deep learning methods for feature extractions to identify precise object boundaries under different conditions. For example, Ref. [43] adopted a k-curvature technique to determine the candidate curve points and further improve the detection of corner points.

4. Ellipse-Fitting Methods

Most of the ellipse-fitting approaches have been proposed to address touching elliptical-shaped objects for segmentation tasks [13][44]. For example, the segmentation task of overlapping elliptical grains [45] and cell nuclei [46] utilizes the multi-ellipse fitting method to set a minimum threshold of the expected area of each cell that can automatically detect and split touching cells. Despite its great potential, the lack of generalizability has hindered the widespread adoption of different applications with specific type objects due to the requirement of task-specific rules and parameters.
Some recent studies have extended ellipse-fitting techniques to achieve more accurate segmentation results. For example, Ref. [13] proposed that the modified ellipse-fitting technique generates candidate ellipses and selects the optimal ellipse among the candidate pool to recognize overlapping elliptical objects in a binary image from given concave points that are extracted by a polygon approximation algorithm. Ref. [47] proposed a parameter-free decremental ellipse fitting algorithm (DEFA) for automatically estimating the number and the parameters of ellipses objects by exploiting the skeleton of a shape. However, the approach requires binarization of elliptical-shaped objects in an image or images with high contrast between foreground and background. Ref. [48] proposed an improved version of DEFA, called region-based fitting of overlapping ellipses (RFOVE), which automatically determines the number of possibly overlapping ellipses by optimizing the area of shape coverage from unsupervised learning and operating with previously unknown shapes. Furthermore, Refs. [46][49] have been successfully implemented to accommodate overlapping cell segmentation tasks to retrieve accurate quantitative measures in SEM images.

5. Deep Learning Methods

Owing to the huge success of deep learning (DL), it has become the de facto choice for advanced segmentation tasks. Recently, because of its capability for better feature extraction and accurate segmentation quality, many cell segmentation methods and applications with DL techniques have been widely adopted in cell segmentation [15][50][51]. DL methods for object segmentation can be broadly classified into two categories:
  • Instance segmentation: mask R-CNN [52] is a well-known deep neural network architecture for multi-objects detection, and extends faster R-CNN [14] by adding an extra branch for predicting segmentation masks while simultaneously recognizing the bounding box from the existing branch. Mask R-CNN uses a region proposal-based object detection and uses a high-quality segmentation mask technique to achieve instance segmentation results. However, this method cannot perform well in situations with heavily overlapping object instances or highly close object occurrences due to greedy non-maximum suppression post-processing.
  • Semantic segmentation: U-Net [53] deep learning architecture is recognized as another popular semantic segmentation approach that neither employs region proposals nor reuses pooling indices. Instead, it uses encoder–decoder-based neural network architecture to predict a class-based object segmentation output. The U-Net architecture has been successfully used in many overlapping cell segmentation tasks [51][54], especially in the medical community, because of its intrinsic capability to perform down sampling–up sampling. For example, research studies by [55][56] have demonstrated how to use the architecture to accurately segment overlapping cervical cells.

This entry is adapted from the peer-reviewed paper 10.3390/make4040052

References

  1. Sampaio, N.M.V.; Dunlop, M.J. Functional roles of microbial cell-to-cell heterogeneity and emerging technologies for analysis and control. Curr. Opin. Microbiol. 2020, 57, 87–94.
  2. Keegstra, J.M.; Kamino, K.; Anquez, F.; Lazova, M.D.; Emonet, T.; Shimizu, T.S. Phenotypic diversity and temporal variability in a bacterial signaling network revealed by single-cell FRET. eLife 2017, 6, e27455.
  3. Westfall, S.; Lomis, N.; Kahouli, I.; Dia, S.Y.; Singh, S.P.; Prakash, S. Microbiome, probiotics and neurodegenerative diseases: Deciphering the gut brain axis. Cell. Mol. Life Sci. 2017, 74, 3769–3787.
  4. Golding, C.G.; Lamboo, L.L.; Beniac, D.R.; Booth, T.F. The scanning electron microscope in microbiology and diagnosis of infectious disease. Sci. Rep. 2016, 6, 26516.
  5. Brahim Belhaouari, D.; Fontanini, A.; Baudoin, J.P.; Haddad, G.; Le Bideau, M.; Bou Khalil, J.Y.; Raoult, D.; La Scola, B. The strengths of scanning electron microscopy in deciphering SARS-CoV-2 infectious cycle. Front. Microbiol. 2020, 11, 2014.
  6. Guo, Y.; Liu, Y.; Georgiou, T.; Lew, M.S. A review of semantic segmentation using deep neural networks. Int. J. Multimed. Inf. Retr. 2018, 7, 87–93.
  7. Zhou, B.; Zhao, H.; Puig, X.; Xiao, T.; Fidler, S.; Barriuso, A.; Torralba, A. Semantic understanding of scenes through the ade20k dataset. Int. J. Comput. Vis. 2019, 127, 302–321.
  8. Arteta, C.; Lempitsky, V.; Noble, J.A.; Zisserman, A. Learning to detect partially overlapping instances. In Proceedings of the IEEE Conference on Computer Vision And Pattern Recognition, Portland, OR, USA, 23–28 June 2013; pp. 3230–3237.
  9. Yan, L.; Park, C.W.; Lee, S.R.; Lee, C.Y. New separation algorithm for touching grain kernels based on contour segments and ellipse fitting. J. Zhejiang Univ. Sci. 2011, 12, 54–61.
  10. Zhang, W.; Li, H. Automated segmentation of overlapped nuclei using concave point detection and segment grouping. Pattern Recognit. 2017, 71, 349–360.
  11. Mosaliganti, K.R.; Noche, R.R.; Xiong, F.; Swinburne, I.A.; Megason, S.G. ACME: Automated cell morphology extractor for comprehensive reconstruction of cell membranes. PLoS Comput. Biol. 2012, 8, e1002780.
  12. Vyas, N.; Sammons, R.; Addison, O.; Dehghani, H.; Walmsley, A. A quantitative method to measure biofilm removal efficiency from complex biomaterial surfaces using SEM and image analysis. Sci. Rep. 2016, 6, 32694.
  13. Zou, T.; Pan, T.; Taylor, M.; Stern, H. Recognition of overlapping elliptical objects in a binary image. Pattern Anal. Appl. 2021, 24, 1193–1206.
  14. Ren, S.; He, K.; Girshick, R.; Sun, J. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. arXiv 2015, arXiv:cs.CV/1506.01497.
  15. Litjens, G.; Kooi, T.; Bejnordi, B.E.; Setio, A.A.A.; Ciompi, F.; Ghafoorian, M.; van der Laak, J.A.; van Ginneken, B.; Sánchez, C.I. A survey on deep learning in medical image analysis. Med. Image Anal. 2017, 42, 60–88.
  16. Nistico, L.; Hall-Stoodley, L.; Stoodley, P. Imaging bacteria and biofilms on hardware and periprosthetic tissue in orthopedic infections. In Microbial Biofilms; Springer: Berlin/Heidelberg, Germany, 2014; pp. 105–126.
  17. Li, J.; Hirota, K.; Goto, T.; Yumoto, H.; Miyake, Y.; Ichikawa, T. Biofilm formation of Candida albicans on implant overdenture materials and its removal. J. Dent. 2012, 40, 686–692.
  18. Hägi, T.T.; Klemensberger, S.; Bereiter, R.; Nietzsche, S.; Cosgarea, R.; Flury, S.; Lussi, A.; Sculean, A.; Eick, S. A biofilm pocket model to evaluate different non-surgical periodontal treatment modalities in terms of biofilm removal and reformation, surface alterations and attachment of periodontal ligament fibroblasts. PLoS ONE 2015, 10, e0131056.
  19. Vincent, L. Morphological grayscale reconstruction in image analysis: Applications and efficient algorithms. IEEE Trans. Image Process. 1993, 2, 176–201.
  20. Cooper, L.A.; Kong, J.; Gutman, D.A.; Wang, F.; Gao, J.; Appin, C.; Cholleti, S.; Pan, T.; Sharma, A.; Scarpace, L.; et al. Integrated morphologic analysis for the identification and characterization of disease subtypes. J. Am. Med. Inform. Assoc. 2012, 19, 317–323.
  21. Vincent, L.; Soille, P. Watersheds in digital spaces: An efficient algorithm based on immersion simulations. IEEE Comput. Archit. Lett. 1991, 13, 583–598.
  22. Nath, S.K.; Palaniappan, K.; Bunyak, F. Cell Segmentation Using Coupled Level Sets and Graph-Vertex Coloring. In Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention 2006, Copenhagen, Denmark, 1–6 October 2006; Larsen, R., Nielsen, M., Sporring, J., Eds.; Springer: Berlin/Heidelberg, Germany, 2006; pp. 101–108.
  23. Dzyubachyk, O.; Niessen, W.; Meijering, E. Advanced level-set based multiple-cell segmentation and tracking in time-lapse fluorescence microscopy images. In Proceedings of the 2008 5th IEEE International Symposium on Biomedical Imaging: From Nano to Macro, Paris, France, 14–17 May 2008; pp. 185–188.
  24. Chang, H.; Han, J.; Borowsky, A.; Loss, L.; Gray, J.W.; Spellman, P.T.; Parvin, B. Invariant delineation of nuclear architecture in glioblastoma multiforme for clinical and molecular association. IEEE Trans. Med. Imaging 2012, 32, 670–682.
  25. Al-Kofahi, Y.; Lassoued, W.; Lee, W.; Roysam, B. Improved automatic detection and segmentation of cell nuclei in histopathology images. IEEE Trans. Biomed. Eng. 2009, 57, 841–852.
  26. Kumar, N.; Verma, R.; Sharma, S.; Bhargava, S.; Vahadane, A.; Sethi, A. A dataset and a technique for generalized nuclear segmentation for computational pathology. IEEE Trans. Med. Imaging 2017, 36, 1550–1560.
  27. Jeulin, D. Morphological Models of Random Structures; Springer: Berlin/Heidelberg, Germany, 2021.
  28. Yin, Z.; Bise, R.; Chen, M.; Kanade, T. Cell segmentation in microscopy imagery using a bag of local Bayesian classifiers. In Proceedings of the 2010 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, Rotterdam, The Netherlands, 14–17 April 2010; pp. 125–128.
  29. Su, H.; Yin, Z.; Huh, S.; Kanade, T. Cell segmentation in phase contrast microscopy images via semi-supervised classification over optics-related features. Med. Image Anal. 2013, 17, 746–765.
  30. Wang, W.; Taft, D.A.; Chen, Y.J.; Zhang, J.; Wallace, C.T.; Xu, M.; Watkins, S.C.; Xing, J. Learn to segment single cells with deep distance estimator and deep cell detector. Comput. Biol. Med. 2019, 108, 133–141.
  31. Al-Rifaie, M.M.; Aber, A.; Hemanth, D.J. Deploying swarm intelligence in medical imaging identifying metastasis, micro-calcifications and brain image segmentation. IET Syst. Biol. 2015, 9, 234–244.
  32. Brezočnik, L.; Fister, I., Jr.; Podgorelec, V. Swarm intelligence algorithms for feature selection: A review. Appl. Sci. 2018, 8, 1521.
  33. Li, C.; Xu, C.; Gui, C.; Fox, M.D. Distance regularized level set evolution and its application to image segmentation. IEEE Trans. Image Process. 2010, 19, 3243–3254.
  34. Zhang, K.; Song, H.; Zhang, L. Active contours driven by local image fitting energy. Pattern Recognit. 2010, 43, 1199–1206.
  35. Niu, S.; Chen, Q.; De Sisternes, L.; Ji, Z.; Zhou, Z.; Rubin, D.L. Robust noise region-based active contour model via local similarity factor for image segmentation. Pattern Recognit. 2017, 61, 104–119.
  36. Zhang, J.; Lu, Z.; Li, M. Active contour-based method for finger-vein image segmentation. IEEE Trans. Instrum. Meas. 2020, 69, 8656–8665.
  37. Fernandez, G.; Kunt, M.; Zryd, J.P. A new plant cell image segmentation algorithm. In Proceedings of the International Conference on Image Analysis and Processing, Washington, DC, USA, 23–26 October 1995; Springer: Berlin/Heidelberg, Germany, 1995; pp. 229–234.
  38. He, Y.; Meng, Y.; Gong, H.; Chen, S.; Zhang, B.; Ding, W.; Luo, Q.; Li, A. An automated three-dimensional detection and segmentation method for touching cells by integrating concave points clustering and random walker algorithm. PLoS ONE 2014, 9, e104437.
  39. Wang, H.; Zhang, H.; Ray, N. Clump splitting via bottleneck detection and shape classification. Pattern Recognit. 2012, 45, 2780–2787.
  40. Xing, F.; Yang, L. Chapter 4—Machine learning and its application in microscopic image analysis. In Machine Learning and Medical Imaging; Wu, G., Shen, D., Sabuncu, M.R., Eds.; The Elsevier and MICCAI Society Book Series; Academic Press: Cambridge, MA, USA, 2016; pp. 97–127.
  41. Pavlidis, T. Algorithms for shape analysis of contours and waveforms. IEEE Trans. Pattern Anal. Mach. Intell. 1980, 4, 301–312.
  42. Harris, C.G.; Stephens, M. A combined corner and edge detector. In Proceedings of the Alvey Vision Conference, Manchester, UK, 9–10 September 1988; Citeseer: University Park, PA, USA, 1988; Volume 15, pp. 10–5244.
  43. Miró-Nicolau, M.; Moyà-Alcover, B.; González-Hidalgo, M.; Jaume-i Capó, A. Segmenting overlapped objects in images. A study to support the diagnosis of sickle cell disease. arXiv 2020, arXiv:2008.00997.
  44. Zafari, S.; Eerola, T.; Sampo, J.; Kälviäinen, H.; Haario, H. Segmentation of overlapping elliptical objects in silhouette images. IEEE Trans. Image Process. 2015, 24, 5942–5952.
  45. Zhang, G.; Jayas, D.S.; White, N.D. Separation of Touching Grain Kernels in an Image by Ellipse Fitting Algorithm. Biosyst. Eng. 2005, 92, 135–142.
  46. Panagiotakis, C.; Argyros, A.A. Cell Segmentation Via Region-Based Ellipse Fitting. In Proceedings of the 2018 25th IEEE International Conference on Image Processing (ICIP), Athens, Greece, 7–10 October 2018; pp. 2426–2430.
  47. Panagiotakis, C.; Argyros, A. Parameter-Free Modelling of 2D Shapes with Ellipses. Pattern Recogn. 2016, 53, 259–275.
  48. Panagiotakis, C.; Argyros, A. Region-based Fitting of Overlapping Ellipses and its application to cells segmentation. Image Vis. Comput. 2020, 93, 103810.
  49. Abeyrathna, D.; Life, T.; Rauniyar, S.; Ragi, S.; Sani, R.; Chundi, P. Segmentation of Bacterial Cells in Biofilms Using an Overlapped Ellipse Fitting Technique. In Proceedings of the 2021 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), Houston, TX, USA, 9–12 December 2021; pp. 3548–3554.
  50. Yang, L.; Zhang, Y.; Guldner, I.H.; Zhang, S.; Chen, D.Z. 3D segmentation of glial cells using fully convolutional networks and k-terminal cut. In Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Athens, Greece, 17–21 October 2016; Springer: Berlin/Heidelberg, Germany, 2016; pp. 658–666.
  51. Saleh, H.M.; Saad, N.H.; Isa, N.A.M. Overlapping chromosome segmentation using U-net: Convolutional networks with test time augmentation. Procedia Comput. Sci. 2019, 159, 524–533.
  52. He, K.; Gkioxari, G.; Dollár, P.; Girshick, R.B. Mask R-CNN. In Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 22–29 October 2017; pp. 2980–2988.
  53. Ronneberger, O.; Fischer, P.; Brox, T. U-Net: Convolutional Networks for Biomedical Image Segmentation. In Proceedings of the Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015, Munich, Germany, 5–9 October 2015; pp. 234–241.
  54. Hu, R.L.; Karnowski, J.; Fadely, R.; Pommier, J.P. Image segmentation to distinguish between overlapping human chromosomes. arXiv 2017, arXiv:1712.07639.
  55. Kurnianingsih.; Allehaibi, K.H.S.; Nugroho, L.E.; Widyawan.; Lazuardi, L.; Prabuwono, A.S.; Mantoro, T. Segmentation and Classification of Cervical Cells Using Deep Learning. IEEE Access 2019, 7, 116925–116941.
  56. Lu, Z.; Carneiro, G.; Bradley, A.P. An Improved Joint Optimization of Multiple Level Set Functions for the Segmentation of Overlapping Cervical Cells. IEEE Trans. Image Process. 2015, 24, 1261–1272.
More
© Text is available under the terms and conditions of the Creative Commons Attribution (CC BY) license; additional terms may apply. By using this site, you agree to the Terms and Conditions and Privacy Policy.
This entry is offline, you can click here to edit this entry!
ScholarVision Creations
Feedback