Approach for Overlapped Segmentation of Bacterial Cell Images: Comparison
Please note this is a comparison between Version 1 by Pei-Chi Huang and Version 2 by Camila Xu.

Scanning electron microscopy (SEM) techniques have been extensively performed to image and study bacterial cells with high-resolution images. Bacterial image segmentation in SEM images is an essential task to distinguish an object of interest and its specific region.

  • overlapping cell segmentation
  • bacterial cell segmentation
  • U-Net

1. Introduction

Microscopic observation of cellular morphology is a preliminary step in microbiology research. Cell size variability in microbes is indicative of cellular response to environmental stimuli through physiological and gene expression changes [1][2][1,2]. It has also been reported that cellular housekeeping, nutrient transport, and cell reproduction are associated with cell size variations [3]. Automated cell segmentation techniques in microscopy imaging are essential for measuring cellular characteristics such as alteration in size for the evaluation of the impact of specific environmental change [4][5][4,5]. Analysis of a large number of cells requires automated cell recognition techniques to enable faster decision-making.
To gain valuable information for semantic understanding [6] of images, object detection techniques should precisely estimate the shapes and locations of objects in each image. In machine learning, semantic understanding is defined as the ability of a machine to understand the meaning and context behind real-world information [7]. Understanding an image containing a large number of objects of interest is a task of great importance for monitoring in multiple domains, such as cells of in vitro cultures and developing embryos [8], grain analysis [9], and nuclei analysis [10]. However, the complexity of the bacterial cell segmentation task is a barrier, as the intensity and texture of the foreground and background are similar, and most clustered bacterial cells in images that are partially overlapping with each other. Many practical approaches have been proposed in the literature to tackle the overlapping objects segmentation problem, such as objects in different colors [11], level sets [12], ellipse fitting [13], and the watershed algorithm [14][15][14,15]. The traditional approaches to identify cell regions in microscopy images are labor intensive and heavily dependent on the professional knowledge of researchers.
Cell segmentation and growth rate measures of bacteria in a changed growth condition are crucial for developing an instrument with different materials for bacteria cell growth, pattern recognition, and further understanding the behavior and impact on cell cycle progression. A considerable research effort is being made to develop methods for extracting cell regions from its surface and background, segmenting the cell surfaces, and counting/measuring the cell growth proliferation [12]. Direct imaging techniques, such as confocal laser microscopy, electron microscopy, light microscopy, bioluminescence imaging, and macroscale photography have been frequently used in experiments on bacteria cell detection and to extract measurements. These imaging techniques have become more convenient to experiment with, rapid, and standard diagnostic methods for analyzing morphological disruption [16]. Scanning electron microscopy (SEM) techniques have been extensively performed to image and study bacterial cells with high-resolution images [17][18][17,18]. However, distinguishing the cells in a cluster remains challenging in many SEM images because of the similar pixel intensities and touching and overlapping cell clusters, leading to significant difficulties in extracting quantitative information.

2. Traditional Approaches for Overlapping Objects Segmentation

Traditional early-stage image segmentation tasks have applied thresholding techniques where each pixel is classified as either an object or as part of the background based on a chosen threshold value. The well-known approaches include morphological operations [19][20][19,20], watershed segmentation [21], levelset methods [22][23][22,23], graph-based approaches [24][25][24,25], and their variations [26][27][26,27]. These methods mainly focus on features, such as gradient, color, and/or structural distributions. However, due to the variability of cell segmentation tasks, e.g., heterogeneous object/cell structures and shared features between foreground and background, the parameters in these methods need to be customized for application by the proper selection of the structuring elements. Similarly, traditional machine learning approaches have been used for cell segmentation [28][29][28,29], but most of the approaches were evaluated in situations with uniform cell appearance and a high contrast between the foreground and background. Furthermore, these approaches are too labor-intensive and time consuming (especially for large datasets) to have generalized segmentation performance and robustness for cell segmentation tasks [30]. Furthermore, optimization algorithms such as swarm intelligence and genetic algorithms have been extended to perform medical image segmentation applications [31][32][31,32]. However, these approaches have not been used in complex segmentation applications.

3. Contour-Based Methods

The contour-based method employs a set of steps to represent the curve evolution of segmented regions in an image, including curvature, skeleton, and polygon approximation. The method treats the instance segmentation as a regression task where a contour can be reduced to a series of discrete vertex coordinates. Contour detection has attracted extensive attention in the field of image segmentation for overlapping or touching object segmentation [33][34][35][36][33,34,35,36]. Many contour detection algorithms have been introduced in the literature to understand the nature of overlapping objects. Refs. [37][38][37,38] adopted sliding window-based approaches to extract the contours and foreground object from the background. Similarly, Ref. [39] proposed a bottleneck detector that identifies a pair of splitting points with a minimum Euclidean distance transform (EDT) while maximizing their distance over the contours for concave corners identification. However, these methods are highly prone to noise [40] and tend to retrieve false corner points, leading to low accuracy. Hence, extensive preprocessing is required to avoid noise before the ellipse fitting stage. Besides the aforementioned corner detection approaches with manual object segmentation, the general corner identification algorithms, such as k-curvature [41][42][41,42], have been successfully used to extract corners for overlapping object instances as well. For example, Ref. [10] proposed a corner point identification that uses the Harris corner detection algorithm to generate candidate points and then extract obvious and uncertain concave points from the candidate points. However, the method is constrained by the parameter settings, which is reflected in the low degree of generalizability of the models. Furthermore, these algorithms cannot perform well when object boundaries appear blurry. Such a situation requires deep learning methods for feature extractions to identify precise object boundaries under different conditions. For example, Ref. [43] adopted a k-curvature technique to determine the candidate curve points and further improve the detection of corner points.

4. Ellipse-Fitting Methods

Most of the ellipse-fitting approaches have been proposed to address touching elliptical-shaped objects for segmentation tasks [13][44][13,44]. For example, the segmentation task of overlapping elliptical grains [45] and cell nuclei [46] utilizes the multi-ellipse fitting method to set a minimum threshold of the expected area of each cell that can automatically detect and split touching cells. Despite its great potential, the lack of generalizability has hindered the widespread adoption of different applications with specific type objects due to the requirement of task-specific rules and parameters. Some recent studies have extended ellipse-fitting techniques to achieve more accurate segmentation results. For example, Ref. [13] proposed that the modified ellipse-fitting technique generates candidate ellipses and selects the optimal ellipse among the candidate pool to recognize overlapping elliptical objects in a binary image from given concave points that are extracted by a polygon approximation algorithm. Ref. [47] proposed a parameter-free decremental ellipse fitting algorithm (DEFA) for automatically estimating the number and the parameters of ellipses objects by exploiting the skeleton of a shape. However, the approach requires binarization of elliptical-shaped objects in an image or images with high contrast between foreground and background. Ref. [48] proposed an improved version of DEFA, called region-based fitting of overlapping ellipses (RFOVE), which automatically determines the number of possibly overlapping ellipses by optimizing the area of shape coverage from unsupervised learning and operating with previously unknown shapes. Furthermore, Refs. [46][49][46,49] have been successfully implemented to accommodate overlapping cell segmentation tasks to retrieve accurate quantitative measures in SEM images.

5. Deep Learning Methods

Owing to the huge success of deep learning (DL), it has become the de facto choice for advanced segmentation tasks. Recently, because of its capability for better feature extraction and accurate segmentation quality, many cell segmentation methods and applications with DL techniques have been widely adopted in cell segmentation [15][50][51][15,50,51]. DL methods for object segmentation can be broadly classified into two categories:
  • Instance segmentation: mask R-CNN [52] is a well-known deep neural network architecture for multi-objects detection, and extends faster R-CNN [14] by adding an extra branch for predicting segmentation masks while simultaneously recognizing the bounding box from the existing branch. Mask R-CNN uses a region proposal-based object detection and uses a high-quality segmentation mask technique to achieve instance segmentation results. However, this method cannot perform well in situations with heavily overlapping object instances or highly close object occurrences due to greedy non-maximum suppression post-processing.
  • Semantic segmentation: U-Net [53] deep learning architecture is recognized as another popular semantic segmentation approach that neither employs region proposals nor reuses pooling indices. Instead, it uses encoder–decoder-based neural network architecture to predict a class-based object segmentation output. The U-Net architecture has been successfully used in many overlapping cell segmentation tasks [51][54][51,54], especially in the medical community, because of its intrinsic capability to perform down sampling–up sampling. For example, research studies by [55][56][55,56] have demonstrated how to use the architecture to accurately segment overlapping cervical cells.
Video Production Service