Search

In this work, an unsupervised volumetric semantic instance segmentation of the plasma membrane of HeLa cells as observed with serial block face scanning electron microscopy is described. The resin background of the images was segmented at different slices of a 3D stack of 518 slices with 8192 × 8192 pixels each. The background was used to create a distance map, which helped identify and rank the cells by their size at each slice. The centroids of the cells detected at different slices were linked to identify them as a single cell that spanned a number of slices. A subset of these cells, i.e., the largest ones and those not close to the edges were selected for further processing. The selected cells were then automatically cropped to smaller regions of interest of 2000 × 2000 × 300 voxels that were treated as cell instances. Then, for each of these volumes, the nucleus was segmented, and the cell was separated from any neighbouring cells through a series of traditional image processing steps that followed the plasma membrane. The segmentation process was repeated for all the regions of interest previously selected. For one cell for which the ground truth was available, the algorithm provided excellent results in Accuracy (AC) and the Jaccard similarity Index (JI): nucleus: JI =0.9665, AC =0.9975, cell including nucleus JI =0.8711, AC =0.9655, cell excluding nucleus JI =0.8094, AC =0.9629. A limitation of the algorithm for the plasma membrane segmentation was the presence of background. In samples with tightly packed cells, this may not be available. When tested for these conditions, the segmentation of the nuclear envelope was still possible. All the code and data were released openly through GitHub, Zenodo and EMPIAR.

This paper describes a methodology that extracts key morphological features from histological breast cancer images in order to automatically assess Tumour Cellularity (TC) in Neo-Adjuvant treatment (NAT) patients. The response to NAT gives information on therapy efficacy and it is measured by the residual cancer burden index, which is composed of two metrics: TC and the assessment of lymph nodes. The data consist of whole slide images (WSIs) of breast tissue stained with Hematoxylin and Eosin (H&amp, E) released in the 2019 SPIE Breast Challenge. The methodology proposed is based on traditional computer vision methods (K-means, watershed segmentation, Otsu&rsquo, s binarisation, and morphological operations), implementing colour separation, segmentation, and feature extraction. Correlation between morphological features and the residual TC after a NAT treatment was examined. Linear regression and statistical methods were used and twenty-two key morphological parameters from the nuclei, epithelial region, and the full image were extracted. Subsequently, an automated TC assessment that was based on Machine Learning (ML) algorithms was implemented and trained with only selected key parameters. The methodology was validated with the score assigned by two pathologists through the intra-class correlation coefficient (ICC). The selection of key morphological parameters improved the results reported over other ML methodologies and it was very close to deep learning methodologies. These results are encouraging, as a traditionally-trained ML algorithm can be useful when limited training data are available preventing the use of deep learning approaches.

In this work, images of a HeLa cancer cell were semantically segmented with one traditional image-processing algorithm and three deep learning architectures: VGG16, ResNet18 and Inception-ResNet-v2. Three hundred slices, each 2000 × 2000 pixels, of a HeLa Cell were acquired with Serial Block Face Scanning Electron Microscopy. The deep learning architectures were pre-trained with ImageNet and then fine-tuned with transfer learning. The image-processing algorithm followed a pipeline of several traditional steps like edge detection, dilation and morphological operators. The algorithms were compared by measuring pixel-based segmentation accuracy and Jaccard index against a labelled ground truth. The results indicated a superior performance of the traditional algorithm (Accuracy = 99%, Jaccard = 93%) over the deep learning architectures: VGG16 (93%, 90%), ResNet18 (94%, 88%), Inception-ResNet-v2 (94%, 89%).

The quantitative study of cell morphology is of great importance as the structure and condition of cells and their structures can be related to conditions of health or disease. The first step towards that, is the accurate segmentation of cell structures. In this work, we compare five approaches, one traditional and four deep-learning, for the semantic segmentation of the nuclear envelope of cervical cancer cells commonly known as HeLa cells. Images of a HeLa cancer cell were semantically segmented with one traditional image-processing algorithm and four three deep learning architectures: VGG16, ResNet18, Inception-ResNet-v2, and U-Net. Three hundred slices, each 2000 × 2000 pixels, of a HeLa Cell were acquired with Serial Block Face Scanning Electron Microscopy. The first three deep learning architectures were pre-trained with ImageNet and then fine-tuned with transfer learning. The U-Net architecture was trained from scratch with 36, 000 training images and labels of size 128 × 128. The image-processing algorithm followed a pipeline of several traditional steps like edge detection, dilation and morphological operators. The algorithms were compared by measuring pixel-based segmentation accuracy and Jaccard index against a labelled ground truth. The results indicated a superior performance of the traditional algorithm (Accuracy = 99%, Jaccard = 93%) over the deep learning architectures: VGG16 (93%, 90%), ResNet18 (94%, 88%), Inception-ResNet-v2 (94%, 89%), and U-Net (92%, 56%).

Accurate measurements of cell morphology and behaviour are fundamentally important for understanding how disease, molecules and drugs affect cell function in vivo. Using muscle stem cell (muSC) responses to injury in zebrafish as our biological paradigm we established a ground truth for muSC behaviour. This revealed segmentation and tracking algorithms from commonly used programs are error-prone, leading us to develop a fast semi-automated image analysis pipeline that allows user defined parameters for segmentation and correction of cell tracking. Cell Tracking Profiler (CTP) is a package that runs two existing programs, HK Means and Phagosight within the Icy image analysis suite, to enable user-managed cell tracking from 3D time-lapsed datasets to provide measures of cell shape and movement. We demonstrate how CTP can be used to reveal changes to cell behaviour of muSCs in response to manipulation of the cell cytoskeleton by small molecule inhibitors. CTP and the associated tools we have developed for analysis of outputs thus provide a powerful framework for analysing complex cell behaviour in vivo from 4D datasets that are not amenable to straightforward analysis.

This paper describes an unsupervised algorithm, which segments the nuclear envelope of HeLa cells imaged by Serial Block Face Scanning Electron Microscopy. The algorithm exploits the variations of pixel intensity in different cellular regions by calculating edges, which are then used to generate superpixels. The superpixels are morphologically processed and those that correspond to the nuclear region are selected through the analysis of size, position, and correspondence with regions detected in neighbouring slices. The nuclear envelope is segmented from the nuclear region. The three-dimensional segmented nuclear envelope is then modelled against a spheroid to create a two-dimensional (2D) surface. The 2D surface summarises the complex 3D shape of the nuclear envelope and allows the extraction of metrics that may be relevant to characterise the nature of cells. The algorithm was developed and validated on a single cell and tested in six separate cells, each with 300 slices of 2000 &times, 2000 pixels. Ground truth was available for two of these cells, i.e., 600 hand-segmented slices. The accuracy of the algorithm was evaluated with two similarity metrics: Jaccard Similarity Index and Mean Hausdorff distance. Jaccard values of the first/second segmentation were 93%/90% for the whole cell, and 98%/94% between slices 75 and 225, as the central slices of the nucleus are more regular than those on the extremes. Mean Hausdorff distances were 9/17 pixels for the whole cells and 4/13 pixels for central slices. One slice was processed in approximately 8 s and a whole cell in 40 min. The algorithm outperformed active contours in both accuracy and time.

Accurate measurements of cell morphology and behaviour are fundamentally important for understanding how disease, molecules and drugs affect cell function in vivo. Using muscle stem cell (muSC) responses to injury in zebrafish as our biological paradigm we have established a ground truth for muSC cell behaviour. This revealed that variability in segmentation and tracking algorithms from commonly used programs are error-prone, leading us to develop a fast semi-automated image analysis pipeline that allows user defined parameters for segmentation and correction of cell tracking. Cell Tracking Profiler (CTP) operates through the freely available Icy platform, and allows user-managed cell tracking from 3D time-lapsed datasets to provide measures of cell shape and movement. Using dimensionality reduction methods, multiple correlation and regression analyses we identify myosin II-dependent parameters of muSC behaviour during regeneration. CTP and the associated statistical tools we have developed thus provide a powerful framework for analysing complex cell behaviour in vivo from 4D datasets.SummaryAnalysis of cell shape and movement from 3D time-lapsed datasets is currently very challenging. We therefore designed Cell Tracking Profiler for analysing cell behaviour from complex datasets and demonstrate its effectiveness by analysing stem cell behaviour during muscle regeneration in zebrafish.

This paper compares a series of traditional and deep learning methodologies for the segmentation of textures. Six well-known texture composites first published by Randen and Hus{\o}y were used to compare traditional segmentation techniques (co-occurrence, filtering, local binary patterns, watershed, multiresolution sub-band filtering) against a deep-learning approach based on the U-Net architecture. For the latter, the effects of depth of the network, number of epochs and different optimisation algorithms were investigated. Overall, the best results were provided by the deep-learning approach. However, the best results were distributed within the parameters, and many configurations provided results well below the traditional techniques.

The segmentation of power lines (PLs) from aerial images is a crucial task for the safe navigation of unmanned aerial vehicles (UAVs) operating at low altitudes. Despite the advances in deep learning-based approaches for PL segmentation, these models are still vulnerable to the class imbalance present in the data. The PLs occupy only a minimal portion (1–5%) of the aerial images as compared to the background region (95–99%). Generally, this class imbalance problem is addressed via the use of PL-specific detectors in conjunction with the popular class balanced cross entropy (BBCE) loss function. However, these PL-specific detectors do not work outside their application areas and a BBCE loss requires hyperparameter tuning for class-wise weights, which is not trivial. Moreover, the BBCE loss results in low dice scores and precision values and thus, fails to achieve an optimal trade-off between dice scores, model accuracy, and precision–recall values. In this work, we propose a generalized focal loss function based on the Matthews correlation coefficient (MCC) or the Phi coefficient to address the class imbalance problem in PL segmentation while utilizing a generic deep segmentation architecture. We evaluate our loss function by improving the vanilla U-Net model with an additional convolutional auxiliary classifier head (ACU-Net) for better learning and faster model convergence. The evaluation of two PL datasets, namely the Mendeley Power Line Dataset and the Power Line Dataset of Urban Scenes (PLDU), where PLs occupy around 1% and 2% of the aerial images area, respectively, reveal that our proposed loss function outperforms the popular BBCE loss by 16% in PL dice scores on both the datasets, 19% in precision and false detection rate (FDR) values for the Mendeley PL dataset and 15% in precision and FDR values for the PLDU with a minor degradation in the accuracy and recall values. Moreover, our proposed ACU-Net outperforms the baseline vanilla U-Net for the characteristic evaluation parameters in the range of 1–10% for both the PL datasets. Thus, our proposed loss function with ACU-Net achieves an optimal trade-off for the characteristic evaluation parameters without any bells and whistles. Our code is available at Github.

This paper describes an algorithm to segment the 3D nuclear envelope of HeLa cancer cells from electron microscopy images and model the volumetric shape of the nuclear envelope against an ellipsoid. The algorithm was trained on a single cell and then tested in six separate cells. To assess the algorithm, Jaccard similarity index and Hausdorff distance against a manually-delineated gold standard were calculated on two cells. The mean Jaccard value and Hausdorff distance that the segmentation achieved for central slices were 98% and 4 pixels for the first cell and 94% and 13 pixels for the second cell and outperformed segmentation with active contours. The modelling projects a 3D to a 2D surface that summarises the complexity of the shape in an intuitive result. Measurements extracted from the modelled surface may be useful to correlate shape with biological characteristics. The algorithm is unsupervised, fully automatic, fast and processes one image in less than 10 seconds. Code and data are freely available at https://github.com/reyesaldasoro/Hela-Cell-Segmentation and http://dx.doi.org/10.6019/EMPIAR-10094.

This paper compares a series of traditional and deep learning methodologies for the segmentation of textures. Six well-known texture composites first published by Randen and Hus&oslash, y were used to compare traditional segmentation techniques (co-occurrence, filtering, local binary patterns, watershed, multiresolution sub-band filtering) against a deep-learning approach based on the U-Net architecture. For the latter, the effects of depth of the network, number of epochs and different optimisation algorithms were investigated. Overall, the best results were provided by the deep-learning approach. However, the best results were distributed within the parameters, and many configurations provided results well below the traditional techniques.

This paper presents a novel software framework, called macrosight, which incorporates routines to detect, track, and analyze the shape and movement of objects, with special emphasis on macrophages. The key feature presented in macrosight consists of an algorithm to assess the changes of direction derived from cell&ndash, cell contact, where an interaction is assumed to occur. The main biological motivation is the determination of certain cell interactions influencing cell migration. Thus, the main objective of this work is to provide insights into the notion that interactions between cell structures cause a change in orientation. Macrosight analyzes the change of direction of cells before and after they come in contact with another cell. Interactions are determined when the cells overlap and form clumps of two or more cells. The framework integrates a segmentation technique capable of detecting overlapping cells and a tracking framework into a tool for the analysis of the trajectories of cells before and after they overlap. Preliminary results show promise into the analysis and the hypothesis proposed, and lays the groundwork for further developments. The extensive experimentation and data analysis show, with statistical significance, that under certain conditions, the movement changes before and after an interaction are different from movement in controlled cases.

This paper describes an image-processing pipeline for the automatic segmentation of the nuclear envelope of HeLa cells observed through Electron Microscopy. The pipeline was applied to a 3D stack of 300 images. The intermediate results of neighbouring slices are further combined to improve the final results. Comparison with a hand-segmented ground truth reported Jaccard similarity values between 94–98% on the central slices with a decrease towards the edges of the cell where the structure was considerably more complex. The processing is unsupervised and each 2D slice is processed in about 5–10 s running on a MacBook Pro. No systematic attempt to make the code faster was made. These encouraging results could be further used to provide data for more complex segmentation techniques like Deep Learning, which require a considerable amount of data to train architectures like Convolutional Neural Networks. The code is freely available from https://github.com/reyesaldasoro/HeLa-Cell-Segmentation.

Understanding the migrating patterns of cells in the immune system is of great importance; especially the changes of direction and its cause. For macrophages and other immune cells, excessive migration could be related to autoimmune diseases and cancer. In this work, an algorithm to analyse the change in direction of cells before and after they interact with another cell is proposed. The main objective is to provide insights into the notion that interactions between cell structures appear to anticipate migration. Such interactions are determined when the cells overlap and form clumps of two or more cells. The algorithm integrates a segmentation technique capable of detecting overlapping cells and a tracking framework into a tool for the analysis of the trajectories of cells before and after they overlap. The preliminary results show promise into the analysis and the hypothesis proposed, and it lays the ground work for further developments.

We present a combined report on the results of three editions of the Cell Tracking Challenge, an ongoing initiative aimed at promoting the development and objective evaluation of cell segmentation and tracking algorithms. With 21 participating algorithms and a data repository consisting of 13 data sets from various microscopy modalities, the challenge displays today's state-of-the-art methodology in the field. We analyzed the challenge results using performance measures for segmentation and tracking that rank all participating methods. We also analyzed the performance of all of the algorithms in terms of biological measures and practical usability. Although some methods scored high in all technical aspects, none obtained fully correct solutions. We found that methods that either take prior information into account using learning strategies or analyze cells in a global spatiotemporal video context performed better than other methods under the segmentation and tracking scenarios included in the challenge.

This paper describes the automatic segmentation of overlapping cells through different algorithms. As the first step, the algorithm detects junctions between the boundaries of overlapping objects based on the angles between points of the overlapping boundary. For this purpose, a novel 2D matrix with multiscale angle variation is introduced, i.e. anglegram. The anglegram is used to find junctions of overlapping cells. The algorithm to retrieve junctions from the boundary was tested and validated with synthetic data and fluorescently labelled macrophages observed on embryos of Drosophila melanogaster. Then, four different segmentation techniques were evaluated: (i) a Voronoi partition based on the nuclei positions, (ii) a slicing method, which joined the clumps together (junction slicing), (iii) a partition based on the following of the edges from the junctions (edge following), and (iv) a custom self-organising map to fit to the area of overlap between the cells. Only (ii)-(iv) were based on the junctions. The segmentation results were compared based on precision, recall and Jaccard similarity. The algorithm that reported the best segmentation was the junction slicing.

State-of-the-art CTA imaging equipment has increased clinician’s ability to make non-invasive diagnoses of coronary heart disease; however, an effective interpretation of the cardiac CTA becomes cumbersome due to large amount of imaged data. Intensity based background suppression is often used to enhance the coronary vasculature but setting a fixed threshold to discriminate coronaries from fatty muscles could be misleading due to non-homogeneous response of contrast medium in CTA volumes. In this work, we propose a volume-specific model of the contrast medium in the coronary segmentation process to improve the segmentation accuracy. The influence of the contrast medium in a CTA volume was modelled by approximating the intensity histogram of the descending aorta with Gaussian approximation. It should be noted that a significant variation in Gaussian mean for 12 CTA volumes validates the need of volume-wise exclusive intensity threshold for accurate coronary segmentation. Moreover, the effectiveness of the adaptive intensity threshold is illustrated with the help of qualitative and quantitative results.

Localized Energy for Non-Homogeneous Image DataGlobal Discontinuity Map of CTA using Probabilistic ModelHybrid Energy for Complex Coronary SegmentationRotterdam Coronary Artery CTA DatasetManual Annotaion-Inter Observer Variability. Background and ObjectiveState-of-the-art medical imaging techniques have enabled non-invasive imaging of the internal organs. However, high volumes of imaging data make manual interpretation and delineation of abnormalities cumbersome for clinicians. These challenges have driven intensive research into efficient medical image segmentation. In this work, we propose a hybrid region-based energy formulation for effective segmentation in computed tomography angiography (CTA) imagery. MethodsThe proposed hybrid energy couples an intensity-based local term with an efficient discontinuity-based global model of the image for optimal segmentation. The segmentation is achieved using a level set formulation due to the computational robustness. After validating the statistical significance of the hybrid energy, we applied the proposed model to solve an important clinical problem of 3D coronary segmentation. An improved seed detection method is used to initialize the level set evolution. Moreover, we employed an auto-correction feature that captures the emerging peripheries during the curve evolution for completeness of the coronary tree. ResultsWe evaluated the segmentation accuracy of the proposed energy model against the existing techniques in two stages. Qualitative and quantitative results demonstrate the effectiveness of the proposed framework with a consistent mean sensitivity and specificity measures of 80% across the CTA data. Moreover, a high degree of agreement with respect to the inter-observer differences justifies the generalization of the proposed method. ConclusionsThe proposed method is effective to segment the coronary tree from the CTA volume based on hybrid image based energy, which can improve the clinicians ability to detect arterial abnormalities.

A fully automatic segmentation and morphological analysis algorithm for the analysis of microvessels from CD31 immunostained histological tumour sections is presented. Development of the algorithm exploited the distinctive hues of stained vascular endothelial cells, cell nuclei and background, to provide the seeds for a 'region-growing' method for object segmentation in the 3D hue, saturation, value (HSV) colour model. The segmented objects, identified as microvessels by CD31 immunostaining, were post-processed with three morphological tasks: joining separate objects that were likely to belong to a single vessel, closing objects that had a narrow gap around their periphery, and splitting objects with multiple lumina into individual vessels. The automatic segmentation was validated against a hand-segmented set of 44 images from three different SW1222 human colorectal carcinomas xenografted into mice. 96.3 ± 0.9% of pixels were found to be correctly classified. Automated segmentation was carried out on a further 53 images from three histologically distinct mouse fibrosarcomas (MFs) for morphological comparison with the SW1222 tumours. Four morphometric measurements were calculated for each segmented vessel: vascular area (VA), ratio of lumen area to vascular area (lu/VA), eccentricity (e), and roundness (ro). In addition, the total vascular area relative to tumour tissue area (rVA) was calculated. lu/VA, e and ro were found to be significantly smaller in MF tumours than in SW1222 tumours (p < 0.05; unpaired t-test). The algorithm is available through the website http://www.caiman.org.uk where images can be uploaded, processed and sent back to users. The output from CAIMAN consists of the original image with boundaries of segmented vessels overlaid, the calculated parameters and a Matlab file, which contains the segmentation that the user can use to derive further results.

The movement of small animals in well-defined environments is increasingly used in ecotoxicology as a sensitive indicator of water contamination with toxicants. This work describes a framework for the analysis of Artermia franciscana within miniaturised chambers. This work combines: (a) a proof-of- concept miniaturised Lab-on-a-Chip (LOC) platform where any changes in swimming patterns exhibited by Artemia franciscana can be related to the toxicity of the environment with (b) a pipeline of image processing steps for the movement analysis of video sequences of test specimens. The analysis pipeline is semi- automatic as it allows a user to define a region of interest and confirm intensity levels that are used for two-level hysteresis segmentation. Tracking is automatic, and a large number of statistical measurements and visualisation tools are provided. In addition, proofreading tools for validation of tracks are provided to link, break or delete tracks. The software consists of Matlab® m-files and is available in the website http://www.objecttrack.org.uk/.

This work describes an automatic algorithm for the segmentation and quantification of focal adhesions from mouse embryonic fibroblasts. The main challenges solved by this algorithm are: the variability of the intensity of the focal adhesions, the detection of an outer ring, which distinguishes the cell periphery responsible for the cell migration, and the quantification of the characteristics of the focal adhesions. The algorithm detects maximal regions through gradients and uses a region-growing algorithm limited by intensity-based edges. The outer ring is calculated based on the average radial intensity from an extended centroid of the cell. Finally, traditional morphological characteristics are obtained to distinguish between two groups of cells. Two of the measurements employed showed statistical difference between two groups of cells.

Neutrophil migration in zebrafish larvae is increasingly used as a model to study the response of these leukocytes to different determinants of the cellular inflammatory response. However, it remains challenging to extract comprehensive information describing the behaviour of neutrophils from the multi-dimensional data sets acquired with widefield or confocal microscopes. Here, we describe PhagoSight, an open-source software package for the segmentation, tracking and visualisation of migrating phagocytes in three dimensions. The algorithms in PhagoSight extract a large number of measurements that summarise the behaviour of neutrophils, but that could potentially be applied to any moving fluorescent cells. To derive a useful panel of variables quantifying aspects of neutrophil migratory behaviour, and to demonstrate the utility of PhagoSight, we evaluated changes in the volume of migrating neutrophils. Cell volume increased as neutrophils migrated towards the wound region of injured zebrafish. PhagoSight is openly available as MATLAB® m-files under the GNU General Public License. Synthetic data sets and a comprehensive user manual are available from http://www.phagosight.org.

A feature selection methodology based on a novel Bhattacharyya space is presented and illustrated with a texture segmentation problem. The Bhattacharyya space is constructed from the Bhattacharyya distances of different measurements extracted with sub-band filters from training samples. The marginal distributions of the Bhattacharyya space present a sequence of the most discrimmant sub-bands that can be used as a path for a wrapper algorithm. When this feature selection is used with a multiresolution classification algorithm on a standard set of texture mosaics, it produces the lowest misclassification errors reported. (c) 2006 Pattern Recognition Society. Published by Elsevier Ltd. All rights reserved.