Search

Fixed underwater observatories (FUO), equipped with digital cameras and other sensors, become more commonly used to record different kinds of time series data for marine habitat monitoring. With increasing numbers of campaigns, numbers of sensors and campaign time, the volume and heterogeneity of the data, ranging from simple temperature time series to series of HD images or video call for new data science approaches to analyze the data. While some works have been published on the analysis of data from one campaign, we address the problem of analyzing time series data from two consecutive monitoring campaigns (starting late 2017 and late 2018) in the same habitat. While the data from campaigns in two separate years provide an interesting basis for marine biology research, it also presents new data science challenges, like the the marine image analysis in data form more than one campaign. In this paper, we analyze the polyp activity of two Paragorgia arborea cold water coral (CWC) colonies using FUO data collected from November 2017 to June 2018 and from December 2018 to April 2019. We successfully apply convolutional neural networks (CNN) for the segmentation and classification of the coral and the polyp activities. The result polyp activity data alone showed interesting temporal patterns with differences and similarities between the two time periods. A one month "sleeping" period in spring with almost no activity was observed in both coral colonies, but with a shift of approximately one month. A time series prediction experiment allowed us to predict the polyp activity from the non-image sensor data using recurrent neural networks (RNN). The results pave a way to a new multi-sensor monitoring strategy for Paragorgia arborea behaviour. [ABSTRACT FROM AUTHOR]

An online platform for the management of digital visual data for the condition monitoring of protective coatings on marine steel structures The paper presents an online platform for the annotation and management of visual digital data, which allows for the condition monitoring and maintenance modelling of protective coatings of marine steel structures by means of digital image data only. A 10‐step procedure is introduced which includes the data acquisition, transfer of the data into the online platform, the annotation and management of the data by means of special tools and methods, and the modelling for condition monitoring and maintenance purposes. Some special applications of the platform are exemplary illustrated. [ABSTRACT FROM AUTHOR]

Hyperspectral imaging (HSI) is a promising technology for environmental monitoring with a lot of undeveloped potential due to the high dimensionality and complexity of the data. If temporal effects are studied, such as in a monitoring context, the analysis becomes more challenging as time is added to the dimensions of space (image coordinates) and wavelengths. We conducted a series of laboratory experiments to investigate the impact of different stressor exposure patterns on the spectrum of the cold water coral Desmophyllum pertusum. 65 coral samples were divided into 12 groups, each group being exposed to different types and levels of particles. Hyperspectral images of the coral samples were collected at four time points from prior to exposure to 6 weeks after exposure. To investigate the relationships between the corals' spectral signatures and controlled experimental parameters, a new software tool for interactive visual exploration was developed and applied, the HypIX (Hyperspectral Image eXplorer) web tool. HypIX combines principles from exploratory data analysis, information visualization and machine learning-based dimension reduction. This combination enables users to select regions of interest (ROI) in all dimensions (2D space, time point and spectrum) for a flexible integrated inspection. We propose two HypIX workflows to find relationships in time series of hyperspectral datasets, namely morphology-based filtering workflow and embedded driven response analysis workflow. With these HypIX workflows three users identified different temporal and spatial patterns in the spectrum of corals exposed to different particle stressor conditions. Corals exposed to particles tended to have a larger change rate than control corals, which was evident as a shifted spectrum. The responses, however, were not uniform for coral samples undergoing the same exposure treatments, indicating individual tolerance levels. We also observed a good inter-observer agreement between the three HyPIX users, indicating that the proposed workflow can be applied to obtain reproducible HSI analysis results. [ABSTRACT FROM AUTHOR]

Data augmentation is an established technique in computer vision to foster the generalization of training and to deal with low data volume. Most data augmentation and computer vision research are focused on everyday images such as traffic data. The application of computer vision techniques in domains like marine sciences has shown to be not that straightforward in the past due to special characteristics, such as very low data volume and class imbalance, because of costly manual annotation by human domain experts, and general low species abundances. However, the data volume acquired today with moving platforms to collect large image collections from remote marine habitats, like the deep benthos, for marine biodiversity assessment and monitoring makes the use of computer vision automatic detection and classification inevitable. In this work, we investigate the effect of data augmentation in the context of taxonomic classification in underwater, i.e., benthic images. First, we show that established data augmentation methods (i.e., geometric and photometric transformations) perform differently in marine image collections compared to established image collections like the Cityscapes dataset, showing everyday traffic images. Some of the methods even decrease the learning performance when applied to marine image collections. Second, we propose new data augmentation combination policies motivated by our observations and compare their effect to those proposed by the AutoAugment algorithm and can show that the proposed augmentation policy outperforms the AutoAugment results for marine image collections. We conclude that in the case of small marine image datasets, background knowledge, and heuristics should sometimes be applied to design an effective data augmentation method. [ABSTRACT FROM AUTHOR]

The evaluation of large amounts of digital image data is of growing importance for biology, including for the exploration and monitoring of marine habitats. However, only a tiny percentage of the image data collected is evaluated by marine biologists who manually interpret and annotate the image contents, which can be slow and laborious. In order to overcome the bottleneck in image annotation, two strategies are increasingly proposed: “citizen science” and “machine learning”. In this study, we investigated how the combination of citizen science, to detect objects, and machine learning, to classify megafauna, could be used to automate annotation of underwater images. For this purpose, multiple large data sets of citizen science annotations with different degrees of common errors and inaccuracies observed in citizen science data were simulated by modifying “gold standard” annotations done by an experienced marine biologist. The parameters of the simulation were determined on the basis of two citizen science experiments. It allowed us to analyze the relationship between the outcome of a citizen science study and the quality of the classifications of a deep learning megafauna classifier. The results show great potential for combining citizen science with machine learning, provided that the participants are informed precisely about the annotation protocol. Inaccuracies in the position of the annotation had the most substantial influence on the classification accuracy, whereas the size of the marking and false positive detections had a smaller influence. [ABSTRACT FROM AUTHOR]

Digital imaging has become one of the most important techniques in environmental monitoring and exploration. In the case of the marine environment, mobile platforms such as autonomous underwater vehicles (AUVs) are now equipped with high-resolution cameras to capture huge collections of images from the seabed. However, the timely evaluation of all these images presents a bottleneck problem as tens of thousands or more images can be collected during a single dive. This makes computational support for marine image analysis essential. Computer-aided analysis of environmental images (and marine images in particular) with machine learning algorithms is promising, but challenging and different to other imaging domains because training data and class labels cannot be collected as efficiently and comprehensively as in other areas. In this paper, we present Machine learning Assisted Image Annotation (MAIA), a new image annotation method for environmental monitoring and exploration that overcomes the obstacle of missing training data. The method uses a combination of autoencoder networks and Mask Region-based Convolutional Neural Network (Mask R-CNN), which allows human observers to annotate large image collections much faster than before. We evaluated the method with three marine image datasets featuring different types of background, imaging equipment and object classes. Using MAIA, we were able to annotate objects of interest with an average recall of 84.1% more than twice as fast as compared to “traditional” annotation methods, which are purely based on software-supported direct visual inspection and manual annotation. The speed gain increases proportionally with the size of a dataset. The MAIA approach represents a substantial improvement on the path to greater efficiency in the annotation of large benthic image collections. [ABSTRACT FROM AUTHOR]

The application of protective coating systems is the major measure against the corrosion of stationary marine steel structures. During their service, protective coatings are exposed to various loads, and their protectiveness deteriorates. Digital visual inspection tools are becoming a standard for the condition monitoring of coatings for large and complex structures. One of the challenges with the use of these tools is the accumulation of large amounts of visual data to be evaluated and assessed in short time intervals. A comprehensive interpretation of such image collections requires detection, classification, and quantification processes, which are usually performed by domain experts (coating inspectors). However, the data volume and the rich content of the images make the support by software tools inevitable. This contribution defines requirements for image data annotation, and it presents for the first time the application of an online exploration and annotation tool to coating condition monitoring data and its introduction into a prescriptive coating maintenance model. The software architecture of the system is described, special features are illustrated with different use-cases, and future developments are discussed. The utilization of machine learning-based assessment tools is presented. • First integrative data-driven approach for online analysis of large volumes of visual monitoring data. • Integration of visual data into a prescriptive maintenance model. • Coating deterioration modeling based on pixel footprints of images. [ABSTRACT FROM AUTHOR]

The study of mapping and interaction of co-localized proteins at a sub-cellular level is important for understanding complex biological phenomena. One of the recent techniques to map co-localized proteins is to use the standard immuno-fluorescence microscopy in a cyclic manner (Nat Biotechnol 24:1270-8, 2006; Proc Natl Acad Sci 110:11982-7,2013). Unfortunately, these techniques suffer from variability in intensity and positioning of signals from protein markers within a run and across different runs. Therefore, it is necessary to standardize protocols for preprocessing of the multiplexed bioimaging (MBI) data from multiple runs to a comparable scale before any further analysis can be performed on the data. In this paper, we compare various normalization protocols and propose on the basis of the obtained results, a robust normalization technique that produces consistent results on the MBI data collected from different runs using the Toponome Imaging System TIS). Normalization results produced by the proposed method on a sample TIS data set for colorectal cancer patients were ranked favorably by two pathologists and two biologists. We show that the proposed method produces higher between class Kullback-Leibler (KL) divergence and lower within class KL divergence on a distribution of cell phenotypes from colorectal cancer and histologically normal samples. [ABSTRACT FROM AUTHOR]

Background With the advent of low cost, fast sequencing technologies metagenomic analyses are made possible. The large data volumes gathered by these techniques and the unpredictable diversity captured in them are still, however, a challenge for computational biology. Results In this paper we address the problem of rapid taxonomic assignment with small and adaptive data models (< 5 MB) and present the accelerated k-mer explorer (AKE). Acceleration in AKE's taxonomic assignments is achieved by a special machine learning architecture, which is well suited to model data collections that are intrinsically hierarchical. We report classification accuracy reasonably well for ranks down to order, observed on a study on real world data (Acid Mine Drainage, Cow Rumen). Conclusion We show that the execution time of this approach is orders of magnitude shorter than competitive approaches and that accuracy is comparable. The tool is presented to the public as a web application (url: https://ani.cebitec.uni-bielefeld.de/ake/, username: bmc, password: bmcbioinfo). [ABSTRACT FROM AUTHOR]

Diatoms represent one of the morphologically and taxonomically most diverse groups of microscopic eukaryotes. Light microscopy-based taxonomic identification and enumeration of frustules, the silica shells of these microalgae, is broadly used in aquatic ecology and biomonitoring. One key step in emerging digital variants of such investigations is segmentation, a task that has been addressed before, but usually in manually captured megapixel-sized images of individual diatom cells with a mostly clean background. In this paper, we applied deep learning-based segmentation methods to gigapixel-sized, high-resolution scans of diatom slides with a realistically cluttered background. This setup requires large slide scans to be subdivided into small images (tiles) to apply a segmentation model to them. This subdivision (tiling), when done using a sliding window approach, often leads to cropping relevant objects at the boundaries of individual tiles. We hypothesized that in the case of diatom analysis, reducing the amount of such cropped objects in the training data can improve segmentation performance by allowing for a better discrimination of relevant, intact frustules or valves from small diatom fragments, which are considered irrelevant when counting diatoms. We tested this hypothesis by comparing a standard sliding window / fixed-stride tiling approach with two new approaches we term object-based tile positioning with and without object integrity constraint. With all three tiling approaches, we trained Mask-R-CNN and U-Net models with different amounts of training data and compared their performance. Object-based tiling with object integrity constraint led to an improvement in pixel-based precision by 12–17 percentage points without substantially impairing recall when compared with standard sliding window tiling. We thus propose that training segmentation models with object-based tiling schemes can improve diatom segmentation from large gigapixel-sized images but could potentially also be relevant for other image domains. [ABSTRACT FROM AUTHOR]

Motivation: Bioimaging techniques rapidly develop toward higher resolution and dimension. The increase in dimension is achieved by different techniques such as multitag fluorescence imaging, Matrix Assisted Laser Desorption / Ionization (MALDI) imaging or Raman imaging, which record for each pixel an N-dimensional intensity array, representing local abundances of molecules, residues or interaction patterns. The analysis of such multivariate bioimages (MBIs) calls for new approaches to support users in the analysis of both feature domains: space (i.e. sample morphology) and molecular colocation or interaction. In this article, we present our approach WHIDE (Web-based Hyperbolic Image Data Explorer) that combines principles from computational learning, dimension reduction and visualization in a free web application.Results: We applied WHIDE to a set of MBI recorded using the multitag fluorescence imaging Toponome Imaging System. The MBI show field of view in tissue sections from a colon cancer study and we compare tissue from normal/healthy colon with tissue classified as tumor. Our results show, that WHIDE efficiently reduces the complexity of the data by mapping each of the pixels to a cluster, referred to as Molecular Co-Expression Phenotypes and provides a structural basis for a sophisticated multimodal visualization, which combines topology preserving pseudocoloring with information visualization. The wide range of WHIDE's applicability is demonstrated with examples from toponome imaging, high content screens and MALDI imaging (shown in the Supplementary Material).Availability and implementation: The WHIDE tool can be accessed via the BioIMAX website http://ani.cebitec.uni-bielefeld.de/BioIMAX/; Login: whidetestuser; Password: whidetest.Supplementary information: Supplementary data are available at Bioinformatics online.Contact: tim.nattkemper@uni-bielefeld.de [ABSTRACT FROM AUTHOR]

Deep convolutional neural networks are emerging as the state of the art method for supervised classification of images also in the context of taxonomic identification. Different morphologies and imaging technologies applied across organismal groups lead to highly specific image domains, which need customization of deep learning solutions. Here we provide an example using deep convolutional neural networks (CNNs) for taxonomic identification of the morphologically diverse microalgal group of diatoms. Using a combination of high-resolution slide scanning microscopy, web-based collaborative image annotation and diatom-tailored image analysis, we assembled a diatom image database from two Southern Ocean expeditions. We use these data to investigate the effect of CNN architecture, background masking, data set size and possible concept drift upon image classification performance. Surprisingly, VGG16, a relatively old network architecture, showed the best performance and generalizing ability on our images. Different from a previous study, we found that background masking slightly improved performance. In general, training only a classifier on top of convolutional layers pre-trained on extensive, but not domain-specific image data showed surprisingly high performance (F1 scores around 97%) with already relatively few (100–300) examples per class, indicating that domain adaptation to a novel taxonomic group can be feasible with a limited investment of effort. [ABSTRACT FROM AUTHOR]

The application of protective coating systems is the major measure against the corrosion of steel structures for onshore wind turbines. The organic coatings are, however, susceptible to atmospheric exposure and tend to deteriorate during the operation. At the same time, onshore turbines become more powerful and require taller and more resistant tower structures. The inspection and condition monitoring of protective coating systems on large onshore turbines (in excess of 120 m height) is a demanding and time-consuming procedure and requires high human effort. The rapid developments in digitization and data analysis offer opportunities to notably increase the efficiency of monitoring processes and to develop (semi-)automated standardized procedures. The paper describes a data-oriented approach to utilize digital data for the monitoring and maintenance planning of surface protection systems of large onshore wind turbines. The proposed approach includes the following steps: the segmentation of an existing wind power structure into a number of reference areas based on an In-situ Virtual Twin ; the definition of a local deterioration degree for each individual reference area; the annotation of image data; the use of heterogenous multi-modal data (image data, geodetical data, meteorological data, profile scanning data) as the sources for condition assessment and monitoring. An example procedure is exercised for a tower structure of an onshore wind power turbine in order to illustrate the practical relevance of the approach. [ABSTRACT FROM AUTHOR]

Digitalization and the use of sensor data for the condition monitoring of surface protection systems of steel towers of onshore wind power structures The paper describes the concept of a fully data‐based, mainly automated, procedure for the condition monitoring of surface protection systems on steel structures. Acquisition, processing and evaluation of relevant data as well as their implementation into a mathematical‐empirical condition monitoring model are discussed. The monitoring model is based on the definition and assessment of a local deterioration degree, which is characteristic for pre‐defined reference areas at the structure. The creation of the reference areas by means of In‐situ Virtual Twin Computation is introduced. The data‐based assessment procedure is practically exemplified on a steel tower of an onshore wind power structure in a rural corrosive environment. The procedure can be applied to steel structures with similar geometries (storage tanks, silos, chimneys) as well as to offshore conditions. [ABSTRACT FROM AUTHOR]