Search

Fixed underwater observatories (FUO), equipped with digital cameras and other sensors, become more commonly used to record different kinds of time series data for marine habitat monitoring. With increasing numbers of campaigns, numbers of sensors and campaign time, the volume and heterogeneity of the data, ranging from simple temperature time series to series of HD images or video call for new data science approaches to analyze the data. While some works have been published on the analysis of data from one campaign, we address the problem of analyzing time series data from two consecutive monitoring campaigns (starting late 2017 and late 2018) in the same habitat. While the data from campaigns in two separate years provide an interesting basis for marine biology research, it also presents new data science challenges, like the the marine image analysis in data form more than one campaign. In this paper, we analyze the polyp activity of two Paragorgia arborea cold water coral (CWC) colonies using FUO data collected from November 2017 to June 2018 and from December 2018 to April 2019. We successfully apply convolutional neural networks (CNN) for the segmentation and classification of the coral and the polyp activities. The result polyp activity data alone showed interesting temporal patterns with differences and similarities between the two time periods. A one month "sleeping" period in spring with almost no activity was observed in both coral colonies, but with a shift of approximately one month. A time series prediction experiment allowed us to predict the polyp activity from the non-image sensor data using recurrent neural networks (RNN). The results pave a way to a new multi-sensor monitoring strategy for Paragorgia arborea behaviour. [ABSTRACT FROM AUTHOR]

Diatoms represent one of the morphologically and taxonomically most diverse groups of microscopic eukaryotes. Light microscopy-based taxonomic identification and enumeration of frustules, the silica shells of these microalgae, is broadly used in aquatic ecology and biomonitoring. One key step in emerging digital variants of such investigations is segmentation, a task that has been addressed before, but usually in manually captured megapixel-sized images of individual diatom cells with a mostly clean background. In this paper, we applied deep learning-based segmentation methods to gigapixel-sized, high-resolution scans of diatom slides with a realistically cluttered background. This setup requires large slide scans to be subdivided into small images (tiles) to apply a segmentation model to them. This subdivision (tiling), when done using a sliding window approach, often leads to cropping relevant objects at the boundaries of individual tiles. We hypothesized that in the case of diatom analysis, reducing the amount of such cropped objects in the training data can improve segmentation performance by allowing for a better discrimination of relevant, intact frustules or valves from small diatom fragments, which are considered irrelevant when counting diatoms. We tested this hypothesis by comparing a standard sliding window / fixed-stride tiling approach with two new approaches we term object-based tile positioning with and without object integrity constraint. With all three tiling approaches, we trained Mask-R-CNN and U-Net models with different amounts of training data and compared their performance. Object-based tiling with object integrity constraint led to an improvement in pixel-based precision by 12–17 percentage points without substantially impairing recall when compared with standard sliding window tiling. We thus propose that training segmentation models with object-based tiling schemes can improve diatom segmentation from large gigapixel-sized images but could potentially also be relevant for other image domains. [ABSTRACT FROM AUTHOR]

Hyperspectral imaging (HSI) is a promising technology for environmental monitoring with a lot of undeveloped potential due to the high dimensionality and complexity of the data. If temporal effects are studied, such as in a monitoring context, the analysis becomes more challenging as time is added to the dimensions of space (image coordinates) and wavelengths. We conducted a series of laboratory experiments to investigate the impact of different stressor exposure patterns on the spectrum of the cold water coral Desmophyllum pertusum. 65 coral samples were divided into 12 groups, each group being exposed to different types and levels of particles. Hyperspectral images of the coral samples were collected at four time points from prior to exposure to 6 weeks after exposure. To investigate the relationships between the corals' spectral signatures and controlled experimental parameters, a new software tool for interactive visual exploration was developed and applied, the HypIX (Hyperspectral Image eXplorer) web tool. HypIX combines principles from exploratory data analysis, information visualization and machine learning-based dimension reduction. This combination enables users to select regions of interest (ROI) in all dimensions (2D space, time point and spectrum) for a flexible integrated inspection. We propose two HypIX workflows to find relationships in time series of hyperspectral datasets, namely morphology-based filtering workflow and embedded driven response analysis workflow. With these HypIX workflows three users identified different temporal and spatial patterns in the spectrum of corals exposed to different particle stressor conditions. Corals exposed to particles tended to have a larger change rate than control corals, which was evident as a shifted spectrum. The responses, however, were not uniform for coral samples undergoing the same exposure treatments, indicating individual tolerance levels. We also observed a good inter-observer agreement between the three HyPIX users, indicating that the proposed workflow can be applied to obtain reproducible HSI analysis results. [ABSTRACT FROM AUTHOR]

Data augmentation is an established technique in computer vision to foster the generalization of training and to deal with low data volume. Most data augmentation and computer vision research are focused on everyday images such as traffic data. The application of computer vision techniques in domains like marine sciences has shown to be not that straightforward in the past due to special characteristics, such as very low data volume and class imbalance, because of costly manual annotation by human domain experts, and general low species abundances. However, the data volume acquired today with moving platforms to collect large image collections from remote marine habitats, like the deep benthos, for marine biodiversity assessment and monitoring makes the use of computer vision automatic detection and classification inevitable. In this work, we investigate the effect of data augmentation in the context of taxonomic classification in underwater, i.e., benthic images. First, we show that established data augmentation methods (i.e., geometric and photometric transformations) perform differently in marine image collections compared to established image collections like the Cityscapes dataset, showing everyday traffic images. Some of the methods even decrease the learning performance when applied to marine image collections. Second, we propose new data augmentation combination policies motivated by our observations and compare their effect to those proposed by the AutoAugment algorithm and can show that the proposed augmentation policy outperforms the AutoAugment results for marine image collections. We conclude that in the case of small marine image datasets, background knowledge, and heuristics should sometimes be applied to design an effective data augmentation method. [ABSTRACT FROM AUTHOR]

The application of protective coating systems is the major measure against the corrosion of steel for tower sections of wind turbines. The inspection, condition monitoring and maintenance of protective coating system is a demanding and time-consuming procedure and requires high human effort. The paper introduces for the first time a Digital Twin concept for the condition monitoring and prescriptive maintenance planning for surface protection systems on wind turbine towers. The initial point of the concept is an in-situ Virtual Twin for the generation of reference areas for condition monitoring. The paper describes the integration of an online image annotation and processing tool, a maintenance model, corrosive resistance parameters, structural load parameters, and sensor data into the Digital Twin concept. The prospects of the concept and its practical relevance are shown for tower structures of large onshore wind turbines. [ABSTRACT FROM AUTHOR]

Originating from the Second International Indian Ocean Expedition (IIOE-2), the main goal of the Western Indian Ocean (WIO) Regional Benthic Imagery Workshop, was to provide information and training on the use of various underwater imagery platforms in benthic research. To date, attempts made to explore the bottom of the ocean range from simple diving bells to more advanced camera systems, and the rapidly expanding field of underwater image-based research has supported marine exploration in many forms, from biodiversity surveys, spatial analyses and temporal studies, to monitoring schemes. Alongside the increasing use of underwater camera systems worldwide, there is an evident need to improve training and access to these techniques for students and researchers from institutes within the WIO. The week-long virtual event was conducted between 30 August and 3 September 2021 with 266 participants. Sessions consisted of lessons, practical demonstrations and interactive discussions which covered the steps required to conduct underwater imagery surveys, taking participants through elements of sampling design, data acquisition and processing, considerations for statistical analysis and, effective managment of data. The session recordings from the workshop are available online as a teaching aid which has the potential to reach marine researchers both regionally and globally. It is crucial that we build on this momentum by continuing to develop and strengthen the network established through this initiative for standardised benthic-image-based research within the WIO. [ABSTRACT FROM AUTHOR]

With the mining of polymetallic nodules from the deep-sea seafloor once more evoking commercial interest, decisions must be taken on how to most efficiently regulate and monitor physical and community disturbance in these remote ecosystems. Image-based approaches allow non-destructive assessment of the abundance of larger fauna to be derived from survey data, with repeat surveys of areas possible to allow time series data collection. At the time of writing, key underwater imaging platforms commonly used to map seafloor fauna abundances are autonomous underwater vehicles (AUVs), remotely operated vehicles (ROVs) and towed camera "ocean floor observation systems" (OFOSs). These systems are highly customisable, with cameras, illumination sources and deployment protocols changing rapidly, even during a survey cruise. In this study, eight image datasets were collected from a discrete area of polymetallic-nodule-rich seafloor by an AUV and several OFOSs deployed at various altitudes above the seafloor. A fauna identification catalogue was used by five annotators to estimate the abundances of 20 fauna categories from the different datasets. Results show that, for many categories of megafauna, differences in image resolution greatly influenced the estimations of fauna abundance determined by the annotators. This is an important finding for the development of future monitoring legislation for these areas. When and if commercial exploitation of these marine resources commences, robust and verifiable standards which incorporate developing technological advances in camera-based monitoring surveys should be key to developing appropriate management regulations for these regions. [ABSTRACT FROM AUTHOR]

With the mining of polymetallic nodules from the deep sea seafloor again approaching commercial viability, decisions must be taken on how to most efficiently regulate and monitor physical and community disturbance in these remote ecosystems. Image based approaches allow non-destructive assessment of larger fauna abundances to be derived from survey data, with repeat surveys of areas possible to allow time series data collection. At time of writing key underwater imaging platforms commonly used to map seafloor fauna abundances are Automated Underwater Vehicles (AUVs), Remotely Operated Vehicles (ROVs) and towed camera Ocean Floor Observation Systems (OFOSs). These systems are highly customisable, with mounted cameras, illumination systems and deployment protocols rapidly changing over time, and even within survey cruises. In this study 8 image datasets were collected from a discrete area of polymetallic nodule rich seafloor by an AUV and several OFOSs deployed at various altitudes above the seafloor. A fauna identification catalogue was used by 5 annotators to estimate the abundances of 20 fauna categories from the different data sets. Results show that for many categories of megafauna differences in image resolution greatly influenced the estimations of fauna abundance determined by the annotators. This is an important finding for the development of future monitoring legislation for these areas. When and if commercial exploitation of these marine resources commences, to ensure best monitoring practice, unambiguous rules on how camera-based monitoring surveys should be conducted, and with what equipment, must be put in place. [ABSTRACT FROM AUTHOR]

The evaluation of large amounts of digital image data is of growing importance for biology, including for the exploration and monitoring of marine habitats. However, only a tiny percentage of the image data collected is evaluated by marine biologists who manually interpret and annotate the image contents, which can be slow and laborious. In order to overcome the bottleneck in image annotation, two strategies are increasingly proposed: “citizen science” and “machine learning”. In this study, we investigated how the combination of citizen science, to detect objects, and machine learning, to classify megafauna, could be used to automate annotation of underwater images. For this purpose, multiple large data sets of citizen science annotations with different degrees of common errors and inaccuracies observed in citizen science data were simulated by modifying “gold standard” annotations done by an experienced marine biologist. The parameters of the simulation were determined on the basis of two citizen science experiments. It allowed us to analyze the relationship between the outcome of a citizen science study and the quality of the classifications of a deep learning megafauna classifier. The results show great potential for combining citizen science with machine learning, provided that the participants are informed precisely about the annotation protocol. Inaccuracies in the position of the annotation had the most substantial influence on the classification accuracy, whereas the size of the marking and false positive detections had a smaller influence. [ABSTRACT FROM AUTHOR]

Digital imaging has become one of the most important techniques in environmental monitoring and exploration. In the case of the marine environment, mobile platforms such as autonomous underwater vehicles (AUVs) are now equipped with high-resolution cameras to capture huge collections of images from the seabed. However, the timely evaluation of all these images presents a bottleneck problem as tens of thousands or more images can be collected during a single dive. This makes computational support for marine image analysis essential. Computer-aided analysis of environmental images (and marine images in particular) with machine learning algorithms is promising, but challenging and different to other imaging domains because training data and class labels cannot be collected as efficiently and comprehensively as in other areas. In this paper, we present Machine learning Assisted Image Annotation (MAIA), a new image annotation method for environmental monitoring and exploration that overcomes the obstacle of missing training data. The method uses a combination of autoencoder networks and Mask Region-based Convolutional Neural Network (Mask R-CNN), which allows human observers to annotate large image collections much faster than before. We evaluated the method with three marine image datasets featuring different types of background, imaging equipment and object classes. Using MAIA, we were able to annotate objects of interest with an average recall of 84.1% more than twice as fast as compared to “traditional” annotation methods, which are purely based on software-supported direct visual inspection and manual annotation. The speed gain increases proportionally with the size of a dataset. The MAIA approach represents a substantial improvement on the path to greater efficiency in the annotation of large benthic image collections. [ABSTRACT FROM AUTHOR]

The study of mapping and interaction of co-localized proteins at a sub-cellular level is important for understanding complex biological phenomena. One of the recent techniques to map co-localized proteins is to use the standard immuno-fluorescence microscopy in a cyclic manner (Nat Biotechnol 24:1270-8, 2006; Proc Natl Acad Sci 110:11982-7,2013). Unfortunately, these techniques suffer from variability in intensity and positioning of signals from protein markers within a run and across different runs. Therefore, it is necessary to standardize protocols for preprocessing of the multiplexed bioimaging (MBI) data from multiple runs to a comparable scale before any further analysis can be performed on the data. In this paper, we compare various normalization protocols and propose on the basis of the obtained results, a robust normalization technique that produces consistent results on the MBI data collected from different runs using the Toponome Imaging System TIS). Normalization results produced by the proposed method on a sample TIS data set for colorectal cancer patients were ranked favorably by two pathologists and two biologists. We show that the proposed method produces higher between class Kullback-Leibler (KL) divergence and lower within class KL divergence on a distribution of cell phenotypes from colorectal cancer and histologically normal samples. [ABSTRACT FROM AUTHOR]

Within the recent years clock rates of modern processors stagnated while the demand for computing power continued to grow. This applied particularly for the fields of life sciences and bioinformatics, where new technologies keep on creating rapidly growing piles of raw data with increasing speed. The number of cores per processor increased in an attempt to compensate for slight increments of clock rates. This technological shift demands changes in software development, especially in the field of high performance computing where parallelization techniques are gaining in importance due to the pressing issue of large sized datasets generated by e.g., modern genomics. This paper presents an overview of state-of-the-art manual and automatic acceleration techniques and lists some applications employing these in different areas of sequence informatics. Furthermore, we provide examples for automatic acceleration of two use cases to show typical problems and gains of transforming a serial application to a parallel one. The paper should aid the reader in deciding for a certain techniques for the problem at hand. We compare four different state-of-the-art automatic acceleration approaches (OpenMP, PluTo-SICA, PPCG, and OpenACC). Their performance as well as their applicability for selected use cases is discussed. While optimizations targeting the CPU worked better in the complex k-mer use case, optimizers for Graphics Processing Units (GPUs) performed better in the matrix multiplication example. But performance is only superior at a certain problem size due to data migration overhead. We show that automatic code parallelization is feasible with current compiler software and yields significant increases in execution speed. Automatic optimizers for CPU are mature and usually no additional manual adjustment is required. In contrast, some automatic parallelizers targeting GPUs still lack maturity and are limited to simple statements and structures. [ABSTRACT FROM AUTHOR]

Background With the advent of low cost, fast sequencing technologies metagenomic analyses are made possible. The large data volumes gathered by these techniques and the unpredictable diversity captured in them are still, however, a challenge for computational biology. Results In this paper we address the problem of rapid taxonomic assignment with small and adaptive data models (< 5 MB) and present the accelerated k-mer explorer (AKE). Acceleration in AKE's taxonomic assignments is achieved by a special machine learning architecture, which is well suited to model data collections that are intrinsically hierarchical. We report classification accuracy reasonably well for ranks down to order, observed on a study on real world data (Acid Mine Drainage, Cow Rumen). Conclusion We show that the execution time of this approach is orders of magnitude shorter than competitive approaches and that accuracy is comparable. The tool is presented to the public as a web application (url: https://ani.cebitec.uni-bielefeld.de/ake/, username: bmc, password: bmcbioinfo). [ABSTRACT FROM AUTHOR]

The application of protective coating systems is the major measure against the corrosion of steel structures for onshore wind turbines. The organic coatings are, however, susceptible to atmospheric exposure and tend to deteriorate during the operation. At the same time, onshore turbines become more powerful and require taller and more resistant tower structures. The inspection and condition monitoring of protective coating systems on large onshore turbines (in excess of 120 m height) is a demanding and time-consuming procedure and requires high human effort. The rapid developments in digitization and data analysis offer opportunities to notably increase the efficiency of monitoring processes and to develop (semi-)automated standardized procedures. The paper describes a data-oriented approach to utilize digital data for the monitoring and maintenance planning of surface protection systems of large onshore wind turbines. The proposed approach includes the following steps: the segmentation of an existing wind power structure into a number of reference areas based on an In-situ Virtual Twin ; the definition of a local deterioration degree for each individual reference area; the annotation of image data; the use of heterogenous multi-modal data (image data, geodetical data, meteorological data, profile scanning data) as the sources for condition assessment and monitoring. An example procedure is exercised for a tower structure of an onshore wind power turbine in order to illustrate the practical relevance of the approach. [ABSTRACT FROM AUTHOR]