Search

Abstract Current convolutional neural networks (CNNs) are not designed for large scientific images with rich multi-scale features, such as in satellite and microscopy domain. A new phase of development of CNNs especially designed for large images is awaited. However, application-independent high-quality and challenging datasets needed for such development are still missing. We present the ‘UltraMNIST dataset’ and associated benchmarks for this new research problem of ‘training CNNs for large images’. The dataset is simple, representative of wide-ranging challenges in scientific data, and easily customizable for different levels of complexity, smallest and largest features, and sizes of images. Two variants of the problem are discussed: standard version that facilitates the development of novel CNN methods for effective use of the best available GPU resources and the budget-aware version to promote the development of methods that work under constrained GPU memory. Several baselines are presented and the effect of reduced resolution is studied. The presented benchmark dataset and baselines will hopefully trigger the development of new CNN methods for large scientific images.

Global contexts in images are quite valuable in image-to-image translation problems. Conventional attention-based and graph-based models capture the global context to a large extent; however, these are computationally expensive. Moreover, existing approaches are limited to only learning the pairwise semantic relation between any two points in the image. In this paper, we present Latent Graph Attention (LGA), a computationally inexpensive (linear to the number of nodes) and stable modular framework for incorporating the global context in existing architectures. This framework particularly empowers small-scale architectures to achieve performance closer to that of large architectures, making the light-weight architectures more useful for edge devices with lower compute power and lower energy needs. LGA propagates information spatially using a network of locally connected graphs, thereby facilitating the construction of a semantically coherent relation between any two spatially distant points that also takes into account the influence of the intermediate pixels. Moreover, the depth of the graph network can be used to adapt the extent of contextual spread to the target dataset, thereby able to explicitly control the added computational cost. To enhance the learning mechanism of LGA, we also introduce a novel contrastive loss term that helps our LGA module to couple well with the original architecture at the expense of minimal additional computational load. We show that incorporating LGA improves performance in three challenging applications, namely transparent object segmentation, image restoration for dehazing and optical flow estimation.

Maritime training can improve safety by equipping seafarers with the knowledge and skills to manage risk. However, designing a quality training program can be challenging and stress can negatively impact performance and safety. To address this, the present study aims to investigate the relationship between stress and training outcomes, with the goal of developing more effective stress-based training systems. Two stressful scenarios were designed with varying safety factors involved during navigation tasks. The study examines the impact of stress levels on training outcomes and performance based on safety factors and the correlation between self-assessed stress levels and objective stress levels obtained from biosignal data. The study was conducted in a simulated bridge environment in Tromsø, Norway, and analyzed using statistical tests and machine learning models. The findings of this study indicate that training scenarios can be classified by stress levels, which were found to be associated with reduced visibility, equipment failures, and severe weather conditions. Additionally, the study revealed that stress levels can negatively impact performance in maritime navigation and sailing route reliability. These findings provide insights into how to improve the quality and effectiveness of maritime training programs and ultimately enhance safety at sea.

Scanning acoustic microscopy (SAM) is a non-ionizing and label-free imaging modality used to visualize the surface and internal structures of industrial objects and biological specimens. The image of the sample under investigation is created using high-frequency acoustic waves. The frequency of the excitation signals, the signal-to-noise ratio, and the pixel size all play a role in acoustic image resolution. We propose a deep learning-enabled image inpainting for acoustic microscopy in this paper. The method is based on training various generative adversarial networks (GANs) to inpaint holes in the original image and generate a 4× image from it. In this approach, five different types of GAN models are used: AOTGAN, DeepFillv2, Edge-Connect, DMFN, and Hypergraphs image inpainting. The trained model’s performance is assessed by calculating the peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM) between network-predicted and ground truth images. The Hypergraphs image inpainting model provided an average SSIM of 0.93 for 2× and up to 0.93 for the final 4×, respectively, and a PSNR of 32.33 for 2× and up to 32.20 for the final 4×. The developed SAM and GAN frameworks can be used in a variety of industrial applications, including bio-imaging.

Federated learning promises an elegant solution for learning global models across distributed and privacy-protected datasets. However, challenges related to skewed data distribution, limited computational and communication resources, data poisoning, and free riding clients affect the performance of federated learning. Selection of the best clients for each round of learning is critical in alleviating these problems. We propose a novel sampling method named the irrelevance sampling technique. Our method is founded on defining a novel irrelevance score that incorporates the client characteristics in a single floating value, which can elegantly classify the client into three numerical sign defined pools for easy sampling. It is a computationally inexpensive, intuitive and privacy preserving sampling technique that selects a subset of clients based on quality and quantity of data on edge devices. It achieves 50–80% faster convergence even in highly skewed data distribution in the presence of free riders based on lack of data and severe class imbalance under both Independent and Identically Distributed (IID) and Non-IID conditions. It shows good performance on practical application datasets.

This work presents a novel approach to detecting stress differences between experts and novices in Situation Awareness (SA) tasks during maritime navigation using one type of wearable sensor, Empatica E4 Wristband. We propose that for a given workload state, the values of biosignal data collected from wearable sensor vary in experts and novices. We describe methods to conduct a designed SA task experiment, and collected the biosignal data on subjects sailing on a 240° view simulator. The biosignal data were analysed by using a machine learning algorithm, a Convolutional Neural Network. The proposed algorithm showed that the biosingal data associated with the experts can be categorized as different from that of the novices, which is in line with the results of NASA Task Load Index (NASA-TLX) rating scores. This study can contribute to the development of a self-training system in maritime navigation in further studies.

Abstract This paper presents eigen-analysis of image stack of blinking fluorophores to identify the components that enable super-resolved imaging of blinking fluorophores. Eigen-analysis reveals that the contributions of spatial distribution of fluorophores and their temporal photon emission characteristics can be completely separated. While cross-emitter cross-pixel information of spatial distribution that permits super-resolution is encoded in two matrices, temporal statistics weigh the contribution of these matrices to the measured data. The properties and conditions of exploitation of these matrices are investigated. Con-temporary super-resolution imaging methods that use blinking for super-resolution are studied in the context of the presented analysis. Besides providing insight into the capabilities and limitations of existing super-resolution methods, the analysis shall help in designing better super-resolution techniques that directly exploit these matrices.

The recent worldwide outbreak of the novel coronavirus (COVID-19) has opened up new challenges to the research community. Artificial intelligence (AI) driven methods can be useful to predict the parameters, risks, and effects of such an epidemic. Such predictions can be helpful to control and prevent the spread of such diseases. The main challenges of applying AI is the small volume of data and the uncertain nature. Here, we propose a shallow long short-term memory (LSTM) based neural network to predict the risk category of a country. We have used a Bayesian optimization framework to optimize and automatically design country-specific networks. The results show that the proposed pipeline outperforms state-of-the-art methods for data of 180 countries and can be a useful tool for such risk categorization. We have also experimented with the trend data and weather data combined for the prediction. The outcome shows that the weather does not have a significant role. The tool can be used to predict long-duration outbreak of such an epidemic such that we can take preventive steps earlier.

This paper introduces a fusion convolutional architecture for efficient learning of spatio-temporal features in video action recognition. Unlike 2D convolutional neural networks (CNNs), 3D CNNs can be applied directly on consecutive frames to extract spatio-temporal features. The aim of this work is to fuse the convolution layers from 2D and 3D CNNs to allow temporal encoding with fewer parameters than 3D CNNs. We adopt transfer learning from pre-trained 2D CNNs for spatial extraction, followed by temporal encoding, before connecting to 3D convolution layers at the top of the architecture. We construct our fusion architecture, semi-CNN, based on three popular models: VGG-16, ResNets and DenseNets, and compare the performance with their corresponding 3D models. Our empirical results evaluated on the action recognition dataset UCF-101 demonstrate that our fusion of 1D, 2D and 3D convolutions outperforms its 3D model of the same depth, with fewer parameters and reduces overfitting. Our semi-CNN architecture achieved an average of 16−30% boost in the top-1 accuracy when evaluated on an input video of 16 frames.

Ultrasound based structural health monitoring of piezoelectric material is challenging if a damage changes at a microscale over time. Classifying geometrically similar damages with a difference in diameter as small as 100 μ m is difficult using conventional sensing and signal analysis approaches. Here, we use an unconventional ultrasound sensing approach that collects information of the entire bulk of the material and investigate the applicability of machine learning approaches for classifying such similar defects. Our results show that appropriate feature design combined with simple k-nearest neighbor classifier can provide up to 98% classification accuracy even though conventional features for time-series data and a variety of classifiers cannot achieve close to 70% accuracy. The newly proposed hybrid feature, which combines frequency domain information in the form of power spectral density and time domain information in the form of sign of slope change, is a suitable feature for achieving the best classification accuracy on this challenging problem.

In the present work, various thermal parameters of an annular fin subjected to thermal loading are inversely estimated using differential evolution (DE) method. In order to obtain the temperature field, the second order nonlinear differential equation for heat transfer with variable thermal conductivity and internal heat generation is solved using Homotopy Perturbation Method (HPM). Classical thermoelasticity approach coupled with an HPM solution for temperature field is employed for the forward solution of thermal stresses. It is interesting that the internal heat generation does not affect the radial stresses, while the temperature field and the tangential stresses are influenced by the heat generation parameters. As the tangential stresses are mainly responsible for mechanical failure due to thermal loading in an annular fin, the unknown thermal parameters are inversely estimated from a prescribed tangential stress field. The reconstructed stress fields obtained from the inverse parameters are found to be in good agreement with the actual solution.

We propose a method for classifying radiometric oceanic color data measured by hyperspectral satellite sensors into known spectral classes, irrespective of the downwelling irradiance of the particular day, i.e., the illumination conditions. The focus is not on retrieving the inherent optical properties but to classify the pixels according to the known spectral classes of the reflectances from the ocean. The method compensates for the unknown downwelling irradiance by white balancing the radiometric data at the ocean pixels using the radiometric data of bright pixels (typically from clouds). The white-balanced data is compared with the entries in a pre-calibrated lookup table in which each entry represents the spectral properties of one class. The proposed approach is tested on two datasets of in situ measurements and 26 different daylight illumination spectra for medium resolution imaging spectrometer (MERIS), moderate-resolution imaging spectroradiometer (MODIS), sea-viewing wide field-of-view sensor (SeaWiFS), coastal zone color scanner (CZCS), ocean and land colour instrument (OLCI), and visible infrared imaging radiometer suite (VIIRS) sensors. Results are also shown for CIMEL’s SeaPRISM sun photometer sensor used on-board field trips. Accuracy of more than 92% is observed on the validation dataset and more than 86% is observed on the other dataset for all satellite sensors. The potential of applying the algorithms to non-satellite and non-multi-spectral sensors mountable on airborne systems is demonstrated by showing classification results for two consumer cameras. Classification on actual MERIS data is also shown. Additional results comparing the spectra of remote sensing reflectance with level 2 MERIS data and chlorophyll concentration estimates of the data are included.