Search

Recent advances in deep neural networks have shed new light on physics, engineering, and scientific computing. Reconciling the data-centered viewpoint with physical simulation is one of the research hotspots. The physics-informed neural network (PINN) is currently the most general framework, which is more popular due to the convenience of constructing NNs and excellent generalization ability. The automatic differentiation (AD)-based PINN model is suitable for the homogeneous scientific problem; however, it is unclear how AD can enforce flux continuity across boundaries between cells of different properties where spatial heterogeneity is represented by grid cells with different physical properties. In this work, we propose a criss-cross physics-informed convolutional neural network (CC-PINN) learning architecture, aiming to learn the solution of parametric PDEs with spatial heterogeneity of physical properties. To achieve the seamless enforcement of flux continuity and integration of physical meaning into CNN, a predefined 2D convolutional layer is proposed to accurately express transmissibility between adjacent cells. The efficacy of the proposed method was evaluated through predictions of several petroleum reservoir problems with spatial heterogeneity and compared against state-of-the-art (PINN) through numerical analysis as a benchmark, which demonstrated the superiority of the proposed method over the PINN. [ABSTRACT FROM AUTHOR]

Recent advances in deep neural networks have shed new light on physics, engineering, and scientific computing. Reconciling the data-centered viewpoint with physical simulation is one of the research hotspots. The physics-informed neural network (PINN) is currently the most general framework, which is more popular due to the convenience of constructing NNs and excellent generalization ability. The automatic differentiation (AD)-based PINN model is suitable for the homogeneous scientific problem; however, it is unclear how AD can enforce flux continuity across boundaries between cells of different properties where spatial heterogeneity is represented by grid cells with different physical properties. In this work, we propose a criss-cross physics-informed convolutional neural network (CC-PINN) learning architecture, aiming to learn the solution of parametric PDEs with spatial heterogeneity of physical properties. To achieve the seamless enforcement of flux continuity and integration of physical meaning into CNN, a predefined 2D convolutional layer is proposed to accurately express transmissibility between adjacent cells. The efficacy of the proposed method was evaluated through predictions of several petroleum reservoir problems with spatial heterogeneity and compared against state-of-the-art (PINN) through numerical analysis as a benchmark, which demonstrated the superiority of the proposed method over the PINN. [ABSTRACT FROM AUTHOR]

Motivation Selective sweeps can successfully be distinguished from neutral genetic data using summary statistics and likelihood-based methods that analyze single nucleotide polymorphisms (SNPs). However, these methods are sensitive to confounding factors, such as severe population bottlenecks and old migration. By virtue of machine learning, and specifically convolutional neural networks (CNNs), new accurate classification models that are robust to confounding factors have been recently proposed. However, such methods are more computationally expensive than summary-statistic-based ones, yielding them impractical for processing large-scale genomic data. Moreover, SNP data are frequently preprocessed to improve classification accuracy, further exacerbating the long analysis times. Results To this end, we propose a 1D CNN-based model, dubbed FAST-NN, that does not require any preprocessing while using only derived allele frequencies instead of summary statistics or raw SNP data, thereby yielding a sample-size-invariant, scalable solution. We evaluated several data fusion approaches to account for the variance of the density of genetic diversity across genomic regions (a selective sweep signature), and performed an extensive neural architecture search based on a state-of-the-art reference network architecture (SweepNet). The resulting model, FAST-NN, outperforms the reference architecture by up to 12% inference accuracy over all challenging evolutionary scenarios with confounding factors that were evaluated. Moreover, FAST-NN is between 30× and 259× faster on a single CPU core, and between 2.0× and 6.2× faster on a GPU, when processing sample sizes between 128 and 1000 samples. Our work paves the way for the practical use of CNNs in large-scale selective sweep detection. Availability and implementation https://github.com/SjoerdvandenBelt/FAST-NN [ABSTRACT FROM AUTHOR]

This paper presents a detailed analysis of a convolutional neural network based on multi-valued neurons (CNNMVN) and a fully connected multilayer neural network based on multi-valued neurons (MLMVN), employed here as a convolutional neural network in the frequency domain. We begin by providing an overview of the fundamental concepts underlying CNNMVN, focusing on the organization of convolutional layers and the CNNMVN learning algorithm. The error backpropagation rule for this network is justified and presented in detail. Subsequently, we consider how MLMVN can be used as a convolutional neural network in the frequency domain. It is shown that each neuron in the first hidden layer of MLMVN may work as a frequency-domain convolutional kernel, utilizing the Convolution Theorem. Essentially, these neurons create Fourier transforms of the feature maps that would have resulted from the convolutions in the spatial domain performed in regular convolutional neural networks. Furthermore, we discuss optimization techniques for both networks and compare the resulting convolutions to explore which features they extract from images. Finally, we present experimental results showing that both approaches can achieve high accuracy in image recognition. [ABSTRACT FROM AUTHOR]

Grapevine rootstocks are gaining importance in viticulture as a strategy to combat abiotic challenges, as well as enhance scion physiology. Direct leaf-level physiological parameters like net assimilation rate, stomatal conductance to water vapor, quantum yield of PSII, and transpiration can illuminate the rootstock effect on scion physiology. However, these measures are time-consuming and limited to leaf-level analysis. This study used different rootstocks to investigate the potential application of aerial hyperspectral imagery in the estimation of canopy level measurements. A statistical framework was developed as an ensemble stacked regression (REGST) that aggregated five different individual machine learning algorithms: Least absolute shrinkage and selection operator (Lasso), Partial least squares regression (PLSR), Ridge regression (RR), Elastic net (ENET), and Principal component regression (PCR) to optimize high-throughput assessment of vine physiology. In addition, a Convolutional Neural Network (CNN) algorithm was integrated into an existing REGST, forming a hybrid CNN-REGST model with the aim of capturing patterns from the hyperspectral signal. Based on the findings, the performance of individual base models exhibited variable prediction accuracies. In most cases, Ridge Regression (RR) demonstrated the lowest test Root Mean Squared Error (RMSE). The ensemble stacked regression model (REGST) outperformed the individual machine learning algorithms with an increase in R2 by (0.03 to 0.1). The performances of CNN-REGST and REGST were similar in estimating the four different traits. Overall, these models were able to explain approximately 55–67% of the variation in the actual ground-truth data. This study suggests that hyperspectral features integrated with powerful AI approaches show great potential in tracing functional traits in grapevines. [ABSTRACT FROM AUTHOR]

In this study, optical photothermal infrared (O-PTIR) spectroscopy combined with machine learning algorithms were used to evaluate 46 tissue cores of surgically resected cervical lymph nodes, some of which harboured oral squamous cell carcinoma nodal metastasis. The ratios obtained between O-PTIR chemical images at 1252 cm−1 and 1285 cm−1 were able to reveal morphological details from tissue samples that are comparable to the information achieved by a pathologist's interpretation of optical microscopy of haematoxylin and eosin (H&E) stained samples. Additionally, when used as input data for a hybrid convolutional neural network (CNN) and random forest (RF) analyses, these yielded sensitivities, specificities and precision of 98.6 ± 0.3%, 92 ± 4% and 94 ± 5%, respectively, and an area under receiver operator characteristic (AUC) of 94 ± 2%. Our findings show the potential of O-PTIR technology as a tool to study cancer on tissue samples. [ABSTRACT FROM AUTHOR]

This paper aims to use deep learning algorithms to identify and study the stability of breakwater armor blocks. It introduces a posture identification model for fender blocks using a Mask Region-based Convolutional Neural Network (R-CNN), which has been enhanced by considering factors affecting breakwater fender blocks. Furthermore, a wave prediction model for breakwaters is developed by integrating Bidirectional Encoder Representations from Transformers (BERTs) with Bidirectional Long Short-Term Memory (BiLSTM). The performance of these models is evaluated. The results show that the accuracy of the Mask R-CNN and its comparison algorithms initially increases and then decreases with higher Intersection Over Union (IOU) thresholds, peaking at 95.16% accuracy at an IOU threshold of 0.5. The BERT-BiLSTM wave prediction model maintains a loss value around 0.01 and an accuracy of approximately 90.00%. These results suggest that the proposed models offer more accurate stability assessments of breakwater armor blocks. By combining the random forest prediction model with BiLSTM, the wave characteristics and fender posture can be predicted better, offering reliable decision support for breakwater engineering. [ABSTRACT FROM AUTHOR]

Myoelectric prostheses have recently shown significant promise for restoring hand function in individuals with upper limb loss or deficiencies, driven by advances in machine learning and increasingly accessible bioelectrical signal acquisition devices. Here, we first introduce and validate a novel experimental paradigm using a virtual reality headset equipped with hand-tracking capabilities to facilitate the recordings of synchronized EMG signals and hand pose estimation. Using both the phasic and tonic EMG components of data acquired through the proposed paradigm, we compare hand gesture classification pipelines based on standard signal processing features, convolutional neural networks, and covariance matrices with Riemannian geometry computed from raw or xDAWN-filtered EMG signals. We demonstrate the performance of the latter for gesture classification using EMG signals. We further hypothesize that introducing physiological knowledge in machine learning models will enhance their performances, leading to better myoelectric prosthesis control. We demonstrate the potential of this approach by using the neurophysiological integration of the "move command" to better separate the phasic and tonic components of the EMG signals, significantly improving the performance of sustained posture recognition. These results pave the way for the development of new cutting-edge machine learning techniques, likely refined by neurophysiology, that will further improve the decoding of real-time natural gestures and, ultimately, the control of myoelectric prostheses. [ABSTRACT FROM AUTHOR]

The use of deep learning (DL) is steadily gaining traction in scientific challenges such as cancer research. Advances in enhanced data generation, machine learning algorithms, and compute infrastructure have led to an acceleration in the use of deep learning in various domains of cancer research such as drug response problems. In our study, we explored tree-based models to improve the accuracy of a single drug response model and demonstrate that tree-based models such as XGBoost (eXtreme Gradient Boosting) have advantages over deep learning models, such as a convolutional neural network (CNN), for single drug response problems. However, comparing models is not a trivial task. To make training and comparing CNNs and XGBoost more accessible to users, we developed an open-source library called UNNT (A novel Utility for comparing Neural Net and Tree-based models). The case studies, in this manuscript, focus on cancer drug response datasets however the application can be used on datasets from other domains, such as chemistry. Author summary: Advancement in data science, machine learning (ML), and artificial intelligence (AI) methods has enabled extraction of meaningful information from large and complex datasets that has assisted in better understanding, diagnosing, and treating cancer. The understanding of the drug response domain in cancer research has been accelerated with developing ML models to aid in predicting the effectiveness of the drugs based on a specific genomic molecular feature. In this study we developed a novel robust framework called UNNT (A novel Utility for comparing Neural Net and Tree-based models) that trains and compares deep learning method such as CNN and tree-based method such as XGBoost on the user input dataset. We applied this software to single drug response problem in cancer to identify the best performing ML method based on the National Cancer Institute 60 (NCI60) dataset. In addition, we studied the computational aspects of training each of these models where our results show that neither is evidently superior on both CPUs and GPUs while training. This shows that when both models have similar error rates for a dataset the hardware available determines the model choice for training. [ABSTRACT FROM AUTHOR]

The growing demand for high-quality video transmission over bandwidth-constrained networks and the increasing availability of video content have led to the need for efficient storage and distribution of large video files. To improve the latter, this article offers a comparison of six video compression methods without loss of quality. Particularly, H.255, VP9, AV1, convolutional neural network (CNN), recurrent neural network (RNN), and deep autoencoder (DAE). The proposed decision is to use a dataset of high-quality videos to implement and compare the performance of classical compression algorithms and algorithms based on machine learning. Evaluations of the compression efficiency and the quality of the received images were made on the basis of two metrics: PSNR and SSIM. This comparison revealed the strengths and weaknesses of each approach and provided insights into how machine learning algorithms can be optimized in future research. In general, it contributed to the development of more efficient and effective video compression algorithms that can be useful for a wide range of applications. [ABSTRACT FROM AUTHOR]

Accurate predictions of wind and other weather phenomena are essential for making informed strategic and tactical decisions in sailing. Sailors worldwide utilize current state-of-the-art forecasts, yet such forecasts are often insufficient because they do not offer the high temporal and geographic resolution required by sailors. This paper examines wind forecasting in competitive sailing and demonstrates that traditional wind forecasts can be improved for sailing events by using an integration of traditional numerical modeling and machine learning (ML) methods. Our primary objective is to provide practical and more precise wind forecasts that will give sailors a competitive edge. As a case study, we demonstrate the capabilities of our proposed methods to improve wind forecasting at Lake Kinneret, a popular sailing site. The lake wind pattern is highly influenced by the area's topographic features and is characterized by unique local and mesoscale phenomena at different times of the day. In this research, we simulate the Kinneret wind during the summers of 2015–2021 in up to one-kilometer resolution using the Weather Research and Forecasting (WRF) atmospheric model. The results are used as input for convolutional neural network (CNN) and multilayer perceptron (MLP) ML models to postprocess and improve the WRF model accuracy. These advanced ML models are trained using training datasets based on the WRF data as well as real data measured by the meteorological service, and subsequently, a validation process of the trained ML model is performed on unseen datasets against site-specific meteorological service observations. Through our experimental analysis, we demonstrate the limitations of the WRF model. It uncovers notable biases in wind direction and velocity, particularly a persistent northern bias in direction and an overestimation of wind strength. Despite its inherent limitations, this study demonstrates that the integration of ML models can potentially improve wind forecasting due to the remarkable prediction accuracy rate achieved by the CNN model, surpassing 95%, while achieving partial success for the MLP model. Furthermore, a successful CNN-based preliminary forecast was effectively generated, suggesting its potential contribution to the future development of a user-friendly tool for sailors. [ABSTRACT FROM AUTHOR]

We parameterize sub‐grid scale (SGS) fluxes in sinusoidally forced two‐dimensional turbulence on the β‐plane at high Reynolds numbers (Re ∼25,000) using simple 2‐layer convolutional neural networks (CNN) having only O(1000) parameters, two orders of magnitude smaller than recent studies employing deeper CNNs with 8–10 layers; we obtain stable, accurate, and long‐term online or a posteriori solutions at 16× downscaling factors. Our methodology significantly improves training efficiency and speed of online large eddy simulations runs, while offering insights into the physics of closure in such turbulent flows. Our approach benefits from extensive hyperparameter searching in learning rate and weight decay coefficient space, as well as the use of cyclical learning rate annealing, which leads to more robust and accurate online solutions compared to fixed learning rates. Our CNNs use either the coarse velocity or the vorticity and strain fields as inputs, and output the two components of the deviatoric stress tensor, Sd. We minimize a loss between the SGS vorticity flux divergence (computed from the high‐resolution solver) and that obtained from the CNN‐modeled Sd, without requiring energy or enstrophy preserving constraints. The success of shallow CNNs in accurately parameterizing this class of turbulent flows implies that the SGS stresses have a weak non‐local dependence on coarse fields; it also aligns with our physical conception that small‐scales are locally controlled by larger scales such as vortices and their strained filaments. Furthermore, 2‐layer CNN‐parameterizations are more likely to be interpretable. Plain Language Summary: In this study, we demonstrate that simple, shallow neural networks can be used to effectively model complex turbulent flows in the atmosphere and oceans. By using these simpler NNs, we can improve the efficiency of our simulations and better understand the underlying physics of turbulent flows. We also explore different training techniques to make these models more accurate and robust. Our findings suggest that the stress in these turbulent flows has only a weak spatial dependence on larger‐scale features, which has important implications for our understanding of how turbulence behaves. Overall, our work can help improve climate and weather models by providing a more efficient and interpretable way to simulate turbulence. Key Points: Shallow convolutional neural networks (CNNs) accurately parameterize high Reynolds number forced2D turbulence, with efficient training and high online large eddy simulations accuracyExtensive hyperparameter searching and cyclical learning rates yield robust and accurate online solutions for CNN‐based turbulence modelsThe success of shallow CNNs implies nearly‐local dependence of SGS stresses providing insights into turbulent flow interactions [ABSTRACT FROM AUTHOR]

This paper investigates the usage of machine learning (ML) algorithms on agricultural images with the aim of extracting information regarding the health of plants. More specifically, a custom convolutional neural network is trained on Google Colab using photos of healthy and unhealthy plants. The trained models are evaluated using various single-board computers (SBCs) that demonstrate different essential characteristics. Raspberry Pi 3 and Raspberry Pi 4 are the current mainstream SBCs that use their Central Processing Units (CPUs) for processing and are used for many applications for executing ML algorithms based on popular related libraries such as TensorFlow. NVIDIA Graphic Processing Units (GPUs) have a different rationale and base the execution of ML algorithms on a GPU that uses a different architecture than a CPU. GPUs can also implement high parallelization on the Compute Unified Device Architecture (CUDA) cores. Another current approach involves using a Tensor Processing Unit (TPU) processing unit carried by the Google Coral Dev TPU Board, which is an Application-Specific Integrated Circuit (ASIC) specialized for accelerating ML algorithms such as Convolutional Neural Networks (CNNs) via the usage of TensorFlow Lite. This study experiments with all of the above-mentioned devices and executes custom CNN models with the aim of identifying plant diseases. In this respect, several evaluation metrics are used, including knowledge extraction time, CPU utilization, Random Access Memory (RAM) usage, swap memory, temperature, current milli Amperes (mA), voltage (Volts), and power consumption milli Watts (mW). [ABSTRACT FROM AUTHOR]

Numerical simulation of fluid flow plays an essential role in modeling many physical phenomena, which enables technological advancements, contributes to sustainable practices, and expands our understanding of various natural and engineered systems. The calculation of heat transfer in fluid flow in simple flat channels is a relatively easy task for various simulation methods. However, once the channel geometry becomes more complex, numerical simulations become a bottleneck in optimizing wall geometries. We present a combination of accurate numerical simulations of arbitrary, flat, and non-flat channels as well as machine learning models trained on simulated data to predict the drag coefficient and Stanton number. We show that convolutional neural networks (CNNs) can accurately predict target properties at a fraction of the computational cost of numerical simulations. We use CNN models in a virtual high-throughput screening approach to explore a large number of possible, randomly generated wall architectures. Data augmentation techniques are incorporated to enforce physical invariances toward shifting and flipping, contributing to precise prediction for fluid flow and heat transfer characteristics. Moreover, we approach the interpretation of the trained model to better understand relevant channel structures and their influence on heat transfer. The general approach is not only applicable to simple flow setups as presented here but can be extended to more complex tasks, such as multiphase or even reactive unit operations in chemical engineering. [ABSTRACT FROM AUTHOR]

This study leverages machine learning to analyze the cross-sectional profiles of materials subjected to wire-rolling processes, focusing on the specific stages of these processes and the characteristics of the resulting microstructural profiles. The convolutional neural network (CNN), a potent tool for visual feature analysis and learning, is utilized to explore the properties and impacts of the cold plastic deformation technique. Specifically, CNNs are constructed and trained using 6400 image segments, each with a resolution of 120 x 90 pixels. The chosen architecture incorporates convolutional layers intercalated with polling layers and the "ReLu" activation function. The results, intriguingly, are derived from the observation of only a minuscule cropped fraction of the material's cross-sectional profile. Following calibration two distinct neural networks, training and validation accuracies of 97.4%/97% and 79%/75% have been achieved. These accuracies correspond to identifying the cropped image's location and the number of passes applied to the material. Further improvements in accuracy are reported upon integrating the two networks using a multiple-output setup, with the overall training and validation accuracies slightly increasing to 98.9%/79.4% and 94.6%/78.1%, respectively, for the two features. The study emphasizes the pivotal role of specific architectural elements, such as the rescaling parameter of the augmentation process, in attaining a satisfactory prediction rate. Lastly, we delve into the potential implications of our findings, which shed light on the potential of machine learning techniques in refining our understanding of wire-rolling processes and guiding the development of more efficient and sustainable manufacturing practices. [ABSTRACT FROM AUTHOR]

In this study, we perform online sea ice bias correction within a Geophysical Fluid Dynamics Laboratory global ice‐ocean model. For this, we use a convolutional neural network (CNN) which was developed in a previous study (Gregory et al., 2023, https://doi.org/10.1029/2023ms003757) for the purpose of predicting sea ice concentration (SIC) data assimilation (DA) increments. An initial implementation of the CNN shows systematic improvements in SIC biases relative to the free‐running model, however large summertime errors remain. We show that these residual errors can be significantly improved with a novel sea ice data augmentation approach. This approach applies sequential CNN and DA corrections to a new simulation over the training period, which then provides a new training data set to refine the weights of the initial network. We propose that this machine‐learned correction scheme could be utilized for generating improved initial conditions, and also for real‐time sea ice bias correction within seasonal‐to‐subseasonal sea ice forecasts. Plain Language Summary: Climate models contain errors which often lead to predictions which are consistently out of agreement with what we observe in reality. In some cases we know the origin of these errors, for example, predicting too much sea ice as a result of consistently cool ocean temperatures. In reality, however, there are typically numerous model errors interacting across the atmosphere, ocean and sea ice, and to manually parse through large volumes of climate model data in an attempt to isolate these errors in time and space is highly impractical. Machine learning on the other hand is a framework which is well‐suited to this task. In this work we take a machine learning model which, at any given moment, ingests information about a climate model's atmosphere, ocean and sea ice conditions, and predicts how much error there is in the climate model's representation of sea ice, without seeing any actual sea ice observations. We use this to adjust the sea ice conditions in one particular climate model as it is running forward in time making predictions, and we find that this significantly reduces the model's sea ice errors globally. Key Points: We use a convolutional neural network (CNN) to perform online sea ice bias correction within global ice‐ocean simulationsThe CNN systematically reduces the free‐running model bias in both the Arctic and AntarcticThe online performance can be improved by combining CNN and data assimilation corrections in order to iteratively augment the training data [ABSTRACT FROM AUTHOR]

One of the most significant research areas in education and Artificial Intelligence (AI) is the earlier prediction of students' academic achievement. Limited studies have been conducted using Deep Learning (DL) in the student domain of Intelligent Tutoring System (ITS). Traditional Machine Learning (ML) techniques have been employed in many earlier publications to predict student performance. This paper investigates the effectiveness of DL algorithms for predicting student academic performance. Three different DL architectures based on the structure of Convolutional Neural Networks (CNN) are presented. Two public datasets are used. Furthermore, two feature selection techniques are utilized in this experiment: Principal Component Analysis (PCA) and Decision Trees (DTs). Also, we applied a resampling technique for the first dataset to address the issue of an imbalanced dataset. According to the experimental findings, the proposed CNN model's success in predicting student performance at early stages reached an accuracy of 94.36% using the first dataset and 84.83% using the second dataset. Comparing the proposed approach with the previous studies, the proposed approach outperformed all previous studies when dataset 2 and part of dataset 1 were used. For the complete dataset 1, the proposed model performed very well. [ABSTRACT FROM AUTHOR]

The accuracy of initial conditions is an important driver of the forecast skill of numerical weather prediction models. Increases in the quantity of available measurements, particularly high‐resolution remote sensing observational data products from satellites, are valuable inputs for improving those initial condition estimates. However, the traditional data assimilation methods for integrating observations into forecast models are computationally expensive. This makes incorporating dense observations into operational forecast systems challenging, and it is often prohibitively time‐consuming. Additionally, high‐resolution observations often have correlated observation errors which are difficult to estimate and create problems for assimilation systems. As a result, large quantities of data are discarded and not used for state initialization. Using the Lorenz‐96 system for testing, we demonstrate that a simple machine learning method can be trained to assimilate high‐resolution data. Using it to do so improves both initial conditions and forecast accuracy. Compared to using the Ensemble Kalman Filter with high‐resolution observations ignored, our augmented method has an average root‐mean‐squared error reduced by 37%. Ensemble forecasts using initial conditions generated by the augmented method are more accurate and reliable at up to 10 days of forecast lead time. Plain Language Summary: Weather forecasts are highly sensitive to the estimate of the current state of the atmosphere, known as initial conditions. The atmosphere is chaotic, meaning that small errors in this estimate can grow quickly as the forecast model predicts events further into the future. The satellite era has contributed to large improvements in weather forecasts by providing additional data that allow for more accurate estimates of initial conditions. However, current methods for generating initial conditions are computationally time‐consuming, and as a result, large fractions of available measurements are not used for this purpose. In a proof‐of‐concept study using a simplified representation of the atmosphere for testing, we train a machine learning (ML) method to replicate the results of a traditional method. Once trained, ML models are usually very fast. Applying the trained model exclusively to measurements that would otherwise be too time‐consuming to use produces better initial conditions and more accurate forecasts. Key Points: Machine Learning (ML) augmented data assimilation (DA) of high‐resolution observations improves the analysis in a nonlinear dynamical modelExplainable Artificial Intelligence identifies system covariances to guide neural network training for analysis state reproductionShort‐term forecasts from the analysis generated by the ML augmented DA are more accurate and more reliable [ABSTRACT FROM AUTHOR]