Search

In this paper, we build and explore supervised learning models of ferromagnetic system behavior, using Monte-Carlo sampling of the spin configuration space generated by the 2D Ising model. Given the enormous size of the space of all possible Ising model realizations, the question arises as to how to choose a reasonable number of samples that will form physically meaningful and non-intersecting training and testing datasets. Here, we propose a sampling technique called “ID-MH” that uses the Metropolis–Hastings algorithm creating Markov process across energy levels within the predefined configuration subspace. We show that application of this method retains phase transitions in both training and testing datasets and serves the purpose of validation of a machine learning algorithm. For larger lattice dimensions, ID-MH is not feasible as it requires knowledge of the complete configuration space. As such, we develop a new “block-ID” sampling strategy: it decomposes the given structure into square blocks with lattice dimension N ≤ 5 and uses ID-MH sampling of candidate blocks. Further comparison of the performance of commonly used machine learning methods such as random forests, decision trees, k nearest neighbors and artificial neural networks shows that the PCA-based Decision Tree regressor is the most accurate predictor of magnetizations of the Ising model. For energies, however, the accuracy of prediction is not satisfactory, highlighting the need to consider more algorithmically complex methods (e.g., deep learning). [ABSTRACT FROM AUTHOR]

This article presents a new method to identify the main patterns describing the flow motion in complex flows. The algorithm is an extension of the higher-order dynamic mode decomposition (HODMD), which compresses the snapshots from the analysed database and progressively updates new compressed snapshots on-the-fly , so it is denoted as HODMD on-the-fly (HODMD-of). This algorithm can be applied in parallel to the numerical simulations (or experiments), and it exhibits two main advantages over offline algorithms: (i) it automatically selects on-the-fly the number of necessary snapshots from the database to identify the relevant dynamics; and (ii) it can be used from the beginning of a numerical simulation (or experiment), since it uses a sliding-window to automatically select, also on-the-fly, the suitable interval to perform the data analysis, i.e. it automatically identifies and discards the transient dynamics. The HODMD-of algorithm is suitable to build reduced order models, which have a much lower computational cost than the original simulation. The performance of the method has been tested in three different cases: the axi-symmetric synthetic jet, the three-dimensional wake of a circular cylinder and the turbulent wake behind a wall-mounted square cylinder. The obtained speed-up factors are around 7 with respect to HODMD; this value depends on the simulation and the configuration of the hyperparameters. HODMD-of also provides a significant reduction of the memory requirements, between 40 − 80 % amongst the two- and three-dimensional cases studied in this paper. • Novel algorithm based on HODMD that combines rank reduction strategies with the on-the-fly methodology. • The algorithm automatically selects the optimal time interval to extract the main flow patterns. • Significant reduction of the memory requirements and speed-up related with standard HODMD. • Suitable to build reduced order models. [ABSTRACT FROM AUTHOR]

In this paper, we propose forward and backward stochastic differential equations (FBSDEs) based deep neural network (DNN) learning algorithms for the solution of high dimensional quasi-linear parabolic partial differential equations (PDEs), which is related to the FBSDEs from the Pardoux-Peng theory. The algorithms rely on a learning process by minimizing the path-wise difference between two discrete stochastic processes, which are defined by the time discretization of the FBSDEs and the DNN representation of the PDE solution, respectively. The proposed algorithms are shown to generate DNN solution for a 100-dimensional Black–Scholes–Barenblatt equation, which is accurate in a finite region in the solution space, and has a convergence rate close to that of the Euler–Maruyama scheme used for discretizing the FBSDEs. • FBSDE deep neural network (DNN) with loss comparing two stochastic processes for the PDEs from Pardoux-Peng theory. • Strong 1/2 Convergence as the Euler-Maruyama method in solving a 100-dimensional Black-Scholes-Barenblatt equation. • Use of Richardson extrapolation for accuracy enhancement. • Multiscale DNN captures temporal high frequency content of the PDEs more accurately. [ABSTRACT FROM AUTHOR]

• Machine learning framework for the reconstruction of unsteady fluid flows fields. • Variational approach for a robust reduction of the high dimensional field. • Investigations on three canonical cases. • Combination of linear or nonlinear reduction with regressive or direct reconstruction. This paper investigates the use of data-driven methods for the reconstruction of unsteady fluid flow fields. The proposed framework is based on the combination of machine learning tools: dimensionality reduction to extract dominant spatial directions from data, reconstruction algorithm to recover encoded data by limited measurements and cross-validation for hyperparameter optimization. For the encoding part, linear and nonlinear extraction of patterns are considered: proper orthogonal decomposition (POD), linear autoencoder (LAE) and variational autoencoder (VAE). For the reconstruction part, regressive reconstruction (neural network, linear, support vector, gradient boosting) and library-based reconstruction are compared, each method being cross-validated to ensure good generalization on testing data. The position of sensors is optimized using an enhanced clustering algorithm. The robustness of regressive reconstructions to noise measurements is also addressed, showing the benefits of variational approaches in the reduction phase. The strategy is tested for three increasing complexity flows: 2D vortex shedding (R e = 200), 2D spatial mixing layer and 3D vortex shedding (R e = 20000). The results suggest that proper machine learning approaches to fluid flow data can lead to effective reconstruction models that can be used for the rapid estimation of complex flows. [ABSTRACT FROM AUTHOR]

• Method (modBIP) presented to pre-train random coefficients of extreme learning machines (ELM). • modBIP can work with any activation function and is more accurate than the BIP method. • ELM/modBIP is highly accurate and accuracy is insensitive to initial random coefficients. In extreme learning machines (ELM) the hidden-layer coefficients are randomly set and fixed, while the output-layer coefficients of the neural network are computed by a least squares method. The randomly-assigned coefficients in ELM are known to influence its performance and accuracy significantly. In this paper we present a modified batch intrinsic plasticity (modBIP) method for pre-training the random coefficients in the ELM neural networks. The current method is devised based on the same principle as the batch intrinsic plasticity (BIP) method, namely, by enhancing the information transmission in every node of the neural network. It differs from BIP in two prominent aspects. First, modBIP does not involve the activation function in its algorithm, and it can be applied with any activation function in the neural network. In contrast, BIP employs the inverse of the activation function in its construction, and requires the activation function to be invertible (or monotonic). The modBIP method can work with the often-used non-monotonic activation functions (e.g. Gaussian, swish, Gaussian error linear unit, and radial-basis type functions), with which BIP breaks down. Second, modBIP generates target samples on random intervals with a minimum size, which leads to highly accurate computation results when combined with ELM. The combined ELM/modBIP method is markedly more accurate than ELM/BIP in numerical simulations. Ample numerical experiments are presented with shallow and deep neural networks for function approximation and boundary/initial value problems with partial differential equations. They demonstrate that the combined ELM/modBIP method produces highly accurate simulation results, and that its accuracy is insensitive to the random-coefficient initializations in the neural network. This is in sharp contrast with the ELM results without pre-training of the random coefficients. [ABSTRACT FROM AUTHOR]