Search

We propose a novel method for fast and accurate training of physics-informed neural networks (PINNs) to find solutions to boundary value problems (BVPs) and initial boundary value problems (IBVPs). By combining the methods of training deep neural networks (DNNs) and Extreme Learning Machines (ELMs), we develop a model which has the expressivity of DNNs with the fine-tuning ability of ELMs. We showcase the superiority of our proposed method by solving several BVPs and IBVPs which include linear and non-linear ordinary differential equations (ODEs), partial differential equations (PDEs) and coupled PDEs. The examples we consider include a stiff coupled ODE system where traditional numerical methods fail, a 3+1D non-linear PDE, Kovasznay flow and Taylor-Green vortex solutions to incompressible Navier-Stokes equations and pure advection solution of 1+1 D compressible Euler equation. The Theory of Functional Connections (TFC) is used to exactly impose initial and boundary conditions (IBCs) of (I)BVPs on PINNs. We propose a modification to the TFC framework named Reduced TFC and show a significant improvement in the training and inference time of PINNs compared to IBCs imposed using TFC. Furthermore, Reduced TFC is shown to be able to generalize to more complex boundary geometries which is not possible with TFC. We also introduce a method of applying boundary conditions at infinity for BVPs and numerically solve the pure advection in 1+1 D Euler equations using these boundary conditions., Comment: Accepted for publication in CNSNS

Artificial neural networks (ANNs) are powerful tools capable of approximating any arbitrary mathematical function, but their interpretability remains limited, rendering them as black box models. To address this issue, numerous methods have been proposed to enhance the explainability and interpretability of ANNs. In this study, we introduce the application of information geometric framework to investigate phase transition-like behavior during the training of ANNs and relate these transitions to overfitting in certain models. The evolution of ANNs during training is studied by looking at the probability distribution of its parameters. Information geometry utilizing the principles of differential geometry, offers a unique perspective on probability and statistics by considering probability density functions as points on a Riemannian manifold. We create this manifold using a metric based on Fisher information to define a distance and a velocity. By parameterizing this distance and velocity with training steps, we study how the ANN evolves as training progresses. Utilizing standard datasets like MNIST, FMNIST and CIFAR-10, we observe a transition in the motion on the manifold while training the ANN and this transition is identified with over-fitting in the ANN models considered. The information geometric transitions observed is shown to be mathematically similar to the phase transitions in physics. Preliminary results showing finite-size scaling behavior is also provided. This work contributes to the development of robust tools for improving the explainability and interpretability of ANNs, aiding in our understanding of the variability of the parameters these complex models exhibit during training., Comment: Accepted for publication in Neurocomputing journal

In this work, we investigate the relation between the concept of ``information rate'', an information geometric method for measuring the speed of the time evolution of the statistical states of a stochastic process, and stochastic thermodynamics quantities like entropy rate and entropy production. Then, we propose the application of entropy rate and entropy production to different practical applications such as abrupt event detection, correlation analysis, and control engineering. Specifically, by utilising the Fokker-Planck equation of multi-variable linear stochastic processes described by Langevin equations, we calculate the exact value for information rate, entropy rate, and entropy production and derive various inequalities among them. Inspired by classical correlation coefficients and control techniques, we create entropic-informed correlation coefficients as abrupt event detection methods and information geometric cost functions as optimal thermodynamic control policies, respectively. The methods are analysed via the numerical simulations of common prototypical systems., Comment: 32 pages

We report a first study of time-dependent Probability Density Functions (PDFs) in the Low-to- High confinement mode (L-H) transition by extending the previous prey-predator-type model (Kim & Diamond, Phys. Rev. Lett. 91, 185006, 2003) to a stochastic model. We highlight the limited utility of mean value and variance in understanding the L-H transition by showing strongly non-Gaussian PDFs, with the number of peaks changing in time. We also propose a new information geometric method by using information length, dynamical time scale, and information phase portrait, and show their utility in forecasting transitions and self-regulation between turbulence and zonal flows. In particular, we demonstrate the importance of intermittency (rare events of large amplitude) of zonal flows that can play an important role in promoting the L-H transition., Comment: 5 figures, 6 pages

With improved measurement and modelling technology, variability has emerged as an essential feature in non-equilibrium processes. While traditionally, mean values and variance have been heavily used, they are not appropriate in describing extreme events where a significant deviation from mean values often occurs. Furthermore, stationary Probability Density Functions (PDFs) miss crucial information about the dynamics associated with variability. It is thus critical to go beyond a traditional approach and deal with time-dependent PDFs. Here, we consider atmospheric data from the Whole Atmosphere Community Climate Model (WACCM) model and calculate time-dependent PDFs and the information length from these PDFs, which is the total number of statistically different states that a system passes through in time. Time-dependent PDFs are shown to be non-Gaussian in general, and the information length calculated from these PDFs shed us a new perspective of understanding variabilities, correlation among different variables and regions. Specifically, we calculate time-dependent PDFs and information length and show that the information length tends to increase with the altitude albeit in a complex form. This tendency is more robust for flows/shears than temperature. Also, much similarity among flows and shears in the information length is found in comparison with the temperature. These results also suggest the importance of high latitude/altitude in the information budge in the Earth's atmosphere, the spatial gradient of the information as a useful proxy for the transport of physical quantities., Comment: 15 pages, 5 figures

The impact of adiabatic electrons on drift-wave turbulence, modelled by the Hasegawa-Wakatani equations, is studied using information length. Information length is a novel theoretical method for measuring distances between statistical states represented by different probability distribution functions (PDFs) along the path of a system. Specifically, the time-dependent PDFs of turbulent fluctuations for a given adiabatic index $A$ is computed. The changes in fluctuation statistics are then quantified in time by using information length. The numerical results provide time traces exhibiting intermittent plasma dynamics, and such behaviour is identified by a rapid change in the information length. The effects of $A$ are discussed., Comment: 23 pages submitted to Physics of Plasmas. arXiv admin note: text overlap with arXiv:1706.02849

We report a non-perturbative study of the effect of different type of shear flows on the evolution of vorticity and particle density fluctuations in interchange turbulence. For the same shear strength, the transport of density is less reduced by streamers than by zonal flows, zonal flows leading to oscillation death. In the inviscid limit, vorticity (density) grows (decays) as a power law due to streamer or zonal flow, and exponentially due to the combined effect of zonal flow and streamer with the same sign of shear. Zonal flow and streamer with the opposite sign of shear lead to oscillation at multiple frequencies., Comment: 4 pages, 6 figures, Physics of Plasmas (2017)

We report a non-perturbative study of the effects of shear flows on turbulence reduction in a decaying turbulence in two dimensions. By considering different initial power spectra and shear flows (zonal flows, combined zonal flows and streamers), we demonstrate how shear flows rapidly generate small scales, leading to a fast damping of turbulence amplitude. In particular, a double exponential decrease in turbulence amplitude is shown to occur due to an exponential increase in wavenumber. The scaling of the effective dissipation time scale $\tau_{e}$, previously taken to be a hybrid time scale $\tau_{e} \propto \tau_{\Omega}^{{2/3}} \tau_{\eta}$, is shown to depend on types of depend on the type of shear flow as well as the initial power spectrum. Here, $\tau_{\Omega}$ and $\tau_{\eta}$ are shearing and molecular diffusion times, respectively. Furthermore, we present time-dependent Probability Density Functions (PDFs) and discuss the effect of enhanced dissipation on PDFs and a dynamical time scale $\tau(t)$, which represents the time scale over which a system passes through statistically different states., Comment: 26 pages, 5 figures

Many systems in nature and laboratories are far from equilibrium and exhibit significant fluctuations, invalidating the key assumptions of small fluctuations and short memory time in or near equilibrium. A full knowledge of Probability Distribution Functions (PDFs), especially time-dependent PDFs, becomes essential in understanding far-from-equilibrium processes. We consider a stochastic logistic model with multiplicative noise, which has gamma distributions as stationary PDFs. We numerically solve the transient relaxation problem, and show that as the strength of the stochastic noise increases the time-dependent PDFs increasingly deviate from gamma distributions. For sufficiently strong noise a transition occurs whereby the PDF never reaches a stationary state, but instead forms a peak that becomes ever more narrowly concentrated at the origin. The addition of an arbitrarily small amount of additive noise regularizes these solutions, and re-establishes the existence of stationary solutions. In addition to diagnostic quantities such as mean value, standard deviation, skewness and kurtosis, the transitions between different solutions are analyzed in terms of entropy and information length, the total number of statistically distinguishable states that a system passes through in time., Comment: 26 pages, 11 figures

A probabilistic description is essential for understanding growth processes far from equilibrium. In this paper, we compute time-dependent Probability Density Functions (PDFs) in order to investigate stochastic logistic and Gompertz models, which are two of the most popular growth models. We consider different types of short-correlated internal (multiplicative) and external (additive) stochastic noises and compare the time-dependent PDFs in the two models, elucidating the effects of the additive and multiplicative noises on the form of PDFs. We demonstrate an interesting transition from a unimodal to a bimodal PDF as the multiplicative noise increases for a fixed value of the additive noise. A much weaker (leaky) attractor in the Gompertz model leads to a significant (singular) growth of the population of a very small size. We point out the limitation of using stationary PDFs, mean value and variance in understanding statistical properties of the growth far from equilibrium, highlighting the importance of time-dependent PDFs. We further compare these two models from the perspective of information change that occurs during the growth process. Specifically, we define an infinitesimal distance at any time by comparing two PDFs at times infinitesimally apart and sum these distances in time. The total distance along the trajectory quantifies the total number of different states that the system undergoes in time, and is called the information length. We show that the time-evolution of the two models become more similar when measured in units of the information length and point out the merit of using the information length in unifying and understanding the dynamic evolution of different growth processes.

We study magnetic Taylor-Couette flow in a system having nondimensional radii $r_i=1$ and $r_o=2$, and periodic in the axial direction with wavelengths $h\ge100$. The rotation ratio of the inner and outer cylinders is adjusted to be slightly in the Rayleigh-stable regime, where magnetic fields are required to destabilize the flow, in this case triggering the axisymmetric helical magnetorotational instability (HMRI). Two choices of imposed magnetic field are considered, both having the same azimuthal component $B_\phi=r^{-1}$, but differing axial components. The first choice has $B_z=0.1$, and yields the familiar HMRI, consisting of unidirectionally traveling waves. The second choice has $B_z\approx0.1\sin(2\pi z/h)$, and yields HMRI waves that travel in opposite directions depending on the sign of $B_z$. The first configuration corresponds to a convective instability, the second to an absolute instability. The two variants behave very similarly regarding both linear onset as well as nonlinear equilibration., Comment: submitted to Fluid Dyn Res