Search

Throughout many fields, practitioners often rely on differential equations to model systems. Yet, for many applications, the theoretical derivation of such equations and/or accurate resolution of their solutions may be intractable. Instead, recently developed methods, including those based on parameter estimation, operator subset selection, and neural networks, allow for the data-driven discovery of both ordinary and partial differential equations (PDEs), on a spectrum of interpretability. The success of these strategies is often contingent upon the correct identification of representative equations from noisy observations of state variables and, as importantly and intertwined with that, the mathematical strategies utilized to enforce those equations. Specifically, the latter has been commonly addressed via unconstrained optimization strategies. Representing the PDE as a neural network, we propose to discover the PDE by solving a constrained optimization problem and using an intermediate state representation similar to a Physics-Informed Neural Network (PINN). The objective function of this constrained optimization problem promotes matching the data, while the constraints require that the PDE is satisfied at several spatial collocation points. We present a penalty method and a widely used trust-region barrier method to solve this constrained optimization problem, and we compare these methods on numerical examples. Our results on the Burgers' and the Korteweg-De Vreis equations demonstrate that the latter constrained method outperforms the penalty method, particularly for higher noise levels or fewer collocation points. For both methods, we solve these discovered neural network PDEs with classical methods, such as finite difference methods, as opposed to PINNs-type methods relying on automatic differentiation. We briefly highlight other small, yet crucial, implementation details., Comment: Minor changes: added references, clarified use of classical solvers over PINNs, minor notation changes. 39 pages, 18 figures

Stochastic collocation (SC) is a well-known non-intrusive method of constructing surrogate models for uncertainty quantification. In dynamical systems, SC is especially suited for full-field uncertainty propagation that characterizes the distributions of the high-dimensional primary solution fields of a model with stochastic input parameters. However, due to the highly nonlinear nature of the parameter-to-solution map in even the simplest dynamical systems, the constructed SC surrogates are often inaccurate. This work presents an alternative approach, where we apply the SC approximation over the dynamics of the model, rather than the solution. By combining the data-driven sparse identification of nonlinear dynamics (SINDy) framework with SC, we construct dynamics surrogates and integrate them through time to construct the surrogate solutions. We demonstrate that the SC-over-dynamics framework leads to smaller errors, both in terms of the approximated system trajectories as well as the model state distributions, when compared against full-field SC applied to the solutions directly. We present numerical evidence of this improvement using three test problems: a chaotic ordinary differential equation, and two partial differential equations from solid mechanics.

The Tucker decomposition, an extension of singular value decomposition for higher-order tensors, is a useful tool in analysis and compression of large-scale scientific data. While it has been studied extensively for static datasets, there are relatively few works addressing the computation of the Tucker factorization of streaming data tensors. In this paper we propose a new streaming Tucker algorithm tailored for scientific data, specifically for the case of a data tensor whose size increases along a single streaming mode that can grow indefinitely, which is typical of time-stepping scientific applications. At any point of this growth, we seek to compute the Tucker decomposition of the data generated thus far, without requiring storing the past tensor slices in memory. Our algorithm accomplishes this by starting with an initial Tucker decomposition and updating its components--the core tensor and factor matrices--with each new tensor slice as it becomes available, while satisfying a user-specified threshold of norm error. We present an implementation within the TuckerMPI software framework, and apply it to synthetic and combustion simulation datasets. By comparing against the standard (batch) decomposition algorithm we show that our streaming algorithm provides significant improvements in memory usage. If the tensor rank stops growing along the streaming mode, the streaming algorithm also incurs less computational time compared to the batch algorithm.

For turbulent reacting flows, identification of low-dimensional representations of the thermo-chemical state space is vitally important, primarily to significantly reduce the computational cost of device-scale simulations. Principal component analysis (PCA), and its variants, is a widely employed class of methods. Recently, an alternative technique that focuses on higher-order statistical interactions, co-kurtosis PCA (CoK-PCA), has been shown to effectively provide a low-dimensional representation by capturing the stiff chemical dynamics associated with spatiotemporally localized reaction zones. While its effectiveness has only been demonstrated based on a priori analysis with linear reconstruction, in this work, we employ nonlinear techniques to reconstruct the full thermo-chemical state and evaluate the efficacy of CoK-PCA compared to PCA. Specifically, we combine a CoK-PCA/PCA based dimensionality reduction (encoding) with an artificial neural network (ANN) based reconstruction (decoding) and examine a priori the reconstruction errors of the thermo-chemical state. In addition, we evaluate the errors in species production rates and heat release rates that are nonlinear functions of the reconstructed state as a measure of the overall accuracy of the dimensionality reduction technique. We employ four datasets to assess CoK-PCA/PCA coupled with ANN-based reconstruction: a homogeneous reactor for autoignition of an ethylene/air mixture that has conventional single-stage ignition kinetics, a dimethyl ether (DME)/air mixture which has two-stage ignition kinetics, a one-dimensional freely propagating premixed ethylene/air laminar flame, and a two-dimensional homogeneous charge compression ignition of ethanol. The analyses demonstrate the robustness of the CoK-PCA based low-dimensional manifold with ANN reconstruction in accurately capturing the data, specifically from the reaction zones., Comment: 19 pages, 18 figures

Principal Component Analysis (PCA) is a dimensionality reduction technique widely used to reduce the computational cost associated with numerical simulations of combustion phenomena. However, PCA, which transforms the thermo-chemical state space based on eigenvectors of co-variance of the data, could fail to capture information regarding important localized chemical dynamics, such as the formation of ignition kernels, appearing as \rev{extreme-valued} samples in a dataset. In this paper, we propose an alternate dimensionality reduction procedure, co-kurtosis PCA (CoK-PCA), wherein the required principal vectors are computed from a high-order joint statistical moment, namely the co-kurtosis tensor, which may better identify directions in the state space that represent stiff dynamics. We first demonstrate the potential of the proposed CoK-PCA method using a synthetically generated dataset that is representative of typical combustion simulations. Thereafter, we characterize and contrast the accuracy of CoK-PCA against PCA for datasets representing spontaneous ignition of premixed ethylene-air in a simple homogeneous reactor and ethanol-fueled homogeneous charged compression ignition (HCCI) engine. Specifically, we compare the low-dimensional manifolds in terms of reconstruction errors of the original thermo-chemical state, and species production and heat release rates computed from the reconstructed state. \rev{The latter -- a comparison of species production and heat release rates -- is a more rigorous assessment of the accuracy of dimensionality reduction.} We find that, even using a simplistic linear reconstruction, the co-kurtosis based reduced manifold represents the original thermo-chemical state more accurately than PCA, especially in the regions where chemical reactions are important.

We propose a strategy to compress and store large volumes of scientific data represented on unstructured grids. Approaches utilizing tensor decompositions for data compression have already been proposed. Here, data on a structured grid is stored as a tensor which is then subjected to appropriate decomposition in suitable tensor formats. Such decompositions are based on generalization of singular value decomposition to tensors and capture essential features in the data with storage cost lower by orders of magnitude. However, tensor based data compression is limited by the fact that one can only consider scientific data represented on structured grids. In case of data on unstructured meshes, we propose to consider data as realizations of a function that is based on functional view of the tensor thus avoiding such limitations. The key is to efficiently estimate the parameters of the function whose complexity is small compared to the cardinality of the dataset (otherwise there is no compression). Here, we introduce the set of functional sparse Tucker tensors and propose a method to construct approximation in this set such that the resulting compact functional tensor can be rapidly evaluated to recover the original data. The compression procedure consists of three steps. In the first step, we consider a fraction of the original dataset for interpolation on a structured grid followed by sequentially truncated higher order singular value decomposition to get a compressed version of the interpolated data.We then fit singular vectors on a set of functional basis using sparse approximation to obtain corresponding functional sparse Tucker tensor representation. Finally, we re-evaluate the coefficients of this functional tensor using randomized least squares at a reduced computational complexity. This strategy leads to compression ratio of orders of magnitude on combustion simulation datasets.

We propose an anomaly detection method for multi-variate scientific data based on analysis of high-order joint moments. Using kurtosis as a reliable measure of outliers, we suggest that principal kurtosis vectors, by analogy to principal component analysis (PCA) vectors, signify the principal directions along which outliers appear. The inception of an anomaly, then, manifests as a change in the principal values and vectors of kurtosis. Obtaining the principal kurtosis vectors requires decomposing a fourth order joint cumulant tensor for which we use a simple, computationally less expensive approach that involves performing a singular value decomposition (SVD) over the matricized tensor. We demonstrate the efficacy of this approach on synthetic data, and develop an algorithm to identify the occurrence of a spatial and/or temporal anomalous event in scientific phenomena. The algorithm decomposes the data into several spatial sub-domains and time steps to identify regions with such events. Feature moment metrics, based on the alignments of the principal kurtosis vectors, are computed at each sub-domain and time step for all features to quantify their relative importance towards the overall kurtosis in the data. Accordingly, spatial and temporal anomaly metrics for each sub-domain are proposed using the Hellinger distance of the feature moment metric distribution from a suitable nominal distribution. We apply the algorithm to a one-dimensional turbulent auto-ignition combustion case and demonstrate that the anomaly metrics reliably capture the occurrence of auto-ignition in relevant spatial sub-domains at the right time steps.