Search

This paper presents the space-split sensitivity or the S3 algorithm to transform Ruelle's linear response formula into a well-conditioned ergodic-averaging computation. We prove a decomposition of Ruelle's formula that is differentiable on the unstable manifold, which we assume to be one-dimensional. This decomposition of Ruelle's formula ensures that one of the resulting terms, the stable contribution, can be computed using a regularized tangent equation, similar to in a non-chaotic system. The remaining term, known as the unstable contribution, is regularized and converted into an efficiently computable ergodic average. In this process, we develop new algorithms, which may be useful beyond linear response, to compute i) a fundamental statistical quantity we introduce called the density gradient, and ii) the unstable derivatives of the regularized tangent vector field and the unstable direction. We prove that the S3 algorithm, which combines these computational ingredients that enter the stable and unstable contribution, converges like a Monte Carlo approximation of Ruelle's formula. The algorithm presented here is hence a first step toward full-fledged applications of sensitivity analysis in chaotic systems, wherever such applications have been limited due to lack of availability of long-term sensitivities., Comment: 40 pages; submitted

An assumption of smooth response to small parameter changes, of statistics or long-time averages of a chaotic system, is generally made in the field of sensitivity analysis, and the parametric derivatives of statistical quantities are critically used in science and engineering. In this paper, we propose a numerical procedure to assess the differentiability of statistics with respect to parameters in chaotic systems. We numerically show that the existence of the derivative depends on the Lebesgue-integrability of a certain density gradient function, which we define as the derivative of logarithmic SRB density along the unstable manifold. We develop a recursive formula for the density gradient that can be efficiently computed along trajectories, and demonstrate its use in determining the differentiability of statistics. Our numerical procedure is illustrated on low-dimensional chaotic systems whose statistics exhibit both smooth and rough regions in parameter space., Comment: 32 pages, 13 figures, submitted to journal, under review

In this computational paper, we perform sensitivity analysis of long-time (or ensemble) averages in the chaotic regime using the shadowing algorithm. We introduce automatic differentiation to eliminate the tangent/adjoint equation solvers used in the shadowing algorithm. In a gradient-based optimization, we use the computed shadowing sensitivity to minimize different long-time averaged functionals of a chaotic time-delayed system by optimal parameter selection. In combined state and parameter estimation for data assimilation, we use the computed sensitivity to predict the optimal trajectory given information from a model and data from measurements beyond the predictability time. The algorithms are applied to a thermoacoustic model. Because the computational framework is rather general, the techniques presented in this paper may be used for sensitivity analysis of ensemble averages, parameter optimization and data assimilation of other chaotic problems, where shadowing methods are applicable., Comment: 50 pages, under review

This paper proves that shadowing solutions can be almost surely nonphysical. This finding invalidates the argument that small perturbations in a chaotic system can only have a small impact on its statistical behavior. This theoretical finding has implications for many applications in which chaotic mechanics plays an important role. It suggests, for example, that we can control the climate through subtle perturbations. It also suggests that numerical simulations of chaotic dynamics, such as turbulent flows, may fail to predict the true long-term or statistical behavior., Comment: 33 pages

Sensitivity analysis in chaotic dynamical systems is a challenging task from a computational point of view. In this work, we present a numerical investigation of a novel approach, known as the space-split sensitivity or S3 algorithm. The S3 algorithm is an ergodic-averaging method to differentiate statistics in ergodic, chaotic systems, rigorously based on the theory of hyperbolic dynamics. We illustrate S3 on one-dimensional chaotic maps, revealing its computational advantage over naive finite difference computations of the same statistical response. In addition, we provide an intuitive explanation of the key components of the S3 algorithm, including the density gradient function., Comment: 20 pages, 17 figures

Covariant Lyapunov vectors or CLVs span the expanding and contracting directions of perturbations along trajectories in a chaotic dynamical system. Due to efficient algorithms to compute them that only utilize trajectory information, they have been widely applied across scientific disciplines, principally for sensitivity analysis and predictions under uncertainty. In this paper, we develop a numerical method to compute the directional derivatives of CLVs along their own directions. Similar to the computation of CLVs, the present method for their derivatives is iterative and analogously uses the second-order derivative of the chaotic map along trajectories, in addition to the Jacobian. We validate the new method on a super-contracting Smale-Williams Solenoid attractor. We also demonstrate the algorithm on several other examples including smoothly perturbed Arnold Cat maps, and the Lorenz attractor, obtaining visualizations of the curvature of each attractor. Furthermore, we reveal a fundamental connection of the CLV self-derivatives with a statistical linear response formula., Comment: 28 pages, 13 figures, under review

We present a computable reformulation of Ruelle's linear response formula for chaotic systems. The new formula, called Space-Split Sensitivity or S3, achieves an error convergence of the order ${\cal O}(1/\sqrt{N})$ using $N$ phase points. The reformulation is based on splitting the overall sensitivity into that to stable and unstable components of the perturbation. The unstable contribution to the sensitivity is regularized using ergodic properties and the hyperbolic structure of the dynamics. Numerical examples of uniformly hyperbolic attractors are used to validate the S3 formula against a na\"ive finite-difference calculation; sensitivities match closely, with far fewer sample points required by S3., Comment: 21 pages, 2 figures, submitted

It is well-known that linearized perturbation methods for sensitivity analysis, such as tangent or adjoint equation-based, finite difference and automatic differentiation are not suitable for turbulent flows. The reason is that turbulent flows exhibit chaotic dynamics, leading to the norm of an infinitesimal perturbation to the state growing exponentially in time. As a result, these conventional methods cannot be used to compute the derivatives of long-time averaged quantities to control or design inputs. The ensemble-based approaches and shadowing-based approaches to circumvent the problems of the conventional methods in chaotic systems, also suffer from computational impracticality and lack of consistency guarantees, respectively. We introduce the space-split sensitivity, or the S3 algorithm, which is a Monte-Carlo approach to the chaotic sensitivity computation problem. In this work, we derive the S3 algorithm under simplifying assumptions on the dynamics and present a numerical validation on a low-dimensional example of chaos., Comment: 12 pages, 1 figure, accepted in the 2019 AIAA Aviation Forum

Chaotic dynamical systems such as turbulent flows are characterized by an exponential divergence of infinitesimal perturbations to initial conditions. Therefore, conventional adjoint/tangent sensitivity analysis methods that are successful with RANS simulations fail in the case of chaotic LES/DNS. In this work, we discuss the limitations of current approaches, including ensemble-based and shadowing-based sensitivity methods, that were proposed as alternatives to conventional sensitivity analysis. We propose a new alternative, called the space-split sensitivity (S3) algorithm, that is computationally efficient and addresses these limitations. In this work, the derivation of the S3 algorithm is presented in the special case where the system converges to a stationary distribution that can be expressed with a probability density function everywhere in phase-space. Numerical examples of low-dimensional chaotic maps are discussed where S3 computation shows good agreement with finite-difference results, indicating potential for the development of the method in more generality., Comment: 10 pages; Center for turbulence research summer program 2018

We present the Finite Difference Non-Intrusive Least Squares Shadowing (FD-NILSS) algorithm for computing sensitivities of long-time averaged quantities in chaotic dynamical systems. FD-NILSS does not require tangent solvers, and can be implemented with little modification to existing numerical simulation software. We also give a formula for solving the least-squares problem in FD-NILSS, which can be applied in NILSS as well. Finally, we apply FD-NILSS for sensitivity analysis of a chaotic flow over a 3-D cylinder at Reynolds number 525, where FD-NILSS computes accurate sensitivities and the computational cost is in the same order as the numerical simulation., Comment: 20 pages, 8 figures

Sensitivity analysis methods are important tools for research and design with simulations. Many important simulations exhibit chaotic dynamics, including scale-resolving turbulent fluid flow simulations. Unfortunately, conventional sensitivity analysis methods are unable to compute useful gradient information for long-time-averaged quantities in chaotic dynamical systems. Sensitivity analysis with least squares shadowing (LSS) can compute useful gradient information for a number of chaotic systems, including simulations of chaotic vortex shedding and homogeneous isotropic turbulence. However, this gradient information comes at a very high computational cost. This paper presents multiple shooting shadowing (MSS), a more computationally efficient shadowing approach than the original LSS approach. Through an analysis of the convergence rate of MSS, it is shown that MSS can have lower memory usage and run time than LSS., Comment: Preprint submitted to the Journal of Computational Physics

Adjoint-based sensitivity analysis methods are powerful tools for engineers who use flow simulations for design. However, the conventional adjoint method breaks down for scale-resolving simulations like large-eddy simulation (LES) or direct numerical simulation (DNS), which exhibit the chaotic dynamics inherent in turbulent flows. Sensitivity analysis based on least-squares shadowing (LSS) avoids the issues encountered by conventional methods, but has a high computational cost. The following report outlines a new, more computationally efficient formulation of LSS, non-intrusive LSS, and estimates its cost for several canonical flows using Lyapunov analysis.

We investigate the Lyapunov spectrum of separated flows and their dependence on the numerical discretization. The chaotic flow around the NACA 0012 airfoil at low Reynolds number and large angle of attack is considered to that end, and t-, h- and p-refinement studies are performed to examine each effect separately. Numerical results show that the time discretization has a small impact on the dynamics of the system, whereas the spatial discretization can dramatically change them. In particular, the asymptotic Lyapunov spectrum for time refinement is achieved for CFL numbers as large as $\mathcal{O}(10^1-10^2)$, whereas the system continues to become more and more chaotic even for meshes that are much finer than the best practice for this type of flows.

This paper develops the non-intrusive formulation of the Least-squares shadowing (LSS) method, for computing the sensitivity of long-time averaged objectives in chaotic dynamical systems. This non-intrusive formulation constrains the computation to only the unstable subspace, greatly reducing the cost of LSS for many problems; moreover, it reparametrizes the LSS problem, requiring only minor modifications to existing tangent solvers. NILSS is demonstrated on a chaotic flow over a backward-facing step simulated with a mesh of 12e3 cells., Comment: 26 pages, 10 figures

Computational methods for sensitivity analysis are invaluable tools for aerodynamics research and engineering design. However, traditional sensitivity analysis methods break down when applied to long-time averaged quantities in turbulent fluid flow fields, specifically those obtained using high-fidelity turbulence simulations. This is because of a number of dynamical properties of turbulent and chaotic fluid flows, most importantly high sensitivity of the initial value problem, popularly known as the "butterfly effect". The recently developed least squares shadowing (LSS) method avoids the issues encountered by traditional sensitivity analysis methods by approximating the "shadow trajectory" in phase space, avoiding the high sensitivity of the initial value problem. The following paper discusses how the least squares problem associated with LSS is solved. Two methods are presented and are demonstrated on a simulation of homogeneous isotropic turbulence and the Kuramoto-Sivashinsky (KS) equation, a 4th order chaotic partial differential equation. We find that while LSS computes fairly accurate gradients, faster, more efficient linear solvers are needed to apply both LSS methods presented in this paper to larger simulations., Comment: Presented at AIAA Scitech 2014

Computational methods for sensitivity analysis are invaluable tools for scientists and engineers investigating a wide range of physical phenomena. However, many of these methods fail when applied to chaotic systems, such as the Kuramoto-Sivashinsky (K-S) equation, which models a number of different chaotic systems found in nature. The following paper discusses the application of a new sensitivity analysis method developed by the authors to a modified K-S equation. We find that least squares shadowing sensitivity analysis computes accurate gradients for solutions corresponding to a wide range of system parameters., Comment: 23 pages, 14 figures. Submitted to Chaos, Solitons and Fractals, in review

The adjoint method, among other sensitivity analysis methods, can fail in chaotic dynamical systems. The result from these methods can be too large, often by orders of magnitude, when the result is the derivative of a long time averaged quantity. This failure is known to be caused by ill-conditioned initial value problems. This paper overcomes this failure by replacing the initial value problem with the well-conditioned "least squares shadowing (LSS) problem". The LSS problem is then linearized in our sensitivity analysis algorithm, which computes a derivative that converges to the derivative of the infinitely long time average. We demonstrate our algorithm in several dynamical systems exhibiting both periodic and chaotic oscillations., Comment: submitted to JCP in revised form

This paper describes a forward algorithm and an adjoint algorithm for computing sensitivity derivatives in chaotic dynamical systems, such as the Lorenz attractor. The algorithms compute the derivative of long time averaged "statistical" quantities to infinitesimal perturbations of the system parameters. The algorithms are demonstrated on the Lorenz attractor. We show that sensitivity derivatives of statistical quantities can be accurately estimated using a single, short trajectory (over a time interval of 20) on the Lorenz attractor., Comment: 3/7/2012: applied chain rule to Equation (7), so that Equations (15), (16) and (17) go through w.o. differentiable Lyapunov vectors. = 4/10/2012: Windowing scheme for forward sensitivity. Reduced heavy-tailedness in computed derivatives. Updated Figs 8 and 9. = 9/7/2012: Minor revision, accepted by JCP