Search

The computation of gradients via the reverse mode of algorithmic differentiation is a valuable technique in modelling many science and engineering applications. This technique is particularly efficient when implemented as a source transformation, as it may use static data-flow analysis. However, some features of the major programming languages are detrimental to the efficiency of the transformed source code. This paper provides an overview of the most common problem scenarios and estimates the cost overhead incurred by using the respective language feature or employing certain common patterns. An understanding of these topics is crucial for the efficiency or even feasibility of adjoint computations, particularly for large-scale numerical simulations, for example, in geosciences. While one cannot hope to cover all effects observable with a given programming language in a given run time environment, the paper aims at providing a reasonable guide for the users of C/C++ and Fortran source transformation tools for algorithmic differentiation. [ABSTRACT FROM AUTHOR]

We apply an optimized method to the adjoint generation of a time-evolving land ice model through algorithmic differentiation (AD). The optimization involves a special treatment of the fixed-point iteration required to solve the nonlinear stress balance, which differs from a straight-forward application of AD software, and leads to smaller memory requirements and in some cases shorter computation times of the adjoint. The optimization is done via implementation of the algorithm of Christianson (1994) for reverse accumulation of fixed-point problems, with the AD tool OpenAD. For test problems, the optimized adjoint is shown to have far lower memory requirements, potentially enabling larger problem sizes on memory-limited machines. In the case of the land ice model, implementation of the algorithm allows further optimization by having the adjoint model solve a sequence of linear systems with identical (as opposed to varying) matrices, greatly improving performance. The methods introduced here will be of value to other efforts applying AD tools to ice models, particularly ones which solve a hybrid shallow ice/shallow shelf approximation to the Stokes equations. [ABSTRACT FROM AUTHOR]

In this paper, we describe a code transformation technique that, given a code for a vector function F, produces a code suitable for computing collections of Jacobian-vector products F′(x)x˙ or Jacobian-transpose-vector products F′(x)T y¯. Exploitation of scarcity – a measure of the degrees of freedom in the Jacobian matrix – requires solving a combinatorial optimization problem that is believed to be hard. Our heuristics transform the computational graph for F, producing, in the form of a transformed graph G′, a representation of the Jacobian F′(x) that is both concise and suitable for evaluating large collections of the Jacobian-vector products or the Jacobian-transpose-vector products. Our heuristics are randomized and compare favourably in all cases with the best known heuristics. [ABSTRACT FROM AUTHOR]

Abstract: An essential performance and correctness factor in numerical simulation and optimization is access to exact derivative information. Adjoint derivative models are particularly useful if a function’s number of inputs far exceeds the number of outputs. The propagation of adjoints requires the data flow to be reversed, implying the reversal of all communication in programs that use message-passing. This paper presents recent advances made in developing the adjoint MPI library AMPI. The described proof of concept aims to serve as the basis for coupling other overloading AD tools with AMPI. We illustrate its use in the context of a specific overloading tool for algorithmic differentiation (AD) for C++ programs. A simplified but representative application problem is discussed as a case study. [ABSTRACT FROM AUTHOR]

Abstract: The computation of derivatives via automatic differentiation is a valuable technique in many science and engineering applications. While the implementation of automatic differentiation via source transformation yields the highest-efficiency results, the implementation via operator overloading remains a viable alternative for some application contexts, such as the computation of higher-order derivatives or in cases where C++ still proves to be too complicated for the currently available source transformation tools. The Rapsodia code generator creates libraries that overload intrinsics for derivative computation. In this paper, we discuss modifications to Rapsodia to improve the efficiency of the generated code, first via limited loop unrolling and second via multithreaded asynchronous derivative computation. We introduce the approaches and present runtime results. [ABSTRACT FROM AUTHOR]