Search

Inverse models arise in various environmental applications, ranging from atmospheric modeling to geosciences. Inverse models can often incorporate predictor variables, similar to regression, to help estimate natural processes or parameters of interest from observed data. Although a large set of possible predictor variables may be included in these inverse or regression models, a core challenge is to identify a small number of predictor variables that are most informative of the model, given limited observations. This problem is typically referred to as model selection. A variety of criterion-based approaches are commonly used for model selection, but most follow a two-step process: first, select predictors using some statistical criteria, and second, solve the inverse or regression problem with these predictor variables. The first step typically requires comparing all possible combinations of candidate predictors, which quickly becomes computationally prohibitive, especially for large-scale problems. In this work, we develop a one-step approach for linear inverse modeling, where model selection and the inverse model are performed in tandem. We reformulate the problem so that the selection of a small number of relevant predictor variables is achieved via a sparsity-promoting prior. Then, we describe hybrid iterative projection methods based on flexible Krylov subspace methods for efficient optimization. These approaches are well-suited for large-scale problems with many candidate predictor variables. We evaluate our results against traditional, criteria-based approaches. We also demonstrate the applicability and potential benefits of our approach using examples from atmospheric inverse modeling based on NASA's Orbiting Carbon Observatory-2 (OCO-2) satellite. [ABSTRACT FROM AUTHOR]

Inverse models arise in various environmental applications, ranging from atmospheric modeling to geosciences. Inverse models can often incorporate predictor variables, similar to regression, to help estimate natural processes or parameters of interest from observed data. Although a large set of possible predictor variables may be included in these inverse or regression models, a core challenge is to identify a small number of predictor variables that are most informative of the model, given limited observations. This problem is typically referred to as model selection. A variety of criterion-based approaches are commonly used for model selection, but most follow a two-step process: first, select predictors using some statistical criteria, and second, solve the inverse or regression problem with these predictor variables. The first step typically requires comparing all possible combinations of candidate predictors, which quickly becomes computationally prohibitive, especially for large-scale problems. In this work, we develop a one-step approach, where model selection and the inverse model are performed in tandem. We reformulate the problem so that the selection of a small number of relevant predictor variables is achieved via a sparsity-promoting prior. Then, we describe hybrid iterative projection methods based on flexible Krylov subspace methods for efficient optimization. These approaches are well-suited for large-scale problems with many candidate predictor variables. We evaluate our results against traditional, criteria-based approaches. We also demonstrate the applicability and potential benefits of our approach using examples from atmospheric inverse modeling based on NASA's Orbiting Carbon Observatory 2 (OCO-2) satellite. [ABSTRACT FROM AUTHOR]

This paper surveys an important class of methods that combine iterative projection methods and variational regularization methods for large-scale inverse problems. Iterative methods such as Krylov subspace methods are invaluable in the numerical linear algebra community and have proved important in solving inverse problems due to their inherent regularizing properties and their ability to handle large-scale problems. Variational regularization describes a broad and important class of methods that are used to obtain reliable solutions to inverse problems, whereby one solves a modified problem that incorporates prior knowledge. Hybrid projection methods combine iterative projection methods with variational regularization techniques in a synergistic way, providing researchers with a powerful computational framework for solving very large inverse problems. Although the idea of a hybrid Krylov method for linear inverse problems goes back to the 1980s, several recent advances on new regularization frameworks and methodologies have made this field ripe for extensions, further analyses, and new applications. In this paper, we provide a practical and accessible introduction to hybrid projection methods in the context of solving large (linear) inverse problems. [ABSTRACT FROM AUTHOR]

We develop hybrid projection methods for computing solutions to large-scale inverse problems, where the solution represents a sum of different stochastic components. Such scenarios arise in many imaging applications (e.g., anomaly detection in atmospheric emissions tomography) where the reconstructed solution can be represented as a combination of two or more components and each component contains different smoothness or stochastic properties. In a deterministic inversion or inverse modeling framework, these assumptions correspond to different regularization terms for each solution in the sum. Although various prior assumptions can be included in our framework, we focus on the scenario where the solution is a sum of a sparse solution and a smooth solution. For computing solution estimates, we develop hybrid projection methods for solution decomposition that are based on a combined flexible and generalized Golub--Kahan process. This approach integrates techniques from the generalized Golub--Kahan bidiagonalization and the flexible Krylov methods. The benefits of the proposed methods are that the decomposition of the solution can be done iteratively, and the regularization terms and regularization parameters are adaptively chosen at each iteration. Numerical results from photoacoustic tomography and atmospheric inverse modeling demonstrate the potential for these methods to be used for anomaly detection. [ABSTRACT FROM AUTHOR]

In this work, we investigate various approaches that use learning from training data to solve inverse problems, following a bi-level learning approach. We consider a general framework for optimal inversion design, where training data can be used to learn optimal regularization parameters, data fidelity terms, and regularizers, thereby resulting in superior variational regularization methods. In particular, we describe methods to learn optimal p and q norms for Lp - Lq regularization and methods to learn optimal parameters for regularization matrices defined by covariance kernels. We exploit efficient algorithms based on Krylov projection methods for solving the regularized problems, both at training and validation stages, making these methods well-suited for large-scale problems. Our experiments show that the learned regularization methods perform well even when there is some inexactness in the forward operator, resulting in a mixture of model and measurement error. [ABSTRACT FROM AUTHOR]

Atmospheric inverse modeling describes the process of estimating greenhouse gas fluxes or air pollution emissions at the Earth's surface using observations of these gases collected in the atmosphere. The launch of new satellites, the expansion of surface observation networks, and a desire for more detailed maps of surface fluxes have yielded numerous computational and statistical challenges for standard inverse modeling frameworks that were often originally designed with much smaller data sets in mind. In this article, we discuss computationally efficient methods for large-scale atmospheric inverse modeling and focus on addressing some of the main computational and practical challenges. We develop generalized hybrid projection methods, which are iterative methods for solving large-scale inverse problems, and specifically we focus on the case of estimating surface fluxes. These algorithms confer several advantages. They are efficient, in part because they converge quickly, they exploit efficient matrix–vector multiplications, and they do not require inversion of any matrices. These methods are also robust because they can accurately reconstruct surface fluxes, they are automatic since regularization or covariance matrix parameters and stopping criteria can be determined as part of the iterative algorithm, and they are flexible because they can be paired with many different types of atmospheric models. We demonstrate the benefits of generalized hybrid methods with a case study from NASA's Orbiting Carbon Observatory 2 (OCO-2) satellite. We then address the more challenging problem of solving the inverse model when the mean of the surface fluxes is not known a priori; we do so by reformulating the problem, thereby extending the applicability of hybrid projection methods to include hierarchical priors. We further show that by exploiting mathematical relations provided by the generalized hybrid method, we can efficiently calculate an approximate posterior variance, thereby providing uncertainty information. [ABSTRACT FROM AUTHOR]

Deep neural networks (DNNs) have shown their success as high-dimensional function approximators in many applications; however, training DNNs can be challenging in general. DNN training is commonly phrased as a stochastic optimization problem whose challenges include nonconvexity, nonsmoothness, insufficient regularization, and complicated data distributions. Hence, the performance of DNNs on a given task depends crucially on tuning hyperparameters, especially learning rates and regularization parameters. In the absence of theoretical guidelines or prior experience on similar tasks, this requires solving a series of repeated training problems which can be time-consuming and demanding on computational resources. This can limit the applicability of DNNs to problems with nonstandard, complex, and scarce datasets, e.g., those arising in many scientific applications. To remedy the challenges of DNN training, we propose slimTrain, a stochastic optimization method for training DNNs with reduced sensitivity to the choice of hyperparameters and fast initial convergence. The central idea of slimTrain is to exploit the separability inherent in many DNN architectures; that is, we separate the DNN into a nonlinear feature extractor followed by a linear model. This separability allows us to leverage recent advances made for solving large-scale, linear, ill-posed inverse problems. Crucially, for the linear weights, slimTrain does not require a learning rate and automatically adapts the regularization parameter. In our numerical experiments using function approximation tasks arising in surrogate modeling and dimensionality reduction, slimTrain outperforms existing DNN training methods with the recommended hyperparameter settings and reduces the sensitivity of DNN training to the remaining hyperparameters. Since our method operates on mini-batches, its computational overhead per iteration is modest and savings can be realized by reducing the number of iterations (due to quicker initial convergence) or the number of training problems that need to be solved to identify effective hyperparameters. [ABSTRACT FROM AUTHOR]

Iterative hybrid projection methods have proven to be very effective for solving large linear inverse problems due to their inherent regularizing properties as well as the added flexibility to select regularization parameters adaptively. In this work, we develop Golub--Kahan-based hybrid projection methods that can exploit compression and recycling techniques in order to solve a broad class of inverse problems where memory requirements or high computational cost may otherwise be prohibitive. For problems that have many unknown parameters and require many iterations, hybrid projection methods with recycling can be used to compress and recycle the solution basis vectors to reduce the number of solution basis vectors that must be stored, while obtaining a solution accuracy that is comparable to that of standard methods. If reorthogonalization is required, this may also reduce computational cost substantially. In other scenarios, such as streaming data problems or inverse problems with multiple datasets, hybrid projection methods with recycling can be used to efficiently integrate previously computed information for faster and better reconstruction. Additional benefits of the proposed methods are that various subspace selection and compression techniques can be incorporated, standard techniques for automatic regularization parameter selection can be used, and the methods can be applied multiple times in an iterative fashion. Theoretical results show that, under reasonable conditions, regularized solutions for our proposed recycling hybrid method remain close to regularized solutions for standard hybrid methods and reveal important connections among the resulting projection matrices. Numerical examples from image processing show the potential benefits of combining recycling with hybrid projection methods. [ABSTRACT FROM AUTHOR]

In many physiological systems, real-time endogeneous and exogenous signals in living organisms provide critical information and interpretations of physiological functions; however, these signals or variables of interest are not directly accessible and must be estimated from noisy, measured signals. In this paper, we study an inverse problem of recovering gas exchange signals of animals placed in a flow-through respirometry chamber from measured gas concentrations. For large-scale experiments (e.g., long scans with high sampling rate) that have many uncertainties (e.g., noise in the observations or an unknown impulse response function), this is a computationally challenging inverse problem. We first describe various computational tools that can be used for respirometry reconstruction and uncertainty quantification when the impulse response function is known. Then, we address the more challenging problem where the impulse response function is not known or only partially known. We describe nonlinear optimization methods for reconstruction, where both the unknown model parameters and the unknown signal are reconstructed simultaneously. Numerical experiments show the benefits and potential impacts of these methods in respirometry. [ABSTRACT FROM AUTHOR]

When solving ill-posed inverse problems, a good choice of the prior is critical for the computation of a reasonable solution. A common approach is to include a Gaussian prior, which is defined by a mean vector and a symmetric and positive definite covariance matrix, and to use iterative projection methods to solve the corresponding regularized problem. However, a main challenge for many of these iterative methods is that the prior covariance matrix must be known and fixed (up to a constant) before starting the solution process. In this paper, we develop hybrid projection methods for inverse problems with mixed Gaussian priors where the prior covariance matrix is a convex combination of matrices and the mixing parameter and the regularization parameter do not need to be known in advance. Such scenarios may arise when data is used to generate a sample prior covariance matrix (e.g., in data assimilation) or when different priors are needed to capture different qualities of the solution. The proposed hybrid methods are based on a mixed Golub–Kahan process, which is an extension of the generalized Golub–Kahan bidiagonalization, and a distinctive feature of the proposed approach is that both the regularization parameter and the weighting parameter for the covariance matrix can be estimated automatically during the iterative process. Furthermore, for problems where training data are available, various data-driven covariance matrices (including those based on learned covariance kernels) can be easily incorporated. Numerical examples from tomographic reconstruction demonstrate the potential for these methods. [ABSTRACT FROM AUTHOR]

Summary: Uncertainty quantification for linear inverse problems remains a challenging task, especially for problems with a very large number of unknown parameters (e.g., dynamic inverse problems) and for problems where computation of the square root and inverse of the prior covariance matrix are not feasible. This work exploits Krylov subspace methods to develop and analyze new techniques for large‐scale uncertainty quantification in inverse problems. In this work, we assume that generalized Golub‐Kahan‐based methods have been used to compute an estimate of the solution, and we describe efficient methods to explore the posterior distribution. In particular, we use the generalized Golub‐Kahan bidiagonalization to derive an approximation of the posterior covariance matrix, and we provide theoretical results that quantify the accuracy of the approximate posterior covariance matrix and of the resulting posterior distribution. Then, we describe efficient methods that use the approximation to compute measures of uncertainty, including the Kullback‐Liebler divergence. We present two methods that use the preconditioned Lanczos algorithm to efficiently generate samples from the posterior distribution. Numerical examples from dynamic photoacoustic tomography demonstrate the effectiveness of the described approaches. [ABSTRACT FROM AUTHOR]

In many modern imaging applications the desire to reconstruct high resolution images, coupled with the abundance of data from acquisition using ultra-fast detectors, have led to new challenges in image reconstruction. A main challenge is that the resulting linear inverse problems are massive. The size of the forward model matrix exceeds the storage capabilities of computer memory, or the observational dataset is enormous and not available all at once. Row-action methods that iterate over samples of rows can be used to approximate the solution while avoiding memory and data availability constraints. However, their overall convergence can be slow. In this paper, we introduce a sampled limited memory row-action method for linear least squares problems, where an approximation of the global curvature of the underlying least squares problem is used to speed up the initial convergence and to improve the accuracy of iterates. We show that this limited memory method is a generalization of the damped block Kaczmarz method, and we prove linear convergence of the expectation of the iterates and of the error norm up to a convergence horizon. Numerical experiments demonstrate the benefits of these sampled limited memory row-action methods for massive 2D and 3D inverse problems in tomography applications. [ABSTRACT FROM AUTHOR]

In this paper we develop flexible Krylov methods for efficiently computing regularized solutions to large-scale linear inverse problems with an ℓ2 fit-to-data term and an £p penalization term, for p > 1. First we approximate the p-norm penalization term as a sequence of 2-norm penalization terms using adaptive regularization matrices in an iterative reweighted norm fashion, and then we exploit flexible preconditioning techniques to efficiently incorporate the weight updates. To handle general (nonsquare) ℓp-regularized least-squares problems, we introduce a flexible Golub--Kahan approach and exploit it within a Krylov-Tikhonov hybrid framework. Furthermore, we show that both the flexible Golub--Kahan and the flexible Arnoldi approaches for p =1 can be used to efficiently compute solutions that are sparse with respect to some transformations. The key benefits of our approach compared to existing optimization methods for £p regularization are that inner-outer iteration schemes are replaced by efficient projection methods on linear subspaces of increasing dimension and that expensive regularization parameter selection techniques can be avoided. Theoretical insights are provided, and numerical results from image deblurring and tomographic reconstruction illustrate the benefits of this approach, compared to well-established methods. [ABSTRACT FROM AUTHOR]

A literary criticism of the book "Oneness: East Asian Conceptions of Virtue, Happiness & How We Are All Connected" by Philip J. Ivanhoe is presented. It outlines the remarkably wide array of sources from neo-Confucian philosophy. It further discusses about opportunities for constructive engagement, particularly between classical-to-contemporary Chinese philosophy and contemporary Anglo-analytic philosophy; and the oneness hypothesis to explicate aesthetic obligation in a different manner.

Much work in epistemology, old and new, is devoted to explaining – or explaining away – the appeal of skeptical arguments (that is, arguments designed to impugn most, if not all, of our claims to know), as well as the fact that it also often seems as if we use knowledge-attributing sentences to express truths. More recently, philosophers have sought to explain other kinds of variability in our use of knowledge-attributing sentences besides, such as the kinds of variability that occur as a result of changing what is at stake or the salience of alternative possibilities, as in scenarios such as Keith DeRose's "bank cases." These cases, and others like them, pose a dilemma very much like that posed by skeptical arguments. For they appear to suggest that we are wrong about something. Either we speak falsely when we attribute knowledge in the situations under consideration, or we speak falsely when we deny it. Or so it would seem. This paper argues that an alternative approach to explaining the relevant variability in our use of knowledge-attributing sentences – namely, a fictionalist one – merits serious consideration. [ABSTRACT FROM AUTHOR]

Copyright of Dialogue: Canadian Philosophical Review is the property of Cambridge University Press and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

We develop a generalized hybrid iterative approach for computing solutions to large-scale Bayesian inverse problems. We consider a hybrid algorithm based on the generalized Golub-Kahan bidiagonalization for computing Tikhonov regularized solutions to problems where explicit computation of the square root and inverse of the covariance kernel for the prior covariance matrix is not feasible. This is useful for large-scale problems where covariance kernels are defined on irregular grids or are available only via matrix-vector multiplication, e.g., those from the Matérn class. We show that iterates are equivalent to LSQR iterates applied to a directly regularized Tikhonov problem, after a transformation of variables, and we provide connections to a generalized singular value decomposition filtered solution. Our approach shares many benefits of standard hybrid methods such as avoiding semiconvergence and automatically estimating the regularization parameter. Numerical examples from image processing demonstrate the effectiveness of the described approaches. [ABSTRACT FROM AUTHOR]

In this paper, we consider optimal low-rank regularized inverse matrix approximations and their applications to inverse problems. We give an explicit solution to a generalized rank-constrained regularized inverse approximation problem, where the key novelties are that we al- low for updates to existing approximations and we allow for incorporation of additional probabilistic information. Since computing optimal regularized inverse matrices under rank constraints can be challenging, especially for problems where matrices are large and sparse or are only accessable via function call, we propose an efficient rank-update approach that decomposes the problem into a sequence of smaller rank problems. Using examples from image deblurring, we demonstrate that more accurate solutions to inverse problems can be achieved by using rank updates to existing regularized inverse approximations. Furthermore, we show the potential benefits of using optimal regularized inverse matrix updates for solving perturbed tomographic reconstruction problems. [ABSTRACT FROM AUTHOR]

Reconstructing images from indirect measurements is a central problem in many applications, including the subject of this special issue, quantitative susceptibility mapping (QSM). The process of image reconstruction typically requires solving an inverse problem that is ill-posed and large-scale and thus challenging to solve. Although the research field of inverse problems is thriving and very active with diverse applications, in this part of the special issue we will focus on recent advances in inverse problems that are specific to deconvolution problems, the class of problems to which QSM belongs. We will describe analytic tools that can be used to investigate underlying ill-posedness and apply them to the QSM reconstruction problem and the related extensively studied image deblurring problem. We will discuss state-of-the-art computational tools and methods for image reconstruction, including regularization approaches and regularization parameter selection methods. We finish by outlining some of the current trends and future challenges. Copyright © 2016 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

Motion, e.g., due to patient movement or improper device calibration, is inevitable in many imaging modalities such as photoacoustic tomography (PAT) by a rotating system and can lead to undesirable motion artifacts in image reconstructions, if ignored. In this paper, we establish a hybrid-type model for PAT that incorporates motion in the model. We introduce an approximate continuous model and establish two uniqueness results for some specific parametrized motion models. Then we formulate the discrete problem of simultaneous motion estimation and image reconstruction as a separable nonlinear least squares problem and describe an automatic approach to detect and eliminate motion artifacts during the reconstruction process. Numerical examples validate our methods. [ABSTRACT FROM AUTHOR]

We develop a hybrid iterative approach for computing solutions to large-scale ill-posed inverse problems via Tikhonov regularization. We consider a hybrid LSMR algorithm, where Tikhonov regularization is applied to the LSMR subproblem rather than the original problem. We show that, contrary to standard hybrid methods, hybrid LSMR iterates are not equivalent to LSMR iterates on the directly regularized Tikhonov problem. Instead, hybrid LSMR leads to a different Krylov subspace problem. We show that hybrid LSMR shares many of the benefits of standard hybrid methods such as avoiding semiconvergence behavior. In addition, since the regularization parameter can be estimated during the iterative process, it does not need to be estimated a priori, making this approach attractive for large-scale problems. We consider various methods for selecting regularization parameters and discuss stopping criteria for hybrid LSMR, and we present results from image processing. [ABSTRACT FROM AUTHOR]

Regularization approaches based on spectral filtering can be highly effective in solving ill-posed inverse problems. These methods, however, require computing the singular value decomposition (SVD) and choosing appropriate regularization parameters. These tasks can be prohibitively expensive for large-scale problems. In this paper, we present a framework that uses operator approximations to efficiently obtain good regularization parameters without an SVD of the original operator. Instead, we approximate the original operator with a nearby structured or separable one whose SVD is easily computable. Highly effective methods can then be used to efficiently compute good regularization parameters for the nearby problem. Then, we solve the original problem iteratively using the regularization determined for the approximate problem. A variety of regularization approaches can be incorporated into this framework, but we focus here on the recently developed windowed regularization, a generalization of Tikhonov regularization in which different regularization parameters are used in different regions of the spectrum. We derive bounds on the perturbation to the computed solution and residual resulting from using the regularization determined for the approximate operator. We demonstrate the effectiveness of our method in computations using operator approximations such as sums of Kronecker products, block circulant with circulant blocks matrices, and Krylov subspace approximations. [ABSTRACT FROM AUTHOR]

Regularization is used in order to obtain a reasonable estimate of the solution to an ill-posed inverse problem. One common form of regularization is to use a filter to reduce the influence of components corresponding to small singular values, perhaps using a Tikhonov least squares formulation. In this work, we break the problem into subproblems with narrower bands of singular values using spectrally defined windows, and we regularize each subproblem individually. We show how to use standard parameter-choice methods, such as the discrepancy principle and generalized cross-validation, in a windowed regularization framework. A perturbation analysis gives sensitivity estimates. We demonstrate the effectiveness of our algorithms on deblurring images and on the backward heat equation. [ABSTRACT FROM AUTHOR]

Spectral filtering suppresses the amplification of errors when computing solutions to ill-posed inverse problems; however, selecting good regularization parameters is often expensive. In many applications, data are available from calibration experiments. In this paper, we describe how to use such data to precompute optimal spectral filters. We formulate the problem in an empirical Bayes risk minimization framework and use efficient methods from stochastic and numerical optimization to compute optimal filters. Our formulation of the optimal filter problem is general enough to use a variety of assessments of goodness of the solution estimate, not just the mean square error. The relationship with the Wiener filter is discussed, and numerical examples from signal and image deconvolution illustrate that our proposed filters perform consistently better than well-established filtering methods. Furthermore, we show how our approach leads to easily computed uncertainty estimates for the pixel values. [ABSTRACT FROM AUTHOR]

Digital tomosynthesis imaging is becoming increasingly significant in a variety of medical imaging applications. Tomosynthesis imaging involves the acquisition of a series of projection images over a limited angular range, which, after reconstruction, results in a pseudo--three-dimensional (3D) representation of the imaged object. The partial separation of features in the third dimension improves the visibility of lesions of interest by reducing the effect of the superimposition of tissues. In breast cancer imaging, tomosynthesis is a viable alternative to standard mammography; however, current algorithms for image reconstruction do not take into account the polyenergetic nature of the x-ray source beam entering the object. This results in inaccuracies in the reconstruction, making quantitative analysis challenging and allowing for beam hardening artifacts. In this paper, we develop a mathematical framework based on a polyenergetic model and develop statistically based iterative methods for digital tomosynthesis reconstruction for breast imaging. By applying our algorithms to simulated data, we illustrate the success of our methods in suppressing beam hardening artifacts and improving the overall quality of the reconstruction. [ABSTRACT FROM AUTHOR]

In many physiological studies, variables of interest are not directly accessible, requiring that they be estimated indirectly from noisy measured signals. Here, we introduce two empirical methods to estimate the true physiological signals from indirectly measured, noisy data. The first method is an extension of Tikhonov regularization to large-scale problems, using a sequential update approach. In the second method, we improve the conditioning of the problem by assuming that the input is uniform over a known time interval, and then use a least-squares method to estimate the input. These methods were validated computationally and experimentally by applying them to flow-through respirometry data. Specifically, we infused CO2 in a flow-through respirometry chamber in a known pattern, and used the methods to recover the known input from the recorded data. The results from these experiments indicate that these methods are capable of sub-second accuracy. We also applied the methods on respiratory data from a grasshopper to investigate the exact timing of abdominal pumping, spiracular opening, and CO2 emission. The methods can be used more generally for input estimation of any linear system., Competing Interests: The authors declare no competing or financial interests., (© 2016. Published by The Company of Biologists Ltd.)