Search

Statistical learning theory and high dimensional statistics have had a tremendous impact on Machine Learning theory and have impacted a variety of domains including systems and control theory. Over the past few years we have witnessed a variety of applications of such theoretical tools to help answer questions such as: how many state-action pairs are needed to learn a static control policy to a given accuracy? Recent results have shown that continuously differentiable and stabilizing control policies can be well-approximated using neural networks with hard guarantees on performance, yet often even the simplest constrained control problems are not smooth. To address this void, in this paper we study smooth approximations of linear Model Predictive Control (MPC) policies, in which hard constraints are replaced by barrier functions, a.k.a. barrier MPC. In particular, we show that barrier MPC inherits the exponential stability properties of the original non-smooth MPC policy. Using a careful analysis of the proposed barrier MPC, we show that its smoothness constant can be carefully controlled, thereby paving the way for new sample complexity results for approximating MPC policies from sampled state-action pairs., Comment: 15 pages, 1 figure

We shed new light on the smoothness of optimization problems arising in prediction error parameter estimation of linear and nonlinear systems. We show that for regions of the parameter space where the model is not contractive, the Lipschitz constant and β-smoothness of the objective function might blow up exponentially with the simulation length, making it hard to numerically find minima within those regions or, even, to escape from them. In addition to providing theoretical understanding of this problem, this paper also proposes the use of multiple shooting as a viable solution. The proposed method minimizes the error between a prediction model and the observed values. Rather than running the prediction model over the entire dataset, multiple shooting splits the data into smaller subsets and runs the prediction model over each subset, making the simulation length a design parameter and making it possible to solve problems that would be infeasible using a standard approach. The equivalence to the original problem is obtained by including constraints in the optimization. The new method is illustrated by estimating the parameters of nonlinear systems with chaotic or unstable behavior, as well as neural networks. We also present a comparative analysis of the proposed method with multi-step-ahead prediction error minimization.

This letter concerns the problem of learning robust LQ-controllers, when the dynamics of the linear system are unknown. First, we propose a robust control synthesis method to minimize the worst-case LQ cost, with probability 1 - delta, given empirical observations of the system. Next, we propose an approximate dual controller that simultaneously regulates the system and reduces model uncertainty. The objective of the dual controller is to minimize the worst-case cost attained by a new robust controller, synthesized with the reduced model uncertainty. The dual controller is subject to an exploration budget in the sense that it has constraints on its worst-case cost with respect to the current model uncertainty. In our numerical experiments, we observe better performance of the proposed robust LQ regulator over the existing methods. Moreover, the dual control strategy gives promising results in comparison with the common greedy random exploration strategies.

We propose an input design method for a general class of parametric probabilistic models, including nonlinear dynamical systems with process noise. The goal of the procedure is to select inputs such that the parameter posterior distribution concentrates about the true value of the parameters; however, exact computation of the posterior is intractable. By representing (samples from) the posterior as trajectories from a certain Hamiltonian system, we transform the input design task into an optimal control problem. The method is illustrated via numerical examples, including magnetic resonance imaging pulse sequence design.

Bayesian modeling has been recognized as a powerful approach to system identification, not least due to its intrinsic uncertainty quantification. However, despite many recent developments, Bayesian identification of nonlinear state space models still poses major computational challenges. We propose a new method to tackle this problem. The technique is based on simulating a so-called thermostat, a stochastic differential equation constructed to have the posterior parameter distribution as its limiting distribution. Simulating the thermostat requires access to unbiased estimates of the gradient of the log-posterior. To handle this, we make use of a recent method for debiasing particle-filter-based smoothing estimates. Numerical results show a clear benefit of this approach compared to a direct application of (biased) particle-filter-based gradient estimates within the thermostat.

In this paper, we consider the inverse optimal control problem for discrete-time Linear Quadratic Regulators (LQR), over finite-time horizons. Given observations of the optimal trajectories, or optimal control inputs, to a linear time-invariant system, the goal is to infer the parameters that define the quadratic cost function. The well-posedness of the inverse optimal control problem is first justified. In the noiseless case, when these observations are exact, we analyze the identifiability of the problem and provide sufficient conditions for uniqueness of the solution. In the noisy case, when the observations are corrupted by additive zero-mean noise, we formulate the problem as an optimization problem and prove that the solution to this problem is statistically consistent. The performance of the proposed method is illustrated through numerical examples.

Estimation of nonlinear dynamic models from data poses many challenges, including model instability and nonconvexity of long-term simulation fidelity. Recently Lagrangian relaxation has been proposed as a method to approximate simulation fidelity and guarantee stability via semidefinite programming (SDP); however, the resulting SDPs have large dimension, limiting their utility in practical problems. In this paper, we develop a path-following interior-point algorithm that takes advantage of special structure in the problem and reduces computational complexity from cubic to linear growth with the length of the dataset. The new algorithm enables empirical comparisons to established methods including nonlinear autoregressive models with exogenous inputs, and we demonstrate superior generalization to new data. We also explore the "regularizing" effect of stability constraints as an alternative to regressor subset selection.

Equation error, also known as one-step-ahead prediction error, is a common quality-of-fit metric in dynamical system identification and learning. In this letter, we use Lagrangian relaxation to construct a convex upper bound on equation error that can be optimized over a convex set of nonlinear models that are guaranteed to be contracting, a strong form of nonlinear stability. We provide theoretical results on the tightness of the relaxation, and show that the method compares favorably to established methods on a variety of case studies.

This paper concerns maximum likelihood identification of linear time invariant state space models, subject to model stability constraints. We combine Expectation Maximization (EM) and Lagrangian relaxation to build tight bounds on the likelihood that can be optimized over a convex parametrization of all stable linear models using semidefinite programming. In particular, we propose two new algorithms: EM with latent States & Lagrangian relaxation (EMSL), and EM with latent Disturbances & Lagrangian relaxation (EMDL). We show that EMSL provides tighter bounds on the likelihood when the effect of disturbances is more significant than the effect of measurement noise, and EMDL provides tighter bounds when the situation is reversed. We also show that EMDL gives the most broadly applicable formulation of EM for identification of models with singular disturbance covariance. The two new algorithms are validated with extensive numerical simulations.

In this paper, we consider the inverse optimal control problem for discrete-time Linear Quadratic Regulators (LQRs), over finite-time horizons. Given observations of the optimal trajectories, or optimal control inputs, to a linear time-invariant system, the goal is to infer the parameters that define the quadratic cost function. The well-posedness of the inverse optimal control problem is first justied. In the noiseless case, when these observations are exact, we analyze the identiability of the problem and provide sufficient conditions for uniqueness of the solution. In the noisy case, when the observations are corrupted by additive zero-mean noise, we formulate the problem as an optimization problem and prove that the solution to this problem is statistically consistent. The performance of the proposed method is illustrated through numerical examples., QC 20190121

In this paper, we consider the inverse optimal control problem for discrete-time Linear Quadratic Regulators (LQRs), over finite-time horizons. Given observations of the optimal trajectories, or optimal control inputs, to a linear time-invariant system, the goal is to infer the parameters that define the quadratic cost function. The well-posedness of the inverse optimal control problem is first justied. In the noiseless case, when these observations are exact, we analyze the identiability of the problem and provide sufficient conditions for uniqueness of the solution. In the noisy case, when the observations are corrupted by additive zero-mean noise, we formulate the problem as an optimization problem and prove that the solution to this problem is statistically consistent. The performance of the proposed method is illustrated through numerical examples., QC 20190121

In this paper, we consider the inverse optimal control problem for discrete-time Linear Quadratic Regulators (LQRs), over finite-time horizons. Given observations of the optimal trajectories, or optimal control inputs, to a linear time-invariant system, the goal is to infer the parameters that define the quadratic cost function. The well-posedness of the inverse optimal control problem is first justied. In the noiseless case, when these observations are exact, we analyze the identiability of the problem and provide sufficient conditions for uniqueness of the solution. In the noisy case, when the observations are corrupted by additive zero-mean noise, we formulate the problem as an optimization problem and prove that the solution to this problem is statistically consistent. The performance of the proposed method is illustrated through numerical examples., QC 20190121

In this paper, we consider the inverse optimal control problem for discrete-time Linear Quadratic Regulators (LQRs), over finite-time horizons. Given observations of the optimal trajectories, or optimal control inputs, to a linear time-invariant system, the goal is to infer the parameters that define the quadratic cost function. The well-posedness of the inverse optimal control problem is first justied. In the noiseless case, when these observations are exact, we analyze the identiability of the problem and provide sufficient conditions for uniqueness of the solution. In the noisy case, when the observations are corrupted by additive zero-mean noise, we formulate the problem as an optimization problem and prove that the solution to this problem is statistically consistent. The performance of the proposed method is illustrated through numerical examples., QC 20190121

In this paper, we consider the inverse optimal control problem for discrete-time Linear Quadratic Regulators (LQRs), over finite-time horizons. Given observations of the optimal trajectories, or optimal control inputs, to a linear time-invariant system, the goal is to infer the parameters that define the quadratic cost function. The well-posedness of the inverse optimal control problem is first justied. In the noiseless case, when these observations are exact, we analyze the identiability of the problem and provide sufficient conditions for uniqueness of the solution. In the noisy case, when the observations are corrupted by additive zero-mean noise, we formulate the problem as an optimization problem and prove that the solution to this problem is statistically consistent. The performance of the proposed method is illustrated through numerical examples., QC 20190121

In this paper, we consider the inverse optimal control problem for discrete-time Linear Quadratic Regulators (LQRs), over finite-time horizons. Given observations of the optimal trajectories, or optimal control inputs, to a linear time-invariant system, the goal is to infer the parameters that define the quadratic cost function. The well-posedness of the inverse optimal control problem is first justied. In the noiseless case, when these observations are exact, we analyze the identiability of the problem and provide sufficient conditions for uniqueness of the solution. In the noisy case, when the observations are corrupted by additive zero-mean noise, we formulate the problem as an optimization problem and prove that the solution to this problem is statistically consistent. The performance of the proposed method is illustrated through numerical examples., QC 20190121

In this paper, we consider the inverse optimal control problem for discrete-time Linear Quadratic Regulators (LQRs), over finite-time horizons. Given observations of the optimal trajectories, or optimal control inputs, to a linear time-invariant system, the goal is to infer the parameters that define the quadratic cost function. The well-posedness of the inverse optimal control problem is first justied. In the noiseless case, when these observations are exact, we analyze the identiability of the problem and provide sufficient conditions for uniqueness of the solution. In the noisy case, when the observations are corrupted by additive zero-mean noise, we formulate the problem as an optimization problem and prove that the solution to this problem is statistically consistent. The performance of the proposed method is illustrated through numerical examples., QC 20190121

In this paper, we consider the inverse optimal control problem for discrete-time Linear Quadratic Regulators (LQRs), over finite-time horizons. Given observations of the optimal trajectories, or optimal control inputs, to a linear time-invariant system, the goal is to infer the parameters that define the quadratic cost function. The well-posedness of the inverse optimal control problem is first justied. In the noiseless case, when these observations are exact, we analyze the identiability of the problem and provide sufficient conditions for uniqueness of the solution. In the noisy case, when the observations are corrupted by additive zero-mean noise, we formulate the problem as an optimization problem and prove that the solution to this problem is statistically consistent. The performance of the proposed method is illustrated through numerical examples., QC 20190121

In this paper, we consider the inverse optimal control problem for discrete-time Linear Quadratic Regulators (LQRs), over finite-time horizons. Given observations of the optimal trajectories, or optimal control inputs, to a linear time-invariant system, the goal is to infer the parameters that define the quadratic cost function. The well-posedness of the inverse optimal control problem is first justied. In the noiseless case, when these observations are exact, we analyze the identiability of the problem and provide sufficient conditions for uniqueness of the solution. In the noisy case, when the observations are corrupted by additive zero-mean noise, we formulate the problem as an optimization problem and prove that the solution to this problem is statistically consistent. The performance of the proposed method is illustrated through numerical examples., QC 20190121

In this paper, we consider the inverse optimal control problem for discrete-time Linear Quadratic Regulators (LQRs), over finite-time horizons. Given observations of the optimal trajectories, or optimal control inputs, to a linear time-invariant system, the goal is to infer the parameters that define the quadratic cost function. The well-posedness of the inverse optimal control problem is first justied. In the noiseless case, when these observations are exact, we analyze the identiability of the problem and provide sufficient conditions for uniqueness of the solution. In the noisy case, when the observations are corrupted by additive zero-mean noise, we formulate the problem as an optimization problem and prove that the solution to this problem is statistically consistent. The performance of the proposed method is illustrated through numerical examples., QC 20190121