Search

This paper derives a complete set of quadratic constraints (QCs) for the repeated ReLU. The complete set of QCs is described by a collection of $2^{n_v}$ matrix copositivity conditions where $n_v$ is the dimension of the repeated ReLU. We also show that only two functions satisfy all QCs in our complete set: the repeated ReLU and a repeated "flipped" ReLU. Thus our complete set of QCs bounds the repeated ReLU as tight as possible up to the sign invariance inherent in quadratic forms. We derive a similar complete set of incremental QCs for repeated ReLU, which can potentially lead to less conservative Lipschitz bounds for ReLU networks than the standard LipSDP approach. Finally, we illustrate the use of the complete set of QCs to assess stability and performance for recurrent neural networks with ReLU activation functions. The stability/performance condition combines Lyapunov/dissipativity theory with the QCs for repeated ReLU. A numerical implementation is given and demonstrated via a simple example.

Online convex optimization (OCO) is a powerful tool for learning sequential data, making it ideal for high precision control applications where the disturbances are arbitrary and unknown in advance. However, the ability of OCO-based controllers to accurately learn the disturbance while maintaining closed-loop stability relies on having an accurate model of the plant. This paper studies the performance of OCO-based controllers for linear time-invariant (LTI) systems subject to disturbance and model uncertainty. The model uncertainty can cause the closed-loop to become unstable. We provide a sufficient condition for robust stability based on the small gain theorem. This condition is easily incorporated as an on-line constraint in the OCO controller. Finally, we verify via numerical simulations that imposing the robust stability condition on the OCO controller ensures closed-loop stability.

This paper presents sufficient conditions for the stability and $\ell_2$-gain performance of recurrent neural networks (RNNs) with ReLU activation functions. These conditions are derived by combining Lyapunov/dissipativity theory with Quadratic Constraints (QCs) satisfied by repeated ReLUs. We write a general class of QCs for repeated RELUs using known properties for the scalar ReLU. Our stability and performance condition uses these QCs along with a "lifted" representation for the ReLU RNN. We show that the positive homogeneity property satisfied by a scalar ReLU does not expand the class of QCs for the repeated ReLU. We present examples to demonstrate the stability / performance condition and study the effect of the lifting horizon.

This letter considers assignment problems consisting of n pursuers attempting to intercept n targets. We consider stationary targets as well as targets maneuvering toward an asset. The assignment algorithm relies on an n x n cost matrix where entry (i, j) is the minimum time for pursuer i to intercept target j. Each entry of this matrix requires the solution of a nonlinear optimal control problem. This subproblem is computationally intensive and hence the computational cost of the assignment is dominated by the construction of the cost matrix. We propose to use neural networks for function approximation of the minimum time until intercept. The neural networks are trained offline, thus allowing for real-time online construction of cost matrices. Moreover, the function approximators have sufficient accuracy to obtain reasonable solutions to the assignment problem. In most cases, the approximators achieve assignments with optimal worst case intercept time. The proposed approach is demonstrated on several examples with increasing numbers of pursuers and targets., Comment: 6 pages, 6 tables, 1 figure. Presented at the 2022 American Control Conference

In this paper, a method is presented to synthesize neural network controllers such that the feedback system of plant and controller is dissipative, certifying performance requirements such as L2 gain bounds. The class of plants considered is that of linear time-invariant (LTI) systems interconnected with an uncertainty, including nonlinearities treated as an uncertainty for convenience of analysis. The uncertainty of the plant and the nonlinearities of the neural network are both described using integral quadratic constraints (IQCs). First, a dissipativity condition is derived for uncertain LTI systems. Second, this condition is used to construct a linear matrix inequality (LMI) which can be used to synthesize neural network controllers. Finally, this convex condition is used in a projection-based training method to synthesize neural network controllers with dissipativity guarantees. Numerical examples on an inverted pendulum and a flexible rod on a cart are provided to demonstrate the effectiveness of this approach., Comment: Submitted to the journal Automatica, 14 pages, 7 figures

In this paper, we explore the capabilities of state-of-the-art large language models (LLMs) such as GPT-4, Claude 3 Opus, and Gemini 1.0 Ultra in solving undergraduate-level control problems. Controls provides an interesting case study for LLM reasoning due to its combination of mathematical theory and engineering design. We introduce ControlBench, a benchmark dataset tailored to reflect the breadth, depth, and complexity of classical control design. We use this dataset to study and evaluate the problem-solving abilities of these LLMs in the context of control engineering. We present evaluations conducted by a panel of human experts, providing insights into the accuracy, reasoning, and explanatory prowess of LLMs in control engineering. Our analysis reveals the strengths and limitations of each LLM in the context of classical control, and our results imply that Claude 3 Opus has become the state-of-the-art LLM for solving undergraduate control problems. Our study serves as an initial step towards the broader goal of employing artificial general intelligence in control engineering.

In this paper, we revisit model-free policy search on an important robust control benchmark, namely $\mu$-synthesis. In the general output-feedback setting, there do not exist convex formulations for this problem, and hence global optimality guarantees are not expected. Apkarian (2011) presented a nonconvex nonsmooth policy optimization approach for this problem, and achieved state-of-the-art design results via using subgradient-based policy search algorithms which generate update directions in a model-based manner. Despite the lack of convexity and global optimality guarantees, these subgradient-based policy search methods have led to impressive numerical results in practice. Built upon such a policy optimization persepctive, our paper extends these subgradient-based search methods to a model-free setting. Specifically, we examine the effectiveness of two model-free policy optimization strategies: the model-free non-derivative sampling method and the zeroth-order policy search with uniform smoothing. We performed an extensive numerical study to demonstrate that both methods consistently replicate the design outcomes achieved by their model-based counterparts. Additionally, we provide some theoretical justifications showing that convergence guarantees to stationary points can be established for our model-free $\mu$-synthesis under some assumptions related to the coerciveness of the cost function. Overall, our results demonstrate that derivative-free policy optimization offers a competitive and viable approach for solving general output-feedback $\mu$-synthesis problems in the model-free setting., Comment: Submitted to L4DC 2024

Quadratic systems with lossless quadratic terms arise in many applications, including models of atmosphere and incompressible fluid flows. Such systems have a trapping region if all trajectories eventually converge to and stay within a bounded set. Conditions for the existence and characterization of trapping regions have been established in prior works for boundedness analysis. However, prior solutions have used non-convex optimization methods, resulting in conservative estimates. In this paper, we build on this prior work and provide a convex semidefinite programming condition for the existence of a trapping region. The condition allows precise verification or falsification of the existence of a trapping region. If a trapping region exists, then we provide a second semidefinite program to compute the least conservative trapping region in the form of a ball. Two low-dimensional systems are provided as examples to illustrate the results. A third high-dimensional example is also included to demonstrate that the computation required for the analysis can be scaled to systems of up to $\sim O(100)$ states. The proposed method provides a precise and computationally efficient numerical approach for computing trapping regions. We anticipate this work will benefit future studies on modeling and control of lossless quadratic dynamical systems.

Complex-valued Gaussian processes are commonly used in Bayesian frequency-domain system identification as prior models for regression. If each realization of such a process were an $H_\infty$ function with probability one, then the same model could be used for probabilistic robust control, allowing for robustly safe learning. We investigate sufficient conditions for a general complex-domain Gaussian process to have this property. For the special case of processes whose Hermitian covariance is stationary, we provide an explicit parameterization of the covariance structure in terms of a summable sequence of nonnegative numbers. We then establish how an $H_\infty$ Gaussian process can serve as a prior for Bayesian system identification and as a probabilistic uncertainty model for probabilistic robust control. In particular, we compute formulas for refining the uncertainty model by conditioning on frequency-domain data and for upper-bounding the probability that the realizations of the process satisfy a given integral quadratic constraint., Comment: 20 pages, 2 figures. Submission to SICON. arXiv admin note: substantial text overlap with arXiv:2211.15923

In this paper, we consider the learning of a Reduced-Order Linear Parameter-Varying Model (ROLPVM) of a nonlinear dynamical system based on data. This is achieved by a two-step procedure. In the first step, we learn a projection to a lower dimensional state-space. In step two, an LPV model is learned on the reduced-order state-space using a novel, efficient parameterization in terms of neural networks. The improved modeling accuracy of the method compared to an existing method is demonstrated by simulation examples., Comment: Accepted to the 20th IFAC Symposium on System Identification (SYSID 2024)

Modern control theory provides us with a spectrum of methods for studying the interconnection of dynamic systems using input-output properties of the interconnected subsystems. Perhaps the most advanced framework for such input-output analysis is the use of Integral Quadratic Constraints (IQCs), which considers the interconnection of a nominal linear system with an unmodelled nonlinear or uncertain subsystem with known input-output properties. Although these methods are widely used for Ordinary Differential Equations (ODEs), there have been fewer attempts to extend IQCs to infinite-dimensional systems. In this paper, we present an IQC-based framework for Partial Differential Equations (PDEs) and Delay Differential Equations (DDEs). First, we introduce infinite-dimensional signal spaces, operators, and feedback interconnections. Next, in the main result, we propose a formulation of hard IQC-based input-output stability conditions, allowing for infinite-dimensional multipliers. We then show how to test hard IQC conditions with infinite-dimensional multipliers on a nominal linear PDE or DDE system via the Partial Integral Equation (PIE) state-space representation using a sufficient version of the Kalman-Yakubovich-Popov lemma (KYP). The results are then illustrated using four example problems with uncertainty and nonlinearity.

This paper presents a synthesis method for robust, regret optimal control. The plant is modeled in discrete-time by an uncertain linear time-invariant (LTI) system. An optimal non-causal controller is constructed using the nominal plant model and given full knowledge of the disturbance. Robust regret is defined relative to the performance of this optimal non-causal control. It is shown that a controller achieves robust regret if and only if it satisfies a robust $H_\infty$ performance condition. DK-iteration can be used to synthesize a controller that satisfies this condition and hence achieve a given level of robust regret. The approach is demonstrated three examples: (i) a simple single-input, single-output classical design, (ii) a longitudinal control for a simplified model for a Boeing 747 model, and (iii) an active suspension for a quarter car model. All examples compare the robust regret optimal against regret optimal controllers designed without uncertainty.

Computing a stabilizing static output-feedback (SOF) controller is an NP-hard problem, in general. Yet, these controllers have amassed popularity in recent years because of their practical use in feedback control applications, such as fluid flow control and sensor/actuator selection. The inherent difficulty of synthesizing SOF controllers is rooted in solving a series of non-convex problems that make the solution computationally intractable. In this note, we show that SOF synthesis is a convex problem for the specific case of systems with a lossless (i.e., energy-conserving) nonlinearity. Our proposed method ensures asymptotic stability of an SOF controller by enforcing the lossless behavior of the nonlinearity using a quadratic constraint approach. In particular, we formulate a bilinear matrix inequality~(BMI) using the approach, then show that the resulting BMI can be recast as a linear matrix inequality (LMI). The resulting LMI is a convex problem whose feasible solution, if one exists, yields an asymptotically stabilizing SOF controller., Comment: Submitted to Automatica as a Technical Communique

In this note, we present an exact solution for the structured singular value (SSV) of rank-one complex matrices with repeated complex full-block uncertainty. A key step in the proof is the use of Von Neumman's trace inequality. Previous works provided exact solutions for rank-one SSV when the uncertainty contains repeated (real or complex) scalars and/or non-repeated complex full-block uncertainties. Our result with repeated complex full-blocks contains, as special cases, the previous results for repeated complex scalars and/or non-repeated complex full-block uncertainties. The repeated complex full-block uncertainty has recently gained attention in the context of incompressible fluid flows. Specifically, it has been used to analyze the effect of the convective nonlinearity in the incompressible Navier-Stokes equation (NSE). SSV analysis with repeated full-block uncertainty has led to an improved understanding of the underlying flow physics. We demonstrate our method on a turbulent channel flow model as an example.

It has recently been shown that the evolution of a linear Partial Differential Equation (PDE) can be more conveniently represented in terms of the evolution of a higher spatial derivative of the state. This higher spatial derivative (termed the `fundamental state') lies in $L_2$ - requiring no auxiliary boundary conditions or continuity constraints. Such a representation (termed a Partial Integral Equation or PIE) is then defined in terms of an algebra of bounded integral operators (termed Partial Integral (PI) operators) and is constructed by identifying a unitary map from the fundamental state to the state of the original PDE. Unfortunately, when the PDE is nonlinear, the dynamics of the associated fundamental state are no longer parameterized in terms of PI operators. However, in this paper, we show that such dynamics can be compactly represented using a new tensor algebra of partial integral operators acting on the tensor product of the fundamental state. We further show that this tensor product of the fundamental state forms a natural distributed equivalent of the monomial basis used in representation of polynomials on a finite-dimensional space. This new representation is then used to provide a simple SDP-based Lyapunov test of stability of quadratic PDEs. The test is applied to three illustrative examples of quadratic PDEs.

This paper considers the robustness of an uncertain nonlinear system along a finite-horizon trajectory. The uncertain system is modeled as a connection of a nonlinear system and a perturbation. The analysis relies on three ingredients. First, the nonlinear system is approximated by a linear time-varying (LTV) system via linearization along a trajectory. This linearization introduces an additional forcing input due to the nominal trajectory. Second, the input/output behavior of the perturbation is described by time-domain, integral quadratic constraints (IQCs). Third, a dissipation inequality is formulated to bound the worst-case deviation of an output signal due to the uncertainty. These steps yield a differential linear matrix inequality (DLMI) condition to bound the worst-case performance. The robustness condition is then converted to an equivalent condition in terms of a Riccati Differential Equation. This yields a computational method that avoids heuristics often used to solve DLMIs, e.g. time gridding. The approach is demonstrated by a two-link robotic arm example.

Complex-valued Gaussian processes are used in Bayesian frequency-domain system identification as prior models for regression. If each realization of such a process were an $H_\infty$ function with probability one, then the same model could be used for probabilistic robust control, allowing for robustly safe learning. We investigate sufficient conditions for a general complex-domain Gaussian process to have this property. For the special case of processes whose Hermitian covariance is stationary, we provide an explicit parameterization of the covariance structure in terms of a summable sequence of nonnegative numbers., Comment: Extended version of a submission to Learning for Dynamics and Control 2023. 18 pages, 2 figures

The structured singular value (SSV), or mu, is used to assess the robust stability and performance of an uncertain linear time-invariant system. Existing algorithms compute upper and lower bounds on the SSV for structured uncertainties that contain repeated (real or complex) scalars and/or non-repeated complex full blocks. This paper presents algorithms to compute bounds on the SSV for the case of repeated complex full blocks. This specific class of uncertainty is relevant for the input output analysis of many convective systems, such as fluid flows. Specifically, we present a power iteration to compute a lower bound on SSV for the case of repeated complex full blocks. This generalizes existing power iterations for repeated complex scalar and non-repeated complex full blocks. The upper bound can be formulated as a semi-definite program (SDP), which we solve using a standard interior-point method to compute optimal scaling matrices associated with the repeated full blocks. Our implementation of the method only requires gradient information, which improves the computational efficiency of the method. Finally, we test our proposed algorithms on an example model of incompressible fluid flow. The proposed methods provide less conservative bounds as compared to prior results, which ignore the repeated full block structure., Comment: Submitted to the International Journal of Robust and Nonlinear Control

The framework of Integral Quadratic Constraints (IQCs) is used to present a performance analysis for gradient descent with varying step sizes. Two performance metrics are considered: convergence rate and noise amplification. We assume that the step size is produced from a line search and varies in a known interval. We model the algorithm as a linear, parameter-varying (LPV) system and construct a parameterized linear matrix inequality (LMI) condition that certifies algorithm performance. This condition is a generalization of prior results for gradient descent with constant step sizes. The parameterized LMI condition is solved using a gridding approach on the step size interval. Our results provide convergence rate guarantees when the step size lies within a restricted interval. Moreover, we recover existing rate bounds when this interval reduces to a single point, i.e. a constant step size. Finally, we note that the convergence rate depends only on the condition number of the problem. In contrast, the noise amplification performance depends on the individual values of the convexity and smoothness parameters, and varies inversely with them for a fixed condition number., Comment: 14 pages, 4 figures

This paper proposes new quadratic constraints (QCs) to bound a quadratic polynomial. Such QCs can be used in dissipation ineqaulities to analyze the stability and performance of nonlinear systems with quadratic vector fields. The proposed QCs utilize the sign-indefiniteness of certain classes of quadratic polynomials. These new QCs provide a tight bound on the quadratic terms along specific directions. This reduces the conservatism of the QC bounds as compared to the QCs in previous work. Two numerical examples of local stability analysis are provided to demonstrate the effectiveness of the proposed QCs., Comment: 6 pages, 4 figures, to be published in IEEE Conference on Decision and Control 2022

We propose a parameterization of a nonlinear dynamic controller based on the recurrent equilibrium network, a generalization of the recurrent neural network. We derive constraints on the parameterization under which the controller guarantees exponential stability of a partially observed dynamical system with sector bounded nonlinearities. Finally, we present a method to synthesize this controller using projected policy gradient methods to maximize a reward function with arbitrary structure. The projection step involves the solution of convex optimization problems. We demonstrate the proposed method with simulated examples of controlling nonlinear plants, including plants modeled with neural networks., Comment: Submitted to IEEE CDC 2022. arXiv admin note: text overlap with arXiv:2109.03861

We present a new data structure for representation of polynomial variables in the parsing of sum-of-squares (SOS) programs. In SOS programs, the variables $s(x;Q)$ are polynomial in the independent variables $x$, but linear in the decision variables $Q$. Current SOS parsers, however, fail to exploit the semi-linear structure of the polynomial variables, treating the decision variables as independent variables in their representation. This results in unnecessary overhead in storage and manipulation of the polynomial variables, prohibiting the parser from addressing larger-scale optimization problems. To eliminate this computational overhead, we introduce a new representation of polynomial variables, the "dpvar" structure, that is affine in the decision variables. We show that the complexity of operations on variables in the dpvar representation scales favorably with the number of decision variables. We further show that the required memory for storing polynomial variables is relatively small using the dpvar structure, particularly when exploiting the MATLAB sparse storage structure. Finally, we incorporate the dpvar data structure into SOSTOOLS 4.00, and test the performance of the parser for several polynomial optimization problems.

Many existing region-of-attraction (ROA) analysis tools find difficulty in addressing feedback systems with large-scale neural network (NN) policies and/or high-dimensional sensing modalities such as cameras. In this paper, we tailor the projected gradient descent (PGD) attack method developed in the adversarial learning community as a general-purpose ROA analysis tool for large-scale nonlinear systems and end-to-end perception-based control. We show that the ROA analysis can be approximated as a constrained maximization problem whose goal is to find the worst-case initial condition which shifts the terminal state the most. Then we present two PGD-based iterative methods which can be used to solve the resultant constrained maximization problem. Our analysis is not based on Lyapunov theory, and hence requires minimum information of the problem structures. In the model-based setting, we show that the PGD updates can be efficiently performed using back-propagation. In the model-free setting (which is more relevant to ROA analysis of perception-based control), we propose a finite-difference PGD estimate which is general and only requires a black-box simulator for generating the trajectories of the closed-loop system given any initial state. We demonstrate the scalability and generality of our analysis tool on several numerical examples with large-scale NN policies and high-dimensional image observations. We believe that our proposed analysis serves as a meaningful initial step toward further understanding of closed-loop stability of large-scale nonlinear systems and perception-based control., Comment: Submitted to L4DC 2022

This paper considers the robust stability of a discrete-time Lurye system consisting of the feedback interconnection between a linear system and a bounded and monotone nonlinearity. It has been conjectured that the existence of a suitable linear time-invariant (LTI) O'Shea-Zames-Falb multiplier is not only sufficient but also necessary. Roughly speaking, a successful proof of the conjecture would require: (a) a conic parameterization of a set of multipliers that describes exactly the set of nonlinearities, (b) a lossless S-procedure to show that the non-existence of a multiplier implies that the Lurye system is not uniformly robustly stable over the set of nonlinearities, and (c) the existence of a multiplier in the set of multipliers used in (a) implies the existence of an LTI multiplier. We investigate these three steps, showing the current bottlenecks for proving this conjecture. In addition, we provide an extension of the class of multipliers which may be used to disprove the conjecture., Comment: 25 Pages

Motivated by the recent empirical success of policy-based reinforcement learning (RL), there has been a research trend studying the performance of policy-based RL methods on standard control benchmark problems. In this paper, we examine the effectiveness of policy-based RL methods on an important robust control problem, namely $\mu$ synthesis. We build a connection between robust adversarial RL and $\mu$ synthesis, and develop a model-free version of the well-known $DK$-iteration for solving state-feedback $\mu$ synthesis with static $D$-scaling. In the proposed algorithm, the $K$ step mimics the classical central path algorithm via incorporating a recently-developed double-loop adversarial RL method as a subroutine, and the $D$ step is based on model-free finite difference approximation. Extensive numerical study is also presented to demonstrate the utility of our proposed model-free algorithm. Our study sheds new light on the connections between adversarial RL and robust control., Comment: Accepted to ACC 2022

Adversarial examples can easily degrade the classification performance in neural networks. Empirical methods for promoting robustness to such examples have been proposed, but often lack both analytical insights and formal guarantees. Recently, some robustness certificates have appeared in the literature based on system theoretic notions. This work proposes an incremental dissipativity-based robustness certificate for neural networks in the form of a linear matrix inequality for each layer. We also propose an equivalent spectral norm bound for this certificate which is scalable to neural networks with multiple layers. We demonstrate the improved performance against adversarial attacks on a feed-forward neural network trained on MNIST and an Alexnet trained using CIFAR-10.

Neural network controllers have become popular in control tasks thanks to their flexibility and expressivity. Stability is a crucial property for safety-critical dynamical systems, while stabilization of partially observed systems, in many cases, requires controllers to retain and process long-term memories of the past. We consider the important class of recurrent neural networks (RNN) as dynamic controllers for nonlinear uncertain partially-observed systems, and derive convex stability conditions based on integral quadratic constraints, S-lemma and sequential convexification. To ensure stability during the learning and control process, we propose a projected policy gradient method that iteratively enforces the stability conditions in the reparametrized space taking advantage of mild additional information on system dynamics. Numerical experiments show that our method learns stabilizing controllers while using fewer samples and achieving higher final performance compared with policy gradient.

This paper focuses on safety critical control with sector-bounded uncertainties at the plant input. The uncertainties can represent nonlinear and/or time-varying components. We propose a new robust control barrier function (RCBF) approach to enforce safety requirements in the presence of these uncertainties. The primary objective is to minimally alter the given baseline control command to guarantee safety in the presence of modeled uncertainty. The resulting min-norm optimization problem can be recast as a Second-Order Cone Program (SOCP) to enable online implementation. Properties of this controller are studied and a numerical example is provided to illustrate the effectiveness of this approach., Comment: 7 Pages, 4 Figures, Submitted to LCSS/ACC 2022

This paper presents a control design method that achieves safety for systems with unmodeled dynamics at the plant input. The proposed method combines control barrier functions (CBFs) and integral quadratic constraints (IQCs). Simplified, low-order models are often used in the design of the controller. Parasitic, unmodeled dynamics (e.g. actuator dynamics, time delays, etc) can lead to safety violations. The proposed method bounds the input-output behavior of these unmodeled dynamics in the time-domain using an alpha-IQC. The alpha-IQC is then incorporated into the CBF constraint to ensure safety. The approach is demonstrated with a simple example.

This letter describes a method for estimating regions of attraction and bounds on permissible perturbation amplitudes in nonlinear fluids systems. The proposed approach exploits quadratic constraints between the inputs and outputs of the nonlinearity on elliptical sets. This approach reduces conservatism and improves estimates for regions of attraction and bounds on permissible perturbation amplitudes over related methods that employ quadratic constraints on spherical sets. We present and investigate two algorithms for performing the analysis: an iterative method that refines the analysis by solving a sequence of semi-definite programs, and another based on solving a generalized eigenvalue problem with lower computational complexity, but at the cost of some precision in the final solution. The proposed algorithms are demonstrated on low-order mechanistic models of transitional flows. We further compare accuracy and computational complexity with analysis based on sum-of-squares optimization and direct-adjoint looping methods.

A method is presented to learn neural network (NN) controllers with stability and safety guarantees through imitation learning (IL). Convex stability and safety conditions are derived for linear time-invariant plant dynamics with NN controllers by merging Lyapunov theory with local quadratic constraints to bound the nonlinear activation functions in the NN. These conditions are incorporated in the IL process, which minimizes the IL loss, and maximizes the volume of the region of attraction associated with the NN controller simultaneously. An alternating direction method of multipliers based algorithm is proposed to solve the IL problem. The method is illustrated on an inverted pendulum system, aircraft longitudinal dynamics, and vehicle lateral dynamics., Comment: 8 pages, 9 figures

We present a numerical approach to finding optimal trajectories for players in a multi-body, asset-guarding game with nonlinear dynamics and non-convex constraints. Using the Iterative Best Response (IBR) scheme, we solve for each player's optimal strategy assuming the other players' trajectories are known and fixed. Leveraging recent advances in Sequential Convex Programming (SCP), we use SCP as a subroutine within the IBR algorithm to efficiently solve an approximation of each player's constrained trajectory optimization problem. We apply the approach to an asset-guarding game example involving multiple pursuers and a single evader (i.e., n-versus-1 engagements). Resulting evader trajectories are tested in simulation to verify successful evasion against pursuers using conventional intercept guidance laws.

This paper considers the Lurye system of a discrete-time, linear time-invariant plant in negative feedback with a nonlinearity. Both monotone and slope-restricted nonlinearities are considered. The main result is a procedure to construct destabilizing nonlinearities for the Lurye system. If the plant satisfies a certain phase condition then a monotone nonlinearity can be constructed so that the Lurye system has a non-trivial periodic cycle. Several examples are provided to demonstrate the construction. This represents a contribution for absolute stability analysis since the constructed nonlinearity provides a less conservative upper bound than existing bounds in the literature., Comment: 6 pages, 8 figures, submitted to IEEE CSL

We present an efficient algorithm to compute the induced norms of finite-horizon Linear Time-Varying (LTV) systems. The formulation includes both induced $\mathcal{L}_2$ and terminal Euclidean norm penalties. Existing computational approaches include the power iteration and bisection of a Riccati Differential Equation (RDE). The power iteration has low computation time per iteration but overall convergence can be slow. In contrast, the RDE condition provides guaranteed bounds on the induced gain but single RDE integration can be slow. The complementary features of these two algorithms are combined to develop a new algorithm that is both fast and provides provable upper and lower bounds on the induced norm within the desired tolerance. The algorithm also provides a worst-case disturbance input that achieves the lower bound on the norm. We also present a new proof which shows that the power iteration for this problem converges monotonically. Finally, we show a controllability Gramian based simpler computational method for induced $\mathcal{L}_2$-to-Euclidean norm. This can be used to compute the reachable set at any time on the horizon. Numerical examples are provided to demonstrate the proposed algorithm., Comment: 17 pages, 8 figures, Submitted to LCSS/ACC 2021

A method is presented to analyze the stability of feedback systems with neural network controllers. Two stability theorems are given to prove asymptotic stability and to compute an ellipsoidal inner-approximation to the region of attraction (ROA). The first theorem addresses linear time-invariant systems, and merges Lyapunov theory with local (sector) quadratic constraints to bound the nonlinear activation functions in the neural network. The second theorem allows the system to include perturbations such as unmodeled dynamics, slope-restricted nonlinearities, and time delay, using integral quadratic constraint (IQCs) to capture their input/output behavior. This in turn allows for off-by-one IQCs to refine the description of activation functions by capturing their slope restrictions. Both results rely on semidefinite programming to approximate the ROA. The method is illustrated on systems with neural networks trained to stabilize a nonlinear inverted pendulum as well as vehicle lateral dynamics with actuator uncertainty., Comment: 8 pages, submitted to IEEE TAC

We present a robust synthesis algorithm for uncertain linear time-varying (LTV) systems on finite horizons. The uncertain system is described as an interconnection of a known LTV system and a perturbation. The input-output behavior of the perturbation is specified by time-domain Integral Quadratic Constraints (IQCs). The objective is to synthesize a controller to minimize the worst-case performance. This leads to a non-convex optimization. The proposed approach alternates between an LTV synthesis step and an IQC analysis step. Both induced $\mathcal{L}_2$ and terminal Euclidean norm penalties on output are considered for finite horizon performance. The proposed algorithm ensures that the robust performance is non-increasing at each iteration step. The effectiveness of this method is demonstrated using numerical examples., Comment: 26 pages, 16 figures, accepted to International Journal of Robust and Nonlinear Control

We study a multi-body asset-guarding game in missile defense where teams of interceptor missiles collaborate to defend a non-manuevering asset against a group of threat missiles. We approach the problem in two steps. We first formulate an assignment problem where we optimally assign subsets of collaborating interceptors to each threat so that all threats are intercepted as far away from the asset as possible. We assume that each interceptor is controlled by a collaborative guidance law derived from linear quadratic dynamic games. Our results include a 6-DOF simulation of a 5-interceptor versus 3-threat missile engagement where each agent is modeled as a missile airframe controlled by an autopilot. Despite the assumption of linear dynamics in our collaborative guidance law and the unmodeled dynamics in the simulation environment (e.g., varying density and gravity), we show that the simulated trajectories match well with those predicted by our approach. Furthermore, we show that a more agile threat, with greater speed and acceleration, can be intercepted by inferior interceptors when they collaborate. We believe the concepts introduced in this paper may be applied in asymmetric missile defense scenarios, including defense against advanced cruise missiles and hypersonic vehicles.

We present a procedure to numerically compute finite step worst case performance guarantees on a given algorithm for the unconstrained optimization of strongly convex functions with Lipschitz continuous gradients. The solution method provided serves as an alternative approach to that derived by Taylor, Hendrickx, and Glineur in [Math. Prog. 161 (1-2), 2017]. The difference lies in the fact that our solution uses conditions for the interpolation of a set of points and gradient evaluations by the gradient of a function in the class of interest, whereas their solution uses conditions for the interpolation of a set of points, gradient evaluations, and function evaluations by a function in the class of interest. The motivation for this alternative solution is that, in many cases, neither the algorithm nor the performance metric of interest rely upon function evaluations. The primary development is a procedure to avoid suffering from the factorial growth in the number of these conditions with the size of the set to be interpolated when solving for the worst case performance.

The dynamics of transitional flows are governed by an interplay between the non-normal linear dynamics and quadratic nonlinearity in the incompressible Navier-Stokes equations. In this work, we propose a framework for nonlinear stability analysis that exploits the fact that nonlinear flow interactions are constrained by the physics encoded in the nonlinearity. In particular, we show that nonlinear stability analysis problems can be posed as convex feasibility and optimization problems based on Lyapunov matrix inequalities, and a set of quadratic constraints that represent the nonlinear flow physics. The proposed framework can be used to conduct global stability, local stability, and transient energy growth analysis. The approach is demonstrated on the low-dimensional Waleffe-Kim-Hamilton model of transition and sustained turbulence. Our analysis correctly determines the critical Reynolds number for global instability. For local stability analysis, we show that the framework can estimate the size of the region of attraction as well as the amplitude of the largest permissible perturbation such that all trajectories converge back to the equilibrium point. Additionally, we show that the framework can predict bounds on the maximum transient energy growth. Finally, we show that careful analysis of the multipliers used to enforce the quadratic constraints can be used to extract dominant nonlinear flow interactions that drive the dynamics and associated instabilities.

A method is proposed to compute robust inner-approximations to the backward reachable set for uncertain nonlinear systems. It also produces a robust control law that drives trajectories starting in these sets to the target set. The method merges dissipation inequalities and integral quadratic constraints (IQCs) with both hard and soft IQC factorizations. Computational algorithms are presented using the generalized S-procedure and sum-of-squares techniques. The use of IQCs in backward reachability analysis allows for a variety of perturbations including parametric uncertainty, unmodeled dynamics, nonlinearities, and uncertain time delays. The method is demonstrated on two examples, including a 6-state quadrotor with actuator uncertainties., Comment: Submitted to L-CSS

This paper provides a tutorial introduction to disk margins. These are robust stability measures that account for simultaneous gain and phase perturbations in a feedback system. The paper first reviews the classical (gain-only and phase-only) margins and their limitations. This motivates the use of disk margins which are defined using a set of perturbations that have simultaneous gain and phase variations. A necessary and sufficient condition is provided to compute the disk margin for a single-input, single-output feedback system. Frequency-dependent disk margins can also be computed yielding additional insight. The paper concludes with a discussion of stability margins for multiple-input, multiple output (MIMO) feedback systems. A typical approach is to assess robust stability "loop-at-a-time" with a perturbation introduced into a single channel and all other channels held at their nominal values. MIMO disk margins provide a useful extension to consider simultaneous variations in multiple channels. This multiple-loop analysis can provide a more accurate robustness assessment as compared to the loop-at-a-time approach.

Signal temporal logic (STL) is an expressive language to specify time-bound real-world robotic tasks and safety specifications. Recently, there has been an interest in learning optimal policies to satisfy STL specifications via reinforcement learning (RL). Learning to satisfy STL specifications often needs a sufficient length of state history to compute reward and the next action. The need for history results in exponential state-space growth for the learning problem. Thus the learning problem becomes computationally intractable for most real-world applications. In this paper, we propose a compact means to capture state history in a new augmented state-space representation. An approximation to the objective (maximizing probability of satisfaction) is proposed and solved for in the new augmented state-space. We show the performance bound of the approximate solution and compare it with the solution of an existing technique via simulations., Comment: Github code repository: https://github.com/kumaa001/Tractable_RL_for_STL_Objectives. arXiv admin note: text overlap with arXiv:1609.07409

A method is presented to obtain an inner-approximation of the backward reachable set (BRS) of a given target tube, along with an admissible controller that maintains trajectories inside this tube. The proposed optimization algorithms are formulated as nonlinear optimization problems, which are decoupled into tractable subproblems and solved by an iterative algorithm using the polynomial S-procedure and sum-of-squares techniques. This framework is also extended to uncertain nonlinear systems with L_2 disturbances and L_{\infty} parametric uncertainties. The effectiveness of the method is demonstrated on several nonlinear robotics and aircraft systems with control saturation., Comment: Submitted to IEEE TAC

Iterative first-order methods such as gradient descent and its variants are widely used for solving optimization and machine learning problems. There has been recent interest in analytic or numerically efficient methods for computing worst-case performance bounds for such algorithms, for example over the class of strongly convex loss functions. A popular approach is to assume the algorithm has a fixed size (fixed dimension, or memory) and that its structure is parameterized by one or two hyperparameters, for example a learning rate and a momentum parameter. Then, a Lyapunov function is sought to certify robust stability and subsequent optimization can be performed to find optimal hyperparameter tunings. In the present work, we instead fix the constraints that characterize the loss function and apply techniques from robust control synthesis to directly search over algorithms. This approach yields stronger results than those previously available, since the bounds produced hold over algorithms with an arbitrary, but finite, amount of memory rather than just holding for algorithms with a prescribed structure., Comment: American Control Conference, 2020

In the existing literature, there are two approaches to estimate tighter bounds of the exponential convergence rate of stable Lure systems. On one hand, the classical integral quadratic constraint (IQC) framework can be applied under loop-transformation, so the stability of the new loop implies the convergence of the original loop. On the other hand, it is possible to modify the IQC framework, the so-called rho-IQC framework, in such a way that the convergence rate is directly obtained over the original loop. In this technical note, we extend the literature results from the search for a causal finite impulse response (FIR) Zames-Falb multiplier to the noncausal case. We show that the multipliers by the two approaches are equivalent by a change of variable. However, the factorisation of the Zames-Falb rho-IQC is restricted compared to the Zames-Falb IQC, so an unified factorisation is proposed. Finally, numerical examples illustrate that noncausal multipliers lead to less-conservative results.

This paper presents an aggregate reduced-order model for a wind farm composed of identical parallel-connected Type-3 wind turbines. The model for individual turbines includes mechanical dynamics (arising from the turbine and doubly fed induction generator) and electrical dynamics (arising from the rotor-side and grid-side converters and associated filters). The proposed aggregate wind-farm model is structure preserving, in the sense that the parameters of the model are derived by scaling corresponding ones from the individual turbines. The aggregate model hence maps to an equivalent--albeit fictitious--wind turbine that captures the dynamics corresponding to the entire wind farm. The reduced-order model has obvious computational advantages, but more importantly, the presented analysis rigorously formalizes parametric scalings for aggregate wind-turbine models that have been applied with limited justification in prior works. Exhaustive numerical simulations validate the accuracy and computational benefits of the proposed reduced-order model.

Reinforcement learning (RL) is used to directly design a control policy using data collected from the system. This paper considers the robustness of controllers trained via model-free RL. The discussion focuses on the standard model-based linear quadratic Gaussian (LQG) problem as a special instance of RL. A simple example, originally formulated for LQG problems, is used to demonstrate that RL with partial observations can lead to poor robustness margins. It is proposed to recover robustness by introducing random perturbations at the system input during the RL training. The perturbation magnitude can be used to trade off performance for robustness. Two simple examples are presented to demonstrate the proposed method for enhancing robustness during RL training., Comment: Github Code Repository: https://github.com/kumaa001/RLRobustness (Note : The files have been named to match with the section names and number. The comments in the code explains the procedure step by step. The data from the .mat file could be pulled into the work-space to avoid the need for complete code execution)