Search

Control barrier functions (CBFs) play a critical role in the design of safe optimization-based controllers for control-affine systems. Given a CBF associated with a desired ``safe'' set, the typical approach consists in embedding CBF-based constraints into the optimization problem defining the control law to enforce forward invariance of the safe set. While this approach effectively guarantees safety for a given CBF, the CBF-based control law can introduce undesirable equilibrium points (i.e., points that are not equilibria of the original system); open questions remain on how the choice of CBF influences the number and locations of undesirable equilibria and, in general, the dynamics of the closed-loop system. This paper investigates how the choice of CBF impacts the dynamics of the closed-loop system and shows that: (i) The CBF does not affect the number, location, and (local) stability properties of the equilibria in the interior of the safe set; (ii) undesirable equilibria only appear on the boundary of the safe set; and, (iii) the number and location of undesirable equilibria for the closed-loop system do not depend of the choice of the CBF. Additionally, for the well-established safety filters and controllers based on both CBF and control Lyapunov functions (CLFs), we show that the stability properties of equilibria of the closed-loop system are independent of the choice of the CBF and of the associated extended class-K function.

This paper studies the dynamical properties of closed-loop systems obtained from control barrier function-based safety filters. We provide a sufficient and necessary condition for the existence of undesirable equilibria and show that the Jacobian matrix of the closed-loop system evaluated at an undesirable equilibrium always has a nonpositive eigenvalue. In the special case of linear planar systems and ellipsoidal obstacles, we give a complete characterization of the dynamical properties of the corresponding closed-loop system. We show that for underactuated systems, the safety filter always introduces a single undesirable equilibrium, which is a saddle-point. We prove that all trajectories outside the global stable manifold of such equilibrium converge to the origin. In the fully actuated case, we discuss how the choice of nominal controller affects the stability properties of the closed-loop system. Various simulations illustrate our results.

This paper considers an incremental Volt/Var control scheme for distribution systems with high integration of inverter-interfaced distributed generation (such as photovoltaic systems). The incremental Volt/Var controller is implemented with the objective of minimizing reactive power usage while maintaining voltages within safe limits sufficiently often. To this end, the parameters of the incremental Volt/Var controller are obtained by solving a chance-constrained optimization problem, where constraints are designed to ensure that voltage violations do not occur more often than a pre-specified probability. This approach leads to cost savings in a controlled, predictable way, while still avoiding significant over- or under-voltage issues. The proposed chance-constrained problem is solved using a successive convex approximation method. Once the gains are broadcast to the inverters, no additional communication is required since the controller is implemented locally at the inverters. The proposed method is successfully tested on a low-voltage 42-nodes network.

This article investigates the problem of controlling linear time-invariant systems subject to time-varying and a priori unknown cost functions, state and input constraints, and exogenous disturbances. We combine the online convex optimization framework with tools from robust model predictive control to propose an algorithm that is able to guarantee robust constraint satisfaction. The performance of the closed loop emerging from application of our framework is studied in terms of its dynamic regret, which is proven to be bounded linearly by the variation of the cost functions and the magnitude of the disturbances. We corroborate our theoretical findings and illustrate implementational aspects of the proposed algorithm by a numerical case study on a tracking control problem of an autonomous vehicle., Comment: 16 pages

This paper tackles the problem of solving stochastic optimization problems with a decision-dependent distribution in the setting of stochastic strongly-monotone games and when the distributional dependence is unknown. A two-stage approach is proposed, which initially involves estimating the distributional dependence on decision variables, and subsequently optimizing over the estimated distributional map. The paper presents guarantees for the approximation of the cost of each agent. Furthermore, a stochastic gradient-based algorithm is developed and analyzed for finding the Nash equilibrium in a distributed fashion. Numerical simulations are provided for a novel electric vehicle charging market formulation using real-world data., Comment: Paper submitted to the IEEE Open Journal of Control Systems

This paper considers the problem of controlling inverter-interfaced distributed energy resources (DERs) in a distribution grid to solve an AC optimal power flow (OPF) problem in real time. The AC OPF includes voltage constraints, and seeks to minimize costs associated with the economic operation, power losses, or the power curtailment from renewables. We develop an online feedback optimization method to drive the DERs' power setpoints to solutions of an AC OPF problem based only on voltage measurements (and without requiring measurements of the power consumption of non-controllable assets). The proposed method - grounded on the theory of control barrier functions - is based on a continuous approximation of the projected gradient flow, appropriately modified to accommodate measurements from the power network. We provide results in terms of local exponential stability, and assess the robustness to errors in the measurements and in the system Jacobian matrix. We show that the proposed method ensures anytime satisfaction of the voltage constraints when no model and measurement errors are present; if these errors are present and are small, the voltage violation is practically negligible. We also discuss extensions of the framework to virtual power plant setups and to cases where constraints on power flows and currents must be enforced. Numerical experiments on a 93-bus distribution system and with realistic load and production profiles show a superior performance in terms of voltage regulation relative to existing methods.

This paper considers the problem of regulating a dynamical system to equilibria that are defined as solutions of an input- and state-constrained optimization problem. To solve this regulation task, we design a state feedback controller based on a continuous approximation of the projected gradient flow. We first show that the equilibria of the interconnection between the plant and the proposed controller correspond to critical points of the constrained optimization problem. We then derive sufficient conditions to ensure that, for the closed-loop system, isolated locally optimal solutions of the optimization problem are locally exponentially stable and show that input constraints are satisfied at all times by identifying an appropriate forward-invariant set., Comment: This paper has been accepted for publication on IEEE Control Systems Letters

In this paper, we provide a novel contraction-theoretic approach to analyze two-time scale systems, including those commonly encountered in Online Feedback Optimization (OFO). Our framework endows these systems with several robustness properties, enabling a more comprehensive characterization of their behaviors. The primary assumptions are the contractivity of the fast sub-system and the reduced model, along with an explicit upper bound on the time-scale parameter. For two-time scale systems subject to disturbances, we show that the distance between solutions of the nominal system and solutions of its reduced model is uniformly upper bounded by a function of contraction rates, Lipschitz constants, the time-scale parameter, and the variability of the disturbances over time. Applying these general results to the OFO context, we establish new individual tracking error bounds, showing that solutions converge to their time-varying optimizer, provided the plant and steady-state feedback controller exhibit contractivity and the controller gain is suitably bounded. Finally, we explore two special cases: for autonomous nonlinear systems, we derive sharper bounds than those in the general results, and for linear time-invariant systems, we present novel bounds based on induced matrix norms and induced matrix log norms., Comment: This paper has been submitted to SIAM Journal on Control and Optimization

Governments, regulatory bodies, and manufacturers are proposing plans to accelerate the adoption of electric vehicles (EVs), with the goal of reducing the impact of greenhouse gases and pollutants from internal combustion engines on human health and climate change. In this context, the paper considers a scenario where ride-sharing enterprises utilize a 100%-electrified fleet of vehicles, and seeks responses to the following key question: How can renewable-based EV charging be maximized without disrupting the quality of the ride-sharing services? We propose a new mechanism to promote EV charging during hours of high renewable generation, and we introduce the concept of charge request, which is issued by a power utility company. Our mechanism is inspired by a game-theoretic approach where the power utility company proposes incentives and the ride-sharing platform assigns vehicles to both ride and charge requests; the bargaining mechanism leads to prices and EV assignments that are aligned with the notion of Nash equilibria. Numerical results show that it is possible to shift the EV charging during periods of high renewable generation and adapt to intermittent generation while minimizing the impact on the quality of service. The paper also investigates how the users' willingness to ride-share affects the charging strategy and the quality of service., Comment: Article published in Scientific Reports

In this paper, we develop a unified framework for analyzing the tracking error and dynamic regret of inexact online optimization methods under a variety of settings. Specifically, we leverage the quadratic constraint approach from control theory to formulate sequential semidefinite programs (SDPs) whose feasible points naturally correspond to tracking error bounds of various inexact online optimization methods including the inexact online gradient descent (OGD) method, the online gradient descent-ascent method, the online stochastic gradient method, and the inexact proximal online gradient method. We provide exact analytical solutions for our proposed sequential SDPs, and obtain fine-grained tracking error bounds for the online algorithms studied in this paper. We also provide a simple routine to convert the obtained tracking error bounds into dynamic regret bounds. The main novelty of our analysis is that we derive exact analytical solutions for our proposed sequential SDPs under various inexact oracle assumptions in a unified manner.

This paper presents a system identification framework -- inspired by multi-task learning -- to estimate the dynamics of a given number of linear time-invariant (LTI) systems jointly by leveraging structural similarities across the systems. In particular, we consider LTI systems that model networked systems with similar connectivity, or LTI systems with small differences in their matrices. The system identification task involves the minimization of the least-squares (LS) fit for individual systems, augmented with a regularization function that enforces structural similarities. The proposed method is particularly suitable for cases when the recorded trajectories for one or more LTI systems are not sufficiently rich, leading to ill-conditioning of LS methods. We analyze the performance of the proposed method when the matrices of the LTI systems feature a common sparsity pattern (i.e., similar connectivity), and provide simulations based on real data for the estimation of the brain dynamics. We show that the proposed method requires a significantly smaller number of fMRI scans to achieve similar error levels of the LS., Comment: Paper to appear in the proceedings of 62nd IEEE Conference on Decision and Control (CDC 2023)

In this work, we consider a time-varying stochastic saddle point problem in which the objective is revealed sequentially, and the data distribution depends on the decision variables. Problems of this type express the distributional dependence via a distributional map, and are known to have two distinct types of solutions--saddle points and equilibrium points. We demonstrate that, under suitable conditions, online primal-dual type algorithms are capable of tracking equilibrium points. In contrast, since computing closed-form gradient of the objective requires knowledge of the distributional map, we offer an online stochastic primal-dual algorithm for tracking equilibrium trajectories. We provide bounds in expectation and in high probability, with the latter leveraging a sub-Weibull model for the gradient error. We illustrate our results on an electric vehicle charging problem where responsiveness to prices follows a location-scale family based distributional map., Comment: Submitted to the Learning for Dynamics and Control Conference (L4DC)

Motivated by perception-based control problems in autonomous systems, this paper addresses the problem of developing feedback controllers to regulate the inputs and the states of a dynamical system to optimal solutions of an optimization problem when one has no access to exact measurements of the system states. In particular, we consider the case where the states need to be estimated from high-dimensional sensory data received only at discrete time intervals. We develop a sampled-data feedback controller that is based on adaptations of a projected gradient descent method, and that includes neural networks as integral components to estimate the state of the system from perceptual information. We derive sufficient conditions to guarantee (local) input-to-state stability of the control loop. Moreover, we show that the interconnected system tracks the solution trajectory of the underlying optimization problem up to an error that depends on the approximation errors of the neural network and on the time-variability of the optimization problem; the latter originates from time-varying safety and performance objectives, input constraints, and unknown disturbances. As a representative application, we illustrate our results with numerical simulations for vision-based autonomous driving., Comment: This is an extended version of the paper accepted to IFAC World Congress 2023 for publication, containing proofs, and recently updated to address a typo in Assumption 3

Given a network of agents, we study the problem of designing a distributed algorithm that computes k independent weighted means of the network's initial conditions (namely, the agents agree on a k-dimensional space). Akin to average consensus, this problem finds applications in distributed computing and sensing, where agents seek to simultaneously evaluate k independent functions at a common point by running a single coordination algorithm. We show that linear algorithms can agree on quantities that are oblique projections of the vector of initial conditions, and we provide techniques to design protocols that are compatible with a pre-specified communication graph. More broadly, our results show that a single agreement algorithm can solve $k$ consensus problems simultaneously at a fraction of the complexity of classical approaches but, in general, it requires higher network connectivity.

This paper considers the problem of controlling a dynamical system when the state cannot be directly measured and the control performance metrics are unknown or partially known. In particular, we focus on the design of data-driven controllers to regulate a dynamical system to the solution of a constrained convex optimization problem where: i) the state must be estimated from nonlinear and possibly high-dimensional data; and, ii) the cost of the optimization problem -- which models control objectives associated with inputs and states of the system -- is not available and must be learned from data. We propose a data-driven feedback controller that is based on adaptations of a projected gradient-flow method; the controller includes neural networks as integral components for the estimation of the unknown functions. Leveraging stability theory for perturbed systems, we derive sufficient conditions to guarantee exponential input-to-state stability (ISS) of the control loop. In particular, we show that the interconnected system is ISS with respect to the approximation errors of the neural network and unknown disturbances affecting the system. The transient bounds combine the universal approximation property of deep neural networks with the ISS characterization. Illustrative numerical results are presented in the context of control of robotics and epidemics., Comment: This is an extended version of the paper published in the IEEE Open Journal of Control Systems - Special Section on Machine Learning with Control, containing proofs with a correction in Section II.A

This paper presents an online algorithm for identification of partial differential equations (PDEs) based on the weak-form sparse identification of nonlinear dynamics algorithm (WSINDy). The algorithm is online in a sense that if performs the identification task by processing solution snapshots that arrive sequentially. The core of the method combines a weak-form discretization of candidate PDEs with an online proximal gradient descent approach to the sparse regression problem. In particular, we do not regularize the $\ell_0$-pseudo-norm, instead finding that directly applying its proximal operator (which corresponds to a hard thresholding) leads to efficient online system identification from noisy data. We demonstrate the success of the method on the Kuramoto-Sivashinsky equation, the nonlinear wave equation with time-varying wavespeed, and the linear wave equation, in one, two, and three spatial dimensions, respectively. In particular, our examples show that the method is capable of identifying and tracking systems with coefficients that vary abruptly in time, and offers a streaming alternative to problems in higher dimensions., Comment: 21 pages, 4 figures

This paper focuses on stochastic saddle point problems with decision-dependent distributions. These are problems whose objective is the expected value of a stochastic payoff function and whose data distribution drifts in response to decision variables--a phenomenon represented by a distributional map. A common approach to accommodating distributional shift is to retrain optimal decisions once a new distribution is revealed, or repeated retraining. We introduce the notion of equilibrium points, which are the fixed points of this repeated retraining procedure, and provide sufficient conditions for their existence and uniqueness. To find equilibrium points, we develop deterministic and stochastic primal-dual algorithms and demonstrate their convergence with constant step-size in the former and polynomial decay step-size schedule in the latter. By modeling errors emerging from a stochastic gradient estimator as sub-Weibull random variables, we provide error bounds in expectation and in high probability that hold for each iteration. Without additional knowledge of the distributional map, computing saddle points is intractable. Thus we propose a condition on the distributional map--which we call opposing mixture dominance--that ensures that the objective is strongly-convex-strongly-concave. Finally, we demonstrate that derivative-free algorithms with a single function evaluation are capable of approximating saddle points, Comment: Completed first revision with SIOPT. A link to the codebase is provided in the document

This paper considers the problem of regulating a linear dynamical system to the solution of a convex optimization problem with an unknown or partially-known cost. We design a data-driven feedback controller - based on gradient flow dynamics - that (i) is augmented with learning methods to estimate the cost function based on infrequent (and possibly noisy) functional evaluations; and, concurrently, (ii) is designed to drive the inputs and outputs of the dynamical system to the optimizer of the problem. We derive sufficient conditions on the learning error and the controller gain to ensure that the error between the optimizer of the problem and the state of the closed-loop system is ultimately bounded; the error bound accounts for the functional estimation errors and the temporal variability of the unknown disturbance affecting the linear dynamical system. Our results directly lead to exponential input-to-state stability of the closed-loop system. The proposed method and the theoretical bounds are validated numerically., Comment: This is an extended version of the paper accepted to the 4th Annual Learning for Dynamics & Control Conference (L4DC 2022), containing proofs

This paper considers a feedback-based projected gradient method for optimizing systems modeled as algebraic maps. The focus is on a setup where the gradient is corrupted by random errors that follow a sub-Weibull distribution, and where the measurements of the output -- which replace the input-output map of the system in the algorithmic updates -- may not be available at each iteration. The sub-Weibull error model is particularly well-suited in frameworks where the cost of the problem is learned via Gaussian Process (GP) regression (from functional evaluations) concurrently with the execution of the algorithm; however, it also naturally models setups where nonparametric methods and neural networks are utilized to estimate the cost. Using the sub-Weibull model, and with Bernoulli random variables modeling missing measurements of the system output, we show that the online algorithm generates points that are within a bounded error from the optimal solutions. In particular, we provide error bounds in expectation and in high probability. Numerical results are presented in the context of a demand response problem in smart power grids., Comment: Version of the IEEE Control Systems Letters (L-CSS) paper with one typo corrected

Since the early months of 2020, non-pharmaceutical interventions (NPIs) -- implemented at varying levels of severity and based on widely-divergent perspectives of risk tolerance -- have been the primary means to control SARS-CoV-2 transmission. We seek to identify how risk tolerance and vaccination rates impact the rate at which a population can return to pre-pandemic contact behavior. To this end, we develop a novel feedback control method for data-driven decision-making to identify optimal levels of NPIs across geographical regions in order to guarantee that hospitalizations will not exceed a given risk tolerance. Results are shown for the state of Colorado, and they suggest that: coordination in decision-making across regions is essential to maintain the daily number of hospitalizations below the desired limits; increasing risk tolerance can decrease the number of days required to discontinue NPIs, at the cost of an increased number of deaths; and if vaccination uptake is less than 70\%, at most levels of risk tolerance, return to pre-pandemic contact behaviors before the early months of 2022 may newly jeopardize the healthcare system., Comment: A preliminary version of this work appeared on medrXiv on April 1, 2021. Available at: https://www.medrxiv.org/content/10.1101/2021.04.20.21255350v1

This paper proposes a data-driven control framework to regulate an unknown, stochastic linear dynamical system to the solution of a (stochastic) convex optimization problem. Despite the centrality of this problem, most of the available methods critically rely on a precise knowledge of the system dynamics (thus requiring off-line system identification and model refinement). To this aim, in this paper we first show that the steady-state transfer function of a linear system can be computed directly from control experiments, bypassing explicit model identification. Then, we leverage the estimated transfer function to design a controller -- which is inspired by stochastic gradient descent methods -- that regulates the system to the solution of the prescribed optimization problem. A distinguishing feature of our methods is that they do not require any knowledge of the system dynamics, disturbance terms, or their distributions. Our technical analysis combines concepts and tools from behavioral system theory, stochastic optimization with decision-dependent distributions, and stability analysis. We illustrate the applicability of the framework on a case study for mobility-on-demand ride service scheduling in Manhattan, NY.

This paper focuses on the online gradient and proximal-gradient methods with stochastic gradient errors. In particular, we examine the performance of the online gradient descent method when the cost satisfies the Polyak-\L ojasiewicz (PL) inequality. We provide bounds in expectation and in high probability (that hold iteration-wise), with the latter derived by leveraging a sub-Weibull model for the errors affecting the gradient. The convergence results show that the instantaneous regret converges linearly up to an error that depends on the variability of the problem and the statistics of the sub-Weibull gradient error. Similar convergence results are then provided for the online proximal-gradient method, under the assumption that the composite cost satisfies the proximal-PL condition. In the case of static costs, we provide new bounds for the regret incurred by these methods when the gradient errors are modeled as sub-Weibull random variables. Illustrative simulations are provided to corroborate the technical findings.

This paper investigates the problem of tracking solutions of stochastic optimization problems with time-varying costs that depend on random variables with decision-dependent distributions. In this context, we propose the use of an online stochastic gradient descent method to solve the optimization, and we provide explicit bounds in expectation and in high probability for the distance between the optimizers and the points generated by the algorithm. In particular, we show that when the gradient error due to sampling is modeled as a sub-Weibull random variable, then the tracking error is ultimately bounded in expectation and in high probability. The theoretical findings are validated via numerical simulations in the context of charging optimization of a fleet of electric vehicles., Comment: To appear in IEEE L-CSS

This paper presents a new regularization approach -- termed OpReg-Boost -- to boost the convergence and lessen the asymptotic error of online optimization and learning algorithms. In particular, the paper considers online algorithms for optimization problems with a time-varying (weakly) convex composite cost. For a given online algorithm, OpReg-Boost learns the closest algorithmic map that yields linear convergence; to this end, the learning procedure hinges on the concept of operator regression. We show how to formalize the operator regression problem and propose a computationally-efficient Peaceman-Rachford solver that exploits a closed-form solution of simple quadratically-constrained quadratic programs (QCQPs). Simulation results showcase the superior properties of OpReg-Boost w.r.t. the more classical forward-backward algorithm, FISTA, and Anderson acceleration., Comment: Code available here https://github.com/nicola-bastianello/reg4opt To be presented at the 4th Annual Learning for Dynamics & Control Conference (L4DC'22)

This paper proposes a framework to study the convergence of stochastic optimization and learning algorithms. The framework is modeled over the different challenges that these algorithms pose, such as (i) the presence of random additive errors (e.g. due to stochastic gradients), and (ii) random coordinate updates (e.g. due to asynchrony in distributed set-ups). The paper covers both convex and strongly convex problems, and it also analyzes online scenarios, involving changes in the data and costs. The paper relies on interpreting stochastic algorithms as the iterated application of stochastic operators, thus allowing us to use the powerful tools of operator theory. In particular, we consider operators characterized by additive errors with sub-Weibull distribution (which parameterize a broad class of errors by their tail probability), and random updates. In this framework we derive convergence results in mean and in high probability, by providing bounds to the distance of the current iteration from a solution of the optimization or learning problem. The contributions are discussed in light of federated learning applications., Comment: To appear in IEEE Transactions on Automatic Control

This paper proposes a data-driven framework to solve time-varying optimization problems associated with unknown linear dynamical systems. Making online control decisions to regulate a dynamical system to the solution of an optimization problem is a central goal in many modern engineering applications. Yet, the available methods critically rely on a precise knowledge of the system dynamics, thus mandating a preliminary system identification phase before a controller can be designed. In this work, we leverage results from behavioral theory to show that the steady-state transfer function of a linear system can be computed from data samples without any knowledge or estimation of the system model. We then use this data-driven representation to design a controller, inspired by a gradient-descent optimization method, that regulates the system to the solution of a convex optimization problem, without requiring any knowledge of the time-varying disturbances affecting the model equation. Results are tailored to cost functions satisfy the Polyak-\L ojasiewicz inequality.

This paper considers a time-varying optimization problem associated with a network of systems, with each of the systems shared by (and affecting) a number of individuals. The objective is to minimize cost functions associated with the individuals' preferences, which are unknown, subject to time-varying constraints that capture physical or operational limits of the network. To this end, the paper develops a distributed online optimization algorithm with concurrent learning of the cost functions. The cost functions are learned on-the-fly based on the users' feedback (provided at irregular intervals) by leveraging tools from shape-constrained Gaussian Process. The online algorithm is based on a primal-dual method, and acts effectively in a closed-loop fashion where: i) users' feedback is utilized to estimate the cost, and ii) measurements from the network are utilized in the algorithmic steps to bypass the need for sensing of (unknown) exogenous inputs of the network. The performance of the algorithm is analyzed in terms of dynamic network regret and constraint violation. Numerical examples are presented in the context of real-time optimization of distributed energy resources.

We study the stability properties of a control system composed of a dynamical plant and a feedback controller, the latter generating control signals that can be compromised by a malicious attacker. We consider two classes of feedback controllers: a static output-feedback controller, and a dynamical gradient-flow controller that seeks to steer the output of the plant towards the solution of a convex optimization problem. In both cases, we analyze the stability properties of the closed-loop system under a class of switching attacks that persistently modify the control inputs generated by the controllers. Our stability analysis leverages the framework of hybrid dynamical systems, Lyapunov-based arguments for switching systems with unstable modes, and singular perturbation theory. Our results reveal that, under a suitable time-scale separation between plant and controllers, the stability of the interconnected system can be preserved when the attack occurs with "sufficiently low frequency" in any bounded time interval. We present simulation results in a power-grid example that corroborate the technical findings., Comment: Appeared in Nonlinear Analysis: Hybrid Systems, pp. 101152, vol. 44, may. 2022

This paper investigates the problem of regulating in real time a linear dynamical system to the solution trajectory of a time-varying constrained convex optimization problem. The proposed feedback controller is based on an adaptation of the saddle-flow dynamics, modified to take into account projections on constraint sets and output-feedback from the plant. We derive sufficient conditions on the tunable parameters of the controller (inherently related to the time-scale separation between plant and controller dynamics) to guarantee exponential and input-to-state stability of the closed-loop system. The analysis is tailored to the case of time-varying strongly convex cost functions and polytopic output constraints. The theoretical results are further validated in a ramp metering control problem in a network of traffic highways.

This paper studies the design of feedback controllers to steer a switching linear time-invariant dynamical system towards the solution trajectory of a time-varying convex optimization problem. We propose two types of controllers: (i) a continuous controller inspired by the online gradient descent method, and (ii) a hybrid controller that can be interpreted as an online version of Nesterov's accelerated gradient method with restarts of the state variables. By design, the controllers continuously steer the system towards the time-varying optimizer without requiring knowledge of exogenous disturbances affecting the system. For cost functions that are smooth and satisfy the Polyak-\L ojasiewicz inequality, we demonstrate that the online gradient-flow controller ensures uniform global exponential stability when the time scales of the system and controller are sufficiently separated and the switching signal of the system varies slowly on average. For cost functions that are strongly convex, we show that the hybrid accelerated controller outperforms the continuous gradient descent method. When the cost function is not strongly convex, we show that the the hybrid accelerated method guarantees global practical asymptotic stability.

Estimation of sensitivity matrices in electrical transmission systems allows grid operators to evaluate in real-time how changes in power injections reflect into changes in power flows. In this paper, we propose a robust low-rank minimization approach to estimate sensitivity matrices based on measurements of power injections and power flows. An online proximal-gradient method is proposed to estimate sensitivities on-the-fly from real-time measurements. The proposed method obtains meaningful estimates with fewer measurements when the regression model is underdetermined, in contrast with existing methods based on least-squares approaches. In addition, our method can also identify faulty measurements and handle missing data. In this work, convergence results in terms of dynamic regret are presented. Numerical tests corroborate the effectiveness of the novel approach and the robustness of missing measurements and outliers., Comment: Paper to appear in the proceedings of 62nd IEEE Conference on Decision and Control (CDC 2023)

Optimization underpins many of the challenges that science and technology face on a daily basis. Recent years have witnessed a major shift from traditional optimization paradigms grounded on batch algorithms for medium-scale problems to challenging dynamic, time-varying, and even huge-size settings. This is driven by technological transformations that converted infrastructural and social platforms into complex and dynamic networked systems with even pervasive sensing and computing capabilities. The present paper reviews a broad class of state-of-the-art algorithms for time-varying optimization, with an eye to both algorithmic development and performance analysis. It offers a comprehensive overview of available tools and methods, and unveils open challenges in application domains of broad interest. The real-world examples presented include smart power systems, robotics, machine learning, and data analytics, highlighting domain-specific issues and solutions. The ultimate goal is to exempify wide engineering relevance of analytical tools and pertinent theoretical foundations., Comment: 14 pages, 6 figures; to appear in the Proceedings of the IEEE

This paper formalizes a demand response task as an optimization problem featuring a known time-varying engineering cost and an unknown (dis)comfort function. Based on this model, this paper develops a feedback-based projected gradient method to solve the demand response problem in an online fashion, where: i) feedback from the user is leveraged to learn the (dis)comfort function concurrently with the execution of the algorithm; and, ii) measurements of electrical quantities are used to estimate the gradient of the known engineering cost. To learn the unknown function, a shape-constrained Gaussian Process is leveraged; this approach allows one to obtain an estimated function that is strongly convex and smooth. The performance of the online algorithm is analyzed by using metrics such as the tracking error and the dynamic regret. A numerical example is illustrated to corroborate the technical findings., Comment: Submitted to IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids (SmartGridComm) 2020

Stochastic gradient descent is one of the most common iterative algorithms used in machine learning and its convergence analysis is a rich area of research. Understanding its convergence properties can help inform what modifications of it to use in different settings. However, most theoretical results either assume convexity or only provide convergence results in mean. This paper, on the other hand, proves convergence bounds in high probability without assuming convexity. Assuming strong smoothness, we prove high probability convergence bounds in two settings: (1) assuming the Polyak-{\L}ojasiewicz inequality and norm sub-Gaussian gradient noise and (2) assuming norm sub-Weibull gradient noise. In the second setting, as an intermediate step to proving convergence, we prove a sub-Weibull martingale difference sequence self-normalized concentration inequality of independent interest. It extends Freedman-type concentration beyond the sub-exponential threshold to heavier-tailed martingale difference sequences. We also provide a post-processing method that picks a single iterate with a provable convergence guarantee as opposed to the usual bound for the unknown best iterate. Our convergence result for sub-Weibull noise extends the regime where stochastic gradient descent has equal or better convergence guarantees than stochastic gradient descent with modifications such as clipping, momentum, and normalization., Comment: V5: reorganization. No new analysis, but the generalization analysis was removed, the post-processing algorithm was emphasized, and new numerical experiments were run

The paper investigates the problem of estimating the state of a time-varying system with a linear measurement model; in particular, the paper considers the case where the number of measurements available can be smaller than the number of states. In lieu of a batch linear least-squares (LS) approach -- well suited for static networks, where a sufficient number of measurements could be collected to obtain a full-rank design matrix -- the paper proposes an online algorithm to estimate the possibly time-varying state by processing measurements as and when available. The design of the algorithm hinges on a generalized LS cost augmented with a proximal-point-type regularization. With the solution of the regularized LS problem available in closed-form, the online algorithm is written as a linear dynamical system where the state is updated based on the previous estimate and based on the new available measurements. Conditions under which the algorithmic steps are in fact a contractive mapping are shown, and bounds on the estimation error are derived for different noise models. Numerical simulations are provided to corroborate the analytical findings.

This paper considers a convex optimization problem with cost and constraints that evolve over time. The function to be minimized is strongly convex and possibly non-differentiable, and variables are coupled through linear constraints. In this setting, the paper proposes an online algorithm based on the alternating direction method of multipliers (ADMM), to track the optimal solution trajectory of the time-varying problem; in particular, the proposed algorithm consists of a primal proximal gradient descent step and an appropriately perturbed dual ascent step. The paper derives tracking results, asymptotic bounds, and linear convergence results. The proposed algorithm is then specialized to a multi-area power grid optimization problem, and our numerical results verify the desired properties.

This paper investigates online algorithms for smooth time-varying optimization problems, focusing first on methods with constant step-size, momentum, and extrapolation-length. Assuming strong convexity, precise results for the tracking iterate error (the limit supremum of the norm of the difference between the optimal solution and the iterates) for online gradient descent are derived. The paper then considers a general first-order framework, where a universal lower bound on the tracking iterate error is established. Furthermore, a method using "long-steps" is proposed and shown to achieve the lower bound up to a fixed constant. This method is then compared with online gradient descent for specific examples. Finally, the paper analyzes the effect of regularization when the cost is not strongly convex. With regularization, it is possible to achieve a non-regret bound. The paper ends by testing the accelerated and regularized methods on synthetic time-varying least-squares and logistic regression problems, respectively., Comment: Main changes: added discussion on regret literature in section 2 and added discussion on lower bounds in section 4

This paper develops and analyzes an online distributed proximal-gradient method (DPGM) for time-varying composite convex optimization problems. Each node of the network features a local cost that includes a smooth strongly convex function and a non-smooth convex function, both changing over time. By coordinating through a connected communication network, the nodes collaboratively track the trajectory of the minimizers without exchanging their local cost functions. The DPGM is implemented in an online fashion, that is, in a setting where only a limited number of steps are implemented before the function changes. Moreover, the algorithm is analyzed in an inexact scenario, that is, with a source of additive noise, that can represent e.g. communication noise or quantization. It is shown that the tracking error of the online inexact DPGM is upper-bounded by a convergent linear system, guaranteeing convergence within a neighborhood of the optimal solution., Comment: To be presented at the European Control Conference 2021 (ECC'21)

There is a growing cross-disciplinary effort in the broad domain of optimization and learning with streams of data, applied to settings where traditional batch optimization techniques cannot produce solutions at time scales that match the inter-arrival times of the data points due to computational and/or communication bottlenecks. Special types of online algorithms can handle this situation, and this article focuses on such time-varying optimization algorithms, with emphasis on Machine Leaning and Signal Processing, as well as data-driven Control. Approaches for the design of time-varying or online first-order optimization methods are discussed, with emphasis on algorithms that can handle errors in the gradient, as may arise when the gradient is estimated. Insights on performance metrics and accompanying claims are provided, along with evidence of cases where algorithms that are provably convergent in batch optimization may perform poorly in an online regime. The role of distributed computation is discussed. Illustrative numerical examples for a number of applications of broad interest are provided to convey key ideas., Comment: Accepted for publication in IEEE Signal Processing Magazine. Limit of 40 references

This paper considers an online proximal-gradient method to track the minimizers of a composite convex function that may continuously evolve over time. The online proximal-gradient method is inexact, in the sense that: (i) it relies on an approximate first-order information of the smooth component of the cost; and, (ii) the proximal operator (with respect to the non-smooth term) may be computed only up to a certain precision. Under suitable assumptions, convergence of the error iterates is established for strongly convex cost functions. On the other hand, the dynamic regret is investigated when the cost is not strongly convex, under the additional assumption that the problem includes feasibility sets that are compact. Bounds are expressed in terms of the cumulative error and the path length of the optimal solutions. This suggests how to allocate resources to strike a balance between performance and precision in the gradient computation and in the proximal operator., Comment: In Proceedings of ACC-2020

This paper presents a convex reformulation of a nonlinear constrained optimization problem for Markov decision processes, and applies the technical findings to optimal control problems for an ensemble of thermostatically controlled loads (TCLs). The paper further explores the formulation and solution of a (linearized) AC optimal power flow problem when one or more ensembles of TCLs are connected to a power network. In particular, a receding horizon controller is proposed, to simultaneously compute the optimal set-points of distributed energy resources (DERs) in the grid and the optimal switching signal for the TCLs. This formulation takes into account hardware constraints of the DERs, operational constraints of the grid (e.g., voltage limits), comfort of the TCL users, and ancillary services provision at the substation. Numerical results are provided to verify the effectiveness of the proposed methodology., Comment: 8 Pages

This paper develops an online algorithm to solve a time-varying optimization problem with an objective that comprises a known time-varying cost and an unknown function. This problem structure arises in a number of engineering systems and cyber-physical systems where the known function captures time-varying engineering costs, and the unknown function models user's satisfaction; in this context, the objective is to strike a balance between given performance metrics and user's satisfaction. Key challenges related to the problem at hand are related to (1) the time variability of the problem, and (2) the fact that learning of the user's utility function is performed concurrently with the execution of the online algorithm. This paper leverages Gaussian processes (GP) to learn the unknown cost function from noisy functional evaluation and build pertinent upper confidence bounds. Using the GP formalism, the paper then advocates time-varying optimization tools to design an online algorithm that exhibits tracking of the oracle-based optimal trajectory within an error ball, while learning the user's satisfaction function with no-regret. The algorithmic steps are inexact, to account for possible limited computational budgets or real-time implementation considerations. Numerical examples are illustrated based on a problem related to vehicle platooning., Comment: 17 pages, 7 figures -- v3 with additional simulation results and comparisons

This paper leverages a framework based on averaged operators to tackle the problem of tracking fixed points associated with maps that evolve over time. In particular, the paper considers the Krasnosel'skii-Mann method in a settings where: (i) the underlying map may change at each step of the algorithm, thus leading to a "running" implementation of the Krasnosel'skii-Mann method; and, (ii) an imperfect information of the map may be available. An imperfect knowledge of the maps can capture cases where processors feature a finite precision or quantization errors, or the case where (part of) the map is obtained from measurements. The analytical results are applicable to inexact running algorithms for solving optimization problems, whenever the algorithmic steps can be written in the form of (a composition of) averaged operators; examples are provided for inexact running gradient methods and the forward-backward splitting method. Convergence of the average fixed-point residual is investigated for the non-expansive case; linear convergence to a unique fixed-point trajectory is showed in the case of inexact running algorithms emerging from contractive operators.

The Robbins-Monro stochastic approximation algorithm is a foundation of many algorithmic frameworks for reinforcement learning (RL), and often an efficient approach to solving (or approximating the solution to) complex optimal control problems. However, in many cases practitioners are unable to apply these techniques because of an inherent high variance. This paper aims to provide a general foundation for "quasi-stochastic approximation," in which all of the processes under consideration are deterministic, much like quasi-Monte-Carlo for variance reduction in simulation. The variance reduction can be substantial, subject to tuning of pertinent parameters in the algorithm. This paper introduces a new coupling argument to establish optimal rate of convergence provided the gain is sufficiently large. These results are established for linear models, and tested also in non-ideal settings. A major application of these general results is a new class of RL algorithms for deterministic state space models. In this setting, the main contribution is a class of algorithms for approximating the value function for a given policy, using a different policy designed to introduce exploration.

This paper focuses on the sparse subspace clustering problem, and develops an online algorithmic solution to cluster data points on-the-fly, without revisiting the whole dataset. The strategy involves an online solution of a sparse representation (SR) problem to build a (sparse) dictionary of similarities where points in the same subspace are considered "similar," followed by a spectral clustering based on the obtained similarity matrix. When the SR cost is strongly convex, the online solution converges to within a neighborhood of the optimal time-varying batch solution. A dynamic regret analysis is performed when the SR cost is not strongly convex., Comment: 4 pages, 4 figures. Copyright 2019 IEEE. Published in the 2019 IEEE Data Science Workshop (DSW 2019), scheduled for June 4-6, 2019 in Minneapolis, Minnesota

The increasing deployment of distribution-level phasor measurement units (PMUs) calls for dynamic distribution state estimation (DDSE) approaches that tap into high-rate measurements to maintain a comprehensive view of the distribution-system state in real time. Accordingly, this paper explores the development of a fast algorithmic framework by casting the DDSE task within the time-varying optimization realm. The time-varying formulation involves a time-varying robustified least-squares approach, and it naturally models optimal trajectories for the estimated states under streaming of measurements. The formulation is based on a linear surrogate of the AC power-flow equations, and it includes an element of robustness with respect to measurement outliers. The paper then leverages a first-order prediction-correction method to achieve simple online updates that can provably track the state variables from heterogeneous measurements. This online algorithm is computationally efficient as it relies on the Hessian of the cost function without computing matrix-inverse. Convergence and bounds on the estimation errors of proposed algorithm can be analytically established., Comment: 8 pages, 7 figures

With a large-scale integration of distributed energy resources (DERs), distribution systems are expected to be capable of providing capacity support for the transmission grid. To effectively harness the collective flexibility from massive DER devices, this paper studies distribution-level power aggregation strategies for transmission-distribution interaction. In particular, this paper proposes a method to model and quantify the aggregate power flexibility, i.e., the net power injection achievable at the substation, in unbalanced distribution systems over time. Incorporating the network constraints and multi-phase unbalanced modeling, the proposed method obtains an effective approximate feasible region of the net power injection. For any aggregate power trajectory within this region, it is proved that there exists a feasible disaggregation solution. In addition, a distributed model predictive control (MPC) framework is developed for the practical implementation of the transmission-distribution interaction. At last, we demonstrate the performances of the proposed method via numerical tests on a real-world distribution feeder with 126 multi-phase nodes., Comment: 12 pages

This paper considers a nonconvex optimization problem that evolves over time, and addresses the synthesis and analysis of regularized primal-dual gradient methods to track a Karush-Kuhn-Tucker (KKT) trajectory. The proposed regularized primal-dual gradient methods are implemented in a running fashion, in the sense that the underlying optimization problem changes during the iterations of the algorithms. For a problem with twice continuously differentiable cost and constraints, and under a generalization of the Mangasarian-Fromovitz constraint qualification, sufficient conditions are derived for the running algorithm to track a KKT trajectory. Further, asymptotic bounds for the tracking error (as a function of the time-variability of a KKT trajectory) are obtained. A continuous-time version of the algorithm, framed as a system of differential inclusions, is also considered and analytical convergence results are derived. For the continuous-time setting, a set of sufficient conditions for the KKT trajectories not to bifurcate or merge is proposed. Illustrative numerical results inspired by a real-world application are provided.

This paper develops and compares algorithms to compute inner approximations of the Minkowski sum of convex polytopes. As an application, the paper considers the computation of the feasibility set of aggregations of distributed energy resources (DERs), such as solar photovoltaic inverters, controllable loads, and storage devices. To fully account for the heterogeneity in the DERs while ensuring an acceptable approximation accuracy, the paper leverages a union-based computation and advocates homothet-based polytope decompositions. However, union-based approaches can in general lead to high-dimensionality concerns; to alleviate this issue, this paper shows how to define candidate sets to reduce the computational complexity. Accuracy and trade-offs are analyzed through numerical simulations for illustrative examples., Comment: Accepted for presentation at the 57th IEEE Conference on Decision and Control