Search

We consider the problem of two-player zero-sum games. This problem is formulated as a min–max Markov game in this article. The solution of this game, which is the min–max payoff, starting from a given state is called the min–max value of the state. In this article, we compute the solution of the two-player zero-sum game, utilizing the technique of successive relaxation that has been successfully applied in this article to compute a faster value iteration algorithm in the context of Markov decision processes. We extend the concept of successive relaxation to the setting of two-player zero-sum games. We show that, under a special structure on the game, this technique facilitates faster computation of the min–max value of the states. We then derive a generalized minimax Q-learning algorithm, which computes the optimal policy when the model information is not known. Finally, we prove the convergence of the proposed generalized minimax Q-learning algorithm utilizing stochastic approximation techniques, under an assumption on the boundedness of iterates. Through experiments, we demonstrate the [ABSTRACT FROM AUTHOR]

Inspired by classical sensitivity results for nonlinear optimization, we derive and discuss new quantitative bounds to characterize the solution map and dual variables of a parametrized nonlinear program. In particular, we derive explicit expressions for the local and global Lipschitz constants of the solution map of nonconvex or convex optimization problems, respectively. Our results are geared towards the study of time-varying optimization problems, which are commonplace in various applications of online optimization, including power systems, robotics, signal processing, and more. In this context, our results can be used to bound the rate of change of the optimizer. To illustrate the use of our sensitivity bounds we generalize existing arguments to quantify the tracking performance of continuous-time, monotone running algorithms. Furthermore, we introduce a new continuous-time running algorithm for time-varying constrained optimization, which we model as a so-called perturbed sweeping process. For this discontinuous scheme we establish an explicit bound on the asymptotic solution tracking for a class of convex problems. [ABSTRACT FROM AUTHOR]

The problems discussed in this article are motivated by general ratio consensus algorithms, introduced by Kempe et al. in 2003 in a simple form as the push-sum algorithm, later extended by Bénézit et al. in 2010 under the name weighted gossip algorithm. We consider a communication protocol described by a strictly stationary, ergodic, sequentially primitive sequence of nonnegative matrices, applied iteratively to a pair of fixed initial vectors, the components of which are called values and weights defined at the nodes of a network. The subject of ratio consensus problems is to study the asymptotic properties of ratios of values and weights at each node, expecting convergence to the same limit for all nodes. The main results of this article provide upper bounds for the rate of the almost sure exponential convergence in terms of the spectral gap associated with the given sequence of random matrices. It will be shown that these upper bounds are sharp. Our results complement previous results of Picci and Taylor in 2013 and Iutzeler et al. in 2013. [ABSTRACT FROM AUTHOR]

An accelerated algorithm is proposed in this article for solving the linear quadratic optimal consensus problem of multiagent systems. To optimize the linear quadratic response and the final consensus state simultaneously, a nonseparable multiobjective optimization problem with coupled constraints on decision variables is formulated. The main difficulty in solving the optimization problem lies in the nonlinear coupling of objectives, which is overcome by separating the problem into two independent and solvable single-objective optimization subproblems using the alternating direction method of multipliers. The proximal gradient decent scheme is then introduced to approximate the precise optimal solutions of the subproblems so as to improve the computing efficiency. Convergence analysis is performed to estimate the convergence rate and derive the convergence condition, which is independent of any global information of the system and, therefore, is fully distributed. Furthermore, the solution of each subproblem is obtained in a distributed form, allowing the multiagent system to achieve optimal consensus. Numerical examples show the effectiveness of the accelerated algorithm. [ABSTRACT FROM AUTHOR]

We consider the problem of tracking the minimum of a time-varying convex optimization problem over a dynamic graph. Motivated by target tracking and parameter estimation problems in intermittently connected robotic and sensor networks, the goal is to design a distributed algorithm capable of handling nondifferentiable regularization penalties. The proposed proximal online gradient descent algorithm is built to run in a fully decentralized manner and utilizes consensus updates over possibly disconnected graphs. The performance of the proposed algorithm is analyzed by developing bounds on its dynamic regret in terms of the cumulative path length of the time-varying optimum. It is shown that as compared to the centralized case, the dynamic regret incurred by the proposed algorithm over $T$ time slots is worse by a factor of $\log (T)$ only, despite the disconnected and time-varying network topology. The empirical performance of the proposed algorithm is tested on the distributed dynamic sparse recovery problem, where it is shown to incur a dynamic regret that is close to that of the centralized algorithm. [ABSTRACT FROM AUTHOR]

This article considers online optimization with a finite prediction window of cost functions and additional switching costs on the decisions. We study the fundamental limits of dynamic regret of any online algorithm for both the with-prediction and the no-prediction cases. Besides, we propose two gradient-based online algorithms: receding horizon gradient descent (RHGD) and receding horizon accelerated gradient (RHAG); and provide their regret upper bounds. RHAG's regret upper bound is close to the lower bound, indicating the tightness of our lower bound and that our RHAG is near-optimal. Finally, we conduct numerical experiments to complement the theoretical results. [ABSTRACT FROM AUTHOR]

This article addresses reset moving horizon estimation for multiple output discrete-time systems with quantized measurements. A new state reset estimator is designed based on a one-dimensional noisy measurement to overcome underestimation or overestimation of the system state, and an iterative algorithm is proposed to deal with multiple output systems. It is shown that with the proposed reset algorithm, the state estimation error is improved in the presence of over or under estimation, and the boundedness of the estimation error is established. The proposed algorithm also achieves a better estimate than the existing one for systems with a scalar measurement in the static case. A simulation of a moving vehicle is provided to demonstrate the advantage of the developed approach. [ABSTRACT FROM AUTHOR]

This article presents a new fast and robust algorithm that provides fuel-optimal impulsive control input sequences that drive a linear time-variant system to a desired state at a specified time. This algorithm is applicable to a broad class of problems where the cost is expressed as a time-varying norm-like function of the control input, enabling inclusion of complex operational constraints in the control planning problem. First, it is shown that the reachable sets for this problem have identical properties to those in prior works using constant cost functions, enabling use of existing algorithms in conjunction with newly derived contact and support functions. By reformulating the optimal control problem as a semi-infinite convex program, it is also demonstrated that the semi-infinite component of the commonly studied primer vector is an outward normal vector to the reachable set at the target state. Using this formulation, a fast and robust algorithm that provides globally optimal impulsive control input sequences is proposed. The algorithm iteratively refines estimates of an outward normal vector to the reachable set at the target state and a minimal set of control input times until the optimality criteria are satisfied to within a user-specified tolerance. Next, optimal control inputs are computed by solving a quadratic program. The algorithm is validated through simulations of challenging example problems based on the recently proposed miniaturized distributed occulter/telescope small satellite mission, which demonstrate that the proposed algorithm converges several times faster than comparable algorithms in the literature. [ABSTRACT FROM AUTHOR]

This article addresses the problem of simultaneous distributed state estimation, and control of linear systems with linear state feedback, subjected to process, and measurement noise, under the constraints of quantized, and rate-limited network data transmission. In the set-up adopted, sensors and actuators communicate through a network with a strongly connected topology. Unlike the case of centralized linear systems, for which the separation principle holds, the above practical assumption prevents the separate design of observers, and controller because each of the nodes does not necessarily have access to the control inputs generated at all the other nodes. We derive a linear distributed Luenberger observer, and a set of sufficient conditions that guarantee ultimate boundedness of the estimation error, and system state vectors, with bounds that depend on the $\mathcal {L}_{\infty }$ norm of the noise signals, and the number of bits used in the transmissions. A numerical example illustrates the performance and effectiveness of the proposed algorithm in controlling a network of open-loop unstable systems. [ABSTRACT FROM AUTHOR]

This article studies a class of multirobot coordination problems where a team of robots aim to reach their goal regions with minimum time and avoid collisions with obstacles and other robots. A novel numerical algorithm is proposed to identify the Pareto optimal solutions where no robot can unilaterally reduce its traveling time without extending others’. The consistent approximation of the algorithm in the epigraphical profile sense is guaranteed using set-valued numerical analysis. Experiments on an indoor multirobot platform and computer simulations show the anytime property of the proposed algorithm, i.e., it is able to quickly return a feasible control policy that safely steers the robots to their goal regions and it keeps improving policy optimality if more time is given. [ABSTRACT FROM AUTHOR]

This article concerns the problem of adaptive output regulation for multivariable nonlinear systems in normal form. We present a regulator employing an adaptive internal model of the exogenous signals based on the theory of nonlinear Luenberger observers. Adaptation is performed by means of discrete-time system identification schemes, in which every algorithm fulfilling some optimality and stability conditions can be used. Practical and approximate regulation results are given relating the prediction capabilities of the identified model to the asymptotic bound on the regulated variables, which become asymptotic whenever a “right” internal model exists in the identifier's model set. The proposed approach, moreover, does not require “high-gain” stabilization actions. [ABSTRACT FROM AUTHOR]

Controlled interacting particle systems such as the ensemble Kalman filter (EnKF) and the feedback particle filter (FPF) are numerical algorithms to approximate the solution of the nonlinear filtering problem in continuous time. The distinguishing feature of these algorithms is that the Bayesian update step is implemented using a feedback control law. It has been noted in the literature that the control law is not unique. This is the main problem addressed in this article. To obtain a unique control law, the filtering problem is formulated here as an optimal transportation problem. An explicit formula for the (mean-field type) optimal control law is derived in the linear Gaussian setting. Comparisons are made with the control laws for different types of EnKF algorithms described in the literature. Via empirical approximation of the mean-field control law, a finite- $N$ controlled interacting particle algorithm is obtained. For this algorithm, the equations for empirical mean and covariance are derived and shown to be identical to the Kalman filter. This allows strong conclusions on convergence and error properties based on the classical filter stability theory for the Kalman filter. It is shown that, under certain technical conditions, the mean squared error converges to zero even with a finite number of particles. A detailed propagation of chaos analysis is carried out for the finite- $N$ algorithm. The analysis is used to prove weak convergence of the empirical distribution as $N\rightarrow \infty$. For a certain simplified filtering problem, analytical comparison of the mse with the importance sampling-based algorithms is described. The analysis helps explain the favorable scaling properties of the control-based algorithms reported in several numerical studies in recent literature. [ABSTRACT FROM AUTHOR]

This article addresses the problem of verifying discrete-time stochastic systems against omega-regular specifications using finite-state abstractions. Omega-regular properties allow specifying complex behavior and encompass, for example, linear temporal logic. We focus on a class of systems with mixed monotone dynamics. This class is shown to be amenable to efficient reachable set computation and models a wide range of physically relevant systems. In general, finite-state abstractions of continuous state stochastic systems give rise to augmented Markov chains wherein the probabilities of transition between states are restricted to an interval. We present a procedure to compute a finite-state interval-valued Markov chain (IMC) abstraction of discrete-time, mixed monotone stochastic systems subject to affine disturbances given a rectangular partition of the state space. Then, we suggest an algorithm for performing verification against omega-regular properties in IMCs. Specifically, we aim to compute bounds on the probability of satisfying a specification from any initial state in the IMC. This is achieved by solving a reachability problem on the sets of so-called winning and losing components in the Cartesian product between the IMC and a Rabin automaton representing the specification. Next, the verification of IMCs may yield a set of states whose acceptance status is undecided with respect to the specification, requiring a refinement of the abstraction. We describe a specification-guided approach that compares the best and worst case behaviors of accepting paths in the IMC and targets the appropriate states accordingly. Finally, we show a case study. [ABSTRACT FROM AUTHOR]