Search

This article is concerned with the distributed Kalman filtering problem for discrete-time linear systems whose measurement information comes from a set of sensor nodes that can communicate with their direct neighbors. Two algorithms for distributed Kalman filtering are proposed, where the first algorithm is based on single-node measurement and the second algorithm is based on neighboring-node measurements. In order to improve the performance, a novel criterion is introduced to the algorithm design, where in the criterion, the proximity of matrix’s trace is used to determine the proximity of positive semidefinite matrix. We prove that all the proposed algorithms for distributed Kalman filtering are unbiased. Numerical examples are provided to demonstrate the correctness of the proposed theoretical method and the performance of the proposed new design technique. [ABSTRACT FROM AUTHOR]

This article is about the data-driven computation of optimal control for a class of control affine deterministic nonlinear systems. We assume that the control dynamical system model is not available, and the only information about the system dynamics is available in the form of time-series data. We provide a convex formulation for the optimal control problem (OCP) of the nonlinear system. The convex formulation relies on the duality result in the dynamical system’s stability theory involving density function and Perron–Frobenius operator. We formulate the OCP as an infinite-dimensional convex optimization program. The finite-dimensional approximation of the optimization problem relies on the recent advances made in the Koopman operator’s data-driven computation, which is dual to the Perron–Frobenius operator. Simulation results are presented to demonstrate the application of the developed framework. [ABSTRACT FROM AUTHOR]

This article addresses the problem of learning the optimal control policy for a nonlinear stochastic dynamical. This problem is subject to the “curse of dimensionality” associated with the dynamic programming method. This article proposes a novel decoupled data-based control (D2C) algorithm that addresses this problem using a decoupled, “open-loop–closed-loop,” approach. First, an open-loop deterministic trajectory optimization problem is solved using a black-box simulation model of the dynamical system. Then, closed-loop control is developed around this open-loop trajectory by linearization of the dynamics about this nominal trajectory. By virtue of linearization, a linear quadratic regulator based algorithm can be used for this closed-loop control. We show that the performance of D2C algorithm is approximately optimal. Moreover, simulation performance suggests a significant reduction in training time compared to other state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]

A fundamental component of the game-theoretic approach to distributed control is the design of local utility functions. Relative to resource allocation problems that are additive over the resources, Part I showed how to design local utilities so as to maximize the associated performance guarantees (Paccagnan, et al., 2020), which we measure by the price of anarchy. The purpose of this article is to specialize these results to the case of submodular, covering, and supermodular problems. In all these cases, we obtain tight expressions for the price of anarchy that often match or improve the guarantees associated with state-of-the-art approximation algorithms. Two applications and corresponding numerics are presented: the vehicle-target assignment problem and a coverage problem arising in wireless data caching. [ABSTRACT FROM AUTHOR]

This article studies the robustness of policy iteration in the context of continuous-time infinite-horizon linear quadratic regulator (LQR) problem. It is shown that Kleinman's policy iteration algorithm is small-disturbance input-to-state stable, a property that is stronger than Sontag's local input-to-state stability but weaker than global input-to-state stability. More precisely, whenever the error in each iteration is bounded and small, the solutions of the policy iteration algorithm are also bounded and enter a small neighborhood of the optimal solution of the LQR problem. Based on this result, an off-policy data-driven policy iteration algorithm for the LQR problem is shown to be robust when the system dynamics are subject to small additive unknown bounded disturbances. The theoretical results are validated by a numerical example. [ABSTRACT FROM AUTHOR]

This article deals with a distributed Mixed-Integer Linear Programming (MILP) setup arising in several control applications. Agents of a network aim to minimize the sum of local linear cost functions subject to both individual constraints and a linear coupling constraint involving all the decision variables. A key, challenging feature of the considered setup is that some components of the decision variables must assume integer values. The addressed MILPs are NP-hard, nonconvex, and large-scale. Moreover, several additional challenges arise in a distributed framework due to the coupling constraint, so that feasible solutions with guaranteed suboptimality bounds are of interest. We propose a fully distributed algorithm based on a primal decomposition approach and an appropriate tightening of the coupling constraint. The algorithm is guaranteed to provide feasible solutions in finite time. Moreover, asymptotic and finite-time suboptimality bounds are established for the computed solution. Monte Carlo simulations highlight the extremely low suboptimality bounds achieved by the algorithm. [ABSTRACT FROM AUTHOR]

In this article, we describe an approximate dynamic programming (ADP) approach to compute lower bounds on the optimal value function for a discrete time, continuous space, and infinite horizon setting. The approach iteratively constructs a family of lower bounding approximate value functions by using the so-called Bellman inequality. The novelty of our approach is that, at each iteration, we aim to compute an approximate value function that maximizes the point-wise maximum taken with the family of approximate value functions computed thus far. This leads to a nonconvex objective, and we propose a gradient ascent algorithm to find stationary points by solving a sequence of convex optimization problems. We provide convergence guarantees for our algorithm and an interpretation for how the gradient computation relates to the state-relevance weighting parameter appearing in related ADP approaches. We demonstrate through numerical examples that, when compared to the existing approaches, the algorithm we propose computes tighter suboptimality bounds with comparable computation time. [ABSTRACT FROM AUTHOR]

Despite the increasing interest in multiagent reinforcement learning (MARL) in multiple communities, understanding its theoretical foundation has long been recognized as a challenging problem. In this article, we address this problem by providing a finite-sample analysis for decentralized batch MARL. Specifically, we consider a type of mixed MARL setting with both cooperative and competitive agents, where two teams of agents compete in a zero-sum game setting, while the agents within each team collaborate by communicating over a time-varying network. This setting covers many conventional MARL settings in the literature. We then develop batch MARL algorithms that can be implemented in a decentralized fashion, and quantify the finite-sample errors of the estimated action-value functions. Our error analysis captures how the function class, the number of samples within each iteration, and the number of iterations determine the statistical accuracy of the proposed algorithms. Our results, compared to the finite-sample bounds for single-agent reinforcement learning, involve additional error terms caused by decentralized computation, which is inherent in our decentralized MARL setting. This article provides the first finite-sample analysis for batch MARL, a step toward rigorous theoretical understanding of general MARL algorithms in the finite-sample regime. [ABSTRACT FROM AUTHOR]

The objective of this article is to introduce a novel data-driven iterative linear quadratic (LQ) control method for solving a class of nonlinear optimal tracking problems. Specifically, an algorithm is proposed to approximate the Q-factors arising from LQ stochastic optimal tracking problems. This algorithm is then coupled with iterative LQ-methods for determining local solutions to nonlinear optimal tracking problems in a purely data-driven setting. Simulation results highlight the potential of this method for field applications. [ABSTRACT FROM AUTHOR]

Value functions derived from Markov decision processes arise as a central component of algorithms as well as performance metrics in many statistics and engineering applications of machine learning. Computation of the solution to the associated Bellman equations is challenging in most practical cases of interest. A popular class of approximation techniques, known as temporal difference (TD) learning algorithms, are an important subclass of general reinforcement learning methods. The algorithms introduced in this article are intended to resolve two well-known issues with TD-learning algorithms. Their slow convergence due to very high central limit theorem variance, and the fact that, for the problem of computing the relative value function, consistent algorithms exist only in special cases. First we show that the gradients of these value functions admit a representation that lends itself to algorithm design. Based on this result, a new class of differential TD-learning algorithms is introduced. For Markovian models on Euclidean space with smooth dynamics, the algorithms are shown to be consistent under general conditions. Numerical results show dramatic variance reduction in comparison to standard methods. [ABSTRACT FROM AUTHOR]

Distributed bandit online convex optimization with time-varying coupled inequality constraints is considered, motivated by a repeated game between a group of learners and an adversary. The learners attempt to minimize a sequence of global loss functions and at the same time satisfy a sequence of coupled constraint functions, where the constraints are coupled across the distributed learners at each round. The global loss and the coupled constraint functions are the sum of local convex loss and constraint functions, respectively, which are adaptively generated by the adversary. The local loss and constraint functions are revealed in a bandit manner, i.e., only the values of loss and constraint functions are revealed to the learners at the sampling instance, and the revealed function values are held privately by each learner. Both one- and two-point bandit feedback are studied with the two corresponding distributed bandit online algorithms used by the learners. We show that sublinear expected regret and constraint violation are achieved by these two algorithms, if the accumulated variation of the comparator sequence also grows sublinearly. In particular, we show that $\mathcal {O}(T^{\theta })$ expected static regret and $\mathcal {O}(T^{7/4-\theta })$ constraint violation are achieved in the one-point bandit feedback setting, and $\mathcal {O}(T^{\max \lbrace \kappa,1-\kappa \rbrace })$ expected static regret and $\mathcal {O}(T^{1-\kappa /2})$ constraint violation in the two-point bandit feedback setting, where $\theta \in (3/4,5/6]$ and $\kappa \in (0,1)$ are user-defined tradeoff parameters. Finally, the tightness of the theoretical results is illustrated by numerical simulations of a simple power grid example, which also compares the proposed algorithms to algorithms existing in the literature. [ABSTRACT FROM AUTHOR]

In this article, we introduce a class of novel distributed algorithms for solving stochastic big-data convex optimization problems over directed graphs. In the addressed set-up, the dimension of the decision variable can be extremely high and the objective function can be nonsmooth. The general algorithm consists of two main steps: a consensus step and an update on a single block of the optimization variable, which is then broadcast to neighbors. Three special instances of the proposed method, involving particular problem structures, are then presented. In the general case, the convergence of a dynamic consensus algorithm over random row stochastic matrices is shown. Then, the convergence of the proposed algorithm to the optimal cost is proven in expected value. Exact convergence is achieved when using diminishing (local) stepsizes, whereas approximate convergence is attained when constant stepsizes are employed. The convergence rate is shown to be sublinear and an explicit rate is provided in the case of constant stepsizes. Finally, the algorithm is tested on a distributed classification problem, first on synthetic data and, then, on a real, high-dimensional, text dataset. [ABSTRACT FROM AUTHOR]

This article studies a class of multirobot coordination problems where a team of robots aim to reach their goal regions with minimum time and avoid collisions with obstacles and other robots. A novel numerical algorithm is proposed to identify the Pareto optimal solutions where no robot can unilaterally reduce its traveling time without extending others’. The consistent approximation of the algorithm in the epigraphical profile sense is guaranteed using set-valued numerical analysis. Experiments on an indoor multirobot platform and computer simulations show the anytime property of the proposed algorithm, i.e., it is able to quickly return a feasible control policy that safely steers the robots to their goal regions and it keeps improving policy optimality if more time is given. [ABSTRACT FROM AUTHOR]

This article considers a distributed Nash equilibrium seeking problem, where the players only have partial access to other players’ actions, such as their neighbors’ actions. Thus, the players are supposed to communicate with each other to estimate other players’ actions. To solve the problem, a leader–following consensus gradient-free distributed Nash equilibrium seeking algorithm is proposed. This algorithm utilizes only the measurements of the player’ local cost function without the knowledge of its explicit expression or the requirement on its smoothness. Hence, the algorithm is gradient-free during the entire updating process. Moreover, the analysis on the convergence of the Nash equilibrium is studied for the algorithm with both diminishing and constant step-sizes, respectively. Specifically, in the case of diminishing step-size, it is shown that the players’ actions converge to the Nash equilibrium almost surely, while in the case of fixed step-size, the convergence to the neighborhood of the Nash equilibrium is achieved. The performance of the proposed algorithm is verified through numerical simulations. [ABSTRACT FROM AUTHOR]

We develop a stochastic approximation-type algorithm to solve finite state/action, infinite-horizon, risk-aware Markov decision processes. Our algorithm has two loops. The inner loop computes the risk by solving a stochastic saddle-point problem. The outer loop performs Q-learning to compute an optimal risk-aware policy. Several widely investigated risk measures (e.g., conditional value-at-risk, optimized certainty equivalent, and absolute semideviation) are covered by our algorithm. Almost sure convergence and the convergence rate of the algorithm are established. For an error tolerance ε > 0 for optimal Q-value estimation gap and learning rate k ∈ (1/2,1], the overall convergence rate of our algorithm is Ω((ln(1/δε)/ε2)1/k + (ln(1/ε))1/(1−k)) with probability at least 1−ε. [ABSTRACT FROM AUTHOR]

This article studies the infinite-horizon adaptive optimal control of continuous-time linear periodic (CTLP) systems. A novel value iteration (VI) based off-policy adaptive dynamic programming (ADP) algorithm is proposed for a general class of CTLP systems, so that approximate optimal solutions can be obtained directly from the collected data, without the exact knowledge of system dynamics. Under mild conditions, the proofs on uniform convergence of the proposed algorithm to the optimal solutions are given for both the model-based and model-free cases. The VI-based ADP algorithm is able to find suboptimal controllers without assuming the knowledge of an initial stabilizing controller. Application to the optimal control of a triple inverted pendulum subjected to a periodically varying load demonstrates the feasibility and effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

This article proposes a data-driven iterative feedback control method to efficiently solve finite time horizon, nonlinear, input constrained optimal control problems. The proposed method introduces a novel approach to combine an inexact system model with measured state information to reduce the cost and provide near-optimal control by approximately solving the optimal control problem along the trajectory of the real system, as opposed to solving it along the trajectory predicted by the inexact model. We present a new algorithm that implements the proposed method, establish the convergence and optimality properties of the proposed algorithm, and compare it to optimal feedback control and model predictive control that solve the same optimal control problem along the trajectory predicted by the inexact model. Finally, we illustrate the generality of the proposed algorithm by approximately solving a challenging optimal control problem with unknown and changing dynamics. [ABSTRACT FROM AUTHOR]

This article presents a multicommodity, discrete-time, distributed, and noncooperative routing algorithm, which is proved to converge to an equilibrium in the presence of heterogeneous, unknown, time-varying but bounded delays. Under mild assumptions on the latency functions, which describe the cost associated with the network paths, two algorithms are proposed: The former assumes that each commodity relies only on measurements of the latencies associated with its own paths; the latter assumes that each commodity has (at least indirectly) access to the measures of the latencies of all the network paths. Both algorithms are proven to drive the system state to an invariant set that approximates and contains the Wardrop equilibrium, defined as a network state in which no traffic flow over the network paths can improve its routing unilaterally, with the latter achieving a better reconstruction of the Wardrop equilibrium. Numerical simulations show the effectiveness of the proposed approach. [ABSTRACT FROM AUTHOR]

This article deals with output feedback selection in linear time-invariant structured systems. We assume that the inputs and the outputs are dedicated, i.e., each input directly actuates a single state and each output directly senses a single state. Given a structured system with dedicated inputs and outputs and a cost matrix that denotes the cost of each feedback connection, our aim is to select a minimum cost set of feedback connections such that the closed-loop system satisfies arbitrary pole-placement. This problem is referred to as the optimal feedback selection problem for dedicated i/o. The optimal feedback selection problem for dedicated i/o is NP-hard and inapproximable to a constant factor of log n, where n denotes the system dimension. To this end, we propose an algorithm to find an approximate solution to the problem. The proposed algorithm consists of a potential function incorporated with a greedy scheme and attains a solution with a guaranteed approximation ratio. We consider two special network topologies of practical importance, referred to as back-edge feedback structure and hierarchical networks. For the first case, which is NP-hard and inapproximable to a multiplicative factor of log n, we provide a log n-approximate solution. For hierarchical networks, we give a dynamic programming based algorithm to obtain an optimal solution in polynomial time. [ABSTRACT FROM AUTHOR]

This article focuses on developing a strategy for control of systems, whose dynamics are almost entirely unknown. This situation arises naturally in a scenario, where a system undergoes a critical failure. In that case, it is imperative to retain the ability to satisfy basic control objectives in order to avert an imminent catastrophe. To deal with limitations on the knowledge of system dynamics, we develop a theory of myopic control. At any given time, myopic control optimizes the current direction of the system trajectory, given solely the knowledge about system dynamics obtained from data until that time. We propose an algorithm that uses small perturbations in the control effort to learn system dynamics around the current system state, while ensuring that the system moves in a nearly optimal direction, and provide bounds for its suboptimality. [ABSTRACT FROM AUTHOR]

Directed acyclic graphs (DAGs) are emerging as an attractive alternative to traditional blockchain architectures for distributed ledger technology. In particular, DAG ledgers with stochastic attachment mechanisms potentially offer many advantages over blockchain, including scalability and faster transaction speeds. However, the random nature of the attachment mechanism coupled with the requirement of protection against double-spending transactions might result in an unstable system in which not all transactions get eventually validated. Such transactions are said to be orphaned, and will never be validated. Our principal contribution is to propose a simple modification to the attachment mechanism for the Tangle (the IOTA DAG architecture). This modification ensures that all transactions are validated in finite time, and preserves essential features of the popular Monte Carlo selection algorithm. In order to demonstrate these results, we derive a fluid approximation for the Tangle (in the limit of infinite arrival rate) and prove that this fluid model exhibits the desired behavior. We also present simulations that validate the results for finite arrival rates. [ABSTRACT FROM AUTHOR]

We propose universal randomized function approximation-based empirical value learning (EVL) algorithms for Markov decision processes. The “empirical” nature comes from each iteration being done empirically from samples available from simulations of the next state. This makes the Bellman operator a random operator. A parametric and a nonparametric method for function approximation using a parametric function space and a reproducing kernel Hilbert space respectively are then combined with EVL. Both function spaces have the universal function approximation property. Basis functions are picked randomly. Convergence analysis is performed using a random operator framework with techniques from the theory of stochastic dominance. Finite time sample complexity bounds are derived for both universal approximate dynamic programming algorithms. Numerical experiments support the versatility and computational tractability of this approach. [ABSTRACT FROM AUTHOR]

We consider the leader selection problem in a network with consensus dynamics where both leader and follower agents are subject to stochastic external disturbances. The performance of the system is quantified by the total steady-state variance of the node states, and the goal is to identify the set of leaders that minimizes this variance. We first show that this performance measure can be expressed as a submodular set function over the nodes in the network. We then use this result to analyze the performance of two greedy, polynomial-time algorithms for leader selection, showing that the leader sets produced by the greedy algorithms are within provable bounds of optimal. [ABSTRACT FROM AUTHOR]

A regularized optimization problem over a large unstructured graph is studied, where the regularization term is tied to the graph geometry. Typical regularization examples include the total variation and the Laplacian regularizations over the graph. When the graph is a simple path without loops, efficient off-the-shelf algorithms can be used. However, when the graph is large and unstructured, such algorithms cannot be used directly. In this paper, an algorithm, referred to as “Snake,” is proposed to solve such regularized problems over general graphs. The algorithm consists in properly selecting random simple paths in the graph and performing the proximal gradient algorithm over these simple paths. This algorithm is an instance of a new general stochastic proximal gradient algorithm, whose convergence is proven. Applications to trend filtering and graph inpainting are provided among others. Numerical experiments are conducted over large graphs. [ABSTRACT FROM AUTHOR]

This paper studies linear stochastic approximation (SA) algorithms and their application to multiagent systems in engineering and sociology. As main contribution, we provide necessary and sufficient conditions for convergence of linear SA algorithms to a deterministic or random final vector. We also characterize the system convergence rate, when the system is convergent. Moreover, differing from non-negative gain functions in traditional SA algorithms, this paper considers also the case when the gain functions are allowed to take arbitrary real numbers. Using our general treatment, we provide necessary and sufficient conditions to reach consensus and group consensus for first-order discrete-time multiagent system over random signed networks and with state-dependent noise. Finally, we extend our results to the setting of multidimensional linear SA algorithms and characterize the behavior of the multidimensional Friedkin–Johnsen model over random interaction networks. [ABSTRACT FROM AUTHOR]

This paper presents a discrete-time distributed and noncooperative routing algorithm, which is proved, via Lyapunov arguments, to asymptotically converge to a specific equilibrium condition among the traffic flows over the network paths, known as Wardrop equilibrium. This convergence result improves the discrete-time algorithms in the literature, which achieve approximate convergence to the Wardrop equilibrium. Numerical simulations show the effectiveness of the proposed approach. [ABSTRACT FROM AUTHOR]

This paper addresses optimal feedback selection for arbitrary pole placement of structured systems when each feedback edge is associated with a cost. Given a structured system and a feedback cost matrix, our aim is to find a feasible feedback matrix of minimum cost that guarantees arbitrary pole placement of the closed-loop structured system. The same problem but with uniform costs has been considered recently, but the claim and the NP-hardness proof there has subtle flaws: we mention this briefly in our paper and then prove the NP-hardness of the feedback selection problem using a reduction from the weighted set cover problem. We next prove the polynomial time constant factor inapproximability of the problem by showing that a constant factor approximation for the problem does not exist, unless the weighted set cover problem can also be approximated within a constant factor. We study a subclass of systems whose directed acyclic graph constructed using the strongly connected components of the state digraph is a line graph and the state bipartite graph has a perfect matching. We propose a polynomial time algorithm based on dynamic programming principles for optimal feedback selection on this class of systems. Further, over the same class of systems we relax the perfect matching assumption, and provide a polynomial time 2-optimal solution using a minimum cost perfect matching algorithm. [ABSTRACT FROM AUTHOR]

We consider large-scale Markov decision processes (MDPs) with a time-consistent risk measure of variability in cost under the risk-aware MDP paradigm. Previous studies showed that risk-aware MDPs, based on a minimax approach to handling risk, can be solved using dynamic programming for small- to medium-sized problems. However, due to the “curse of dimensionality,” MDPs that model real-life problems are typically prohibitively large for such approaches. In this technical note, we employ an approximate dynamic programming approach and develop a family of simulation-based algorithms to approximately solve large-scale risk-aware MDPs with time-consistent risk measures. In parallel, we develop a unified convergence analysis technique to derive sample complexity bounds for this new family of algorithms. [ABSTRACT FROM AUTHOR]

We study distributed non-convex optimization on a time-varying multi-agent network. Each node has access to its own smooth local cost function, and the collective goal is to minimize the sum of these functions. The perturbed push-sum algorithm was previously used for convex distributed optimization. We generalize the result obtained for the convex case to the case of non-convex functions. Under some additional technical assumptions on the gradients we prove the convergence of the distributed push-sum algorithm to some critical point of the objective function. By utilizing perturbations on the update process, we show the almost sure convergence of the perturbed dynamics to a local minimum of the global objective function, if the objective function has no saddle points. Our analysis shows that this perturbed procedure converges at a rate of $O(1/t)$ . [ABSTRACT FROM AUTHOR]

We consider controllable linear discrete-time systems with bounded perturbations and present two methods to compute robust controlled invariant sets. The first method tolerates an arbitrarily small constraint violation to compute an arbitrarily precise outer approximation of the maximal robust controlled invariant set, while the second method provides an inner approximation. The outer approximation scheme is $\delta$ -complete, given that the constraint sets are formulated as finite unions of polytopes. [ABSTRACT FROM AUTHOR]

We present an on-line sensor selection strategy (SSS) for the Sequential Probability Ratio Test (SPRT) with multiple sensors. Each sensor incurs an associated observation cost. We aim to design an SSS, in which the sensor selection may depend causally on the measurement values, that minimizes the expected total observation cost. In general, the optimal SSS can be obtained by solving a dynamic program; however, the problem is computationally quite demanding. We propose a computationally efficient algorithm in which we partition the state space into three regions and solve for the SSS in each region. The computational complexity of the proposed algorithm is linear in the number of sensors and numerical results show that it can well approximate the optimal SSS. [ABSTRACT FROM PUBLISHER]

Actor-critic algorithms solve dynamic decision making problems by optimizing a performance metric of interest over a user-specified parametric class of policies. They employ a combination of an actor, making policy improvement steps, and a critic, computing policy improvement directions. Many existing algorithms use a steepest ascent method to improve the policy, which is known to suffer from slow convergence for ill-conditioned problems. In this paper, we first develop an estimate of the (Hessian) matrix containing the second derivatives of the performance metric with respect to policy parameters. Using this estimate, we introduce a new second-order policy improvement method and couple it with a critic using a second-order learning method. We establish almost sure convergence of the new method to a neighborhood of a policy parameter stationary point. We compare the new algorithm with some existing algorithms in two applications and demonstrate that it leads to significantly faster convergence. [ABSTRACT FROM AUTHOR]

The problem of designing a switching and control policy for regulating the state of a switched linear system to zero while minimizing a quadratic cost appears in numerous applications. However, obtaining the optimal policy is in general computationally intractable. Here, we propose a class of suboptimal policies that exploit information, in terms of upper or lower bounds, on the optimal cost. We analyze the performance of these novel policies, obtaining new bounds on the optimal cost which are tighter than the initial ones. The usefulness of these policies and performance bounds is illustrated in the context of resource-aware control. [ABSTRACT FROM AUTHOR]

In this paper, a time-varying distributed convex optimization problem is studied for continuous-time multi-agent systems. The objective is to minimize the sum of local time-varying cost functions, each of which is known to only an individual agent, through local interaction. Here, the optimal point is time varying and creates an optimal trajectory. Control algorithms are designed for the cases of single-integrator and double-integrator dynamics. In both cases, a centralized approach is first introduced to solve the optimization problem. Then, this problem is solved in a distributed manner and a discontinuous algorithm based on the signum function is proposed in each case. In the case of single-integrator (respectively, double-integrator) dynamics, each agent relies only on its own position and the relative positions (respectively, positions and velocities) between itself and its neighbors. A gain adaption scheme is introduced in both algorithms to eliminate certain global information requirement. To relax the restricted assumption imposed on feasible cost functions, an estimator based algorithm using the signum function is proposed, where each agent uses dynamic average tracking as a tool to estimate the centralized control input. As a tradeoff, the estimator-based algorithm necessitates communication between neighbors. Then, in the case of double-integrator dynamics, the proposed algorithms are further extended. Two continuous algorithms based on, respectively, a time-varying and a fixed boundary layer are proposed as continuous approximations of the signum function. To account for interagent collision for physical agents, a distributed convex optimization problem with swarm tracking behavior is introduced for both single-integrator and double-integrator dynamics. It is shown that the center of the agents tracks the optimal trajectory, the connectivity of the agents is maintained, and interagent collision is avoided. [ABSTRACT FROM AUTHOR]

Motivated by the recent applications of game-theoretical learning to the design of distributed control systems, we study a class of control problems that can be formulated as potential games with continuous action sets. We propose an actor-critic reinforcement learning algorithm that adapts mixed strategies over continuous action spaces. To analyze the algorithm, we extend the theory of finite-dimensional two-timescale stochastic approximation to a Banach space setting, and prove that the continuous dynamics of the process converge to equilibrium in the case of potential games. These results combine to give a provably-convergent learning algorithm in which players do not need to keep track of the controls selected by other agents. [ABSTRACT FROM PUBLISHER]

We present and examine a novel method for obtaining solutions to specific discrete-time optimal control problems. Our approach is based on linear state dynamics and convexity assumptions commonly satisfied in practical applications. We show that the important class of optimal switching problems under partial observation is covered by our methodology, and we exploit specific model features to achieve simple algorithmic form of a numerical solution. [ABSTRACT FROM AUTHOR]

We consider an optimization problem for a dynamical system whose evolution depends on a collection of binary decision variables. We develop scalable approximation algorithms with provable suboptimality bounds to provide computationally tractable solution methods even when the dimension of the system and the number of the binary variables are large. The proposed method employs a linear approximation of the objective function such that the approximate problem is defined over the feasible space of the binary decision variables, which is a discrete set. To define such a linear approximation, we propose two different variation methods: one uses continuous relaxation of the discrete space and the other uses convex combinations of the vector field and running payoff. The approximate problem is a 0-1 linear program, which can be solved by existing polynomial-time exact or approximation algorithms, and does not require the solution of the dynamical system. Furthermore, we characterize a sufficient condition ensuring the approximate solution has a provable suboptimality bound. We show that this condition can be interpreted as the concavity of the objective function or that of a reformulated objective function. [ABSTRACT FROM PUBLISHER]

Reachability analysis of nonlinear uncertain hybrid systems, i.e., continuous-discrete dynamical systems whose continuous dynamics, guard sets and reset functions are defined by nonlinear functions, can be decomposed in three algorithmic steps: computing the reachable set when the system is in a given operation mode, computing the discrete transitions, i.e., detecting and localizing when (and where) the continuous flowpipe intersects the guard sets, and aggregating the multiple trajectories that result from an uncertain transition once the whole flow-pipe has transitioned so that the algorithm can resume. This paper proposes a comprehensive method that provides a nicely integrated solution to the hybrid reachability problem. At the core of the method is the concept of MSPB, i.e., geometrical object obtained as the Minkowski sum of a parallelotope and an axes aligned box. MSPB are a way to control the over-approximation of the Taylor's interval integration method. As they happen to be a specific type of zonotope, they articulate perfectly with the zonotope bounding method that we propose to enclose in an optimal way the set of flowpipe trajectories generated by the transition process. The method is evaluated both theoretically by analyzing its complexity and empirically by applying it to well-chosen hybrid nonlinear examples. [ABSTRACT FROM AUTHOR]

Multi-dimensional stochastic optimization plays an important role in analysis and control of many technical systems. To solve the challenging multidimensional problems of nonstationary optimization, it is suggested to use a stochastic approximation algorithm (like SPSA) with perturbed input and constant step-size which has simple form. We get a finite bound of residual between estimates and time-varying unknown parameters when observations are made under an unknown but bounded noise. Applications of the algorithm are considered for a random walk, an optimization of UAV's flight, and a load balancing problem. [ABSTRACT FROM AUTHOR]

The mixed-norm cost functions arise in many applied optimization problems. As an important example, we consider the state estimation problem for a linear dynamic system under a nonclassical assumption that some entries of state vector admit jumps in their trajectories. The estimation problem is solved by means of mixed l1/l2 -norm approximation. This approach combines the advantages of the well-known quadratic smoothing and the robustness of the least absolute deviations method. For the implementation of the mixed-norm approximation, a dynamic iterative estimation algorithm is proposed. This algorithm is based on weight and time recursions and demonstrates the high efficiency. It well identifies the rare jumps in the state vector and has some advantages over more customary methods in the typical case of a large amount of measurements. Nonoptimality levels for current iterations of the algorithm are constructed. Computation of these levels allows to check the accuracy of iterations. [ABSTRACT FROM AUTHOR]

The control vector parameterization (CVP) approach is popular to solve nonlinear dynamic optimization problems, but handling inequality path constraints within its framework is a challenge. This study deals with inequality path constraints by using the l1 penalty function and a novel smoothing technique. After clarifying some theories of the proposed approach, a concomitant algorithm is also put forward and verified. [ABSTRACT FROM PUBLISHER]

In a leader-follower multi-agent system (MAS), the leader agents act as control inputs and influence the states of the remaining follower agents. The rate at which the follower agents converge to their desired states, as well as the errors in the follower agent states prior to convergence, are determined by the choice of leader agents. In this paper, we study leader selection in order to minimize convergence errors experienced by the follower agents, which we define as a norm of the distance between the follower agents' intermediate states and the convex hull of the leader agent states. By introducing a novel connection to random walks on the network graph, we show that the convergence error has an inherent supermodular structure as a function of the leader set. Supermodularity enables development of efficient discrete optimization algorithms that directly approximate the optimal leader set, provide provable performance guarantees, and do not rely on continuous relaxations. We formulate two leader selection problems within the supermodular optimization framework, namely, the problem of selecting a fixed number of leader agents in order to minimize the convergence error, as well as the problem of selecting the minimum-size set of leader agents to achieve a given bound on the convergence error. We introduce algorithms for approximating the optimal solution to both problems in static networks, dynamic networks with known topology distributions, and dynamic networks with unknown and unpredictable topology distributions. Our approach is shown to provide significantly lower convergence errors than existing random and degree-based leader selection methods in a numerical study. [ABSTRACT FROM PUBLISHER]

We study the traveling salesman problem for a Dubins vehicle. We prove that this problem is NP-hard, and provide lower bounds on the approximation ratio achievable by some recently proposed heuristics. We also describe new algorithms for this problem based on heading discretization, and evaluate their performance numerically. [ABSTRACT FROM PUBLISHER]