Search

In this article, we propose a symplectic algorithm for the stable manifolds of the Hamilton–Jacobi equations combined with an iterative procedure in [N. Sakamoto and A. J. van der Schaft, “Analytical approximation methods for the stabilizing solution of the Hamilton–Jacobi equation,” IEEE Trans. Autom. Control, 2008]. Our algorithm includes two key aspects. The first one is to prove a precise estimate for radius of convergence and the errors of local approximate stable manifolds. The second one is to extend the local approximate stable manifolds to larger ones by symplectic algorithms, which have better long-time behaviors than general-purpose schemes. Our approach avoids the case of divergence of the iterative sequence of approximate stable manifolds and reduces the computation cost. We illustrate the effectiveness of the algorithm by an optimal control problem with exponential nonlinearity. [ABSTRACT FROM AUTHOR]

The objective of this article is to introduce a novel data-driven iterative linear quadratic (LQ) control method for solving a class of nonlinear optimal tracking problems. Specifically, an algorithm is proposed to approximate the Q-factors arising from LQ stochastic optimal tracking problems. This algorithm is then coupled with iterative LQ-methods for determining local solutions to nonlinear optimal tracking problems in a purely data-driven setting. Simulation results highlight the potential of this method for field applications. [ABSTRACT FROM AUTHOR]

Economic model predictive control (EMPC) has recently become popular because of its ability to control constrained nonlinear systems while explicitly optimizing a prescribed performance criterion. Large performance gains have been reported for many applications and closed-loop stability has been recently investigated. However, computational performance still remains an open issue and only few contributions have proposed real-time algorithms tailored to EMPC. We perform a step towards computationally cheap algorithms for EMPC by proposing a new positive-definite Hessian approximation which does not hinder fast convergence and is suitable for being used within the real-time iteration (RTI) scheme. We provide two simulation examples to demonstrate the effectiveness of RTI-based EMPC relying on the proposed Hessian approximation. [ABSTRACT FROM AUTHOR]

In this article, we introduce a class of novel distributed algorithms for solving stochastic big-data convex optimization problems over directed graphs. In the addressed set-up, the dimension of the decision variable can be extremely high and the objective function can be nonsmooth. The general algorithm consists of two main steps: a consensus step and an update on a single block of the optimization variable, which is then broadcast to neighbors. Three special instances of the proposed method, involving particular problem structures, are then presented. In the general case, the convergence of a dynamic consensus algorithm over random row stochastic matrices is shown. Then, the convergence of the proposed algorithm to the optimal cost is proven in expected value. Exact convergence is achieved when using diminishing (local) stepsizes, whereas approximate convergence is attained when constant stepsizes are employed. The convergence rate is shown to be sublinear and an explicit rate is provided in the case of constant stepsizes. Finally, the algorithm is tested on a distributed classification problem, first on synthetic data and, then, on a real, high-dimensional, text dataset. [ABSTRACT FROM AUTHOR]

This article studies a class of multirobot coordination problems where a team of robots aim to reach their goal regions with minimum time and avoid collisions with obstacles and other robots. A novel numerical algorithm is proposed to identify the Pareto optimal solutions where no robot can unilaterally reduce its traveling time without extending others’. The consistent approximation of the algorithm in the epigraphical profile sense is guaranteed using set-valued numerical analysis. Experiments on an indoor multirobot platform and computer simulations show the anytime property of the proposed algorithm, i.e., it is able to quickly return a feasible control policy that safely steers the robots to their goal regions and it keeps improving policy optimality if more time is given. [ABSTRACT FROM AUTHOR]

Controlled interacting particle systems such as the ensemble Kalman filter (EnKF) and the feedback particle filter (FPF) are numerical algorithms to approximate the solution of the nonlinear filtering problem in continuous time. The distinguishing feature of these algorithms is that the Bayesian update step is implemented using a feedback control law. It has been noted in the literature that the control law is not unique. This is the main problem addressed in this article. To obtain a unique control law, the filtering problem is formulated here as an optimal transportation problem. An explicit formula for the (mean-field type) optimal control law is derived in the linear Gaussian setting. Comparisons are made with the control laws for different types of EnKF algorithms described in the literature. Via empirical approximation of the mean-field control law, a finite- $N$ controlled interacting particle algorithm is obtained. For this algorithm, the equations for empirical mean and covariance are derived and shown to be identical to the Kalman filter. This allows strong conclusions on convergence and error properties based on the classical filter stability theory for the Kalman filter. It is shown that, under certain technical conditions, the mean squared error converges to zero even with a finite number of particles. A detailed propagation of chaos analysis is carried out for the finite- $N$ algorithm. The analysis is used to prove weak convergence of the empirical distribution as $N\rightarrow \infty$. For a certain simplified filtering problem, analytical comparison of the mse with the importance sampling-based algorithms is described. The analysis helps explain the favorable scaling properties of the control-based algorithms reported in several numerical studies in recent literature. [ABSTRACT FROM AUTHOR]

This article studies a class of nonsmooth decentralized multiagent optimization problems where the agents aim at minimizing a sum of local strongly-convex smooth components plus a common nonsmooth term. We propose a general primal-dual algorithmic framework that unifies many existing state-of-the-art algorithms. We establish linear convergence of the proposed method to the exact minimizer in the presence of the nonsmooth term. Moreover, for the more general class of problems with agent specific nonsmooth terms, we show that linear convergence cannot be achieved (in the worst case) for the class of algorithms that uses the gradients and the proximal mappings of the smooth and nonsmooth parts, respectively. We further provide a numerical counterexample that shows how some state-of-the-art algorithms fail to converge linearly for strongly convex objectives and different local non smooth terms. [ABSTRACT FROM AUTHOR]

This article proposes a novel information-theoretic joint probabilistic data association filter for tracking unknown number of targets. The proposed information-theoretic joint probabilistic data association algorithm is obtained by the minimization of a weighted reverse Kullback–Leibler divergence to approximate the posterior Gaussian mixture probability density function. Theoretical analysis of mean performance and error covariance performance with ideal detection probability is presented to provide insights of the proposed approach. Extensive empirical simulations are undertaken to validate the performance of the proposed multitarget tracking algorithm. [ABSTRACT FROM AUTHOR]

Temporal logic provides a rigorous and expressive way to model specifications for cyber-physical systems (CPS). Such systems, which are playing increasingly important roles in various domains, are also subject to malicious attacks initiated by intelligent adversaries. Malicious attacks may prevent the system from satisfying the desired specification. In this article, we introduce a control synthesis method to minimize deviations from the desired specification due to adversaries. We consider the scenario where the adversary has limited capability of observing the controller's strategy. We synthesize a minimum violation control strategy for a finite-state stochastic game under cosafe linear temporal logic constraints. We formulate a mixed-integer nonlinear program for computing the optimal controller strategy and present two algorithms. First, we propose an exact algorithm that computes the optimal control strategy with probability one but without guarantees on the convergence rate. Second, we present an approximate algorithm that computes a suboptimal control strategy. The approximate algorithm converges faster compared to the exact algorithm; however, it only returns an optimal solution when the adversary has full observation over the controller's strategy. A numerical case study is presented to evaluate the proposed framework. [ABSTRACT FROM AUTHOR]

This article deals with output feedback selection in linear time-invariant structured systems. We assume that the inputs and the outputs are dedicated, i.e., each input directly actuates a single state and each output directly senses a single state. Given a structured system with dedicated inputs and outputs and a cost matrix that denotes the cost of each feedback connection, our aim is to select a minimum cost set of feedback connections such that the closed-loop system satisfies arbitrary pole-placement. This problem is referred to as the optimal feedback selection problem for dedicated i/o. The optimal feedback selection problem for dedicated i/o is NP-hard and inapproximable to a constant factor of log n, where n denotes the system dimension. To this end, we propose an algorithm to find an approximate solution to the problem. The proposed algorithm consists of a potential function incorporated with a greedy scheme and attains a solution with a guaranteed approximation ratio. We consider two special network topologies of practical importance, referred to as back-edge feedback structure and hierarchical networks. For the first case, which is NP-hard and inapproximable to a multiplicative factor of log n, we provide a log n-approximate solution. For hierarchical networks, we give a dynamic programming based algorithm to obtain an optimal solution in polynomial time. [ABSTRACT FROM AUTHOR]

With the development of large-scale sensor networks, there is an increasing interest for high dimensional distributed decision fusion problem in statistical decision, machine learning, control theory, etc. In this article, we investigate the optimal sensor quantized rules of the distributed decision under a given fusion rule for conditionally dependent and high dimensional sensor observations. We provide a Monte Carlo importance sampling method to reduce the computational complexity of the distributed decision fusion. For the K-out-of-L rule, we derive a group of analytic sensor rules based on the necessary and sufficient condition of the optimal sensor rules. In term of efficiency, it is significantly better than the traditional iterative search algorithms, such as the Riemann sum approximation iteration algorithm. The analytic solution is also a general result since it does not rely on the specific K value of the K-out-of-L rule, the specific conditional probability density function and the dimension of sensor observations. Thus, it can be applied to multisensor decision fusion problems with high dimensional dependent sensor observations. For the general fusion rules rather than the K-out-of-L rule, the Monte Carlo Gauss–Seidel algorithm is also provided. The numerical examples demonstrate the effectiveness of the analytic solution and the Monte Carlo Gauss–Seidel algorithm. [ABSTRACT FROM AUTHOR]

This article focuses on developing a strategy for control of systems, whose dynamics are almost entirely unknown. This situation arises naturally in a scenario, where a system undergoes a critical failure. In that case, it is imperative to retain the ability to satisfy basic control objectives in order to avert an imminent catastrophe. To deal with limitations on the knowledge of system dynamics, we develop a theory of myopic control. At any given time, myopic control optimizes the current direction of the system trajectory, given solely the knowledge about system dynamics obtained from data until that time. We propose an algorithm that uses small perturbations in the control effort to learn system dynamics around the current system state, while ensuring that the system moves in a nearly optimal direction, and provide bounds for its suboptimality. [ABSTRACT FROM AUTHOR]

Security issues in cyber-physical systems (CPSs) have gained increasing attention in recent years due to the importance and unavoidable vulnerability of CPSs. This article focuses on designing an intelligent online attack, which can compromise a sensor, eavesdrop measurements, and inject false feedback information, against remote state estimation. From the viewpoint of the attacker, we design an event-based attack strategy to degrade the estimation quality with an arbitrary communication rate stealthy constraint. The approximate minimum mean-squared error estimation algorithm from the viewpoint of the attacker is derived under a Gaussian assumption. Furthermore, the relation between the attack threshold and the scheduling threshold is obtained in a closed form. We show that the mean-squared stability condition of the estimation is weakened under the attack. Two examples are provided to demonstrate the main results. [ABSTRACT FROM AUTHOR]

Directed acyclic graphs (DAGs) are emerging as an attractive alternative to traditional blockchain architectures for distributed ledger technology. In particular, DAG ledgers with stochastic attachment mechanisms potentially offer many advantages over blockchain, including scalability and faster transaction speeds. However, the random nature of the attachment mechanism coupled with the requirement of protection against double-spending transactions might result in an unstable system in which not all transactions get eventually validated. Such transactions are said to be orphaned, and will never be validated. Our principal contribution is to propose a simple modification to the attachment mechanism for the Tangle (the IOTA DAG architecture). This modification ensures that all transactions are validated in finite time, and preserves essential features of the popular Monte Carlo selection algorithm. In order to demonstrate these results, we derive a fluid approximation for the Tangle (in the limit of infinite arrival rate) and prove that this fluid model exhibits the desired behavior. We also present simulations that validate the results for finite arrival rates. [ABSTRACT FROM AUTHOR]

An online reinforcement learning algorithm called renewal Monte Carlo (RMC) is presented. RMC works for infinite horizon Markov decision processes with a designated start state. RMC is a Monte Carlo algorithm that retains the key advantages of Monte Carlo—viz., simplicity, ease of implementation, and low bias—while circumventing the main drawbacks of Monte Carlo—viz., high variance and delayed updates. Given a parameterized policy $\pi _\theta$ , the algorithm consists of three parts: estimating the expected discounted reward $\mathsf {R}_{\theta }$ and the expected discounted time $\mathsf {T}_\theta$ over a regenerative cycle; estimating the derivatives $\nabla _\theta \mathsf {R}_\theta$ and $\nabla _\theta \mathsf {T}_\theta$ ; and updating the policy parameters using stochastic approximation to find the roots of $\mathsf {R}_\theta \nabla _\theta \mathsf {T}_\theta - \mathsf {T}_\theta \nabla _\theta \mathsf {R}_\theta$. It is shown that under mild technical conditions, RMC converges to a locally optimal policy. It is also shown that RMC works for postdecision state models as well. An approximate version of RMC is proposed where a regenerative cycle is defined as successive visits to a prespecified “renewal set”. It is shown that if the value function of the system is locally Lipschitz on the renewal set, then RMC converges to an approximate locally optimal policy. Three numerical experiments are presented to illustrate RMC and compare it with other state-of-the-art reinforcement learning algorithms. [ABSTRACT FROM AUTHOR]

This paper studies consensus problems of multi-agent systems with binary-valued communications. Different from most existing works, the agents considered in this paper can only get binary-valued observations of its neighbors’ states with random noises. A consensus algorithm is proposed: first, each agent estimates its neighbors’ states by the recursive projection algorithm; then, each agent designs the control timely based on the estimates. It is proved that the estimates of the states can converge to the true states with a faster convergence rate than that in the parameter estimation. Moreover, the states of the agents can achieve mean-square consensus, and the corresponding consensus speed can achieve $O(1/t)$ under certain conditions. Finally, simulations are given to demonstrate the theoretical results. [ABSTRACT FROM AUTHOR]

Many problems of interest for cyber-physical network systems can be formulated as mixed-integer linear programs in which the constraints are distributed among the agents. In this paper, we propose a distributed algorithmic framework to solve this class of optimization problems in a peer-to-peer network with no coordinator and with limited computation and communication capabilities. At each communication round, agents locally solve a small linear program, generate suitable cutting planes, and communicate a fixed number of active constraints. Within the distributed framework, we first propose an algorithm that, under the assumption of integer-valued optimal cost, guarantees finite-time convergence to an optimal solution. Second, we propose an algorithm for general problems that provides a suboptimal solution up to a given tolerance in a finite number of communication rounds. Both algorithms work under asynchronous, directed, unreliable networks. Finally, through numerical computations, we analyze the algorithm scalability in terms of the network size. Moreover, for a multi-agent multi-task assignment problem, we show, consistently with the theory, its robustness to packet loss. [ABSTRACT FROM AUTHOR]

This paper proposes an iterative learning control (ILC) algorithm with gain adaptation for discrete-time stochastic systems. The algorithm is based on Kesten's accelerated stochastic approximation (SA) algorithm. The gain adaptation uses only tracking error information, and, hence, is a data-driven adaptation approach. If stochastic noises account for a small proportion of the tracking error, the learning gain matrix remains constant with a high probability. If stochastic noises dominate the tracking error, the learning gain matrix is decreasing. Therefore, the new ILC algorithm converges more quickly than existing SA-based algorithms. In addition, the classic P-type ILC law for noise-free systems is a special case of the new ILC algorithm. The behaviors of the proposed ILC algorithm are demonstrated through examples. [ABSTRACT FROM AUTHOR]

In this paper, we propose a distributed Newton method for decenteralized optimization of large sums of convex functions. Our proposed method is based on creating a set of separable finite sum minimization problems by utilizing a decomposition technique known as Global Consensus that distributes the computation across nodes of a graph and enforces a consensus constraint among the separated variables. The key idea is to exploit the sparsity of the dual Hessian and recast the computation of the Newton step as one of the efficiently solving symmetric diagonally dominant linear equations. We show that our method outperforms the state-of-the-art algorithms, including ADMM. We validate our algorithm both theoretically and empirically. On the theory side, we demonstrate that our algorithm exhibits superlinear convergence within a neighborhood of optimality. Empirically, we show the superiority of this new method on a variety of large-scale optimization problems. The proposed approach is scalable to large problems and has a low communication overhead. [ABSTRACT FROM AUTHOR]

This paper studies linear stochastic approximation (SA) algorithms and their application to multiagent systems in engineering and sociology. As main contribution, we provide necessary and sufficient conditions for convergence of linear SA algorithms to a deterministic or random final vector. We also characterize the system convergence rate, when the system is convergent. Moreover, differing from non-negative gain functions in traditional SA algorithms, this paper considers also the case when the gain functions are allowed to take arbitrary real numbers. Using our general treatment, we provide necessary and sufficient conditions to reach consensus and group consensus for first-order discrete-time multiagent system over random signed networks and with state-dependent noise. Finally, we extend our results to the setting of multidimensional linear SA algorithms and characterize the behavior of the multidimensional Friedkin–Johnsen model over random interaction networks. [ABSTRACT FROM AUTHOR]

This paper considers a class of stochastic source-seeking problems to drive a mobile robot to the minimizer of a source signal. Our approach is first analyzed in an obstacle-free scenario, where measurements of the signal at the robot location and information of a contact sensor are required. We extend our results to environments with obstacles under mild assumptions on the step size. Our approach builds on the simultaneous perturbation stochastic approximation idea to obtain information of the signal field. We prove the practical convergence of the algorithms to a ball whose size depends on the step size that contains the location of the source. The novelty relies in that we consider nondifferentiable convex functions, a fixed step size, and the environment may contain obstacles. Our proof methods employ nonsmooth Lyapunov function theory, tools from convex analysis, and stochastic difference inclusions. Finally, we illustrate the applicability of the proposed algorithms in a two-dimensional scenarios. [ABSTRACT FROM AUTHOR]

This paper considers the problem of a leader that seeks to optimally influence the opinions of agents in a directed network through connecting with a limited number of the agents (“direct followers”), possibly in the presence of a fixed competing leader. The settings involving a single leader and two competing leaders are unified into a general combinatoric optimization problem, for which two heuristic approaches are developed. The first approach is based on a convex relaxation scheme, possibly in combination with the $\ell _1$ -norm regularization technique, and the second is based on a greedy selection strategy. The main technical novelties of this work are in the establishment of supermodularity of the objective function and convexity of its continuous relaxation. The greedy approach is guaranteed to have a lower bound on the approximation ratio sharper than $(1-1/e)$ , while the convex approach can benefit from efficient (customized) numerical solvers to have practically comparable solutions possibly with faster computation times. The two approaches can be combined to provide improved results. In numerical examples, the approximation ratio can be made to reach ${\text{90}\%}$ or higher depending on the number of direct followers. [ABSTRACT FROM AUTHOR]

This paper presents a discrete-time distributed and noncooperative routing algorithm, which is proved, via Lyapunov arguments, to asymptotically converge to a specific equilibrium condition among the traffic flows over the network paths, known as Wardrop equilibrium. This convergence result improves the discrete-time algorithms in the literature, which achieve approximate convergence to the Wardrop equilibrium. Numerical simulations show the effectiveness of the proposed approach. [ABSTRACT FROM AUTHOR]

Cumulative prospect theory (CPT) is a popular approach for modeling human preferences. It is based on probabilistic distortions and generalizes the expected utility theory. We bring the CPT to a stochastic optimization framework and propose algorithms for both estimation and optimization of CPT-value objectives. We propose an empirical distribution function-based scheme to estimate the CPT value, and then, use this scheme in the inner loop of a CPT-value optimization procedure. We propose both gradient based as well as gradient-free CPT-value optimization algorithms that are based on two well-known simulation optimization ideas: simultaneous perturbation stochastic approximation and model-based parameter search, respectively. We provide theoretical convergence guarantees for all the proposed algorithms and also illustrate the potential of CPT-based criteria in a traffic signal control application. [ABSTRACT FROM AUTHOR]

We propose an axiomatic approach for design and performance analysis of noisy linear consensus networks by introducing a notion of systemic performance measure. This class of measures are spectral functions of Laplacian eigenvalues of the network that are monotone, convex, and orthogonally invariant with respect to the Laplacian matrix of the network. It is shown that several existing gold-standard and widely used performance measures in the literature belong to this new class of measures. We build upon this new notion and investigate a general form of the combinatorial problem of growing a linear consensus network via minimizing a given systemic performance measure. Two efficient polynomial-time approximation algorithms are devised to tackle this network synthesis problem: a linearization-based method and a simple greedy algorithm based on rank-one updates. Several theoretical fundamental limits on the best achievable performance for the combinatorial problem are derived that assist us to evaluate optimality gaps of our proposed algorithms. A detailed complexity analysis confirms the effectiveness and viability of our algorithms to handle large-scale consensus networks. [ABSTRACT FROM AUTHOR]

We develop algorithms that find and track the optimal solution trajectory of time-varying convex optimization problems that consist of local and network-related objectives. The algorithms are derived from the prediction-correction methodology, which corresponds to a strategy where the time-varying problem is sampled at discrete time instances, and then, a sequence is generated via alternatively executing predictions on how the optimizers at the next time sample are changing and corrections on how they actually have changed. Prediction is based on how the optimality conditions evolve in time, while correction is based on a gradient or Newton method, leading to decentralized prediction-correction gradient and decentralized prediction-correction Newton. We extend these methods to cases where the knowledge on how the optimization programs are changing in time is only approximate and propose decentralized approximate prediction-correction gradient and decentralized approximate prediction-correction Newton. Convergence properties of all the proposed methods are studied and empirical performance is shown on an application of a resource allocation problem in a wireless network. We observe that the proposed methods outperform existing running algorithms by orders of magnitude. The numerical results showcase a tradeoff between convergence accuracy, sampling period, and network communications. [ABSTRACT FROM PUBLISHER]

In this paper, a time-varying distributed convex optimization problem is studied for continuous-time multi-agent systems. The objective is to minimize the sum of local time-varying cost functions, each of which is known to only an individual agent, through local interaction. Here, the optimal point is time varying and creates an optimal trajectory. Control algorithms are designed for the cases of single-integrator and double-integrator dynamics. In both cases, a centralized approach is first introduced to solve the optimization problem. Then, this problem is solved in a distributed manner and a discontinuous algorithm based on the signum function is proposed in each case. In the case of single-integrator (respectively, double-integrator) dynamics, each agent relies only on its own position and the relative positions (respectively, positions and velocities) between itself and its neighbors. A gain adaption scheme is introduced in both algorithms to eliminate certain global information requirement. To relax the restricted assumption imposed on feasible cost functions, an estimator based algorithm using the signum function is proposed, where each agent uses dynamic average tracking as a tool to estimate the centralized control input. As a tradeoff, the estimator-based algorithm necessitates communication between neighbors. Then, in the case of double-integrator dynamics, the proposed algorithms are further extended. Two continuous algorithms based on, respectively, a time-varying and a fixed boundary layer are proposed as continuous approximations of the signum function. To account for interagent collision for physical agents, a distributed convex optimization problem with swarm tracking behavior is introduced for both single-integrator and double-integrator dynamics. It is shown that the center of the agents tracks the optimal trajectory, the connectivity of the agents is maintained, and interagent collision is avoided. [ABSTRACT FROM AUTHOR]

We develop a gradient algorithm for optimizing the performance of product-form networks through online adjustment of control parameters. The use of standard algorithms for finding optimal parameter settings is hampered by the prohibitive computational burden of calculating the gradient in terms of the stationary probabilities. The proposed approach instead relies on measuring empirical frequencies of the various states through simulation or online operation so as to obtain estimates for the gradient. Besides the reduction in computational effort, a further benefit of the online operation lies in the natural adaptation to slow variations in ambient parameters as commonly occurring in dynamic environments. On the downside, the measurements result in inherently noisy and biased estimates. We exploit mixing time results in order to overcome the impact of the bias and establish sufficient conditions for convergence to a globally optimal solution. We discuss our algorithm in the context of different systems, including queueing networks, loss networks, and wireless networks. We also illustrate how the algorithm can be used in such systems to optimize a service/cost trade-off, to map parameter regions that lead to systems meeting specified constraints, and to achieve target performance measures. For the latter application, we first identify which performance measures can be controlled depending on the set of configurable parameters. We then characterize the achievable region of performance measures in product-form networks, and finally we describe how our algorithm can be used to achieve the target performance in an online, distributed fashion, depending on the application context. [ABSTRACT FROM PUBLISHER]

This technical note extends the feedback particle filter (FPF) methodology and algorithms to the problem of filtering a continuous-time Markov chain. The main contribution is the development of a feedback control-based transformation of the Wonham filter, where the control input is realized via a time-modulated Poisson counter process. A complete characterization of the feedback mechanism that defines the FPF is obtained, which leads to tractable algorithms for the nonlinear filtering problem even for large state spaces. Numerical examples are introduced to help illustrate the application of these techniques. [ABSTRACT FROM PUBLISHER]

This technical note presents a method for the identification of Hammerstein Output Error (HOE) models corrupted by bounded noise. The method is based on an iterative scheme using two Optimal Bounding Ellipsoid (OBE) type algorithms. The proposed approach allows the joint estimation of the linear part and of the nonlinear part with a low computational burden. A numerical example is given so as to show the efficiency of the method. [ABSTRACT FROM PUBLISHER]

We apply first- and second-order splitting schemes to the differential Riccati equation. Such equations are very important in, e.g., linear quadratic regulator (LQR) problems, where they provide a link between the state of the system and the optimal input. The methods can also be extended to generalized Riccati equations, e.g., arising from LQR problems given in implicit form. In contrast to previously proposed schemes such as BDF or Rosenbrock methods, the splitting schemes exploit the fact that the nonlinear and affine parts of the problem, when considered in isolation, have closed-form solutions. We show that if the solution possesses low-rank structure, which is frequently the case, then this is preserved by the method. This feature is used to implement the methods efficiently for large-scale problems. The proposed methods are expected to be competitive, as they at most require the solution of a small number of linear equation systems per time step. Finally, we apply our low-rank implementations to the Riccati equations arising from two LQR problems. The results show that the rank of the solutions stay low, and the expected orders of convergence are observed. [ABSTRACT FROM PUBLISHER]

We investigate the connections between compression learning and scenario based optimization. We first show how to strengthen, or relax the consistency assumption at the basis of compression learning and provide novel learnability conditions for the underlying algorithms. We then consider different constrained optimization problems affected by uncertainty represented by means of scenarios. We show that the compression learning perspective provides a unifying framework for scenario based optimization, since the issue of providing guarantees on the probability of constraint violation reduces to a learning problem for an appropriately chosen algorithm that satisfies some consistency assumption. To illustrate this, we revisit the scenario approach within the developed context. Moreover, using the compression learning machinery we provide novel results on the probability of constraint violation for the class of cascading optimization problems. [ABSTRACT FROM PUBLISHER]

Recently, Chen (2010) presented a new approach to recursive identification for ARMAX systems, which is a three-stage recursive scheme and assumes independent and identically distributed input signals. Here, we observe that, unless the time scale of the algorithm at one stage is reasonably faster than those at the previous stages, convergence to the true value may not take place. To remedy this issue, this note proposes a multi-time-scale modification of the algorithm in [5] such that convergence is achieved. In addition, the new scheme handles a wider class of input signals so that the input can be designed for some purpose.The advantage of the multi-time scale algorithm is verified with numerical examples. [ABSTRACT FROM PUBLISHER]

This paper deals with the update step of Gaussian MAP filtering. In this framework, we seek a Gaussian approximation to the posterior probability density function (PDF) whose mean is given by the maximum a posteriori (MAP) estimator. We propose two novel optimization algorithms which are quite suitable for finding the MAP estimate although they can also be used to solve general optimization problems. These are based on the design of a sequence of PDFs that become increasingly concentrated around the MAP estimate. The resulting algorithms are referred to as Kalman optimization (KO) methods. We also provide the important relations between these KO methods and their conventional optimization algorithms (COAs) counterparts, i.e., Newton's and Levenberg-Marquardt algorithms. Our simulations indicate that KO methods are more robust than their COA equivalents. [ABSTRACT FROM AUTHOR]

We apply diffusion strategies to develop a fully-distributed cooperative reinforcement learning algorithm in which agents in a network communicate only with their immediate neighbors to improve predictions about their environment. The algorithm can also be applied to off-policy learning, meaning that the agents can predict the response to a behavior different from the actual policies they are following. The proposed distributed strategy is efficient, with linear complexity in both computation time and memory footprint. We provide a mean-square-error performance analysis and establish convergence under constant step-size updates, which endow the network with continuous learning capabilities. The results show a clear gain from cooperation: when the individual agents can estimate the solution, cooperation increases stability and reduces bias and variance of the prediction error; but, more importantly, the network is able to approach the optimal solution even when none of the individual agents can (e.g., when the individual behavior policies restrict each agent to sample a small portion of the state space). [ABSTRACT FROM AUTHOR]

Most of the controllers designed for systems with timedelays involve finite impulse response (FIR) filters in their structure, in particular prediction based controllers for systems with input/output delays. We present theory, algorithms and software to evaluate the frequency response, to approximate the filters by stable linear timeinvariant systems and to compute their induced \cal L2 gain. The approach is based on the theory of functions of matrices. In the approach pole zero cancelations in frequency domain representations are explicitly taken into account. [ABSTRACT FROM AUTHOR]

We present a fast distributed solution to the convex network flow optimization problem. Our approach uses a family of dual descent algorithms that approximate the Newton direction to achieve faster convergence rates than existing distributed methods. The approximate Newton directions are obtained through matrix splitting techniques and sparse Taylor approximations of the inverse Hessian. We couple this descent direction with a distributed line search algorithm which requires the same information as our descent direction to compute. We show that, similarly to conventional Newton methods, the proposed algorithm exhibits super-linear convergence within a neighborhood of the optimal value. Numerical experiments corroborate that convergence times are between one to two orders of magnitude faster than existing distributed optimization methods. A connection with recent developments that use consensus to compute approximate Newton directions is also presented. [ABSTRACT FROM PUBLISHER]

We reformulate a deterministic optimization problem as a filtering problem, where the goal is to compute the conditional distribution of the unobserved state given the observation history. We prove that in our formulation the conditional distribution converges asymptotically to a degenerate distribution concentrated on the global optimum. Hence, the goal of searching for the global optimum can be achieved by computing the conditional distribution. Since this computation is often analytically intractable, we approximate it by particle filtering, a class of sequential Monte Carlo methods for filtering, which has proven convergence in “tracking” the conditional distribution. The resultant algorithmic framework unifies some randomized optimization algorithms and provides new insights into their connection. [ABSTRACT FROM PUBLISHER]

We study the traveling salesman problem for a Dubins vehicle. We prove that this problem is NP-hard, and provide lower bounds on the approximation ratio achievable by some recently proposed heuristics. We also describe new algorithms for this problem based on heading discretization, and evaluate their performance numerically. [ABSTRACT FROM PUBLISHER]

We consider the problem of monitoring spatial phenomena, such as road speeds on a highway, using wireless sensors with limited battery life. A central question is to decide where to locate these sensors to best predict the phenomenon at the unsensed locations. However, given the power constraints, we also need to determine when to activate these sensors in order to maximize the performance while satisfying lifetime requirements. Traditionally, these two problems of sensor placement and scheduling have been considered separately; one first decides where to place the sensors, and then when to activate them. [ABSTRACT FROM AUTHOR]

This paper deals with a method for the approximation of a spectral density function among the solutions of a generalized moment problem à la Byrnes/Georgiou/Lindquist. The approximation is pursued with respect to the Kullback–Leibler pseudo-distance, which gives rise to a convex optimization problem. After developing the variational analysis, we discuss the properties of an efficient algorithm for the solution of the corresponding dual problem, based on the iteration of a nonlinear map in a bounded subset of the dual space. Our main result is the proof of local convergence of the latter, established as a consequence of the central manifold theorem. Supported by numerical evidence, we conjecture that, in the mentioned bounded set, the convergence is actually global. [ABSTRACT FROM AUTHOR]