Search

We consider an environmentwhere players need to decide whether to buy a certain product (or adopt a technology) or not. The product is either good or bad, but its true value is unknown to the players. Instead, each player has her own private information on its quality. Each player can observe the previous actions of other players and estimate the quality of the product. A classic result in the literature shows that in similar settings, informational cascades occur, where learning stops for the whole network and players repeat the actions of their predecessors. In contrast to this literature, in this paper, players get more than one opportunity to act. In each turn, a player is chosen uniformly at random from all the players and can decide to buy the product and leave the market or wait. Her utility is the total expected discounted reward, and thus, myopic strategies may not constitute equilibria. We provide a characterization of perfect Bayesian equilibria (PBEs) with forward-looking strategies through a fixed-point equation of dimensionality that grows only quadratically with the number of players. Using this tractable fixed-point equation, we show the existence of a PBE and characterize PBEs with threshold strategies. Based on this characterization, we study informational cascades in two regimes. First, we show that for a discount factor $\delta$ strictly smaller than 1, informational cascades happen with high probability as the number of players $N$ increases. Furthermore, only a small portion of the total information in the system is revealed before a cascade occurs. Second, and more surprisingly, we show that for a fixed $N$ , and for a sufficiently large $\delta < 1$ , when the product is bad, there exists an equilibrium where an informational cascade can happen only after at least half of the players revealed their private information, and consequently, the probability for a “bad cascade” where all the players buy the product vanishes exponentially with $N$. Finally, when $\delta =1$ and the product is bad, there exists an equilibrium where informational cascades do not happen at all. [ABSTRACT FROM AUTHOR]

This paper studies the optimal decentralized state-feedback control of large-scale systems. The large-scale system is composed of subsystems and defined over a connected digraph. One step time is required for information to travel across an edge in the graph. Under the above-mentioned setup, when subsystem process noises are uncorrelated, the explicit optimal state-feedback controller can be designed by independence decomposition based on information hierarchy graph. However, this decomposition method fails when the subsystem process noises are correlated. In this paper, we propose a new decomposition method for system state and control input, and split the optimal state-feedback control problem with correlated process noises into two subproblems that can be solved separately. The solution to the first subproblem can be obtained by solving a linear matrix equation. The second subproblem is solved by algebraic Ricatti equation. [ABSTRACT FROM AUTHOR]

This paper studies two-player zero-sum repeated Bayesian games in which every player has a private type that is unknown to the other player, and the initial probability of the type of every player is publicly known. The types of players are independently chosen according to the initial probabilities, and are kept the same all through the game. At every stage, players simultaneously choose actions, and announce their actions publicly. For finite horizon cases, an explicit linear program is provided to compute players’ security strategies. Moreover, this paper shows that a player's sufficient statistics, which is independent of the strategy of the other player, consists of the belief over the player's own type, the regret over the other player's type, and the stage. Explicit linear programs, whose size is linear in the size of the game tree, are provided to compute the initial regrets, and the security strategies that only depends on the sufficient statistics. For discounted cases, following the same idea in the finite horizon, this paper shows that a player's sufficient statistics consists of the belief of the player's own type and the antidiscounted regret with respect to the other player's type. Besides, an approximated security strategy depending on the sufficient statistics is provided, and an explicit linear program to compute the approximated security strategy is given. This paper also obtains a bound on the performance difference between the approximated security strategy and the security strategy, and shows that the bound converges to 0 exponentially fast. [ABSTRACT FROM AUTHOR]

In this paper, the robustness to model mismatch of a preexisting collocated boundary adaptive feedback law is investigated. This control law was originally designed for an antidamped pure wave partial differential equation (PDE). Actuation and measurements are located at the same boundary. Adaptive terms account for uncertain parameters located at the antidamped boundary, opposite to the collocated actuation and measurement. By extending and transforming the system state using, in particular, backstepping, this paper establishes that this controller is robust to sufficiently small in-domain damping. In particular, both stability and attractivity (convergence) are established similarly as in the nominal case. Note moreover that, assuming that some parameters are known, the exponential stability to an attractor holds. Simulations are performed to illustrate the interest of this study to attenuate mechanical vibrations in an oil-drilling context. [ABSTRACT FROM AUTHOR]

This paper introduces an explicit reference governor to supervise closed-loop linear time-delay systems. The proposed scheme enforces state and input constraints by modifying the reference of the supervised system so that the state vector always belongs to admissible sublevel sets of a suitably defined Lyapunov–Krasovskii functional. To accomplish this, this paper extends the existing definition of “dynamic safety margin” to a time-delay setting and illustrates how to employ classic Lyapunov–Krasovskii functionals even though the reference is time varying. Constraint enforcement for arbitrary reference signals and asymptotic convergence to any strictly steady-state admissible set point is rigorously proven. Experimental results are reported to demonstrate the simplicity, practicality, and robustness of the proposed method. [ABSTRACT FROM AUTHOR]

In this paper, we apply Thompson sampling to a class of average reward stochastic control problems with parameter uncertainty. Specifically, we study an average reward stochastic control problem over an infinite horizon in which both the reward and state transition distributions are parameterized by an unknown parameter taking values in a finite space. The main result of this paper is a proof showing that Thompson sampling achieves a worst case average per period regret of O(T^-1), which is asymptotically optimal. [ABSTRACT FROM PUBLISHER]

Zero-sum asymmetric information games model decision-making scenarios involving two competing players who have different information about the game being played. A particular case is that of nested information, where one (informed) player has superior information over the other (uninformed) player. This paper considers the case of nested information in repeated zero-sum games and studies the computation of strategies for both the informed and uninformed players for finite-horizon and discounted infinite-horizon nested information games. For finite-horizon settings, we exploit that for both players, the security strategy, and also the opponent's corresponding best response, depend only on the informed player's history of actions. Using this property, we formulate an linear program (LP) computation of player strategies that is linear in the size of the uninformed player's action set. For the infinite-horizon discounted game, we construct LP formulations to compute the approximated security strategies for both players, and show that the worst-case performance difference between the approximated security strategies and the security strategies converges to zero exponentially. Finally, we illustrate the results on a network interdiction game between an informed system administrator and an uniformed intruder. [ABSTRACT FROM AUTHOR]

This paper is concerned with the first passage discounted payoff criterion for continuous-time $N$ -person nonzero-sum stochastic games in countable states and Borel action spaces, where the discount factor is nonconstant. For the game with unbounded reward functions and transition rates, the optimality is over the set of all randomized history-dependent policies. After showing the uniqueness of the solution to the Shapley equation, we give some suitable conditions imposed on the primitive data for the existence of Nash equilibria. Then, a financial system is introduced to illustrate the applications of the main result here. [ABSTRACT FROM AUTHOR]

We consider both finite-horizon and infinite-horizon versions of a dynamic game with $N$ selfish players who observe their types privately and take actions that are publicly observed. Players’ types evolve as conditionally independent Markov processes, conditioned on their current actions. Their actions and types jointly determine their instantaneous rewards. In dynamic games with asymmetric information, a widely used concept of equilibrium is perfect Bayesian equilibrium (PBE) which consists of a strategy and belief pair that simultaneously satisfy sequential rationality and belief consistency. In general, there does not exist a universal algorithm that decouples the interdependence of strategies and beliefs over time in calculating PBE. In this paper, for the finite-horizon game with independent types, we develop a two-step backward–forward recursive algorithm that sequentially decomposes the problem (w.r.t. time) to obtain a subset of PBEs, which we refer to as structured Bayesian perfect equilibria (SPBE). In such equilibria, a player's strategy depends on his/her history only through a common public belief and its current private type. The backward recursive part of this algorithm defines an equilibrium generating function. Each period in the backward recursion involves solving a fixed-point equation on the space of probability simplexes for every possible belief on types. Using this function, equilibrium strategies and beliefs are generated through a forward recursion. We then extend this methodology to the infinite-horizon model, where we propose a time-invariant single-shot fixed-point equation, which in conjunction with a forward recursive step, generates the SPBE. Sufficient conditions for the existence of SPBE are provided. With our proposed method, we find equilibria that exhibit signaling behavior. This is illustrated with the help of a concrete public goods example. [ABSTRACT FROM AUTHOR]

In this paper, we consider a delay differential inclusion $\dot{x}(t)\in F(t,x_t)$ , where $x_t$ denotes the history function of $x(\cdot)$ along an interval of time. We extend the celebrated Filippov's theorem to this case. Then, we further generalize this theorem to the case when the state variable $x$ is constrained to the closure of an open subset $K\subset \mathbb {R}^n$. Under a new “inward pointing condition,” we give a relaxation result stating that the set of trajectories lying in the interior of the state constraint is dense in the set of constrained trajectories of the convexified inclusion ${\dot{x}(t)\in \mbox{co}\,F(t,x_t)}$. [ABSTRACT FROM AUTHOR]

In decentralized control systems with linear dynamics, quadratic cost, and Gaussian disturbances (also called decentralized LQG systems) linear control strategies are not always optimal. Nonetheless, linear control strategies are appealing due to their analytic and implementation simplicity. In this paper, we investigate decentralized LQG systems with partial history sharing information structure and identify finite-dimensional sufficient statistics for such systems. Unlike prior work on decentralized LQG systems, we do not assume partial nestedness or quadratic invariance. Our approach is based on the common information approach of Nayyar et al., 2013 and exploits the linearity of the system dynamics and control strategies. To illustrate our methodology, we identify sufficient statistics for linear strategies in decentralized systems where controllers communicate over a strongly connected graph with finite delays, and for decentralized systems consisting of coupled subsystems with control sharing or one-sided one step delay sharing information structures. [ABSTRACT FROM PUBLISHER]

We consider online learning in finite stochastic Markovian environments where in each time step a new reward function is chosen by an oblivious adversary. The goal of the learning agent is to compete with the best stationary policy in hindsight in terms of the total reward received. Specifically, in each time step the agent observes the current state and the reward associated with the last transition, however, the agent does not observe the rewards associated with other state-action pairs. The agent is assumed to know the transition probabilities. The state of the art result for this setting is an algorithm with an expected regret of O(T^2/3\ln T). In this paper, assuming that stationary policies mix uniformly fast, we show that after T time steps, the expected regret of this algorithm (more precisely, a slightly modified version thereof) is O(T^1/2\ln T), giving the first rigorously proven, essentially tight regret bound for the problem. [ABSTRACT FROM AUTHOR]

Subsystems that are coupled due to dynamics and costs arise naturally in various communication applications. In many such applications the control actions are shared between different control stations giving rise to a control sharing information structure. Previous studies of control-sharing have concentrated on the linear quadratic Gaussian setup and a solution approach tailored to continuous valued control actions. In this paper a three step solution approach for finite valued control actions is presented. In the first step, a person-by-person approach is used to identify redundant data or a sufficient statistic for local information at each control station. In the second step, the common-information based approach of Nayyar (2013) is used to find a sufficient statistic for the common information shared between all control stations and to obtain a dynamic programming decomposition. In the third step, the specifics of the model are used to simplify the sufficient statistic and the dynamic program. [ABSTRACT FROM AUTHOR]

This paper considers a class of scenarios where targets emerge from some known location and move towards some unknown destinations in a weighted acyclic digraph. A decision maker with knowledge of the target positions must decide when preparations should be made at any given destination for their arrival. The decision maker faces a timing tradeoff: early decisions mean more time for preparation at the cost of higher uncertainty in the target's true destination while later decisions mean less uncertainty at the cost of having less time to prepare. We show how this problem can be formulated as an optimal stopping problem on a Markov chain. This sets the basis for the introduction of the best investment algorithm which prescribes when investments must be made conditioned on the target's motion along the digraph. We establish the optimality of this prescription and examine its robustness against changes in the problem parameters, identifying sufficient conditions to determine whether the solution computed by the best investment algorithm remains optimal. Based on this analysis, we develop the self-triggered acquisition&decision algorithm that allows the decision maker, under partial knowledge of the parameter dynamics, to schedule in advance when to check if the control policy in its memory remains optimal and, if this test fails, when to recompute it. Finally, we obtain worst case lower bounds on the maximum time that can elapse under arbitrary parameter dynamics before the optimal solution must be recomputed. Simulations illustrate our results. [ABSTRACT FROM AUTHOR]

In this paper, we consider the safety control problem for hidden mode hybrid systems (HMHSs), which are a special class of hybrid automata in which the mode is not available for control. For these systems, safety control is a problem with imperfect state information. We tackle this problem by introducing the notion of nondeterministic discrete information state and by translating the problem to one with perfect state information. The perfect state information control problem is obtained by constructing a new hybrid automaton, whose discrete state is an estimate of the HMHS mode and is, as such, available for control. This problem is solved by computing the capture set and the least restrictive control map for the new hybrid automaton. Sufficient conditions for the termination of the algorithm that computes the capture set are provided. Finally, we show that the solved perfect state information control problem is equivalent to the original problem with imperfect state information under suitable assumptions. We illustrate the application of the proposed technique to a collision avoidance problem between an autonomous vehicle and a human driven vehicle at a traffic intersection. [ABSTRACT FROM PUBLISHER]

In this paper, we propose a new tractable framework for dealing with linear dynamical systems affected by uncertainty, applicable to multistage robust optimization and stochastic programming. We introduce a hierarchy of near-optimal polynomial disturbance-feedback control policies, and show how these can be computed by solving a single semidefinite programming problem. The approach yields a hierarchy parameterized by a single variable (the degree of the polynomial policies), which controls the trade-off between the optimality gap and the computational requirements. We evaluate our framework in the context of three classical applications—two in inventory management, and one in robust regulation of an active suspension system—in which very strong numerical performance is exhibited, at relatively modest computational expense. [ABSTRACT FROM AUTHOR]

Information delays exist when the most recent inventory information available to the Inventory Manager (IM) is dated. In other words, the IM observes only the inventory level that belongs to an earlier period. Such situations are not uncommon, and they arise when it takes a while to process the demand data and pass the results to the IM. In this paper, we establish that the average cost optimal policy is of state-dependent basestock type with respect to the reference inventory position. We show that the optimal base stock depends on the age and the magnitude of the latest observed delay. We illustrate the results by solving an example with delays of 0 and 1, for which we are able to obtain formulas/bounds for the basestock levels. [ABSTRACT FROM AUTHOR]

In this paper, we derive an efficient Chebyshev algorithm for solving optimal control problems. The Chebyshev expansions are employed to approximate both the control and the state functions. The discretizing process and the related techniques are unique compared to existing methods. The optimal control problems are transformed into the resulting mathematical programming problems. Theoretical analysis is given to support the method. Further numerical examples and comparisons are presented to illustrate the efficiency of the method. [ABSTRACT FROM AUTHOR]

This paper deals with a method for the approximation of a spectral density function among the solutions of a generalized moment problem à la Byrnes/Georgiou/Lindquist. The approximation is pursued with respect to the Kullback–Leibler pseudo-distance, which gives rise to a convex optimization problem. After developing the variational analysis, we discuss the properties of an efficient algorithm for the solution of the corresponding dual problem, based on the iteration of a nonlinear map in a bounded subset of the dual space. Our main result is the proof of local convergence of the latter, established as a consequence of the central manifold theorem. Supported by numerical evidence, we conjecture that, in the mentioned bounded set, the convergence is actually global. [ABSTRACT FROM AUTHOR]