Search

This paper proposes several novel optimization algorithms for minimizing a nonlinear objective function. The algorithms are enlightened by the optimal state trajectory of an optimal control problem closely related to the minimized objective function. They are superlinear convergent when appropriate parameters are selected as required. Unlike Newton's method, all of them can be also applied in the case of a singular Hessian matrix. More importantly, by reduction, some of them avoid calculating the inverse of the Hessian matrix or an identical dimension matrix and some of them need only the diagonal elements of the Hessian matrix. In these cases, these algorithms still outperform the gradient descent method. The merits of the proposed optimization algorithm are illustrated by numerical experiments.

This paper studies the linear-quadratic (LQ) optimal control problem of a class of systems governed by the first-order hyperbolic partial differential equations (PDEs) with final state constraints. The main contribution is to present the solvability condition and the corresponding explicit optimal controller by using the Lagrange multiplier method and the technique of solving forward and backward partial differential equations (FBPDEs). In particular, the result is reduced to the case with zero-valued final state constraints. Several numerical examples are provided to demonstrate the performance of the designed optimal controller.

This paper is concerned with the exact controllability of discrete-time stochastic system which is one of the basic problems of modern control theory. Though the exact controllability of continuous-time system governed by Ito stochastic differential equations has been well studied in S. Peng, Progress in Natural Science, 1994, the counterpart of the discrete-time case is still open due to the adaptiveness constraint of the controllers and the solvability challenging of stochastic difference equation with terminal value. The main contribution in this paper is to present both the Gramian matrix criterion and the Rank criterion for the exact controllability of discrete-time stochastic system. The novelty lies in the transformation of the forward stochastic difference equation into a novel backward one.

In the paper, we propose solving optimization problems (OPs) and understanding the Newton method from the optimal control view. We propose a new optimization algorithm based on the optimal control problem (OCP). The algorithm features converging more rapidly than gradient descent, meanwhile, it is superior to Newton's method because it is not divergent in general and can be applied in the case of a singular Hessian matrix. These merits are supported by the convergence analysis for the algorithm in the paper. We also point out that the convergence rate of the proposed algorithm is inversely proportional to the magnitude of the control weight matrix and proportional to the control terminal time inherited from OCP.

This paper studies the consensus problem of heterogeneous multi-agent systems by the feedforward control and linear quadratic (LQ) optimal control theory. Different from the existing consensus control algorithms, which require to design an additional distributed observer for estimating the leader's information and to solve a set of regulator equations. In this paper, by designing a distributed feedforward controller, a non-standard neighbor error system is transformed into a standard linear system, and then an optimal consensus controller is designed by minimizing a combined state error with neighbour agents. The proposed optimal controller is obtained by solving Riccati equations, and it is shown that the corresponding cost function under the proposed distributed controllers is asymptotically optimal. The proposed consensus algorithm can be directly applicable to solve the consensus problem of homogeneous systems. Simulation example indicates the effectiveness of the proposed scheme and a much faster convergence speed than the existing algorithm., Comment: arXiv admin note: substantial text overlap with arXiv:2309.12577

This paper develops a novel approach to the consensus problem of multi-agent systems by minimizing a weighted state error with neighbor agents via linear quadratic (LQ) optimal control theory. Existing consensus control algorithms only utilize the current state of each agent, and the design of distributed controller depends on nonzero eigenvalues of the communication topology. The presented optimal consensus controller is obtained by solving Riccati equations and designing appropriate observers to account for agents' historical state information. It is shown that the corresponding cost function under the proposed controllers is asymptotically optimal. Simulation examples demonstrate the effectiveness of the proposed scheme, and a much faster convergence speed than the conventional consensus methods. Moreover, the new method avoids computing nonzero eigenvalues of the communication topology as in the traditional consensus methods.

In this paper, the two-player leader-follower game with private inputs for feedback Stackelberg strategy is considered. In particular, the follower shares its measurement information with the leader except its historical control inputs while the leader shares none of the historical control inputs and the measurement information with the follower. The private inputs of the leader and the follower lead to the main obstacle, which causes the fact that the estimation gain and the control gain are related with each other, resulting that the forward and backward Riccati equations are coupled and making the calculation complicated. By introducing a kind of novel observers through the information structure for the follower and the leader, respectively, a kind of new observer-feedback Stacklberg strategy is designed. Accordingly, the above-mentioned obstacle is also avoided. Moreover, it is found that the cost functions under the presented observer-feedback Stackelberg strategy are asymptotically optimal to the cost functions under the optimal feedback Stackelberg strategy with the feedback form of the state. Finally, a numerical example is given to show the efficiency of this paper.

In this paper, we focus on a method based on optimal control to address the optimization problem. The objective is to find the optimal solution that minimizes the objective function. We transform the optimization problem into optimal control by designing an appropriate cost function. Using Pontryagin's Maximum Principle and the associated forward-backward difference equations (FBDEs), we derive the iterative update gain for the optimization. The steady system state can be considered as the solution to the optimization problem. Finally, we discuss the compelling characteristics of our method and further demonstrate its high precision, low oscillation, and applicability for finding different local minima of non-convex functions through several simulation examples.

This paper focuses on optimal mismatched disturbance rejection control for linear continuoustime uncontrollable systems. Different from previous studies, by introducing a new quadratic performance index to transform the mismatched disturbance rejection control into a linear quadratic tracking problem, the regulated state can track a reference trajectory and minimize the influence of disturbance. The necessary and sufficient conditions for the solvability and the disturbance rejection controller are obtained by solving a forward-backward differential equation over a finite horizon. A sufficient condition for system stability is obtained over an infinite horizon under detectable condition. This paper details our novel approach for transforming disturbance rejection into a linear quadratic tracking problem. The effectiveness of the proposed method is provided with two examples to demonstrate., Comment: arXiv admin note: substantial text overlap with arXiv:2209.07014

In this paper, we study the linear quadratic (LQ) optimal control problem of linear systems with private input and measurement information. The main challenging lies in the unavailability of other regulators' historical input information. To overcome this difficulty, we introduce a kind of novel observers by using the private input and measurement information and accordingly design a kind of new decentralized controllers. In particular, it is verified that the corresponding cost function under the proposed decentralized controllers are asymptotically optimal as comparison with the optimal cost under optimal state-feedback controller. The presented results in this paper are new to the best of our knowledge, which represent the fundamental contribution to classical decentralized control.

This study focuses on the problem of optimal mismatched disturbance rejection control for uncontrollable linear discrete-time systems. In contrast to previous studies, by introducing a quadratic performance index such that the regulated state can track a reference trajectory and minimize the effects of disturbances, mismatched disturbance rejection control is transformed into a linear quadratic tracking problem. The necessary and sufficient conditions for the solvability of this problem over a finite horizon and a disturbance rejection controller are derived by solving a forward-backward difference equation. In the case of an infinite horizon, a sufficient condition for the stabilization of the system is obtained under the detectable condition. This paper details our novel approach to disturbance rejection. Four examples are provided to demonstrate the effectiveness of the proposed method.

In this paper, we study the social optimality for mean field linear quadratic control systems following the direct approach, where subsystems are coupled via individual dynamics and costs according to a network topology. A graph is introduced to represent the network topology of the large-population system, where nodes represent subpopulations called clusters and edges represent communication relationship. By the direct approach, we first seek the optimal controller under centralized information structure, which characterized by a set of forward-backward stochastic differential equations. Then the feedback controller is obtained with the help of Riccat equations. Finally, we design the distributed controller with mean field approximations, which has the property of asymptotically social optimality.

This paper is concerned with a new class of mean-field games which involve a finite number of agents. Necessary and sufficient conditions are obtained for the existence of the decentralized open-loop Nash equilibrium in terms of non-standard forward-backward stochastic differential equations (FBSDEs). By solving the FBSDEs, we design a set of decentralized strategies by virtue of two differential Riccati equations. Instead of the $\varepsilon$-Nash equilibrium in classical mean-field games, the set of decentralized strategies is shown to be a Nash equilibrium. For the infinite-horizon problem, a simple condition is given for the solvability of the algebraic Riccati equation arising from consensus. Furthermore, the social optimal control problem is studied. Under a mild condition, the decentralized social optimal control and the corresponding social cost are given., Comment: 18 pages, 5 figures

We investigate the linear quadratic Gaussian Stackelberg game under a class of nested observation information pattern. Two decision makers implement control strategies relying on different information sets: The follower uses its observation data to design its strategy, whereas the leader implements its strategy using global observation data. We show that the solution requires solving a new type of forward-backward stochastic differential equations whose drift terms contain two types of conditional expectation terms associated to the adjoint variables. We then propose a method to find the functional relations between each adjoint pair, i.e., each pair formed by an adjoint variable and the conditional expectation of its associated state. The proposed method follows a layered pattern. More precisely, in the inner layer, we seek the functional relation for the adjoint pair under the sigma-sub-algebra generated by follower's observation information; and in the outer layer, we look for the functional relation for the adjoint pair under the sigma-sub-algebra generated by leader's observation information. Our result shows that the optimal open-loop solution admits an explicit feedback type representation. More precisely, the feedback coefficient matrices satisfy tuples of coupled forward-backward differential Riccati equations, and feedback variables are computed by Kalman-Bucy filtering.

This paper is concerned with mismatched disturbance rejection control for the second-order discrete-time systems.Different from previous work, the controllability of the system is applied to design the disturbance compensation gain, which does not require any coordinate transformations. Via this new idea, it is shown that disturbance in the regulated output is immediately and directly compensated in the case that the disturbance is known. When the disturbance is unknown, an extra generalized extended state observer is applied to design the controller. Two examples are given to show the effectiveness of the proposed methods. Numerical simulation shows that the designed controller has excellent disturbance rejection effect when the disturbance is known. The example with respect to the permanent-magnet direct current motor illustrates that the proposed control method for unknown disturbance rejection is effective.

In this paper, we are concerned with the stabilizatbility of Stackelberg game-based systems. In particular, two players are involved in the system where one is the follower to minimize the related cost function and the other is the leader to stabilize the system. The main contribution is to derive the necessary and sufficient condition for the stabilization of the game-based system. The key technique is to explicitly solve the forward and backward difference equations (FBDEs) based on the maximum principle and give the optimal feedback gain matrix of the leader by using the matrix maximum principle.

This paper studies optimal control and stabilization problems for continuous-time mean-field systems with input delay, which are the fundamental development of control and stabilization problems for mean-field systems. There are two main contributions: 1) To the best of our knowledge, the present paper is first to establish the necessary and sufficient solvability condition for this kind of optimal control problem with delay, and to derive an optimal controller through overcoming the obstacle that separation principle no longer holds for multiplicative-noise systems; 2) For the stabilization problem, under the assumption of exact observability, we strictly prove thatthe system is stabilizable if and only if the algebraic Riccati equation has a unique positive definite solution.

This paper is concerned with the linear quadratic optimal control problem for networked system simultaneously with input delay and Markovian dropout. Different from the results in the literature, we consider the hold-input strategy, which is much more computationally complicated than zero-input strategy, but much better in most cases especially in the transition phase. Necessary and sufficient conditions for the solvability of optimal control problem over a finite horizon are given by the coupled difference Riccati-type equations. Moreover, the networked control system is mean-square stability if and only if the coupled algebraic Riccati-type equations have a particular solution. The key technique in this paper is to tackle the forward and backward difference equations, which are more difficult to be dealt with, due to the adaptability of controller and the temporal correlation caused by simultaneous input delay and Markovian jump.

This paper studies uniform stabilization and social optimality for linear quadratic (LQ) mean field control problems with multiplicative noise, where agents are coupled via dynamics and individual costs. The state and control weights in cost functionals are not limited to be positive semi-definite. This leads to an indefinite LQ mean field control problem, which may still be well-posed due to deep nature of multiplicative noise. We first obtain a set of forward-backward stochastic differential equations (FBSDEs) from variational analysis, and construct a feedback control by decoupling the FBSDEs. By using solutions to two Riccati equations, we design a set of decentralized control laws, which is further shown to be asymptotically social optimal. Some equivalent conditions are given for uniform stabilization of the systems with the help of linear matrix inequalities. A numerical example is given to illustrate the effectiveness of the proposed control laws., Comment: arXiv admin note: text overlap with arXiv:1904.07522, arXiv:2009.09864

This paper is concerned with the decoupling of delayed linear forward-backward stochastic differential equations (D-FBSDEs), which is much more involved than the delay-free case due to the infinite dimension caused by the delay. A new approach of `discretization' is proposed to obtain the explicit solution to the D-FBSDEs. Firstly, we transform the continuous-time D-FBSDEs into the discrete-time form by using discretization. Secondly, we derive the solution of the discrete-time D-FBSDEs by applying backward iterative induction. Finally the explicit solution of the continuous-time D-FBSDEs is obtained by taking the limit to the solution of discrete-time form. The proposed approach can be applied to solve more general FBSDEs with delay, which would provide a complete solution to the stochastic LQ control with time delay., Comment: 16 pages

In this paper, the optimal local and remote linear quadratic (LQ) control problem is studied for a networked control system (NCS) which consists of multiple subsystems and each of which is described by a general multiplicative noise stochastic system with one local controller and one remote controller. Due to the unreliable uplink channels, the remote controller can only access unreliable state information of all subsystems, while the downlink channels from the remote controller to the local controllers are perfect. The difficulties of the LQ control problem for such a system arise from the different information structures of the local controllers and the remote controller. By developing the Pontyagin maximum principle, the necessary and sufficient solvability conditions are derived, which are based on the solution to a group of forward and backward difference equations (G-FBSDEs). Furthermore, by proposing a new method to decouple the G-FBSDEs and introducing new coupled Riccati equations (CREs), the optimal control strategies are derived where we verify that the separation principle holds for the multiplicative noise NCSs with packet dropouts. This paper can be seen as an important contribution to the optimal control problem with asymmetric information structures.

This paper is concerned with uniform stabilization and social optimality for general mean field linear quadratic control systems, where subsystems are coupled via individual dynamics and costs, and the state weight is not assumed with the definiteness condition. For the finite-horizon problem, we first obtain a set of forward-backward stochastic differential equations (FBSDEs) from variational analysis, and construct a feedback-type control by decoupling the FBSDEs. For the infinite-horizon problem, by using solutions to two Riccati equations, we design a set of decentralized control laws, which is further proved to be asymptotically social optimal. Some equivalent conditions are given for uniform stabilization of the systems in different cases, respectively. Finally, the proposed decentralized controls are compared to the asymptotic optimal strategies in previous works., Comment: arXiv admin note: substantial text overlap with arXiv:1904.07522

Irregular linear quadratic control (LQ, was called Singular LQ) has been a long-standing problem since 1970s. This paper will show that an irregular LQ control (deterministic) is solvable (for arbitrary initial value) if and only if the LQ cost can be rewritten as a regular one by changing the terminal cost $x'(T)Hx(T)$ to $x'(T)[H+P_1(T)]x(T)$, while the optimal controller can achieve $P_1(T)x(T)=0$ at the same time. In other words, the irregular controller (if exists) needs to do two things at the same time, one thing is to minimize the cost and the other is to achieve the terminal constraint $P_1(T)x(T)=0$, which clarifies the essential difference of irregular LQ from the standard LQ control where the controller is to minimize the cost only. With this breakthrough, we further study the irregular LQ control for stochastic systems with multiplicative noise. A sufficient solving condition and the optimal controller is presented based on Riccati equations., Comment: 10 pages, 0 figure

This paper considers the optimal decentralized control for networked control systems (NCSs) with asymmetric partial information sharing between two controllers. In this NCSs model, the controller 2 (C2) shares its observations and part of its historical control inputs with the controller 1 (C1), whereas C2 cannot obtain the information of C1 due to network constraints. We present the optimal estimators for C1 and C2 respectively based on asymmetric observations. Since the information for C1 and C2 are asymmetric, the estimation error covariance (EEC) is coupled with the controller which means that the classical separation principle fails. By applying the Pontryagin's maximum principle, we obtain a solution to the forward and backward stochastic difference equations. Based on this solution, we derive the optimal controllers to minimize a quadratic cost function. Combining the optimal controllers with the EEC, the controller C1 is decoupled from the ECC. It should be emphasized that the control gain is dependent on the estimation gain. What's more, the estimation gain satisfies the forward Riccati equation and the control gain satisfies the backward Riccati equation which makes the problem more challenging. We propose iterative solutions to the Riccati equations and give a suboptimal solution to the optimal decentralized control problem.

This paper considers the optimal control and stabilization problems for networked control systems (NCSs) with asymmetric information. In this NCSs model, the remote controller can receive packet-dropout states of the plant, and the available information for the embedded controller are observations of states and packet-dropout states sent from the remote controller. The two controllers operate the plant simultaneously to make the quadratic performance minimized and stabilize the linear plant. For the finite-horizon case, since states of the plant cannot be obtained perfectly, we develop the optimal estimators for the embedded and remote controllers based on asymmetric information respectively. Then we give the necessary and sufficient condition for the optimal control based on the solution to the forward-backward stochastic difference equations (FBSDEs). For the infinite-horizon case, on one hand, the necessary and sufficient condition is given for the stabilization in the mean-square sense of the system without the additive noise. On the other hand, it is shown that the system with the additive noise is bounded in the mean-square sense if and only if there exist the solutions to the two coupled algebraic Riccati equations. Numerical examples on the unmanned underwater vehicle are presented to show the effectiveness of the given algorithm.

In standard linear quadratic (LQ) control, the first step in investigating infinite-horizon optimal control is to derive the stabilization condition with the optimal LQ controller. This paper focuses on the stabilization of an Ito stochastic system with indefinite control and state weighting matrices in the cost functional. A generalized algebraic Riccati equation (GARE) is obtained via the convergence of the generalized differential Riccati equation (GDRE) in the finite-horizon case. More importantly, the necessary and sufficient stabilization conditions for indefinite stochastic control are obtained. One of the key techniques is that the solution of the GARE is decomposed into a positive semi-definite matrix that satisfies the singular algebraic Riccati equation (SARE) and a constant matrix that is an element of the set satisfying certain linear matrix inequality conditions. Using the equivalence between the GARE and SARE, we reduce the stabilization of the general indefinite case to that of the definite case, in which the stabilization is studied using a Lyapunov functional defined by the optimal cost functional subject to the SARE., Comment: 8 pages, 3 figures, This paper has been submitted to Automatica and the current publication decision is Accept provisionally as Brief Paper

The optimal control problem for discrete-time systems with colored multiplicative noise is discussed in this paper. The problem will be more difficult to deal with than the case of white noise due to the correlation of the adjoining state. By solving the forward and backward stochastic difference equations (FBSDEs), the necessary and sufficient conditions for the solvability of the optimal control problems in both delay-free and one-step input delay case are given., Comment: 5 pages, 0 figure

This paper studies mean field games for multi-agent systems with control-dependent multiplicative noises. For the general systems with nonuniform agents, we obtain a set of decentralized strategies by solving an auxiliary limiting optimal control problem subject to consistent mean field approximations. The set of decentralized strategies is further shown to be an $\varepsilon$-Nash equilibrium. For the integrator multiagent systems, we design a set of $\varepsilon$-Nash strategies by exploiting the convexity property of the limiting problem. It is shown that under the mild conditions all the agents achieve mean-square consensus.

In this paper, we study the irregular output feedback linear quadratic (LQ) control problem, which is a continuous work of previous works for irregular LQ control [33] where the state is assumed to be exactly known priori. Different from the classic output feedback LQ control problem, the cost function must be modified to guarantee the solvability. In the framework of the modified cost function, it is shown that the separation principle holds and the explicitly optimal controller is given in the feedback form of the Kalman filtering. In particular, the feedback gain is calculated by two Riccati equations independently of the Kalman filtering. The key technique is the "two-layer optimization" approach. We also emphasize that the optimal controller at the terminal time is required to be deterministic., Comment: 7 pages, 0 figure

This paper studies social optima and Nash games for mean field linear quadratic control systems, where subsystems are coupled via dynamics and individual costs. For the social control problem, we first obtain a set of forward-backward stochastic differential equations (FBSDE) from variational analysis, and construct a feedback-type control by decoupling the FBSDE. By using solutions of two Riccati equations, we design a set of decentralized control laws, which is further proved to be asymptotically social optimal. Two equivalent conditions are given for uniform stabilization of the systems in different cases. For the game problem, we first design a set of decentralized control from variational analysis, and then show that such set of decentralized control constitute an asymptotic Nash equilibrium by exploiting the stabilizing solution of a nonsymmetric Riccati equation. It is verified that the proposed decentralized control laws are equivalent to the feedback strategies of mean field control in previous works. This may illustrate the relationship between open-loop and feedback solutions of mean field control (games).

This paper is concerned with the deterministic optimal control of Ito stochastic systems with random coefficients. The necessary and sufficient conditions for the unique solvability of the optimal control problem with random coefficients are derived via the solution to the coupled stochastic Riccati-type equations. An explicit expression of the deterministic optimal controller for this problem is given. The presented results include the case of deterministic coefficient [14] as special case., Comment: 11 pages, 0 figures

This paper will investigate the infinite horizon optimal control and stabilization problems for the Markov jump linear system (MJLS) subject to control input delay. Different from previous works, for the first time, the necessary and sufficient stabilization conditions are explored under explicit expressions, and the optimal controller for infinite horizon is designed with a coupled algebraic Riccati equation. By introducing a new type of Lyapunov equation, we show that under the exact observability assumption, the MJLS with control input delay is stabilizable in the mean square sense with the optimal controller if and only if a coupled algebraic Riccati equation has a unique positive definite solution. The presented results are parallel to the optimal control and stabilization for standard system with input delay.

In this paper, we mainly investigate an integrated system operating under a software defined network (SDN) protocol. SDN is a new networking paradigm in which network intelligence is centrally administered and data is communicated via channels that are physically separated from those conveying user data. Under the SDN architecture, it is feasible to set up multiple flows for transmitting control signals to an actuator with high priority for each individual application. While each flow may suffer random transmission delay, we focus on the stabilization problem under the joint design of the event-driven strategy in actuator and the control policy in decision-maker. By introducing a predefined application time, the integrated system can be reformulated as the form of stochastic system with input delay and multiplicative noise. For such system, we propose a set of necessary and sufficient stabilization conditions. Specifically, for the scalar system, we derive the allowable sampling period bound that can guarantee stabilization in terms of the probability distributions of the random transmission delays. A simple example is included to show the performance of our theoretic results.

The mean square stabilization problem for discrete-time networked control systems (NCSs) is investigated in this article. What the difference from most previous works is that input delay and packet losses occur simultaneously in the communication channel, moreover, the data packet dropout is modeled as a time-homogeneous Markov process which will bring some difficulties in solving the problem due to the temporal correlation. The contributions in this paper can be summarized as two points. Firstly, the equivalence condition for the solvability of linear quadratic optimal problem in finite horizon subject to the discrete-time NCSs is expressed by solving the forward and backward stochastic difference equations (FBSDEs-M) which is derived from the maximum principle involving Markov jump and delay. Secondly, under basic assumption, the necessary and sufficient condition of mean square stabilization is given by the solutions to the coupled algebraic Riccati equations with Markov jump (CAREs-M). To our best knowledge, the problems studied in this paper are new because most previous works mainly discussed the case of only delay or packet dropout in NCSs., Comment: 12 pages, 6 figures

This paper deals with the finite horizon optimal control problem for discrete-time Markov jump linear system with input delay. The correlation among the jumping parameters and the input delay are considered simultaneously, which forms the basic difficulty of the design. one of the key techniques is to solve a delayed forward and backward jumping parameter difference equation which is obtained by an improved maximum principle, and the other is the introduction of a "d-step backward formula". Based on the proposed techniques, a necessary and sufficient condition for the existence of the optimal controller is given in an explicit form and an analytical solution to the optimal controller is supplied. The optimal controller is a linear function of the current time state and the historical time control input, where the feedback gains are a set of jumping parameter matrices derived by solving a new type of coupled difference Riccati equation. The key step in the derivation is to establish the relationship between the costate and the real state of the system. The result obtained in this paper can be viewed as a generalization of the standard case, in which there is only one mode of operation., Comment: arXiv admin note: text overlap with arXiv:1803.05121

In this paper, we study the stabilization problem for the Ito systems with both multiplicative noise and multiple delays which exist widely in applications such as networked control systems. Sufficient and necessary conditions are obtained for the exponential stabilization problem of Ito stochastic systems with multiple delays. On one hand, we derive the solvability of the modified Riccati equation in case of the mean-square exponential stabilization. On the other hand, the mean-square exponential stabilization is guaranteed by the solvability of a modified Riccati equation. A novel stabilizing controller is shown in the feedback from of the conditional expectation in terms of the modified algebraic Riccati equation. The main technique is to reduce the original system with multiple delays to a pseudo delay-free system., Comment: 14 pages, 1 figure

This paper considers the consensusability of multi-agent systems with delay and packet dropout. By proposing a kind of predictor-like protocol, sufficient and necessary conditions are given for the mean-square consensusability in terms of system matrices, time delay, communication graph and the packetdrop probability. Moreover, sufficient and necessary conditions are also obtained for the formationability of multi-agent systems., Comment: 9 pages, 1 figure

The consensus control with optimal cost remains major challenging although consensus control problems have been well studied in recent years. In this paper, we study the consensus control of multi-agent system associated with a given cost function. The main contribution is to present the distributed control protocol while minimizing the given cost function with positive semi-definite weighting matrix of control. The derived controller is composed of two parts: One part is the feedback of the individual state which minimizes the cost function and the other part is the feedback of the relative states between the neighbours which guarantees the consensus. The presented results are new to the best of our knowledge., Comment: 6 pages, 2 figures

It is well known that stability is the most fundamental nature with regard to a control system, in view of this, the stabilization becomes an inevitable control problem. This article mainly discusses the optimal control and stabilization problem for discrete-time systems involving Markov jump and multiplicative noise. The state and control weighting matrices in the cost function are allowed to be indefinite. By solving the forward-backward stochastic difference equations with Markov jump (FBSDEs-MJ) derived from the maximum principle, we conclude that the necessary and sufficient conditions of the solvability of indefinite optimal control problem in finite-horizon, whose method is different from most previous works [13], etc. Furthermore, necessary and sufficient conditions that stabilize the Markov jump discrete- time systems in the mean square sense with indefinite weighting matrices in the cost are first developed under the basic assumption of exactly observable, which is different from the previous works [12], [14] where an additional assumption of stabilization of systems is made. The key points of this article can be summed up as that an analytic solution to FBSDEs- MJ which makes the optimal controller to be explicitly expressed and the method of trans- formation, i.e., the stabilization problem of indefinite case is boiled down to a definite one whose stabilization is expressed by defining Lyapunov function via the optimal cost subject to a new algebraic Riccati equation involving Markov jump (NGARE-MJ).

This paper mainly investigates the optimal control and stabilization problems for linear discrete-time Markov jump systems. The general case for the finite-horizon optimal controller is considered, where the input weighting matrix in the performance index is just required to be positive semi-definite. The necessary and sufficient condition for the existence of the optimal controller in finite-horizon is given explicitly from a set of coupled difference Riccati equations (CDRE). One of the key techniques is to solve the forward and backward stochastic difference equation (FDSDE) which is obtained by the maximum principle. As to the infinite-horizon case, we establish the necessary and sufficient condition to stabilize the Markov jump linear system in the mean square sense. It is shown that the Markov jump linear system is stabilizable under the optimal controller if and only if the associated couple algebraic Riccati equation (CARE) has a unique positive solution. Meanwhile, the optimal controller and optimal cost function in infinite-horizon case are expressed explicitly.

This paper is concerned with the distributed control and stabilization problems for linear discrete-time large scale systems with imposed constraints. The main contributions of this paper are: Firstly, by using the maximum principle (necessary condition) for the finite horizon optimal control developed in this paper, the optimal distributed controller is thus derived, which can be easily calculated; Secondly, by defining the Lyapunov function with the optimal cost function, we show that the systems with imposed constraints can be stabilized by the optimal distributed controller for the infinite horizon case. The main techniques adopted in this paper are the maximum principle and the solution to the forward and backward difference equations (FBDE), which are the basis in solving distributed control and stabilization problems for linear systems with constraints.

As it is popular known, Riccati equation is the key basic tool for optimal control in the modern control theory. The solvability conditions of optimal control, stabilization conditions and controller design are all based on the Riccati equation. However, these results highly depends on a strictly assumption that the Riccati equation is regular. If the Riccati equation is irregular, the controller could not be derived from the equilibrium condition. This paper is concerned with the general stochastic linear quadratic (LQ) with irregular Riccati equation. Different from the classical control theory for regular LQ problems, a new approach of `multi-layer optimization' is proposed. With the approach, we show that different controller entries of irregular-LQ controller need to be derived from different equilibrium conditions and a specified terminal constraint condition in different layers, which is much different from the classical regular LQ control where all the controller entries can be obtained from equilibrium condition in one layer based on regular Riccati equation. The presented results clarify the differences of open-loop control from closed-loop control in the aspects of solvability and controller design and also explores in essentially the differences of regular control from irregular control. Several examples are presented to show the effectiveness of the proposed approach., Comment: 17 pages, 0 figure

In this paper, we solve the long-standing fundamental problem of irregular linear--quadratic (LQ) optimal control, which has received significant attention since the 1960s. We derive the optimal controllers via the key technique of finding the analytical solutions to two different forward and backward differential equations (FBDEs). We give a complete solution to the finite-horizon irregular LQ control problem using a new `two-layer optimization' approach. We also obtain the necessary and sufficient condition for the existence of optimal and stabilizing solutions in the infinite-horizon case in terms of solutions to two Riccati equations and the stabilization of one specific system. For the first time, we explore the essential differences between irregular and standard LQ control, making a fundamental contribution to classical LQ control theory. We show that irregular LQ control is totally different from regular control as the irregular controller must guarantee the terminal state constraint of $P_1(T)x(T)=0$., Comment: 14 pages, 0 figure

A discrete-time stochastic LQ problem with multiplicative noises and state transmission delay is studied in this paper, which does not require any definiteness constraint on the cost weighting matrices. From some abstract representations of the system and cost functional, the solvability of this LQ problem is characterized by some conditions with operator form. Based on these, necessary and sufficient conditions are derived for the case with a fixed time-state initial pair and the general case with all the time-state initial pairs. For both cases, a set of coupled discrete-time Riccati-like equations can be derived to characterize the existence and the form of the delayed optimal control. In particular, for the general case with all the initial pairs, the existence of the delayed optimal control is equivalent to the solvability of the Riccati-like equations with some algebraic constraints, and both of them are also equivalent to the solvability of a set of coupled linear matrix equality-inequalities. Note that both the constrained Riccati-like equations and the linear matrix equality-inequalities are introduced for the first time in the literature for the proposed LQ problem. Furthermore, the convexity and the {uniform convexity} of the cost functional are fully characterized via certain properties of the solution of the Riccati-like equations.

This paper investigates the stabilization and control problems for linear continuous-time mean-field systems (MFS). Under standard assumptions, necessary and sufficient conditions to stabilize the mean-field systems in the mean square sense are explored for the first time. It is shown that, under the assumption of exact detectability (exact observability), the mean-field system is stabilizable if and only if a coupled algebraic Riccati equation (ARE) admits a unique positive semi-definite solution (positive definite solution), which coincides with the classical stabilization results for standard deterministic systems and stochastic systems. One of the key techniques in the paper is the obtained solution to the forward and backward stochastic differential equation (FBSDE) associated with the maximum principle for an optimal control problem. Actually, with the analytical FBSDE solution, a necessary and sufficient solvability condition of the optimal control, under mild conditions, is derived. Accordingly, the stabilization condition is presented by defining an Lyaponuv functional via the solution to the FBSDE and the optimal cost function. It is worth of pointing out that the presented results are different from the previous works for stabilization and also different from the works on optimal control., Comment: 16 pages, 3 figures

Different from most of the previous works, this paper provides a thorough solution to the fundamental problems of linear-quadratic (LQ) control and stabilization for discrete-time mean-field systems under basic assumptions. Firstly, the sufficient and necessary condition for the solvability of mean-field LQ control problem is firstly presented in analytic expression based on the maximum principle developed in this paper, which is compared with the results obtained in literatures where only operator type solvability conditions were given. The optimal controller is given in terms of a coupled Riccati equation which is derived from the solution to forward and backward stochastic difference equation (FBSDE). Secondly, the sufficient and necessary stabilization conditions are explored. It is shown that, under exactly observability assumption, the mean-field system is stabilizable in mean square sense if and only if a coupled algebraic Riccati equation (ARE) has a unique solution $P$ and $\bar{P}$ satisfying $P>0$ and $P+\bar{P}>0$. Furthermore, under the exactly detectability assumption, which is a weaker assumption than exactly observability, we show that the mean-field system is stabilizable in mean square sense if and only if the coupled ARE has a unique solution $P$ and $\bar{P}$ satisfying $P\geq 0$ and $P+\bar{P}\geq 0$. The key techniques adopted in this paper are the maximum principle and the solution to the FBSDE obtained in this paper. The derived results in this paper forms the basis to solve the mean-field control problem for continuous-time systems and other related problems., Comment: 20 pages, 3 figures

This paper is concerned with a stochastic recursive optimal control problem with time delay, where the controlled system is described by a stochastic differential delayed equation (SDDE) and the cost functional is formulated as the solution to a backward SDDE (BSDDE). When there are only the pointwise and distributed time delays in the state variable, a generalized Hamilton-Jacobi-Bellman (HJB) equation for the value function in finite dimensional space is obtained, applying dynamic programming principle. This generalized HJB equation admits a smooth solution when the coefficients satisfy a particular system of first order partial differential equations (PDEs). A sufficient maximum principle is derived, where the adjoint equation is a forward-backward SDDE (FBSDDE). Under some differentiability assumptions, the relationship between the value function, the adjoint processes and the generalized Hamiltonian function is obtained. A consumption and portfolio optimization problem with recursive utility in the financial market, is discussed to show the applications of our result. Explicit solutions in a finite dimensional space derived by the two different approaches, coincide., Comment: 34pages; Presented in Fourth IMS-FPS Workshop, July 3, 2014, Sydney, Australia