Search

This article investigates optimal control problem for a class of discrete-time (DT) nonlinear systems with unknown dynamics. With the help of a broad learning system (BLS), a novel online adaptive dynamic programming (ADP) controller is presented. First, to approximate the unknown system dynamics, an approximator based on BLS is presented. The connection weights are calculated by the data of the system by using the ridge regression algorithm. Then, two BLSs are adopted to approximate the optimal cost function and optimal control law, respectively. The connection weights of these two BLSs are updated using the given weights tuning law at each sampling instant. The proposed optimal controller is proved to ensure that all the system states and estimation errors are uniform ultimate bounded. Finally, simulation examples are carried out to further demonstrate the effectiveness of the proposed BLS-based approximator and optimal controller. [ABSTRACT FROM AUTHOR]

We consider decentralized large-scale continuous submodular constrained optimization problems over networks, where the goal is to maximize a sum of nonconvex functions with diminishing returns property. However, the computations of the projection step and the whole gradient can become prohibitive in high-dimensional constrained optimization problems. For this reason, a decentralized randomized block-coordinate Frank-Wolfe algorithm is proposed for submoduar maximization over networks by local communication and computation, which adopts the randomized block-coordinate descent and the Frank-Wolfe technique. We also show that the proposed algorithm converges to an approximation fact $(1-e^{-p_{\max }/p_{\min }})$ of the global maximal points at a rate of $\mathcal {O}(1/T)$ by choosing a suitable stepsize, where $T$ is the number of iterations. In addition, we confirm the theoretical results by experiments. [ABSTRACT FROM AUTHOR]

The minimum dominating set (MDS) of the network is a node subset of smallest size that every node in the network is either in this subset or is adjacent to one or more nodes of this subset. MDS has found wide applications, ranging from network monitoring, routing, to epidemic control, and text processing. However, the majority of existing studies on MDS problem are confined to single networks. In real world, more and more complex systems consist of a set of elements linked up by different types of connections, which are best modeled as multiplex networks with interacting network layers. Though vastly important, the MDS of the multiplex networks has not yet been formally defined and its application and identification remain open issues. In this article, we present the definition of the MDS of the multiplex network and show some of its possible applications. For solving the MDS problem of the multiplex network, we built a spin-glass model and solve it through the belief-propagation (BP) equations under the replica symmetry mean-field theory. As a consequence, we can predict the relative size of the MDS of the multiplex network theoretically and we can propose a BP-guided decimation algorithm to construct an approximate optimal dominating set in practice. Then the algorithm is improved in both accuracy and efficiency by embedding a novel multiplex network-oriented leaf-removal strategy. The effectiveness of the proposed algorithms is finally verified by comparing with other methods on a number of the multiplex network examples. [ABSTRACT FROM AUTHOR]

Path tracking control of intelligent vehicles has to deal with the difficulties of model uncertainties and nonlinearities. As a class of adaptive optimal control methods, reinforcement learning (RL) has received increasing attention in solving difficult control problems. However, feature representation and online learning ability are two major problems to be solved for learning control of uncertain dynamic systems. In this article, we propose a multi-kernel online RL approach for path tracking control of intelligent vehicles. In the proposed approach, a multiple kernel feature learning framework is designed for online learning control based on dual heuristic programming (DHP) and the new online learning control algorithm is called multi-kernel DHP (MKDHP). In MKDHP, instead of the expert knowledge for selecting and fine-tuning of a suitable kernel function, only a set of basic kernel functions is required to be predefined and the multi-kernel features can be learned for value function approximation in the critic. The simulation studies on path tracking control for intelligent vehicles have been conducted under $S$ -curve and urban road conditions. The results demonstrated that compared with other typical path tracking controllers for intelligent vehicles, such as the linear quadratic regulator (LQR), the pure pursuit controller and the ribbon-based controller, the proposed multi-kernel learning controller can achieve better performance in terms of tracking precision and smoothness. [ABSTRACT FROM AUTHOR]

This article investigates the optimal control problem for nonlinear nonzero-sum differential game in the environment of no initial admissible policies while considering the control constraint. An adaptive learning algorithm is thus developed based on policy iteration technique to approximately obtain the Nash equilibrium using real-time data. A two-player continuous-time system is used to present this approximate mechanism, which is implemented as a critic–actor architecture for every player. The constraint is incorporated into this optimization by introducing the nonquadratic value function, and the associated constrained Hamilton–Jacobi equation is derived. The critic neural network (NN) and actor NN are utilized to learn the value function and the optimal control policy, respectively, in the light of novel weight tuning laws. In order to tackle the stability during the learning phase, two stable operators are designed for two actors. The proposed algorithm is proved to be convergent as a Newton’s iteration, and the stability of this closed-loop system is also ensured by Lyapunov analysis. Finally, two simulation examples demonstrate the effectiveness of the proposed learning scheme by considering different constraint scenes. [ABSTRACT FROM AUTHOR]

As a class of batch-mode reinforcement learning (RL) methods for Markov decision problems with large or continuous state spaces, approximate policy iteration (API) has received increasing attention in the past decades. One open problem in the design of API algorithms is how to construct the basis functions or features for value function approximation (VFA). In this paper, we propose a novel batch-mode RL approach with randomly projected features for VFA. The proposed approach can be viewed as an extension of extreme learning machines (ELMs) to RL problems so it can be called ELM-API. The ELMs have been popularly studied in supervised learning problems, but there is not much work on the extension of ELMs to learning control problems. The proposed approach has advantages over the previous API algorithms in that the features for VFA can be quickly generated without complex parameter selection and the performance will be adaptive to different sample sets in batch-mode RL. In particular, the ELM-API approach can realize fast and efficient feature reconstruction when training sample sets are relatively small. Comprehensive simulation studies on two benchmark learning control problems were carried out to test the performance of API algorithms with different feature construction methods. It is shown that the ELM-API algorithm can obtain comparable or better performance than the previous API approaches. To further show the effectiveness of ELM-API in real-world applications, the simulation results on a more challenging high-dimensional lane-changing decision problem in dynamic traffic environment are also reported, which show the capability of the ELM-API algorithm in learning satisfactory lane-changing policies with high data efficiency. [ABSTRACT FROM AUTHOR]

Nonlinear optimization problems with dynamical parameters are widely arising in many practical scientific and engineering applications, and various computational models are presented for solving them under the hypothesis of short-time invariance. To eliminate the large lagging error in the solution of the inherently dynamic nonlinear optimization problem, the only way is to estimate the future unknown information by using the present and previous data during the solving process, which is termed the future dynamic nonlinear optimization (FDNO) problem. In this paper, to suppress noises and improve the accuracy in solving FDNO problems, a novel noise-tolerant neural (NTN) algorithm based on zeroing neural dynamics is proposed and investigated. In addition, for reducing algorithm complexity, the quasi-Newton Broyden–Fletcher–Goldfarb–Shanno (BFGS) method is employed to eliminate the intensively computational burden for matrix inversion, termed NTN-BFGS algorithm. Moreover, theoretical analyses are conducted, which show that the proposed algorithms are able to globally converge to a tiny error bound with or without the pollution of noises. Finally, numerical experiments are conducted to validate the superiority of the proposed NTN and NTN-BFGS algorithms for the online solution of FDNO problems. [ABSTRACT FROM AUTHOR]

This article reviews the recent development of adaptive dynamic programming (ADP) with applications in control. First, its applications in optimal regulation are introduced, and some skilled and efficient algorithms are presented. Next, the use of ADP to solve game problems, mainly nonzero-sum game problems, is elaborated. It is followed by applications in large-scale systems. Note that although the functions presented in this article are based on continuous-time systems, various applications of ADP in discrete-time systems are also analyzed. Moreover, in each section, not only some existing techniques are discussed, but also possible directions for future work are pointed out. Finally, some overall prospects for the future are given, followed by conclusions of this article. Through a comprehensive and complete investigation of its applications in many existing fields, this article fully demonstrates that the ADP intelligent control method is promising in today’s artificial intelligence era. Furthermore, it also plays a significant role in promoting economic and social development. [ABSTRACT FROM AUTHOR]

This article presents a novel event-driven guaranteed cost control method for nonlinear systems subject to actuator faults. For the purpose of handling the problem of actuator faults and obtaining the event-driven approximate optimal guaranteed cost control approach for general nonlinear dynamics, the reinforcement learning (RL) algorithm is utilized to develop a sliding-mode control (SMC) strategy. To begin with, the unknown faults can be estimated by designing a fault observer. Meanwhile, an SMC technique is presented aiming at countering the effect of abrupt faults. In addition, the optimal performance of the equivalent sliding mode dynamics is considered, then an event-driven guaranteed cost control mechanism is implemented by using RL principle. In the control process, a general cost function, which has a simpler structure, is given to reduce the computation complexity. At the same time, a modified cost function is approximated to obtain optimal guaranteed cost control by using a single critic neural network (NN). In addition, a modified weight update law for critic NN is presented to relax the persistence of excitation (PE) condition. Moreover, a newly triggering condition, which is easy to be implemented, is designed, and the critic NN update law makes sure that the system states are stable. Furthermore, in light of the Lyapunov analysis, it is demonstrated that the developed event-driven control method guarantees the uniformly ultimately bounded (UUB) property of all the signals. Finally, three simulation results are given to validate the designed control method. [ABSTRACT FROM AUTHOR]

In this paper, a novel adaptive dynamic programming (ADP) algorithm is developed for the infinite-horizon ($H_{\infty}$) optimal control problems with unknown continuous-time (CT) nonlinear systems subject to external disturbances. To facilitate the implementation of the algorithm, generalized fuzzy hyperbolic models (GFHMs) are utilized to establish an identifier–critic architecture, where the identifier is designed to reconstruct the unknown system dynamics, and the GFHM-based critic network is employed to approximate the value functions. The CT $H_{\infty}$ optimal control issue is converted into a two-player zero-sum game and the corresponding Hamilton–Jacobi–Isaacs equation is derived. The learning procedure of the critic design is adaptively implemented with the help of the reconstructed model, thus the requirement of the complete system dynamics is relaxed. Furthermore, by the means of Lyapunov direct method, the uniform ultimate boundedness stability analysis of the closed-loop control system is explicitly provided. Finally, to compare the control performances and disturbance attenuation properties of the proposed method and the existing ADP algorithms, two numerical examples are given. [ABSTRACT FROM AUTHOR]

In this paper, a novel event-sampled robust optimal controller is proposed for a class of continuous-time constrained-input nonlinear systems with unknown dynamics. In order to solve the robust optimal control problem, an online data-driven identifier is established to construct the system dynamics, and an event-sampled critic-only adaptive dynamic programming method is developed to replace the conventional time-driven actor–critic structure. The designed online identification method runs during the solving process and is not applied as a priori part for the solutions, which simplifies the architecture and reduces computational load. The proposed robust optimal control algorithm tunes the parameters of critic-only neural network (NN) by event-triggering condition and runs in a plug-n-play framework without system functions, where fewer transmissions and less computation are required as all the measurements received simultaneously. Based on the novel design, the stability of system and the convergence of critic NN are demonstrated by Lyapunov theory, where the state is asymptotically stable and weight error is guaranteed to be uniformly ultimately bounded. Finally, the applications in a basic nonlinear system and the complex rotational–translational actuator problem demonstrate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Inspired by Nash game theory, a multiplayer mixed-zero-sum (MZS) nonlinear game considering both two situations [zero-sum and nonzero-sum (NZS) Nash games] is proposed in this paper. A synchronous reinforcement learning (RL) scheme based on the identifier-critic structure is developed to learn the Nash equilibrium solution of the proposed MZS game. First, the MZS game formulation is presented, where the performance indexes for players 1 to ${N}$ − 1 and ${N}$ NZS Nash game are presented, and another performance index for players ${N}$ and ${N}$ + 1 zero-sum game is presented, such that player ${N}$ cooperates with players 1 to ${N}$ − 1, while competes with player ${N}$ + 1, which leads to a Nash equilibrium of all players. A single-layer neural network (NN) is then used to approximate the unknown dynamics of the nonlinear game system. Finally, an RL scheme based on NNs is developed to learn the optimal performance indexes, which can be used to produce the optimal control policy of every player such that Nash equilibrium can be obtained. Thus, the widely used actor NN in RL literature is not needed. To this end, a recently proposed adaptive law is used to estimate the unknown identifier coefficient vectors, and an improved adaptive law with the error performance index is further developed to update the critic coefficient vectors. Both linear and nonlinear simulations are presented to demonstrate the existence of Nash equilibrium for MZS game and performance of the proposed algorithm. [ABSTRACT FROM AUTHOR]

This paper presents a parameterized batch reinforcement learning algorithm for near-optimal longitudinal control of autonomous land vehicles (ALVs). The proposed approach uses an actor-critic architecture, where parameterized feature vectors based on kernels are learned from collected samples for approximating the value functions and policies. One difference between the parameterized batch actor-critic (PBAC) algorithm and previous actor-critic learning approaches is that the critic and actor in PBAC share the same linear features, which has been theoretically proved to be a beneficial property for the convergence of actor-critic learning approaches. In order to obtain better learning efficiency, least-squares-based batch updating rules are designed for the critic and actor, respectively. Based on the PBAC learning algorithm, a data-driven longitudinal control method is presented for ALVs to obtain near-optimal control policies which adaptively tune the fuel/brake control signals to track different speeds. A multiobjective reward function is designed so that both tracking precision and driving smoothness are considered. Extensive experiments were conducted on a real ALV platform while driving on flat, slippery, sloping, and bumpy roads. The experimental results illustrate the superiority of the PBAC-based self-learning controller over conventional longitudinal control methods such as proportional-integral (PI) control and learning-based PI control. [ABSTRACT FROM AUTHOR]

We consider a model of competition on complex networks, in which two competitors are fixed to opposite states while other agents, called normal agents, adjust their states according to a distributed consensus protocol. Suppose that one of the competitors could enhance its influence by creating new links. A natural question is, when the number of new links is limited due to the limited resource, how to add these links so as to maximize the influence of the given competitor over the other one (called competitiveness). We consider two competitiveness maximization problems: Problem 1 tries to maximize the number of supporters of the competitor, while Problem 2 tries to maximize the total supporting degree of normal agents toward the competitor. We prove that Problem 1 is NP-hard. We also show that the objective function of Problem 2 is monotonous and submodular, and hence there exists a polynomial-time greedy algorithm (GA) approximately solving Problem 2. Several centrality-based heuristic algorithms of less computational burden are also designed to provide approximate solutions to these two problems. We carry out extensive simulations to check the performances of these algorithms in six real networks. We find that GA always provides the best approximate solution to Problem 2, while for Problem 1, GA only has the best performance in directed networks. Furthermore, among those heuristic algorithms, an algorithm based on centrality in descending order is better than its counterpart in ascending order in solving Problem 2 in statistical sense. But for Problem 1, the performance of the centrality-based heuristic algorithms is more sensitive to the network structure and the locations of competitors. [ABSTRACT FROM AUTHOR]

Decomposed fuzzy system (DFS) is a fuzzy system with a novel structure. Due to its excellent learning performance, DFS is originally proposed for an online learning control scheme and is shown to have effective learning performance. This paper is about the use of DFS for modeling dynamic systems. Since the learning mechanism used in online learning control is not suitable for modeling tasks, a commonly used back propagation learning algorithm is adapted for the use of DFS in modeling dynamic systems. The structure of DFS is to decompose each fuzzy variable into fuzzy subsystems that are called component fuzzy systems. Owing to the independency among component fuzzy systems, the learning for those parameters is also independent among different component fuzzy systems and thus, the learning can become more efficient. From the simulation results, it is evident that the proposed DFS can have much faster convergent speed. In addition, the DFS has a smaller testing error than those of other fuzzy systems. [ABSTRACT FROM PUBLISHER]

In this paper, we present error bound analysis of the Q -function for the action-dependent adaptive dynamic programming for solving discounted optimal control problems of unknown discrete-time nonlinear systems. The convergence of Q -functions derived by a policy iteration algorithm under ideal conditions is given. Considering the approximated errors of the Q -function and control policy in the policy evaluation step and policy improvement step, we establish error bounds of approximate Q -functions in each iteration. With the given boundedness conditions, the approximate Q -function will converge to a finite neighborhood of the optimal Q -function. To implement the presented algorithm, two three-layer neural networks are employed to approximate the Q -function and the control policy, respectively. Finally, a simulation example is utilized to verify the validity of the presented algorithm. [ABSTRACT FROM AUTHOR]

Recently, Shabtay considered a scheduling problem on a single serial-batching machine with rejection to minimize the dual criteria of total completion time and total rejection cost, where the number of jobs to be included in each batch is not restricted. He studied four variants of the problem: the first is to minimize the sum of the two criteria; the second and third are to minimize one criterion, subject to the other criterion not exceeding a given value; and the last is to find the Pareto-optimal solutions for the bicriterion problem. Shabtay provided an O ( \textitn^\mathbf 5 ) algorithm for the first variant and an O ( n ^\mathbf 6/\boldsymbol \varepsilon ^\mathbf 2 ) fully polynomial-time approximation scheme (FPTAS) for the fourth variant. In this paper, we provide an alternative O ( n ^\mathbf 4 ) algorithm to solve the first variant and an O ( n ^\mathbf 5/\boldsymbol \varepsilon ) FPTAS for the fourth variant, which are more efficient than those developed by Shabtay from a theoretical perspective. However, when the size of each batch is bounded by a given number b~\boldsymbol >1 , the corresponding time complexities of our algorithms for the first and fourth variants reduce to O (bn ^\mathbf 3 ) and O (bn ^\mathbf 4/\boldsymbol \varepsilon ), respectively. [ABSTRACT FROM AUTHOR]

In this paper, we develop an online synchronous approximate optimal learning algorithm based on policy iteration to solve a multiplayer nonzero-sum game without the requirement of exact knowledge of dynamical systems. First, we prove that the online policy iteration algorithm for the nonzero-sum game is mathematically equivalent to the quasi-Newton's iteration in a Banach space. Then, a model neural network is established to identify the unknown continuous-time nonlinear system using input-output data. For each player, a critic neural network and an action neural network are used to approximate its value function and control policy, respectively. Our algorithm only needs to tune the weights of critic neural networks, so there will be less computational complexity during the learning process. All the neural network weights are updated online in real-time, continuously and synchronously. Furthermore, the uniform ultimate bounded stability of the closed-loop system is proved based on Lyapunov approach. Finally, two simulation examples are given to demonstrate the effectiveness of the developed scheme. [ABSTRACT FROM AUTHOR]