Echo state network-based online optimal control for discrete-time nonlinear systems.

• This article develops an online ADP method for discrete nonlinear systems, it can reduce the computation burden. • Two ESNs are employed to implement the online learning and the output weights are obtained simultaneously. • The hidden layer of ESN is generated randomly rather than designed with much effort, which reduces the application complexity. This paper investigates the online optimal control problem of discrete-time nonlinear systems using echo state network (ESN)-based adaptive dynamic programming (ADP) method. An online iterative learning algorithm is proposed to solve the partial differential Hamilton–Jacobi–Bellman (HJB) equation in real time. A novel neural networks (NN) critic-actor architecture is presented using two ESNs to implement the ADP method. Then, two online learning laws of the output weights are designed for searching the optimal cost function and control policy. The stability of system and output weights is analysed using Lyapunov approach. Three simulations are given to show the feasibility and effectiveness of the designed algorithm. [ABSTRACT FROM AUTHOR]