Online reinforcement learning with passivity-based stabilizing term for real time overhead crane control without knowledge of the system model.

Due to the existing uncertainties such as the payload mass and unmodeled dynamics in the overhead crane system, classical model-based control methods yielding fixed control gain can exhibit certain limitations. In this study, a novel model-free online Reinforcement Learning (RL) control method is proposed for the real-time overhead crane position regulation and anti-swing control problem, combining the benefits of adaptive control and optimal control. The crane control problem is first formulated as an optimal regulation problem with a user-specified objective function. Then, an improved neural-network updating rule with an additional passivity-based stabilizing term is developed to ensure the system stability during learning based on the overhead crane system passivity analysis. The proposed method, unlike other online RL algorithms, does not require the initial stabilizing control policy or prior knowledge of the crane mathematical model. Lyapunov approach is used to prove the closed-loop system stability, and the learned controller is shown to be near-optimal within a finite bound. Finally, simulation studies are conducted to demonstrate the effectiveness of the proposed method in the presence of system parameter variations and external disturbances, exhibiting satisfactory performance when compared to LQR and passivity-based control. • An online RL method is proposed for real-time overhead crane optimal control without knowing the crane system model. • A stabilizing term is added to the network updating rule based on the crane system passivity analysis, which can relax the initial stabilizing controller requirement. • The closed-loop system stability and the RL algorithm convergence are proved through the Lyapunov approach. [ABSTRACT FROM AUTHOR]