Search

This paper presents a model-free approximation for the Hessian of the performance of deterministic policies to use in the context of Reinforcement Learning based on Quasi-Newton steps in the policy parameters. We show that the approximate Hessian converges to the exact Hessian at the optimal policy, and allows for a superlinear convergence in the learning, provided that the policy parametrization is rich. The natural policy gradient method can be interpreted as a particular case of the proposed method. We analytically verify the formulation in a simple linear case and compare the convergence of the proposed method with the natural policy gradient in a nonlinear example., This paper has been accepted to 2022 American Control Conference (ACC). 6 pages

The aim of this paper is to propose a high performance control approach for trajectory tracking of Autonomous Underwater Vehicles (AUVs). However, the controller performance can be affected by the unknown perturbations including model uncertainties and external time-varying disturbances in an undersea environment. To address this problem, a Backstepping-based Integral Sliding Mode Control with Time Delay Estimation (BS-ISMC-TDE) is designed. To improve the performance of a conventional backstepping control algorithm, an Integral Sliding Mode Control (ISMC) approach is adopted in the backstepping design to attenuate the steady-state error. Moreover, an adaptive Time Delay Estimation (TDE) strategy is proposed to provide an estimation of perturbations by observing the inputs and the states of the AUV one step into the past without an exact knowledge of the dynamics and the upper bound of uncertainties. From the simulation results, it is shown that the proposed control approach using both adaptive and conventional TDE parts outperforms a Backstepping-based Integral Sliding Mode Control (BS-ISMC)., This paper was accepted to 20th International Conference on ADVANCED ROBOTICS 2021

In this paper, we discuss the deterministic policy gradient using the Actor-Critic methods based on the linear compatible advantage function approximator, where the input spaces are continuous. When the policy is restricted by hard constraints, the exploration may not be Centred or Isotropic (non-CI). As a result, the policy gradient estimation can be biased. We focus on constrained policies based on Model Predictive Control (MPC) schemes and to address the bias issue, we propose an approximate Robust MPC approach accounting for the exploration. The RMPC-based policy ensures that a Centered and Isotropic (CI) exploration is approximately feasible. A posterior projection is used to ensure its exact feasibility, we formally prove that this approach does not bias the gradient estimation., This paper has been accepted to ECC 2021. 6 pages

We present a Reinforcement Learning-based Robust Nonlinear Model Predictive Control (RL-RNMPC) framework for controlling nonlinear systems in the presence of disturbances and uncertainties. An approximate Robust Nonlinear Model Predictive Control (RNMPC) of low computational complexity is used in which the state trajectory uncertainty is modelled via ellipsoids. Reinforcement Learning is then used in order to handle the ellipsoidal approximation and improve the closed-loop performance of the scheme by adjusting the MPC parameters generating the ellipsoids. The approach is tested on a simulated Wheeled Mobile Robot (WMR) tracking a desired trajectory while avoiding static obstacles., This paper has been accepted to 2021 European Control Conference (ECC)