Search

• A surrogate model based on IRL is suggested, which applies the max-margin method to approximate the reward function. • Under different dynamic environments, IRL learns the reward function and helps accelerate the convergence of the algorithm. • On the basis of learned reward function, IRL-DMOEA generates promising non-dominated solutions during the evolutionary process. Due to the widespread interest in dynamic multiobjective optimization in real-world applications, more and more approaches exploiting machine learning are deployed to tackle this type of problems. Unfortunately, recent works do not make full use of the data obtained during the optimization process, which could be benefit for model training thereby mining the dynamic characteristics of the underlying problem. To address this issue, this paper proposes a dynamic multiobjective evolutionary algorithm driven by inverse reinforcement learning to solve the dynamic multiobjective optimization problems. IRL is widely used to recover the unknown reward function, making it possible to perform at an expert level. The notable features of the proposed algorithm mainly consist of data-driven evolutionary technique, which uses inverse reinforcement learning as a surrogate-assisted model for model training. This design makes full use of the surrogate management strategy based on inverse reinforcement learning to optimize the reward function within a reinforcement learning framework. At the same time, the algorithm can generate a promising policy based on limited training data during the optimization process to achieve better algorithm evolution and guide the search. The experimental results on the benchmark problems validate that the proposed algorithm is effective in dealing with dynamic multiobjective optimization. [ABSTRACT FROM AUTHOR]

In inverse reinforcement learning (IRL), a reward function is learnt to generalize experts' behavior. This paper proposes a model-free IRL algorithm based on an ensemble method, where the reward function is regarded as a parametric function of expected features. In other words, the parameters are updated based on a weak classification method. The IRL is formulated as a problem of a boosting classifier, akin to the renowned Adaboost algorithm for classification, feature expectations from experts' demonstration, and the trajectory induced by an agent's current policy. The proposed approach takes individual feature expectation as attractor or expeller, depending on the sign of the residuals of the state trajectories between expert's demonstration and the one induced by RL with the currently approximated reward function, so as to tackle its central challenges of accurate inference, generalizability, and correctness of prior knowledge. Then, the proposed method is applied further to approximate an abstract reward function from observations of more complex behavior composed of several basic actions. The results of the simulations in a labyrinth are shown to validate the proposed algorithm. Furthermore, behaviors composed of a set of primitive actions on a soccer robot field are examined for the applicability of the proposed method. [ABSTRACT FROM AUTHOR]