Estimating Biped Gait Using Spline-Based Probability Distribution Function With Q-Learning.

This paper studies the probability distribution functions of the parameters to be learned and optimized in biped gait generation. By formulating the gait pattern generation into a multiobjective optimization problem with consideration of geometric and state constraints, dynamically stable and low energy cost biped gaits are generated and optimized by the proposed method, namely Spline-based Estimation of Distribution Algorithm (EDA) with Q-learning updating rule (EDA̱S̱Q). Instead of assuming variables as independent ones, the relationship between them is exploited by formulating the corresponding probability models with the Catmull-Rom cubic spline function. Such kind of function is proved to be a suboptimal and adaptive realization of the cubic spline function and is capable of providing high- precision description. Moreover, the probability models are up- dated autonomously by Q-learning method, which is model-free and adaptive. Thus, EDA̱S̱Q can deal with complex probability distribution functions without a prior knowledge about the distribution. The biped gait generated by EDA̱S̱Q has been verified using the simulation model of a humanoid soccer robot Robo-Erectus. It also shows that EDA̱S̱Q can generate the desired biped gaits autonomously in short learning epochs. An interpretation of the transition probability distribution achieved by EDA̱S̱Q provides us easy understanding for biped locomotion and better control in humanoid robots. [ABSTRACT FROM AUTHOR]