Search

In dynamic outdoor environments characterized by turbulent airflow and intermittent odor plumes, robotic odor plume tracking remains challenging, because existing algorithms heavily rely on manually tuning or learning from expert experience, which are hard to implement in an unknown environment. In this paper, a multi-continuous-output Takagi–Sugeno–Kang fuzzy system was designed and tuned with reinforcement learning to solve the robotic odor source localization problem in dynamic odor plumes. Based on the Lévy Taxis plume tracking controller, the proposed fuzzy system determined the parameters of the controller based on the robot's observation and guided the robot to turn and move towards the odor source at each searching step. The trained fuzzy system was tested in simulated filament-based odor plumes dispersed by a changing wind field. The results showed that the performance of the proposed fuzzy system-based controller trained with reinforcement learning can achieve a similar success rate and higher efficiency compared with a manually tuned and well-designed fuzzy system-based controller. The fuzzy system-based plume tracking controller was also validated through real robotic experiments. [ABSTRACT FROM AUTHOR]

Model-free reinforcement learning methods have successfully been applied to practical applications such as decision-making problems in Atari games. However, these methods have inherent shortcomings, such as a high variance and low sample efficiency. To improve the policy performance and sample efficiency of model-free reinforcement learning, we propose proximal policy optimization with model-based methods (PPOMM), a fusion method of both model-based and model-free reinforcement learning. PPOMM not only considers the information of past experience but also the prediction information of the future state. PPOMM adds the information of the next state to the objective function of the proximal policy optimization (PPO) algorithm through a model-based method. This method uses two components to optimize the policy: the error of PPO and the error of model-based reinforcement learning. We use the latter to optimize a latent transition model and predict the information of the next state. For most games, this method outperforms the state-of-the-art PPO algorithm when we evaluate across 49 Atari games in the Arcade Learning Environment (ALE). The experimental results show that PPOMM performs better or the same as the original algorithm in 33 games. [ABSTRACT FROM AUTHOR]

Fuzzy inference systems have been widely applied in robotic control. Previous studies proposed various methods to tune the fuzzy rules and the parameters of the membership functions (MFs). Training the systems with only supervised learning requires a large amount of input-output data, and the performance of the trained system is confined by that of the target system. Training the systems with only reinforcement learning (RL) does not require prior knowledge but is time-consuming, and the initialization of the system remains a problem. In this paper, a supervised-reinforced successive training framework is proposed for a multi-continuous-output fuzzy inference system (MCOFIS). The parameters of the fuzzy inference system are first tuned by a limited number of input-output data from an existing controller with supervised training and then are utilized to initialize the system in the reinforcement training stage. The proposed framework is applied in a robotic odor source searching task and the evaluation results demonstrate that the performance of the fuzzy inference system trained by the successive framework is superior to the systems trained by only supervised learning or RL. The system trained by the proposed framework can achieve around a 10% higher success rate compared to the systems trained by only supervised learning or RL. [ABSTRACT FROM AUTHOR]

Environmental information plays an important role in deep reinforcement learning (DRL). However, many algorithms do not pay much attention to environmental information. In multi-agent reinforcement learning decision-making, because agents need to make decisions combined with the information of other agents in the environment, this makes the environmental information more important. To prove the importance of environmental information, we added environmental information to the algorithm. We evaluated many algorithms on a challenging set of StarCraft II micromanagement tasks. Compared with the original algorithm, the standard deviation (except for the VDN algorithm) was smaller than that of the original algorithm, which shows that our algorithm has better stability. The average score of our algorithm was higher than that of the original algorithm (except for VDN and COMA), which shows that our work significantly outperforms existing multi-agent RL methods. [ABSTRACT FROM AUTHOR]