Search

The application of Reinforcement Learning (RL) algorithms to learn tasks for robots is often limited by the large dimension of the state space, which may make prohibitive its application on a tabular model. In this paper, we describe LEAP (Learning Entities Adaptive Partitioning), a model-free learning algorithm that uses overlapping partitions which are dynamically modified to learn near-optimal policies with a small number of parameters. Starting from a coarse aggregation of the state space, LEAP generates refined partitions whenever it detects an incoherence between the current action values and the actual rewards from the environment. Since in highly stochastic problems the adaptive process can lead to over-refinement, we introduce a mechanism that prunes the macrostates without affecting the learned policy. Through refinement and pruning, LEAP builds a multi-resolution state representation specialized only where it is actually needed. In the last section, we present some experimental evaluation on a grid world and a complex simulated robotic soccer task. [ABSTRACT FROM AUTHOR]

Fuzzy Q-learning extends the Q-learning algorithm to work in presence of continuous state and action spaces. A Takagi-Sugeno Fuzzy Inference System (FIS) is used to infer the continuous executed action and its action-value, by means of cooperation of several rules. Different kinds of evolution of the parameters of the FIS are possible, depending on different strategies of distribution of the reinforcement signal. In this paper, we compare two strategies: the classical one, focusing on rewarding the rules that have proposed the actions composed to produce the actual action, and a new one we are introducing, where reward goes to the rules proposing actions closest the ones actually executed. Keywords: Reinforcement Learning, Fuzzy Q-learning, Fuzzy logic, continuous state-action space, reinforcement distribution. [ABSTRACT FROM AUTHOR]

In this paper, we propose a modeling paradigm that uses fuzzy sets to represent concepts on which control modules of a behavior-based autonomous robot operate. The primitives defined in the modeling paradigm are expressive enough to represent the knowledge needed by planning, coordination, and reactive control of a multi-robot control system. At the same time, it provides a well-founded tool to represent in a compact way the data interpretations, needed to reason effectively about what is happening in the world and what is desired to happen. This modeling paradigm makes the design of behavior, planning, and coordination modules easy, since its primitives are simple and expressive. [ABSTRACT FROM AUTHOR]