Search

Temporal difference learning and eligibility traces are two mechanisms for solving reinforcement learning problems. The temporal difference technique bootstraps the state value or state-action value at every step as with dynamic programming, and learns by sampling episodes from experience as in the Monte Carlo approach. Eligibility traces is a mechanism that offers a means for recording the degree of which state is eligible for undergoing learning process. This paper aims to investigate the underlying mechanism of eligibility traces strategies using on-policy and off-policy learning algorithms. In doing so, the performance metrics can be obtained by defining the learning problem in a simulation environment, in conjunction with different learning algorithms. However, measuring learning performance and analysing sensibility are very expensive because such performance metrics can only be obtained by running an experiment with different parameter values. This paper proposes a comparative study for analysing the mechanism of eligibility traces. The objective of this paper is to compare and investigate the influences on performance caused by those different approaches. [ABSTRACT FROM AUTHOR]

Abstract: This paper systematically proposed a multi-population agent co-genetic algorithm with double chain-like agent structure (MPATCGA) to solve the problem of the low optimization precision and long optimization time of simple genetic algorithm in terms of two coding strategy. This algorithm adopted multi-population parallel searching mode, close chain-like agent structure, cycle chain-like agent structure, dynamic neighborhood competition, and improved crossover strategy to realize parallel optimization, and has the characteristics of high optimization precision and short optimization time. Besides, the size of each sub-population is adaptive. The characteristic is very competitive when dealing with imbalanced workload. In order to verify the optimization precision of this algorithm with binary coding, some popular benchmark test functions were used for comparing this algorithm and a popular agent genetic algorithm (MAGA). The experimental results show that MPATCGA has higher optimization precision and shorter optimization time than MAGA. Besides, in order to show the optimization performance of MPATCGA with real coding, the authors used it for feature selection problems as optimization algorithm and compared it with some other well-known GAs. The experimental results show that MPATCGA has higher optimization precision (feature selection precision). In order to show the performance of the adaptability of size of sub-populations, MPATCGA with sub-populations with same size and MPATCGA with sub-populations with different size are compared. The experimental results show that when the workload on different sub-populations becomes not same, the adaptability will adaptively change the size of different sub-population to obtain precision as high as possible. [Copyright &y& Elsevier]