Search

In this paper, a learning algorithm combining memory-less learning and memory-based learning is proposed for agents operating under POMDP. In the first stage of the proposed algorithm, memory-less learning is applied. The stochastic gradient method is employed as a memory-less learning algorithm. In the first stage, a state-action set series that accomplishes the task is stored in memory. In the second stage, memory-based learning is applied. In this process, only the series obtained in the first stage is used, so that this method is able to reduce significantly the amount of required memory. The proposed algorithm is applied to three simulations for comparison with the memory-less learning algorithm. Through computer simulations, it is shown that the proposed algorithm works more effectively in POMDP than ordinary memory-less learning. © 2010 Wiley Periodicals, Inc. Electr Eng Jpn, 173(1): 32–40, 2010; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/eej.20963 [ABSTRACT FROM AUTHOR]

We have proposed the multirate discrete Fourier transform (MR-DFT) as one of the DFT algorithms. The feature of the MR-DFT is that only one multiplication is used for each Fourier coefficient. Therefore, the number of multiplications can be decreased compared with the conventional DFT. In this paper, for applications of the MR-DFT, we consider how the noise affects the estimation performance of the MR-DFT. Then we propose two improved algorithms of the estimation performance, that is, a shifted multirate discrete Fourier transform (SMR-DFT) and a parallel shifted multirate discrete Fourier transform (PSMR-DFT). The SMR-DFT prevents an aliasing of the additive noise by a frequency modulation. The PSMR-DFT performs the parallel processing of the SMR-DFT with different sample points using the periodicity of an input signal. The estimation performances of the proposed algorithms are compared with MR-DFT by computer simulations. The SMR-DFT could prevent the degradation of the estimation performance of the MR-DFT in the lower components caused by the wide bandwidth noise. Hence its estimation variance could become 0.61 times that of the MR-DFT. Furthermore, the PSMR-DFT could improve the estimation variance of the SMR-DFT to 0.46 times. Comparing the number of operations of the proposed algorithms with the MR-DFT, the number of additions has increased, however, the number of multiplications has the same complexity. © 2004 Wiley Periodicals, Inc. Electr Eng Jpn, 150(2): 20–27, 2005; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/eej.20044 [ABSTRACT FROM AUTHOR]