Search

Stochastic approximation techniques play an important role in solving many problems encountered in machine learning or adaptive signal processing. In these contexts, the statistics of the data are often unknown a priori or their direct computation is too intensive, and they have thus to be estimated online from the observed signals. For batch optimization of an objective function being the sum of a data fidelity term and a penalization (e.g., a sparsity promoting function), Majorize-Minimize (MM) methods have recently attracted much interest since they are fast, highly flexible, and effective in ensuring convergence. The goal of this paper is to show how these methods can be successfully extended to the case when the data fidelity term corresponds to a least squares criterion and the cost function is replaced by a sequence of stochastic approximations of it. In this context, we propose an online version of an MM subspace algorithm and we study its convergence by using suitable probabilistic tools. Simulation results illustrate the good practical performance of the proposed algorithm associated with a memory gradient subspace, when applied to both nonadaptive and adaptive filter identification problems. [ABSTRACT FROM PUBLISHER]

A multiband-structured subband adaptive filter (MSAF) algorithm was introduced to achieve a fast convergence rate for the correlated input signal. The convergence analysis of the adaptive filter algorithm is an important concept because it provides a guideline to design the adaptive filter. However, the convergence analysis of the MSAF algorithm has not been researched as extensively as that of the normalized least-mean-square algorithm. Therefore, it needs to be researched. In this paper, we present a new approach to the mean-square deviation (MSD) analysis of the MSAF algorithm by using the persistently exciting input and the practical assumption that the stopband attenuation of the prototype filter is high. Unlike the previous analysis, the proposed analysis is possible to be applied to the long-length adaptive filter such as the acoustic echo cancellation. The proposed analysis is also applied to a non-stationary model with a random walk of the optimal weight vector. The simulation results match with the theoretical results in both the transient-state and steady-state MSD. [ABSTRACT FROM AUTHOR]