Search

Dynamic system modeling plays a crucial role in the development of techniques for stationary and nonstationary signal processing. Due to the inherent physical characteristics of systems under investigation, nonnegativity is a desired constraint that can usually be imposed on the parameters to estimate. In this paper, we propose a general method for system identification under nonnegativity constraints. We derive the so-called nonnegative least-mean-square algorithm (NNLMS) based on stochastic gradient descent, and we analyze its convergence. Experiments are conducted to illustrate the performance of this approach and consistency with the analysis. [ABSTRACT FROM AUTHOR]

The super-resolution reconstruction (SRR) of images is an ill posed problem. Traditionally, it is treated as a regularized minimization problem. Moreover, one of the major problems concerning SRR is its dependence on an accurate registration. In this paper, we show that a certain amount of registration error may, in fact, be beneficial for the performance of the least mean square SRR (LMS-SRR) adaptive algorithm. In these cases, the regularization term may be avoided, leading to reduction in computational cost that can be important in real-time SRR applications. [ABSTRACT FROM AUTHOR]

Kernel-based algorithms have been a topic of considerable interest in the machine learning community over the last ten years. Their attractiveness resides in their elegant treatment of nonlinear problems. They have been successfully applied to pattern recognition, regression and density estimation. A common characteristic of kernel-based methods is that they deal with kernel expansions whose number of terms equals the number of input data, making them unsuitable for online applications. Recently, several solutions have been proposed to circumvent this computational burden in time series prediction problems. Nevertheless, most of them require excessively elaborate and costly operations. In this paper, we investigate a new model reduction criterion that makes computationally demanding sparsification procedures unnecessary. The increase in the number of variables is controlled by the coherence parameter, a fundamental quantity that characterizes the behavior of dictionaries in sparse approximation problems. We incorporate the coherence criterion into a new kernel-based affine projection algorithm for time series prediction. We also derive the kernel-based normalized LMS algorithm as a particular case. Finally, experiments are conducted to compare our approach to existing methods. [ABSTRACT FROM AUTHOR]

Echo cancellers (EC) are required in both electrical (impedance mismatch) and acoustic (speaker-microphone coupling) applications. One of the main design problems is the control logic for adaptation. Basically, the algorithm weights should be frozen in the presence of double-talk and adapt quickly in the absence of double-talk. The optimum likelihood ratio test (LRT) for this problem was studied in a recent paper. The LRT requires a priori knowledge of the background noise and double-talk power levels. Instead, this paper derives a generalized log likelihood ratio test (GLRT) that does not require this knowledge. The probability density function of a sufficient statistic under each hypothesis is obtained and the performance of the test is evaluated as a function of the system parameters. The receiver operating characteristics (ROCs) indicate that it is difficult to correctly decide between double-talk and a channel change, based upon a single look. However, detection based on about 200 successive samples yields a detection probability close to unity (0.99) with a small false alarm probability (0.01) for the theoretical GLRT model. Application of a GLRT-based EC to real voice data shows comparable performance to that of the LRT-based EC given in a recent paper. [ABSTRACT FROM AUTHOR]

This paper presents a statistical analysis of a Pseudo Affine Projection (PAP) algorithm, obtained from the Affine Projection algorithm (AP) for a step size α < 1 and a scalar error signal in the weight update. Deterministic recursive equations are derived for the mean weight and for the mean square error (MSE) for a large number of adaptive taps N compared to the order P of the algorithm. Simulations are presented which show good to excellent agreement with the theory in the transient and steady states. The PAP learning behavior is of special interest in applications where tradeoffs are necessary between convergence speed and steady-state misadjustment. [ABSTRACT FROM AUTHOR]

This paper studies the statistical behavior of an affine combination of the outputs of two least mean-square (LMS) adaptive filters that simultaneously adapt using the same white Gaussian inputs. The purpose of the combination is to obtain an LMS adaptive filter with fast convergence and small steady-state mean-square deviation (MSD). The linear combination studied is a generalization of the convex combination, in which the combination factor λ(n) is restricted to the interval (0,1). The viewpoint is taken that each of the two filters produces dependent estimates of the unknown channel. Thus, there exists a sequence of optimal affine combining coefficients which minimizes the mean-square error (MSE). First, the optimal unrealizable affine combiner is studied and provides the best possible performance for this class. Then two new schemes are proposed for practical applications. The mean-square performances are analyzed and validated by Monte Carlo simulations. With proper design, the two practical schemes yield an overall MSD that is usually less than the MSDs of either filter. [ABSTRACT FROM AUTHOR]

This paper studies the behavior of the low-rank least mean squares (LMS) adaptive algorithm for the general case in which the input transformation may not capture the exact input subspace. It is shown that the Independence Theory and the independent additive noise model are not applicable to this case. A new theoretical model for the weight mean and fluctuation behaviors is developed which incorporates the correlation between successive data vectors (as opposed to the Independence Theory model). The new theory is applied to a network echo cancellation scheme which uses partial-Haar input vector transformations. Comparison of the new model predictions with Monte Carlo simulations shows good-to-excellent agreement, certainly much better than predicted by the Independence Theory based model available in the literature. [ABSTRACT FROM AUTHOR]

Super-resolution reconstruction of image sequences is highly dependent on data outliers and on the quality of the motion estimation. This paper addresses the design of the least mean square algorithm applied to super-resolution reconstruction (LMS-SRR). Based on a statistical model for the algorithm behavior, we propose a design strategy to reduce the effects of outliers on the reconstructed image sequence. We show that the proposed strategy leads the algorithm to a close-to-optimum performance in both the transient and the steady-state phases of adaptation in practical situations in which registration errors occur. The analysis also shows that lower values of the step size do not necessarily lead to a better steady-state mean-square error, differently from the traditional LMS behavior. [ABSTRACT FROM AUTHOR]

In this paper, it is shown that the least-mean fourth (LMF) adaptive algorithm is not mean-square stable when the regressor input is not strictly bounded (as happens, for example, if the input has a Gaussian distribution). For input distributions with infinite support, even for the Gaussian distribution, the LMF always has a nonzero probability of divergence, no matter how small the step-size is chosen. This result is proven for a slight modification of the Gaussian distribution in a one-tap filter and corroborated with several simulations. In addition, an upper bound is given for the probability of divergence of LMF as a function of the filter length, input power, step-size, and noise variance, for the case of Gaussian regressors. The results reported in this paper provide tools for designers to better understand the behavior of the LMF algorithm and decide on the convenience or not of its use for a given application. [ABSTRACT FROM AUTHOR]

This paper presents an improved statistical analysis of the least mean fourth (LMF) adaptive algorithm behavior for a stationary Gaussian input. The analysis improves previous resuits in that higher order moments of the weight error vector are not neglected and that it is not restricted to a specific noise distribution. The analysis is based on the independence theory and assumes reasonably slow learning and a large number of adaptive filter coefficients. A new analytical model is derived, which is able to accurately predict the algorithm behavior both during transient and in steady-state for small step sizes and long impulse responses. The new model is valid for any zero-mean symmetric noise density function and for any signal-to-noise ratio (SNR). Computer simulations illustrate the accuracy of the new model in predicting the algorithm behavior in several different situations. [ABSTRACT FROM AUTHOR]

Investigates the properties of the performance surface for the problem of linearly constrained nonlinear mean-square estimation of a random sequence. Derivation of the deterministic expression for the mean-square error surface; Design of an adaptive estimation systems; Presentation of models for the nonlinear adaptive path.