Search

Abstract: Linear system identification with noisy input/output is a critical problem in signal processing and control. Conventional techniques based on the mean squared-error (MSE) criterion can at best provide a biased parameter estimate of the unknown system being modeled. Recently, we proposed a new criterion called the error whitening criterion (EWC) to solve the problem of linear parameter estimation in the presence of additive white noise. Accordingly, the central idea is to partially whiten the error signal beyond a predetermined correlation lag. In the first-half of the paper, we will derive a fast Quasi-Newton type recursive algorithm to compute the optimal EWC solution in an online manner. The algorithm has O(N 2) complexity where, N represents the length of the parameter vector to be estimated. One of the primary limitations of EWC is the assumption that the input noise must be white. In the second-half of this paper, we will introduce a modified cost function that overcomes this assumption and allows the noise in the input to be colored. The analysis of this modified cost function is then presented followed by a sample-by-sample stochastic gradient algorithm to optimally compute the analytical solution. Finally, we will show the experimental results with EWC as well as the modified criterion in system identification and controller design problems. [Copyright &y& Elsevier]

Mean squared error (MSE) has been the dominant criterion in adaptive filter theory. A major drawback of the MSE criterion in linear filter adaptation is the parameter bias in the Wiener solution when the input data are contaminated with noise. In this paper, we propose and analyze a new augmented MSE criterion called the Error Whitening Criterion (EWC). EWC is able to eliminate this bias when the noise is white. We will determine the analytical solution of the EWC, discuss some interesting properties, and develop stochastic gradient and other fast algorithms to calculate the EWC solution in an online fashion. The stochastic algorithms are locally computable and have structures and complexities similar to their MSE-based counterparts (LMS and NLMS). Convergence of the stochastic gradient algorithm is established with mild assumptions, and upper bounds on the step sizes are deduced for guaranteed convergence. We will briefly discuss an RLS-like Recursive Error Whitening (REW) algorithm and a minor components analysis (MCA) based EWC-total least squares (TLS) algorithm and further draw parallels between the REW algorithm and the Instrumental Variables (lv) method for system identification. Finally, we will demonstrate the noise-rejection capability of the EWC by comparing the performance with MSE criterion and TLS. [ABSTRACT FROM AUTHOR]

Mean squared error (MSE) has been the most widely used tool to solve the linear filter estimation or system identification problem. However, MSE gives biased results when the input signals are noisy. This paper presents a novel stochastic gradient algorithm based on the recently proposed error whitening criterion (EWC) to tackle the problem of linear filter estimation in the presence of additive white disturbances. We will briefly motivate the theory behind the new criterion and derive an online stochastic gradient algorithm. Convergence proof of the stochastic gradient algorithm is derived making mild assumptions. Further, we will propose some extensions to the stochastic gradient algorithm to ensure faster, step-size independent convergence. We will perform extensive simulations and compare the results with MSE as well as total-least squares in a parameter estimation problem. The stochastic EWC algorithm has many potential applications. We will use this in designing robust inverse controllers with noisy data. [Copyright &y& Elsevier]

Minimum output mutual information is regarded as a natural criterion for independent component analysis (ICA) and is used as the performance measure in many ICA algorithms. Two common approaches in information-theoretic ICA algorithms are minimum mutual information and maximum output entropy approaches. In the former approach, we substitute some form of probability density function (pdf) estimate into the mutual information expression, and in the latter we incorporate the source pdf assumption in the algorithm through the use of nonlinearities matched to the corresponding cumulative density functions (cdf). Alternative solutions to ICA use higher-order cumulant-based optimization criteria, which are related to either one of these approaches through truncated series approximations for densities. In this article, we propose a new ICA algorithm motivated by the maximum entropy principle (for estimating signal distributions). The optimality criterion is the minimum output mutual information, where the estimated pdfs are from the exponential family and are approximate solutions to a constrained entropy maximization problem. This approach yields an upper bound for the actual mutual information of the output signals—hence, the name minimax mutual information ICA algorithm. In addition, we demonstrate that for a specific selection of the constraint functions in the maximum entropy density estimation procedure, the algorithm relates strongly to ICA methods using higher-order cumulants. [ABSTRACT FROM AUTHOR]

This paper proposes a divide-and-conquer strategy for designing brain machine interfaces. A nonlinear combination of competitively trained local linear models (experts) is used to identify the mapping from neuronal activity in cortical areas associated with arm movement to the hand position of a primate. The proposed architecture and the training algorithm are described in detail and numerical performance comparisons with alternative linear and nonlinear modeling approaches, including time-delay neural networks and recursive multilayer perceptrons, are presented. This new strategy allows training the local linear models using normalized LMS and using a relatively smaller nonlinear network to efficiently combine the predictions of the linear experts. This leads to savings in computational requirements, while the performance is still similar to a large fully nonlinear network. [Copyright &y& Elsevier]