Search

The combination of the famed kernel trick and the least-mean-square (LMS) algorithm has provided a sample-by-sample update for an adaptive filter in reproducing kernel Hilbert spaces (RKHS). The results have shown that with finite data the KLMS algorithm can be used in high dimensional spaces and particularly in RKHS to derive nonlinear, stable algorithms with comparable performance to batch, regularized solutions.

The probabilistic and geometric meaning of the correntropy function as a localized similarity measure is discussed. Findings reveal that the vastly different properties of correntropy are useful in non-Gaussian signal processing.

A novel classifier is developed in a kernel feature space related to the eigenspectrum of the Laplacian data matrix. It is shown that the new classifier has performed better than the Parzen window Bayes classifier and in many cases it is comparable to the support vector machine, at a computationally lower cost.

A new generalized correlation function is developed that includes the information of both the distribution and that of the time structure of a stochastic process. An attempt is made to find out how this measure is interpreted from a kernel method as well as from information theoretic learning points of view, demonstrating some relevant properties.

An experimental upper bound on the separation performance for a class of convolutive blind source separation structures, which can be used to approximate the optimal performance, is introduced. It is seen that structures other than the finite-impulsive-response (FIR) structure should be considered for real (convolutive) mixtures.

A new augmented mean squared error (MSE) criterion called the Error Whitening Criterion (EWC) is proposed and analyzed. The analytical solution of the EWC is determined and stochastic gradient and other fast algorithms are developed to calculate the EWC solution in an online fashion.

A new blind equalization approach that aims to force the probability density function (PDF) at the equalizer output to match the known constellation PDF is presented. The proposed method relies on the Parzen window method to estimate the data PDF and is implemented by a stochastic gradient descent algorithm.

The suitability of a class of Parzen-window-based entropy estimates, namely Renyi's entropy, as a criterion for blind deconvolution of linear channels are investigated. The results indicate that this nonparametric entropy estimation approach outperforms the standard Bell-Sejnowski and normalized kurtosis algorithms in blind deconvolution.

The minimum error entropy (MEE) criterion, proposed as an information theoretic alternative to the mean square error criterion in supervised adaptive system training, is examined and discussed.

An information-theoretic supervised learning criterion for adaptive systems, minimum error entropy (MEE), is discussed. Equivalence is demonstrated between minimization of error's Renyi entropy of order omega and minimization of a Csiszar distance measure between the densities of desired system outputs.

The application of error-entropy minimization algorithms to digital communications channel equalization is examined. A new optimization criterion, based on the Renyi's error entropy, is used.

A numerically stable formulation supporting the convolution of a sinc kernel with the infinite sequence of a periodic function is presented. The formulation, which supports indeterminate points, was shown to be equivalent to the inverse discrete Fourier transform. The relation between the Fourier series of a signal and the discrete Fourier transform of samples correlated with an expression similar to the inverse discrete Fourier transform.

The study of the problem of linear approximation of a signal with parametric gamma bases in the L(Z) spaces reveals that these bases possess a time scale parameter. The parameter modifies the relative angle between the projection space and the signal. This results in the origin of an additional degree of freedom in the approximation.

A new adaptive filter with restricted feedback architectureis presented. The filter, called the generalized feedforward or gamma filter, combines the beneficial properties of the finite impulse response filter and infinite impulse filter. It features an independently adjustable memory depth and high stability and adaptation characteristics. Experimental and analytical results show that the generalized feedforward filter outperforms its adaptive transversal counterpart.

We propose a novel nearest-neighbors approach to organize and curb the growth of radial basis function network in kernel adaptive filtering (KAF). The nearest-instance-centroid-estimation (NICE) kernel least-mean-square (KLMS) algorithm provides an appropriate time-space tradeoff with good performance. Its centers in the input/feature space are organized by quasi-orthogonal regions for greatly simplified filter evaluation. Instead of using all centers to evaluate/update the function approximation at every new point, a linear search among the iteratively-updated centroids determines the partial function to be used, naturally forming locally-supported partial functionals. Under this framework, partial functionals that compose the adaptive filter are quickly stored/retrieved based on input, each corresponding to a specialized “spatial-band” subfilter. The filter evaluation becomes the update of one of the subfilters, creating a content addressable filter bank (CAFB). This CAFB is incrementally updated for new signal applications with mild constraints, always using the past-learned partial filter sums, opening the door for transfer learning and significant efficiency for new data scenarios, avoiding training from scratch as have been done since the invention of adaptive filtering. Using energy conservation relation, we show the sufficient condition for mean square convergence of the NICE-KLMS algorithm and establish the upper and lower bounds of steady-state excess-mean-square-error (EMSE). Simulations on chaotic time-series prediction demonstrate similar levels of accuracy as existing methods, but with much faster computation involving fewer input samples. Simulations on transfer learning using both synthetic and real-world data demonstrate that NICE CAFB can leverage previously learned knowledge to related task or domain. [ABSTRACT FROM AUTHOR]

Principal component analysis (PCA) of a time series is performed using a novel unsupervised algorithm. A mixture of experts formalism is the parameter through which a set of PCA networks competes based on their ability to reconstruct the original signal. The optimal projection of the input on a reduced dimensional space is determined using the resulting network. The performance of this algorithm is discussed using a number of examples.

Spectrum sensing is a crucial issue in cognitive radio networks. To improve the sensing performance, cooperation among the secondary users can be utilized to collect space diversity. In this paper, we quantify the diversity order for various cooperative spectrum sensing strategies. We first notice that, in a spectrum sensing problem, diversity should reflect both the false alarm and missed detection behaviors since they respectively capture the efficiency and reliability of the overall cognitive system. Hence, we consider both false alarm and missed detection probabilities individually and jointly via the average error probability. With the knowledge of both the noise strength and the primary user signal strength, the threshold of the Neyman-Pearson detector can be varied to control the system performance in desirable ways. Interesting tradeoffs between system efficiency and reliability are found in various cooperative strategies and analytical results are presented to guide practical system designs with different preferences. [ABSTRACT FROM AUTHOR]

With an abundance of tools based on kernel methods and information theoretic learning, a void still exists in incorporating both the time structure and the statistical distribution of the time series in the same functional measure. In this paper, a new generalized correlation measure is developed that includes the information of both the distribution and that of the time structure of a stochastic process. It is shown how this measure can be interpreted from a kernel method as well as from an information theoretic learning points of view, demonstrating some relevant properties. To underscore the effectiveness of the new measure, a simple blind equalization problem is considered using a coded signal. [ABSTRACT FROM AUTHOR]

Presents a study which investigated error-entropy-minimization algorithm for supervised training of nonlinear adaptive systems. Method of the study; Results and discussion; Conclusion.

Investigates the application of error-entropy minimization algorithms to digital communications channel equalization. Characteristics of modern digital communications systems; Overview of linear and nonlinear equalizers; Discussion of quadratic error-entropy for equalization; Analysis of the optimal solutions.

Focuses on the convolution of sinc kernel with infinite sequence of a periodic function for discrete signals. Numerically stable formulation of sinc kernel; Conclusions.