Search

In this paper, the relationships between the eigen-values of the m x m Gram matrix K for a kernel K(.,.) corresponding to a sample [x.sub.1], . [x.sub.m] drawn from a density p(x) and the eigenvalues of the corresponding continuous eigen-problem is analyzed. The differences between the two spectra are bounded and a performance bound on kernel principal component analysis (PCA) is provided showing that good performance can be expected even in very-high-dimensional feature spaces provided the sample eigenvalues fall sufficiently quickly. Index Terms--Concentration bounds, Gram matrices, kernel methods, principal components analysis (PCA), Rademacher complexity, spectra of random matrices, statistical learning theory.

Canonical correlation analysis is a family of multivariate statistical methods for the analysis of paired sets of variables. Since its proposition, canonical correlation analysis has, for instance, been extended to extract relations between two sets of variables when the sample size is insufficient in relation to the data dimensionality, when the relations have been considered to be non-linear, and when the dimensionality is too large for human interpretation. This tutorial explains the theory of canonical correlation analysis, including its regularised, kernel, and sparse variants. Additionally, the deep and Bayesian CCA extensions are briefly reviewed. Together with the numerical examples, this overview provides a coherent compendium on the applicability of the variants of canonical correlation analysis. By bringing together techniques for solving the optimisation problems, evaluating the statistical significance and generalisability of the canonical correlation model, and interpreting the relations, we hope that this article can serve as a hands-on tool for applying canonical correlation methods in data analysis. [ABSTRACT FROM AUTHOR]

Introduces articles published in the August 2003 issue of the periodical "Journal of Machine Learning Research." Machine learning methods for text and images; Machine learning algorithms; Kernel methods for relation extraction.