Search

We consider the problem of performing interpretable classification in the high-dimensional setting, in which the number of features is very large and the number of observations is limited. This setting has been studied extensively in the chemometrics literature, and more recently has become commonplace in biological and medical applications. In this setting, a traditional approach involves performing feature selection before classification. We propose sparse discriminant analysis, a method for performing linear discriminant analysis with a sparseness criterion imposed such that classification and feature selection are performed simultaneously. Sparse discriminant analysis is based on the optimal scoring interpretation of linear discriminant analysis, and can be extended to perform sparse discrimination via mixtures of Gaussians if boundaries between classes are nonlinear or if subgroups are present within each class. Our proposal also provides low-dimensional views of the discriminative directions. [ABSTRACT FROM AUTHOR]

Presents a reaction to the paper by statisticians Ildiko E. Frank and Jerome H. Friedman regarding two methods used in chemometrics for predictive modeling, partial least squares and principal components regression, from a statistical perspective. Penalized least squares; Smooth basis expansions.

Comments on the article about fitting regression models, smoothing and additive. Difficulty of the task of selecting the number and position of the 'knots' for a regression spline; Common method for fitting nonparametric additive models; Noise examples showing Turbo to be far superior to the alternative methods.

Comments on Leo Breiman's article `The II Method for Estimating Multivariate Functions from Noisy Data.' Applaud on Breiman for providing detailed benchmarks as a basis for comparison.