Search

In clinical practice and biomedical research, measurements are often collected sparsely and irregularly in time, while the data acquisition is expensive and inconvenient. Examples include measurements of spine bone mineral density, cancer growth through mammography or biopsy, a progression of defective vision, or assessment of gait in patients with neurological disorders. Practitioners often need to infer the progression of diseases from such sparse observations. A classical tool for analyzing such data is a mixed-effect model where time is treated as both a fixed effect (population progression curve) and a random effect (individual variability). Alternatively, researchers use Gaussian processes or functional data analysis, assuming that observations are drawn from a certain distribution of processes. While these models are flexible, they rely on probabilistic assumptions, require very careful implementation, and tend to be slow in practice. In this study, we propose an alternative elementary framework for analyzing longitudinal data motivated by matrix completion. Our method yields estimates of progression curves by iterative application of the Singular Value Decomposition. Our framework covers multivariate longitudinal data, and regression and can be easily extended to other settings. As it relies on existing tools for matrix algebra, it is efficient and easy to implement. We apply our methods to understand trends of progression of motor impairment in children with Cerebral Palsy. Our model approximates individual progression curves and explains 30% of the variability. Low-rank representation of progression trends enables identification of different progression trends in subtypes of Cerebral Palsy. [ABSTRACT FROM AUTHOR]

AbstractForecasting methodologies have always attracted a lot of attention and have become an especially hot topic since the beginning of the COVID-19 pandemic. In this article we consider the problem of multi-period forecasting that aims to predict several horizons at once. We propose a novel approach that forces the prediction to be “smooth” across horizons and apply it to two tasks: point estimation via regression and interval prediction via quantile regression. This methodology was developed for real-time distributed COVID-19 forecasting. We illustrate the proposed technique with the COVIDcast dataset as well as a small simulation example. Supplementary materials for this article are available online. [ABSTRACT FROM AUTHOR]

Despite its potentials benefits, using prediction targets generated based on latent variable (LV) modeling is not a common practice in supervised learning, a dominating framework for developing prediction models. In supervised learning, it is typically assumed that the outcome to be predicted is clear and readily available, and therefore validating outcomes before predicting them is a foreign concept and an unnecessary step. The usual goal of LV modeling is inference, and therefore using it in supervised learning and in the prediction context requires a major conceptual shift. This study lays out methodological adjustments and conceptual shifts necessary for integrating LV modeling into supervised learning. It is shown that such integration is possible by combining the traditions of LV modeling, psychometrics, and supervised learning. In this interdisciplinary learning framework, generating practical outcomes using LV modeling and systematically validating them based on clinical validators are the two main strategies. In the example using the data from the Longitudinal Assessment of Manic Symptoms (LAMS) Study, a large pool of candidate outcomes is generated by flexible LV modeling. It is demonstrated that this exploratory situation can be used as an opportunity to tailor desirable prediction targets taking advantage of contemporary science and clinical insights. [ABSTRACT FROM AUTHOR]

Abstract Cross-validation is a widely used technique to estimate prediction error, but its behavior is complex and not fully understood. Ideally, one would like to think that cross-validation estimates the prediction error for the model at hand, fit to the training data. We prove that this is not the case for the linear model fit by ordinary least squares; rather it estimates the average prediction error of models fit on other unseen training sets drawn from the same population. We further show that this phenomenon occurs for most popular estimates of prediction error, including data splitting, bootstrapping, and Mallow’s C p . Next, the standard confidence intervals for prediction error derived from cross-validation may have coverage far below the desired level. Because each data point is used for both training and testing, there are correlations among the measured accuracies for each fold, and so the usual estimate of variance is too small. We introduce a nested cross-validation scheme to estimate this variance more accurately, and show empirically that this modification leads to intervals with approximately correct coverage in many examples where traditional cross-validation intervals fail. Lastly, our analysis also shows that when producing confidence intervals for prediction accuracy with simple data splitting, one should not refit the model on the combined data, since this invalidates the confidence intervals. Supplementary materials for this article are available online. [ABSTRACT FROM AUTHOR]

Three-dimensional (3D) genome spatial organization is critical for numerous cellular processes, including transcription, while certain conformation-driven structural alterations are frequently oncogenic. Genome architecture had been notoriously difficult to elucidate, but the advent of the suite of chromatin conformation capture assays, notably Hi-C, has transformed understanding of chromatin structure and provided downstream biological insights. Although many findings have flowed from direct analysis of the pairwise proximity data produced by these assays, there is added value in generating corresponding 3D reconstructions deriving from superposing genomic features on the reconstruction. Accordingly, many methods for inferring 3D architecture from proximity data have been advanced. However, none of these approaches exploit the fact that single chromosome solutions constitute a one-dimensional (1D) curve in 3D. Rather, this aspect has either been addressed by imposition of constraints, which is both computationally burdensome and cell type specific, or ignored with contiguity imposed after the fact. Here, we target finding a 1D curve by extending principal curve methodology to the metric scaling problem. We illustrate how this approach yields a sequence of candidate solutions, indexed by an underlying smoothness or degrees-of-freedom parameter, and propose methods for selection from this sequence. We apply the methodology to Hi-C data obtained on IMR90 cells and so are positioned to evaluate reconstruction accuracy by referencing orthogonal imaging data. The results indicate the utility and reproducibility of our principal curve approach in the face of underlying structural variation. [ABSTRACT FROM AUTHOR]

Conditional density estimation is a fundamental problem in statistics, with scientific and practical applications in biology, economics, finance and environmental studies, to name a few. In this paper, we propose a conditional density estimator based on gradient boosting and Lindsey's method (LinCDE). LinCDE admits exible modeling of the density family and can capture distributional characteristics like modality and shape. In particular, when suitably parametrized, LinCDE will produce smooth and non-negative density estimates. Furthermore, like boosted regression trees, LinCDE does automatic feature selection. We demonstrate LinCDE's efficacy through extensive simulations and three real data examples. [ABSTRACT FROM AUTHOR]

We study the assessment of the accuracy of heterogeneous treatment effect (HTE) estimation, where the HTE is not directly observable so standard computation of prediction errors is not applicable. To tackle the difficulty, we propose an assessment approach by constructing pseudo‐observations of the HTE based on matching. Our contributions are three‐fold: first, we introduce a novel matching distance derived from proximity scores in random forests; second, we formulate the matching problem as an average minimum‐cost flow problem and provide an efficient algorithm; third, we propose a match‐then‐split principle for the assessment with cross‐validation. We demonstrate the efficacy of the assessment approach using simulations and a real dataset. [ABSTRACT FROM AUTHOR]

Ridge or more formally l 2 regularization shows up in many areas of statistics and machine learning. It is one of those essential devices that any good data scientist needs to master for their craft. In this brief ridge fest, I have collected together some of the magic and beauty of ridge that my colleagues and I have encountered over the past 40 years in applied statistics. [ABSTRACT FROM AUTHOR]

Background: Standard pediatric growth curves cannot be used to impute missing height or weight measurements in individual children. The Michaelis–Menten equation, used for characterizing substrate-enzyme saturation curves, has been shown to model growth in many organisms including nonhuman vertebrates. We investigated whether this equation could be used to interpolate missing growth data in children in the first three years of life and compared this interpolation to several common interpolation methods and pediatric growth models. Methods: We developed a modified Michaelis–Menten equation and compared expected to actual growth, first in a local birth cohort (N = 97) then in a large, outpatient, pediatric sample (N = 14,695). Results: The modified Michaelis–Menten equation showed excellent fit for both infant weight (median RMSE: boys: 0.22 kg [IQR:0.19; 90% < 0.43]; girls: 0.20 kg [IQR:0.17; 90% < 0.39]) and height (median RMSE: boys: 0.93 cm [IQR:0.53; 90% < 1.0]; girls: 0.91 cm [IQR:0.50;90% < 1.0]). Growth data were modeled accurately with as few as four values from routine well-baby visits in year 1 and seven values in years 1–3; birth weight or length was essential for best fit. Interpolation with this equation had comparable (for weight) or lower (for height) mean RMSE compared to the best performing alternative models. Conclusions: A modified Michaelis–Menten equation accurately describes growth in healthy babies aged 0–36 months, allowing interpolation of missing weight and height values in individual longitudinal measurement series. The growth pattern in healthy babies in resource-rich environments mirrors an enzymatic saturation curve. [ABSTRACT FROM AUTHOR]

We propose the nuclear norm penalty as an alternative to the ridge penalty for regularized multinomial regression. This convex relaxation of reduced-rank multinomial regression has the advantage of leveraging underlying structure among the response categories to make better predictions. We apply our method, nuclear penalized multinomial regression (NPMR), to Major League Baseball play-by-play data to predict outcome probabilities based on batter–pitcher matchups. The interpretation of the results meshes well with subject-area expertise and also suggests a novel understanding of what differentiates players. [ABSTRACT FROM AUTHOR]

We extend the adaptive regression spline model by incorporating saturation, the natural requirement that a function extend as a constant outside a certain range. We fit saturating splines to data via a convex optimization problem over a space of measures, which we solve using an efficient algorithm based on the conditional gradient method. Unlike many existing approaches, our algorithm solves the original infinite-dimensional (for splines of degree at least two) optimization problem without pre-specified knot locations. We then adapt our algorithm to fit generalized additive models with saturating splines as coordinate functions and show that the saturation requirement allows our model to simultaneously perform feature selection and nonlinear function fitting. Finally, we briey sketch how the method can be extended to higher order splines and to different requirements on the extension outside the data range. [ABSTRACT FROM AUTHOR]

In all sorts of regression problems, it has become more and more important to deal with high-dimensional data with lots of potentially influential covariates. A possible solution is to apply estimation methods that aim at the detection of the relevant effect structure by using penalization methods. In this article, the effect structure in the Cox frailty model, which is the most widely used model that accounts for heterogeneity in survival data, is investigated. Since in survival models one has to account for possible variation of the effect strength over time the selection of the relevant features has to distinguish between several cases, covariates can have time-varying effects, time-constant effects, or be irrelevant. A penalization approach is proposed that is able to distinguish between these types of effects to obtain a sparse representation that includes the relevant effects in a proper form. It is shown in simulations that the method works well. The method is applied to model the time until pregnancy, illustrating that the complexity of the influence structure can be strongly reduced by using the proposed penalty approach. [ABSTRACT FROM AUTHOR]

Canonical correlation analysis (CCA) is a technique for measuring the association between two multivariate data matrices. A regularized modification of canonical correlation analysis (RCCA) which imposes an ℓ 2 penalty on the CCA coefficients is widely used in applications with high-dimensional data. One limitation of such regularization is that it ignores any data structure, treating all the features equally, which can be ill-suited for some applications. In this article we introduce several approaches to regularizing CCA that take the underlying data structure into account. In particular, the proposed group regularized canonical correlation analysis (GRCCA) is useful when the variables are correlated in groups. We illustrate some computational strategies to avoid excessive computations with regularized CCA in high dimensions. We demonstrate the application of these methods in our motivating application from neuroscience, as well as in a small simulation example. [ABSTRACT FROM AUTHOR]

We introduce a method for learning pairwise interactions in a linear regression or logistic regression model in a manner that satisfies strong hierarchy: whenever an interaction is estimated to be nonzero, both its associated main effects are also included in the model. We motivate our approach by modeling pairwise interactions for categorical variables with arbitrary numbers of levels, and then show how we can accommodate continuous variables as well. Our approach allows us to dispense with explicitly applying constraints on the main effects and interactions for identifiability, which results in interpretable interaction models. We compare our method with existing approaches on both simulated and real data, including a genome-wide association study, all using our R package glinternet. [ABSTRACT FROM PUBLISHER]

We consider the problem of learning the structure of a pairwise graphical model over continuous and discrete variables. We present a new pairwise model for graphical models with both continuous and discrete variables that is amenable to structure learning. In previous work, authors have considered structure learning of Gaussian graphical models and structure learning of discrete models. Our approach is a natural generalization of these two lines of work to the mixed case. The penalization scheme involves a novel symmetric use of the group-lasso norm and follows naturally from a particular parameterization of the model. Supplementary materials for this article are available online. [ABSTRACT FROM AUTHOR]

We study the variability of predictions made by bagged learners and random forests, and show how to estimate standard errors for these methods. Our work builds on variance estimates for bagging proposed by Efron (1992, 2013) that are based on the jackknife and the infinitesimal jackknife (IJ). In practice, bagged predictors are computed using a finite number B of bootstrap replicates, and working with a large B can be computationally expensive. Direct applications of jackknife and IJ estimators to bagging require B = 1.5 Θ (n1.5) bootstrap replicates to converge, where n is the size of the training set. We propose improved versions that only require B = Θ(n) replicates. Moreover, we show that the IJ estimator requires 1.7 times less bootstrap replicates than the jackknife to achieve a given accuracy. Finally, we study the sampling distributions of the jackknife and IJ variance estimates themselves. We illustrate our findings with multiple experiments and simulation studies. [ABSTRACT FROM AUTHOR]

Estimating the aggregated market demand for a product in a dynamic market is critical to manufacturers and retailers. Motivated by the need for a statistical demand prediction model for laptop pricing at Hewlett-Packard, we have developed a novel boosting-based varying-coefficient regression model. The developed model uses regression trees as the base learner, and is generally applicable to varying-coefficient models with a large number of mixed-type varying-coefficient variables, which proves to be challenging for conventional nonparametric smoothing methods. The proposed method works well in both predicting the response and estimating the coefficient surface, based on a simulation study. Finally, we have applied this methodology to real-world mobile computer sales data, and demonstrated its superiority by comparing with elastic net- and kernel regression-based varying-coefficient model. Computer codes for boosted varying-coefficient regression are available online as supplementary materials. [ABSTRACT FROM PUBLISHER]

Presence-only data abounds in ecology, often accompanied by a background sample. Although many interesting aspects of the species' distribution can be learned from such data, one cannot learn the overall species occurrence probability, or prevalence, without making unjustified simplifying assumptions. In this forum article we question the approach of Royle et al. (2012) that claims to be able to do this. [ABSTRACT FROM AUTHOR]

Summary: Professor Efron has presented us with a thought‐provoking paper on the relationship between prediction, estimation, and attribution in the modern era of data science. While we appreciate many of his arguments, we see more of a continuum between the old and new methodology, and the opportunity for both to improve through their synergy. [ABSTRACT FROM AUTHOR]

We consider the sparse inverse covariance regularization problem or graphical lasso with regularization parameter λ. Suppose the sample covariance graph formed by thresholding the entries of the sample covariance matrix at λ is decomposed into connected components. We show that the vertex-partition induced by the connected components of the thresholded sample covariance graph (at λ) is exactly equal to that induced by the connected components of the estimated concentration graph, obtained by solving the graphical lasso problem for the same λ. This characterizes a very interesting property of a path of graphical lasso solutions. Furthermore, this simple rule, when used as a wrapper around existing algorithms for the graphical lasso, leads to enormous performance gains. For a range of values of λ, our proposal splits a large graphical lasso problem into smaller tractable problems, making it possible to solve an otherwise infeasible large-scale problem. We illustrate the graceful scalability of our proposal via synthetic and real-life microarray examples. [ABSTRACT FROM AUTHOR]

We consider the problem of performing interpretable classification in the high-dimensional setting, in which the number of features is very large and the number of observations is limited. This setting has been studied extensively in the chemometrics literature, and more recently has become commonplace in biological and medical applications. In this setting, a traditional approach involves performing feature selection before classification. We propose sparse discriminant analysis, a method for performing linear discriminant analysis with a sparseness criterion imposed such that classification and feature selection are performed simultaneously. Sparse discriminant analysis is based on the optimal scoring interpretation of linear discriminant analysis, and can be extended to perform sparse discrimination via mixtures of Gaussians if boundaries between classes are nonlinear or if subgroups are present within each class. Our proposal also provides low-dimensional views of the discriminative directions. [ABSTRACT FROM AUTHOR]

Array-based comparative genomic hybridization (aCGH) enables the measurement of DNA copy number across thousands of locations in a genome. The main goals of analyzing aCGH data are to identify the regions of copy number variation (CNV) and to quantify the amount of CNV. Although there are many methods for analyzing single-sample aCGH data, the analysis of multi-sample aCGH data is a relatively new area of research. Further, many of the current approaches for analyzing multi-sample aCGH data do not appropriately utilize the additional information present in the multiple samples. We propose a procedure called the Fused Lasso Latent Feature Model (FLLat) that provides a statistical framework for modeling multi-sample aCGH data and identifying regions of CNV. The procedure involves modeling each sample of aCGH data as a weighted sum of a fixed number of features. Regions of CNV are then identified through an application of the fused lasso penalty to each feature. Some simulation analyses show that FLLat outperforms single-sample methods when the simulated samples share common information. We also propose a method for estimating the false discovery rate. An analysis of an aCGH data set obtained from human breast tumors, focusing on chromosomes 8 and 17, shows that FLLat and Significance Testing of Aberrant Copy number (an alternative, existing approach) identify similar regions of CNV that are consistent with previous findings. However, through the estimated features and their corresponding weights, FLLat is further able to discern specific relationships between the samples, for example, identifying 3 distinct groups of samples based on their patterns of CNV for chromosome 17. [ABSTRACT FROM PUBLISHER]

In ecological modeling of the habitat of a species, it can be prohibitively expensive to determine species absence. Presence-only data consist of a sample of locations with observed presences and a separate group of locations sampled from the full landscape, with unknown presences. We propose an expectation–maximization algorithm to estimate the underlying presence–absence logistic model for presence-only data. This algorithm can be used with any off-the-shelf logistic model. For models with stepwise fitting procedures, such as boosted trees, the fitting process can be accelerated by interleaving expectation steps within the procedure. Preliminary analyses based on sampling from presence–absence records of fish in New Zealand rivers illustrate that this new procedure can reduce both deviance and the shrinkage of marginal effect estimates that occur in the naive model often used in practice. Finally, it is shown that the population prevalence of a species is only identifiable when there is some unrealistic constraint on the structure of the logistic model. In practice, it is strongly recommended that an estimate of population prevalence be provided. [ABSTRACT FROM AUTHOR]

Abstract: Introduction: Gene expression profiling has been extensively used to predict outcome in breast cancer patients. We have previously reported on biological hypothesis-driven analysis of gene expression profiling data and we wished to extend this approach through the combinations of various gene signatures to improve the prediction of outcome in breast cancer. Methods: We have used gene expression data (25.000 gene probes) from a previously published study of tumours from 295 early stage breast cancer patients from the Netherlands Cancer Institute using updated follow-up. Tumours were assigned to three prognostic groups using the previously reported Wound-response and hypoxia-response signatures, and the outcome in each of these subgroups was evaluated. Results: We have assigned invasive breast carcinomas from 295 stages I and II breast cancer patients to three groups based on gene expression profiles subdivided by the wound-response signature (WS) and hypoxia-response signature (HS). These three groups are (1) quiescent WS/non-hypoxic HS; (2) activated WS/non-hypoxic HS or quiescent WS/hypoxic tumours and (3) activated WS/hypoxic HS. The overall survival at 15 years for patients with tumours in groups 1, 2 and 3 are 79%, 59% and 27%, respectively. In multivariate analysis, this signature is not only independent of clinical and pathological risk factors; it is also the strongest predictor of outcome. Compared to a previously identified 70-gene prognosis profile, obtained with supervised classification, the combination of signatures performs roughly equally well and might have additional value in the ER-negative subgroup. In the subgroup of lymph node positive patients, the combination signature outperforms the 70-gene signature in multivariate analysis. In addition, in multivariate analysis, the WS/HS combination is a stronger predictor of outcome compared to the recently reported invasiveness gene signature combined with the WS. Conclusion: A combination of biological gene expression signatures can be used to identify a powerful and independent predictor for outcome in breast cancer patients. [Copyright &y& Elsevier]

The authors comment on the article "Evidence Contrary to Statistical View of Boosting," by David Mease and Abraham Wyner that was published within the issue. The three important properties that contribute to the success of boosting methods are mentioned. The authors claim that the paper attributes consequences of statistical view that are not the case, including the requirement that test error have a unique minimum as a function of the number of included terms.

For 1 dimension reduction in the l1 norm, the method of Cauchy random projections multiplies the original data matrix A ∈ RnxD with a random matrix R ∈ RDxk (k « D) whose entries are i.i.d. samples of the standard Cauchy C(0, 1). Because of the impossibility result, one can not hope to recover the pairwise l1 distances in A from B = A x R ∈ Rnxk, using linear estimators without incurring large errors. However, nonlinear estimators are still useful for certain applications in data stream computations, information retrieval, learning, and data mining. We study three types of nonlinear estimators: the sample median estimators, the geometric mean estimators, and the maximum likelihood estimators (MLE). We derive tail bounds for the geometric mean estimators and establish that k = O (log n/ε²) suffices with the constants explicitly given. Asymptotically (as k → &infinity;), both the sample median and the geometric mean estimators are about 80% efficient compared to the MLE. We analyze the moments of the MLE and propose approximating its distribution of by an inverse Gaussian. [ABSTRACT FROM AUTHOR]

We propose a method for the classification of more than two classes, from high-dimensional features. Our approach is to build a binary decision tree in a top-down manner, using the optimal margin classifier at each split. We implement an exact greedy algorithm for this task, and compare its performance to less greedy procedures based on clustering of the matrix of pairwise margins. We compare the performance of the "margin tree" to the closely related "all-pairs" (one versus one) support vector machine, and nearest centroids on a number of cancer microarray data sets. We also develop a simple method for feature selection. We find that the margin tree has accuracy that is competitive with other methods and offers additional interpretability in its putative grouping of the classes. [ABSTRACT FROM AUTHOR]

Professor Efron has presented us with a thought-provoking paper on the relationship between prediction, estimation, and attribution in the modern era of data science. While we appreciate many of his arguments, we see more of a continuum between the old and new methodology, and the opportunity for both to improve through their synergy. [ABSTRACT FROM AUTHOR]

Principal component analysis (PCA) is widely used in data processing and dimensionality reduction. However, PCA suffers from the fact that each principal component is a linear combination of all the original variables, thus it is often difficult to interpret the results. We introduce a new method called sparse principal component analysis (SPCA) using the lasso (elastic net) to produce modified principal components with sparse loadings. We first show that PCA can be formulated as a regression-type optimization problem; sparse loadings are then obtained by imposing the lasso (elastic net) constraint on the regression coefficients. Efficient algorithms are proposed to fit our SPCA models for both regular multivariate data and gene expression arrays. We also give a new formula to compute the total variance of modified principal components. As illustrations, SPCA is applied to real and simulated data with encouraging results. [ABSTRACT FROM AUTHOR]

In regression problems where the number of predictors greatly exceeds the number of observations, conventional regression techniques may produce unsatisfactory results. We describe a technique called supervised principal components that can be applied to this type of problem. Supervised principal components is similar to conventional principal components analysis except that it uses a subset of the predictors selected based on their association with the outcome. Supervised principal components can be applied to regression and generalized regression problems, such as survival analysis. It compares favorably to other techniques for this type of problem, and can also account for the effects of other covariates and help identify which predictor variables are most important. We also provide asymptotic consistency results to help support our empirical findings. These methods could become important tools for DNA microarray data, where they may be used to more accurately diagnose and treat cancer. [ABSTRACT FROM AUTHOR]

Abstract: Canonical correspondence analysis (CCA) is perhaps the most popular multivariate technique used by environmental ecologists for constrained ordination; it is an approximation to the maximum likelihood solution of the Gaussian response model. In this article, we look at the constrained ordination problem from a slightly different point of view and argue that it is this particular point of view that CCA implicitly adopts. This gives us additional insights into the nature of CCA. We then exploit the new perspective to generalize the Gaussian response model to incorporate more flexible response functions. A real example is presented to illustrate the use of the more flexible model. [Copyright &y& Elsevier]

The support vector machine (SVM) is known for its good performance in two-class classification, but its extension to multiclass classification is still an ongoing research is- sue. In this article, we propose a new approach for classification, called the import vector machine (IVM), which is built on kernel logistic regression (KLR). We show that the IVM not only performs as well as the SVM in two-class classification, but also can naturally be generalized to the multiclass case. Furthermore, the IVM provides an estimate of the underlying probability. Similar to the support points of the SVM, the IVM model uses only a fraction of the training data to index kernel basis functions, typically a much smaller fraction than the SVM. This gives the IVM a potential computational advantage over the SVM. [ABSTRACT FROM AUTHOR]

While Cherkassky and Ma (2003) raise some interesting issues in comparing techniques for model selection, their article appears to be written largely in protest of comparisons made in our book, Elements of Statistical Learning (2001). Cherkassky and Ma feel that we falsely represented the structural risk minimization (SRM) method, which they defend strongly here. In a two-page section of our book (pp. 212-213), we made an honest attempt to compare the SRM method with two related techniques, Aikaike information criterion (AIC) and Bayesian information criterion (BIC). Apparently, we did not apply SRM in the optimal way. We are also accused of using contrived examples, designed to make SRM look bad. Alas, we did introduce some careless errors in our original simulation - errors that were corrected in the second and subsequent printings. Some of these errors were pointed out to us by Cherkassky and Ma (we supplied them with our source code), and as a result we replaced the assessment "SRM performs poorly overall" with a more moderate "the performance of SRM is mixed" (p. 212). These and other corrections can be seen in the errata section on-line at http://www-stat.stanford.edu/ElemStatLearn. [ABSTRACT FROM AUTHOR]

Develops a general method for finding important discriminant directions without assuming the class densities belong to any particular parametric family using a likelihood-based interpretation of linear discriminant analysis criterion. Importance of discriminant directions; Existence of local maxima; Misclassification errors for different one-dimensional Bayes rules.

Reduced-rank regression is a method with great potential for dimension reduction but has found few applications in applied statistics. To address this, reduced-rank regression is proposed for the class of vector generalized linear models (VGLMs), which is very large. The resulting class, which we call reduced-rank VGLMs (RR-VGLMs), enables the benefits of reduced-rank regression to be conveyed to a wide range of data types, including categorical data. RR-VGLMs are illustrated by focussing on models for categorical data, and especially the multinomial logit model. General algorithmic details are provided and software written by the first author is described. The reduced-rank multinomial logit model is illustrated with real data in two contexts: a regression analysis of workforce data and a classification problem. [ABSTRACT FROM AUTHOR]

When using smoothing splines to estimate a function, the user faces the problem of choosing the smoothing parameter. Several techniques are available for selecting this parameter according to certain optimality criteria. Here, we take a different point of view and we propose a technique for choosing between two alternatives, for example allowing for two different levels of degrees of freedom. The problem is addressed in the framework of a mixed‐effects model, whose assumptions ensure that the resulting estimator is unbiased. A likelihood‐ratio‐type test statistic is proposed, and its exact distribution is derived. Tests of linearity and overall effect follow directly. We then extend this idea to additive models where it provides a more attractive alternative than multi‐parameter optimisation, and where it gives exact distributional results that can be used in an analysis‐of‐deviance‐type approach. Examples on real data and a simulation study of level and power complete the paper. [ABSTRACT FROM PUBLISHER]

Background: T7 based linear amplification of RNA is used to obtain sufficient antisense RNA for microarray expression profiling. We optimized and systematically evaluated the fidelity and reproducibility of different amplification protocols using total RNA obtained from primary human breast carcinomas and high-density cDNA microarrays. Results: Using an optimized protocol, the average correlation coefficient of gene expression of 11,123 cDNA clones between amplified and unamplified samples is 0.82 (0.85 when a virtual array was created using repeatedly amplified samples to minimize experimental variation). Less than 4% of genes show changes in expression level by 2-fold or greater after amplification compared to unamplified samples. Most changes due to amplification are not systematic both within one tumor sample and between different tumors. Amplification appears to dampen the variation of gene expression for some genes when compared to unamplified poly(A)+ RNA. The reproducibility between repeatedly amplified samples is 0.97 when performed on the same day, but drops to 0.90 when performed weeks apart. The fidelity and reproducibility of amplification is not affected by decreasing the amount of input total RNA in the 0.3-3 µg range. Adding template-switching primer, DNA ligase, or column purification of double-stranded cDNA does not improve the fidelity of amplification. The correlation coefficient between amplified and unamplified samples is higher when total RNA is used as template for both experimental and reference RNA amplification. Conclusion: T7 based linear amplification reproducibly generates amplified RNA that closely approximates original sample for gene expression profiling using cDNA microarrays. [ABSTRACT FROM AUTHOR]

The elements of a multivariate dataset are often curves rather than single points. Functional principal components can be used to describe the modes of variation of such curves. If one has complete measurements for each individual curve or, as is more common, one has measurements on a fine grid taken at the same time points for all curves, then many standard techniques may be applied. However, curves are often measured at an irregular and sparse set of time points which can differ widely across individuals. We present a technique for handling this more difficult case using a reduced rank mixed effects framework. [ABSTRACT FROM PUBLISHER]

We propose statistical measures for finding masses in mammograms. The measures are based on fitting broken line regressions to local intensity plots of the images. The method is illustrated on a small database of mammograms that have been read by a radiologist and confirmed by operative data. This work illustrates some of the statistical challenges in working with large diagnostic images, [ABSTRACT FROM AUTHOR]

Provides information on plug-in classification technique (PICT), a method that takes a standard classifier and plugs it into an algorithm to produce a new classifier. Suggested PICT for solving class classification problems using binary classifiers; Discussion on coding matrix; Functions of ECOC PICT.

Presents a reaction to the paper by statisticians Ildiko E. Frank and Jerome H. Friedman regarding two methods used in chemometrics for predictive modeling, partial least squares and principal components regression, from a statistical perspective. Penalized least squares; Smooth basis expansions.

Comments on the article about fitting regression models, smoothing and additive. Difficulty of the task of selecting the number and position of the 'knots' for a regression spline; Common method for fitting nonparametric additive models; Noise examples showing Turbo to be far superior to the alternative methods.

Examines the link between least measures and maximum likelihood estimation. Use of Prediction error and Kulback-Leibler (KL) distance; Reference to the deviance function popular in a generalized linear models literature; Details on the pythagorean property of both KL distance and squared root.

Discusses an alternative algorithm in which the smooth functions are estimated using smoothing spline. The generalized cross-validation (GSV) based techniques used to select the smoothing parameters and number of terms; Structure of the automatic smoothing spline projection pursuit (ASP) algorithm; Examples comparing ASP with the standard supersmoother-based PPR algorithm of Friedman.

Fisher's linear discriminant analysis is a valuable tool for multigroup classification With a large number of predictors, one can find a reduced number of discriminant coordinate functions that are "optimal' for separating the groups With two such functions, one can produce a classification map that partitions the reduced space into regions that are identified with group membership, and the decision boundaries are linear This article is about richer nonlinear classification schemes Linear discriminant analysis is equivalent to multiresponse linear regression using optimal scorings to represent the groups In this paper, we obtain nonparametnc versions of discriminant analysis by replacing linear regression by any nonparameinc regression method In this way, any multiresponse regression technique (such as MARS or neural networks) can be postprocessed to improve its classification performance. [ABSTRACT FROM AUTHOR]

Principal curves are smooth one-dimensional curves that pass through the middle of a p-dimensional data set, providing a nonlinear summary of the data. They are nonparametric, and their shape is suggested by the data. The algorithm for constructing principal curves starts with some prior summary, such as the usual principal-component line. The curve in each successive iteration is a smooth or local average of the p-dimensional points, where the definition of local is based on the distance in arc length of the projections of the points onto the curve found in the previous iteration. In this article principal curves are defined, an algorithm for their construction is given, some theoretical results arc presented, and the procedure is compared to other generalizations of principal components. Two applications illustrate the use of principal curves. The first describes how the principal-curve procedure was used to align the magnets of the Stanford linear collider. The collider uses about 950 magnets in a roughly circular arrangement to bend electron and positron beams and bring them to collision. After construction, it was found that some of the magnets had ended up significantly out of place. As a result, the beams had to be bent too sharply and could not be focused. The engineers realized that the magnets did not have to be moved to their originally planned locations, but rather to a sufficiently smooth are through the middle of the existing positions. This arc was found using the principal curve procedure. In the second application, two different assays for gold content in several samples of computer chip waste appear to show some systematic differences that are blurred by measurement error. The classical approach using linear errors in variables regression can detect systematic linear differences but is not able to account for nonlinearities. When the first linear principal component is replaced with a... [ABSTRACT FROM AUTHOR]

A scatterplot smoother is applied to data of the form {(x[sub 1], y[sub 1]), (x[sub 2], y[sub 2]),..., (x[sub n], y[sub n])} and uses local fitting to estimate the dependence of Y on X. A simple example is the running lines smoother, which fits a least squares line to the y values falling in a window around each x value. The value of the estimated function at x is given by the value of the least squares line at x. A smoother generalizes the least squares line, which assumes that the dependence of Y on X is linear. In this article, we extend the idea of local fitting to likelihood-based regression models. One such application is to the class of generalized linear models (Nelder and Wedderburn 1972). We enlarge this class by replacing the covariate form beta[sub 0] + x beta[sub 1] with an unspecified smooth function s(x). This function is estimated from the data by a technique we call local likelihood estimation. The method consists of maximum likelihood estimation for beta[sub 0] and beta[sub 1], applied in a window around each x value. Multiple covariates are incorporated through an iterative algorithm. We also apply the local likelihood technique to the proportional hazards model of Cox (1972), a model for censored data. The proportional hazards assumption lambda(t | x) = lambda[sub 0](t)exp(x beta) is replaced by lambda(t | x) = lambda[sub 0](t)exp(s(x)), and the function s(x) is estimated from the data by the local likelihood method. In some real data examples, the local likelihood technique proves to be effective in uncovering nonlinear dependencies. It is useful as a descriptive tool or to suggest transformations of the covariates. We also discuss some methods for inference. [ABSTRACT FROM AUTHOR]

Correspondence analysis is an exploratory multivariate technique that converts a data matrix into a particular type of graphical display in which the rows and columns are depicted as points. The method has a long and varied history and has appeared in different forms in the psychometric and ecological literature, among others. In this article we review the geometry of correspondence analysis and its geometric interpretation. We also discuss various extensions of correspondence analysis to multivariate categorical data (multiple correspondence analysis) and a variety of other data types. [ABSTRACT FROM AUTHOR]