Search

Mean squared error is used to compare five regression estimators: Least Squares, Principal Components, Ridge Regression, Latent Root, and a Shrunken estimator. Each of the biased estimators is shown to offer improvement in mean squared error over Least Squares for a wide range of choices of the parameters of the model. Using the results of a simulation involving all five estimators, the Principal Components and Latent Root estimators are seen to perform best overall but the Ridge Regression estimator has the potential of a smaller mean squared error than either of these providing a better estimator of the ridge parameter can be found.

Ridge regression, as developed by Hoerl and Kennard has been extended to apply to situations where a priori expected values and variances are available for the elements of the unknown parameter vector being estimated. An application involving orbit determination with estimation of various radar coefficients is presented.

The report begins with a brief description of the analytic setting in which regression methods are applied in defense systems costing. Because of the characteristics of the data with which cost analysts typically work, classical least Squares can be a very unsatisfactory estimation procedure. This has prompted a number of analysts to turn to Hoerl and Kennard's relatively new procedure known as Ridge Analysis, or Ridge Regression. It is shown that straightforward application of this method involves an implicit prior assignment of zero values to each of the unknown regression parameters being estimated. A rationale and method for introducing explicit non-zero priors is developed, and the theoretical properties of these estimates are examined. (Modified author abstract)

The report explores the statistical properties of a class of estimators, known as either ridge analysis or ridge regression, proposed as an alternative to ordinary least squares (OLS) regression in analyzing sample data that are collinear. Using Monte Carlo techniques, various ridge estimation procedures were evaluated. All the ridge estimators did worse than OLS for at least some choices of the true regression coefficient in the models considered. It thus appears that the ridge estimators proposed to date are not a viable alternative to OLS. However, the results show that it might be possible to define a ridge estimator that would be better than OLS. Until the properties of such an estimator are rigorously derived, the authors caution against using ridge analysis to estimate regression coefficients. (Author)