1. Bootstrapping with Noise: An Effective Regularization Technique
- Author
-
Yuval Raviv and Nathan Intrator
- Subjects
Artificial neural network ,business.industry ,Computer science ,Bootstrap aggregating ,Generalized additive model ,Ensemble averaging ,Feed forward ,Estimator ,Regularization (mathematics) ,Human-Computer Interaction ,Bootstrapping (electronics) ,Artificial Intelligence ,Artificial intelligence ,business ,Algorithm ,Software - Abstract
Bootstrap samples with noise are shown to be an effective smoothness and capacity control technique for training feedforward networks and for other statistical methods such as generalized additive models. It is shown that noisy bootstrap performs best in conjunction with weight-decay regularization and ensemble averaging. The two-spiral problem, a highly non-linear, noise-free data, is used to demonstrate these findings. The combination of noisy bootstrap and ensemble averaging is also shown useful for generalized additive modelling, and is also demonstrated on the well-known Cleveland heart data.
- Published
- 1996