Second-Order Training Methods for MLPs

The idea of using second-order methods to train multi-layer perceptrons is far from a new one. The earliest publications on this subject to reach a wide audience were probably those of Parker (1987) and Watrous (1987); now, a decade later, the catalogue of published MLP research concerned with second-order training methods is substantial1. The aim of this chapter is to explain the major ways in which neural net researchers have sought to adapt the various second-order classical techniques of Chapters 2 and 3 to the particular requirements of MLP training.