Approaches to Combining Local and Evolutionary Search for Training Neural Networks: A Review and Some New Results

Training of neural networks by local search such as gradient-based algorithms could be difficult. This calls for the development of alternative training algorithms such as evolutionary search. However, training by evolutionary search often requires long computation time. In this chapter, we investigate the possibilities of reducing the time taken by combining the efforts of local search and evolutionary search. There are a number of attempts to combine these search strategies, but not all of them are successful. This chapter provides a critical review of these attempts. Moreover, different approaches to combining evolutionary search and local search are compared. Experimental results indicate that while the Baldwinian and the two-phase approaches are inefficient in improving the evolution process for difficult problems, the Lamarckian approach is able to speed up the training process and to improve the solution quality. In this chapter, the strength and weakness of these approaches are illustrated, and the factors affecting their efficiency and applicability are discussed.