1. Mitigating Catastrophic Forgetting In Adaptive Class Incremental Extreme Learning Machine Through Neuron Clustering
- Author
-
Ghalib Ahmed Tahir and Chu Kiong Loo
- Subjects
0303 health sciences ,Forgetting ,Quantitative Biology::Neurons and Cognition ,Artificial neural network ,business.industry ,Computer science ,Activation function ,02 engineering and technology ,03 medical and health sciences ,medicine.anatomical_structure ,Generalization (learning) ,Incremental learning ,0202 electrical engineering, electronic engineering, information engineering ,medicine ,020201 artificial intelligence & image processing ,Neuron ,Artificial intelligence ,Representation (mathematics) ,business ,Cluster analysis ,030304 developmental biology ,Extreme learning machine - Abstract
Catastrophic forgetting is a major problem that affects neural networks during progressive learning. In it, the previously learned representation vanishes as the network learns new information. The extreme learning machine is one of the variants of the neural network. It is used in many domains due to fast training and good generalization ability. However, like other neural networks, it suffers from catastrophic forgetting and negative forward and backward transfer during the progression of neurons in incremental learning. The study hypothesizes that it is due to overlapping in hidden neurons and output weights. The global representation by an activation function further supports this hypothesis. To address this, the study proposes a neuron clustering approach to mitigate it in an adaptive class incremental extreme learning machine. The neuron clustering method activates k nearest neurons during learning and testing. It helps to partition the network to select overlapping subnetwork. Experimental results on four food datasets show that the proposed approach reduces negative forward and backward transfer when neurons are added incrementally during progressive learning.
- Published
- 2020