Back to Search
Start Over
Forward layer-wise learning of convolutional neural networks through separation index maximizing.
- Source :
- Scientific Reports; 4/13/2024, Vol. 14 Issue 1, p1-12, 12p
- Publication Year :
- 2024
-
Abstract
- This paper proposes a forward layer-wise learning algorithm for CNNs in classification problems. The algorithm utilizes the Separation Index (SI) as a supervised complexity measure to evaluate and train each layer in a forward manner. The proposed method explains that gradually increasing the SI through layers reduces the input data's uncertainties and disturbances, achieving a better feature space representation. Hence, by approximating the SI with a variant of local triplet loss at each layer, a gradient-based learning algorithm is suggested to maximize it. Inspired by the NGRAD (Neural Gradient Representation by Activity Differences) hypothesis, the proposed algorithm operates in a forward manner without explicit error information from the last layer. The algorithm's performance is evaluated on image classification tasks using VGG16, VGG19, AlexNet, and LeNet architectures with CIFAR-10, CIFAR-100, Raabin-WBC, and Fashion-MNIST datasets. Additionally, the experiments are applied to text classification tasks using the DBPedia and AG's News datasets. The results demonstrate that the proposed layer-wise learning algorithm outperforms state-of-the-art methods in accuracy and time complexity. [ABSTRACT FROM AUTHOR]
Details
- Language :
- English
- ISSN :
- 20452322
- Volume :
- 14
- Issue :
- 1
- Database :
- Complementary Index
- Journal :
- Scientific Reports
- Publication Type :
- Academic Journal
- Accession number :
- 176583991
- Full Text :
- https://doi.org/10.1038/s41598-024-59176-3