E-Tanh: a novel activation function for image processing neural network models.

Artificial neural network (ANN) is one of the technologies used for emerging real-world problems. Activation functions (AF) are used in deep learning architectures to make decisions in the hidden and output layers. An AF influences the dynamics of training and performance of an ANN. This paper proposes a novel AF called E-Tanh by extending the Tanh AF for ANN models. When used in three shallow ANN models with 2, 4, and 6 hidden layers with the Modified National Institute of Standards and Technology database (MNIST), it gave a maximum accuracy of 99.5% for handwritten digits classification than the results produced by the existing AFs. The proposed AF E-Tanh outperformed other existing and well-known AFs ReLU, Sigmoid, Tanh, Swish, and E-Swish. In deep ANNs with 5–50 hidden layers, the proposed AF's prediction accuracy of up to 25 hidden layers was higher than the existing AFs on the MNIST database. When used in shallow CNN for the CIFAR10 database, it outperformed other AFs. When used in wide residual network, it performed poorly in classifying images in CIFAR10 and CIFAR100 database but performed competitively with all other AFs in CNN models on the MNIST dataset. [ABSTRACT FROM AUTHOR]