Search

Object detection technology is a popular research direction which is widely used in areas such as autonomous driving and medical diagnosis. At this stage mobile devices often have limited storage resources to deploy large object detection networks and need to meet real-time requirements. This paper proposes a lightweight and efficient object detection model based on YOLOv4, first using the lightweight network GhostNet to extract image features and reduce the number of parameters and computation of the backbone structure; then combining AFmodule and Meta-ACON activation function to enhance the feature extraction capability of the backbone network, which strengthen the mode's ability to capture image spatial feature information; this paper also designs the RL-PAFPN feature fusion structure is with the Reslayer module to further improve the model's ability to extract and fuse image features. By comparing other mainstream object detection models, the YOLOv4-Ghost-AMR network in this paper has less computation and fewer parameters, and the accuracy of the model reaches 86.83%, which is suitable for deployment in mobile devices with limited storage. The model proposed in this paper can be applied to medical, traffic and fault detection fields, which changes the traditional manual detection method and saves manpower and time costs, achieving high precision real-time object detection. [ABSTRACT FROM AUTHOR]

The activation function introduces nonlinearity into convolutional neural network, which greatly promotes the development of computer vision tasks. This paper proposes elastic adaptively parametric compounded units to improve the performance of convolutional neural networks for image recognition. The activation function takes the structural advantages of two mainstream functions as the function's fundamental architecture. The SENet model is embedded in the proposed activation function to adaptively recalibrate the feature mapping weight in each channel, thereby enhancing the fitting capability of the activation function. In addition, the function has an elastic slope in the positive input region by simulating random noise to improve the generalization capability of neural networks. To prevent the generated noise from producing overly large variations during training, a special protection mechanism is adopted. In order to verify the effectiveness of the activation function, this paper uses CIFAR-10 and CIFAR-100 image datasets to conduct comparative experiments of the activation function under the exact same model. Experimental results show that the proposed activation function showed superior performance beyond other functions. [ABSTRACT FROM AUTHOR]

Funahashi established that the space of two-layer feedforward neural networks is dense in the space of all continuous functions defined over compact sets in n-dimensional Euclidean space. The purpose of this short survey is to reexamine the proof of Theorem 1 in Funahashi [3]. The Tietze extension theorem, whose proof is contained in the appendix, will be used. This paper is based on harmonic analysis, real analysis, and Fourier analysis. However, the audience in this paper is supposed to be researchers who do not specialize in these fields of mathematics. Some fundamental facts that are used in this paper without proofs will be collected after we present some notation in this paper. [ABSTRACT FROM AUTHOR]

The selection of weight initialization in an artificial neural network is one of the key aspects and affects the learning speed, convergence rate and correctness of classification by an artificial neural network. In this paper, we investigate the effects of weight initialization in an artificial neural network. Nguyen-Widrow weight initialization, random initialization, and Xavier initialization method are paired with five different activation functions. This paper deals with a feedforward neural network, consisting of an input layer, a hidden layer, and an output layer. The paired combination of weight initialization methods with activation functions are examined and tested and compared based on their best achieved loss rate in training. This work aims to better understand how weight initialization methods in neural networks, in combination with activation functions, affect the learning speed in comparison after a fixed number of training epochs. [ABSTRACT FROM AUTHOR]

Compared with other facial expression recognition, classroom facial expression recognition should pay more attention to the feature extraction of a specific region to reflect the attention of students. However, most features are extracted with complete facial images by deep neural networks. In this paper, we proposed a new expression recognition based on attention mechanism, where more attention would be paid in the channel information which have much relationship with the expression classification instead of depending on all channel information. A new classroom expression classification has also been concluded with considering the concentration. Moreover, activation function is modified to reduce the number of parameters and computations, at the same time, dropout regularization is added after the pool layer to prevent overfitting of the model. The experiments show that the accuracy of our method named Ixception has an maximize improvement of 5.25% than other algorithms. It can well meet the requirements of the analysis of classroom concentration. [ABSTRACT FROM AUTHOR]

Precise and rapid detection of seatbelts is an essential research field for intelligent traffic management. In order to improve the detection precision of seatbelts and speed up algorithm inference velocity, a lightweight seatbelt detection algorithm is proposed. Firstly, by adding the G-ELAN module designed in this paper to the YOLOv7-tiny network, the optimization of construction and reduction of parameters are accomplished, and the ResNet is compressed with the channel pruning approach to decrease computational overheads. Then, the Mish activation function is utilized to replace the Leaky Relu in the neck to enhance the non-linear competence of the network. Finally, the triplet attention module is integrated into the model after pruning to make up for the underlying performance reduction caused by the previous stage and upgrade overall detection precision. The experimental results based on the self-built seatbelt dataset showed that, compared to the initial network, the Mean Average Precision (mAP) achieved by the proposed GM-YOLOv7 was improved by 3.8%, while the volume and the computation amount were lowered by 20% and 24.6%, respectively. Compared with YOLOv3, YOLOX, and YOLOv5, the mAP of GM-YOLOv7 increased by 22.4%, 4.6%, and 4.2%, respectively, and the number of computational operations decreased by 25%, 63%, and 38%, respectively. In addition, the accuracy of the improved RST-Net increased to 98.25%, while the parameter value was reduced by 48% compared to the basic model, effectively improving the detection performance and realizing a lightweight structure. [ABSTRACT FROM AUTHOR]

Semantic segmentation of remote sensing (RS) images is a pivotal branch in the realm of RS image processing, which plays a significant role in urban planning, building extraction, vegetation extraction, etc. With the continuous advancement of remote sensing technology, the spatial resolution of remote sensing images is progressively improving. This escalation in resolution gives rise to challenges like imbalanced class distributions among ground objects in RS images, the significant variations of ground object scales, as well as the presence of redundant information and noise interference. In this paper, we propose a multi-scale context extraction network, ASPP+-LANet, based on the LANet for semantic segmentation of high-resolution RS images. Firstly, we design an ASPP+ module, expanding upon the ASPP module by incorporating an additional feature extraction channel, redesigning the dilation rates, and introducing the Coordinate Attention (CA) mechanism so that it can effectively improve the segmentation performance of ground object targets at different scales. Secondly, we introduce the Funnel ReLU (FReLU) activation function for enhancing the segmentation effect of slender ground object targets and refining the segmentation edges. The experimental results show that our network model demonstrates superior segmentation performance on both Potsdam and Vaihingen datasets, outperforming other state-of-the-art (SOTA) methods. [ABSTRACT FROM AUTHOR]

Here we study the quantitative multivariate approximation of perturbed hyperbolic tangent-activated singular integral operators to the unit operator. The engaged neural network activation function is both parametrized and deformed, and the related kernel is a density function on R N . We exhibit uniform and L p , p ≥ 1 approximations via Jackson-type inequalities involving the first L p modulus of smoothness, 1 ≤ p ≤ ∞ . The differentiability of our multivariate functions is covered extensively in our approximations. We continue by detailing the global smoothness preservation results of our operators. We conclude the paper with the simultaneous approximation and the simultaneous global smoothness preservation by our multivariate perturbed activated singular integrals. [ABSTRACT FROM AUTHOR]

The paper introduces two novel activation functions known as modExp and modExpm. The activation functions possess several desirable properties, such as being continuously differentiable, bounded, smooth, and non-monotonic. Our studies have shown that modExp and modExpm consistently outperform ReLU and other activation functions across a range of challenging datasets and complex models. Initially, the experiments involve training and classifying using a multi-layer perceptron (MLP) on benchmark data sets like the Diagnostic Wisconsin Breast Cancer and Iris Flower datasets. Both modExp and modExpm demonstrate impressive performance, with modExp achieving 94.15 and 95.56% and modExpm achieving 94.15 and 95.56% respectively, when compared to ReLU, ELU, Tanh, Mish, Softsign, Leaky ReLU, and TanhExp. In addition, a series of experiments were carried out on five different depths of deeper neural networks, ranging from five to eight layers, using MNIST datasets. The modExpm activation function demonstrated superior performance accuracy on various neural network configurations, achieving 95.56, 95.43, 94.72, 95.14, and 95.61% on wider 5 layers, slimmer 5 layers, 6 layers, 7 layers, and 8 layers respectively. The modExp activation function also performed well, achieving the second highest accuracy of 95.42, 94.33, 94.76, 95.06, and 95.37% on the same network configurations, outperforming ReLU, ELU, Tanh, Mish, Softsign, Leaky ReLU, and TanhExp. The results of the statistical feature measures show that both activation functions have the highest mean accuracy, the lowest standard deviation, the lowest Root Mean squared Error, the lowest variance, and the lowest Mean squared Error. According to the experiment, both functions converge more quickly than ReLU, which is a significant advantage in Neural network learning. [ABSTRACT FROM AUTHOR]

End-to-end disparity estimation algorithms based on cost volume deployed in edge-end neural network accelerators have the problem of structural adaptation and need to ensure accuracy under the condition of adaptation operator. Therefore, this paper proposes a novel disparity calculation algorithm that uses low-rank approximation to approximately replace 3D convolution and transposed 3D convolution, WReLU to reduce data compression caused by the activation function, and unimodal cost volume filtering and a confidence estimation network to regularize cost volume. It alleviates the problem of disparity-matching cost distribution being far away from the true distribution and greatly reduces the computational complexity and number of parameters of the algorithm while improving accuracy. Experimental results show that compared with a typical disparity estimation network, the absolute error of the proposed algorithm is reduced by 38.3%, the three-pixel error is reduced to 1.41%, and the number of parameters is reduced by 67.3%. The calculation accuracy is better than that of other algorithms, it is easier to deploy, and it has strong structural adaptability and better practicability. [ABSTRACT FROM AUTHOR]

We propose a neural-network-based watermarking method that introduces the quantized activation function that approximates the quantization of JPEG compression. Many neural-network-based watermarking methods have been proposed. Conventional methods have acquired robustness against various attacks by introducing an attack simulation layer between the embedding network and the extraction network. The quantization process of JPEG compression is replaced by the noise addition process in the attack layer of conventional methods. In this paper, we propose a quantized activation function that can simulate the JPEG quantization standard as it is in order to improve the robustness against the JPEG compression. Our quantized activation function consists of several hyperbolic tangent functions and is applied as an activation function for neural networks. Our network was introduced in the attack layer of ReDMark proposed by Ahmadi et al. to compare it with their method. That is, the embedding and extraction networks had the same structure. We compared the usual JPEG compressed images and the images applying the quantized activation function. The results showed that a network with quantized activation functions can approximate JPEG compression with high accuracy. We also compared the bit error rate (BER) of estimated watermarks generated by our network with those generated by ReDMark. We found that our network was able to produce estimated watermarks with lower BERs than those of ReDMark. Therefore, our network outperformed the conventional method with respect to image quality and BER. [ABSTRACT FROM AUTHOR]

Recent research has found that the activation function (AF) plays a significant role in introducing non-linearity to enhance the performance of deep learning networks. Researchers recently started developing activation functions that can be trained throughout the learning process, known as trainable, or adaptive activation functions (AAF). Research on AAF that enhances the outcomes is still in its early stages. In this paper, a novel activation function ‘ErfReLU’ has been developed based on the erf function and ReLU. This function leverages the advantages of both the Rectified Linear Unit (ReLU) and the error function (erf). A comprehensive overview of activation functions like Sigmoid, ReLU, Tanh, and their properties have been briefly explained. Adaptive activation functions like Tanhsoft1, Tanhsoft2, Tanhsoft3, TanhLU, SAAF, ErfAct, Pserf, Smish, and Serf is also presented. Lastly, comparative performance analysis of 9 trainable activation functions namely Tanhsoft1, Tanhsoft2, Tanhsoft3, TanhLU, SAAF, ErfAct, Pserf, Smish, and Serf with the proposed one has been performed. These activation functions are used in MobileNet, VGG16, and ResNet models and their performance is evaluated on benchmark datasets such as CIFAR-10, MNIST, and FMNIST. [ABSTRACT FROM AUTHOR]

As one of the important parts of Neural Network, activation function plays a very important role in model training in Neural Network. In this paper, the status quo, advantages and disadvantages of the existing common activation functions are analysed, and a new activation function is proposed and applied to target detection. To test the performance of the new activation function, this paper compares it with the ReLU activation functions on a variety of Neural Networks and data sets, and not only analyses the performance of the activation function itself but also verifies the effectiveness of the activation function in target detection. [ABSTRACT FROM AUTHOR]

Unsupervised learning based on restricted Boltzmann machine or autoencoders has become an important research domain in the area of neural networks. In this paper mathematical expressions to adaptive learning step calculation for RBM with ReLU transfer function are proposed. As a result, we can automatically estimate the step size that minimizes the loss function of the neural network and correspondingly update the learning step in every iteration. We give a theoretical justification for the proposed adaptive learning rate approach, which is based on the steepest descent method. The proposed technique for adaptive learning rate estimation is compared with the existing constant step and Adam methods in terms of generalization ability and loss function. We demonstrate that the proposed approach provides better performance. [ABSTRACT FROM AUTHOR]

Copyright of Automotive Digest is the property of Automotive Digest Editorial Office and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Generative Adversarial Networks (GANs) have surfaced as a transformative approach in the domain of cryptography, introducing a novel paradigm where two neural networks, the generator (akin to Alice) and the discriminator (akin to Bob), are pitted against each other in a cryptographic setting. A third network, representing Eve, attempts to decipher the encrypted information. The efficacy of this encryption–decryption process is deeply intertwined with the choice of activation functions employed within these networks. This study conducted a comparative analysis of four widely used activation functions within a standardized GAN framework. Our recent explorations underscore the superior performance achieved when utilizing the Rectified Linear Unit (ReLU) in the hidden layers combined with the Sigmoid activation function in the output layer. The non-linear nature introduced by the ReLU provides a sophisticated encryption pattern, rendering the deciphering process for Eve intricate. Simultaneously, the Sigmoid function in the output layer guarantees that the encrypted and decrypted messages are confined within a consistent range, facilitating a straightforward comparison with original messages. The amalgamation of these activation functions not only bolsters the encryption strength but also ensures the fidelity of the decrypted messages. These findings not only shed light on the optimal design considerations for GAN-based cryptographic systems but also underscore the potential of investigating hybrid activation functions for enhanced system optimization. In our exploration of cryptographic strength and training efficiency using various activation functions, we discovered that the "ReLU and Sigmoid" combination significantly outperforms the others, demonstrating superior security and a markedly efficient mean training time of 16.51 s per 2000 steps. This highlights the enduring effectiveness of established methodologies in cryptographic applications. This paper elucidates the implications of these choices, advocating for their adoption in GAN-based cryptographic models, given the superior results they yield in ensuring security and accuracy. [ABSTRACT FROM AUTHOR]

Detecting and classifying vehicles as objects from images and videos is challenging in appearance-based representation, yet plays a significant role in the substantial real-time applications of Intelligent Transportation Systems (ITSs). The rapid development of Deep Learning (DL) has resulted in the computer-vision community demanding efficient, robust, and outstanding services to be built in various fields. This paper covers a wide range of vehicle detection and classification approaches and the application of these in estimating traffic density, real-time targets, toll management and other areas using DL architectures. Moreover, the paper also presents a detailed analysis of DL techniques, benchmark datasets, and preliminaries. A survey of some vital detection and classification applications, namely, vehicle detection and classification and performance, is conducted, with a detailed investigation of the challenges faced. The paper also addresses the promising technological advancements of the last few years. [ABSTRACT FROM AUTHOR]

Recently, deep learning has achieved remarkable results in fields that require human cognitive ability, learning ability, and reasoning ability. Activation functions are very important because they provide the ability of artificial neural networks to learn complex patterns through nonlinearity. Various activation functions are being studied to solve problems such as vanishing gradients and dying nodes that may occur in the deep learning process. However, it takes a lot of time and effort for researchers to use the existing activation function in their research. Therefore, in this paper, we propose a universal activation function (UA) so that researchers can easily create and apply various activation functions and improve the performance of neural networks. UA can generate new types of activation functions as well as functions like traditional activation functions by properly adjusting three hyperparameters. The famous Convolutional Neural Network (CNN) and benchmark dataset were used to evaluate the experimental performance of the UA proposed in this study. We compared the performance of the artificial neural network to which the traditional activation function is applied and the artificial neural network to which the UA is applied. In addition, we evaluated the performance of the new activation function generated by adjusting the hyperparameters of the UA. The experimental performance evaluation results showed that the classification performance of CNNs improved by up to 5% through the UA, although most of them showed similar performance to the traditional activation function. [ABSTRACT FROM AUTHOR]

Image captioning extracts multiple semantic features from an image and integrates them into a sentence-level description. For efficient description of the captions, it becomes necessary to learn higher order interactions between detected objects and the relationship among them. Most of the existing systems take into account the first order interactions while ignoring the higher order ones. It is challenging to extract discriminant higher order semantics visual features in images with highly populated objects for caption generation. In this paper, an efficient higher order interaction learning framework is proposed using encoder-decoder based image captioning. A scaled version of Gaussian Error Linear Unit (GELU) activation function, x-GELU is introduced that controls the vanishing gradients and enhances the feature learning. To leverage higher order interactions among multiple objects, an efficient XGL Transformer (XGL-T) model is introduced that exploits both spatial and channel-wise attention by integrating four XGL attention modules in image encoder and one in Bilinear Long Short-Term Memory guided sentence decoder. The proposed model captures rich semantic concepts from objects, attributes, and their relationships. Extensive experiments are conducted on publicly available MSCOCO Karapathy test split and the best performance of the work is observed as 81.5 BLEU@1, 67.1 BLEU@2, 51.6 BLEU@3, 39.9 BLEU@4, 134 CIDEr, 59.9 ROUGE-L, 29.8 METEOR, 23.8 SPICE using CIDEr-D Score Optimization Strategy. The scores validate the significant improvements over state-of-the-art results. An ablation study is also carried out to support the experimental observations. [ABSTRACT FROM AUTHOR]

In this paper, our main aim is to present innovative fixed-point theorems that provide solutions to the fixed-circle problem with symmetric contractions. We accomplish this by employing operator enrichment techniques within the context of Banach spaces. Furthermore, we demonstrate the practical application of these theorems by showcasing their relevance to the rectified linear unit (ReLU) activation function. By exploring the connection between fixed points and activation functions, our work contributes to a deeper understanding of the behavior and properties of these fundamental mathematical concepts. [ABSTRACT FROM AUTHOR]

The main objective of this paper is to improve the efficiency and accuracy of convolutional neural network models for image classification and denoising tasks. The focus of the study is on enhancing the activation layer of these models, which is a critical component that determines the output of each neuron in the network. To achieve this goal, we propose a novel activation function based on Chebyshev polynomials, which is both data-driven and self-learnable. In addition to proposing the LIP model, the authors investigate its performance in approximating various nonlinearities and determine its Lipschitz bound. The study then evaluates the performance of the proposed activation function by conducting experiments on multiple datasets using different convolutional neural network models. The results show that the proposed activation function outperforms other activation layers and significantly enhances the accuracy of image classification and denoising tasks. [ABSTRACT FROM AUTHOR]