Search

Object detection technology is a popular research direction which is widely used in areas such as autonomous driving and medical diagnosis. At this stage mobile devices often have limited storage resources to deploy large object detection networks and need to meet real-time requirements. This paper proposes a lightweight and efficient object detection model based on YOLOv4, first using the lightweight network GhostNet to extract image features and reduce the number of parameters and computation of the backbone structure; then combining AFmodule and Meta-ACON activation function to enhance the feature extraction capability of the backbone network, which strengthen the mode's ability to capture image spatial feature information; this paper also designs the RL-PAFPN feature fusion structure is with the Reslayer module to further improve the model's ability to extract and fuse image features. By comparing other mainstream object detection models, the YOLOv4-Ghost-AMR network in this paper has less computation and fewer parameters, and the accuracy of the model reaches 86.83%, which is suitable for deployment in mobile devices with limited storage. The model proposed in this paper can be applied to medical, traffic and fault detection fields, which changes the traditional manual detection method and saves manpower and time costs, achieving high precision real-time object detection. [ABSTRACT FROM AUTHOR]

The activation function introduces nonlinearity into convolutional neural network, which greatly promotes the development of computer vision tasks. This paper proposes elastic adaptively parametric compounded units to improve the performance of convolutional neural networks for image recognition. The activation function takes the structural advantages of two mainstream functions as the function's fundamental architecture. The SENet model is embedded in the proposed activation function to adaptively recalibrate the feature mapping weight in each channel, thereby enhancing the fitting capability of the activation function. In addition, the function has an elastic slope in the positive input region by simulating random noise to improve the generalization capability of neural networks. To prevent the generated noise from producing overly large variations during training, a special protection mechanism is adopted. In order to verify the effectiveness of the activation function, this paper uses CIFAR-10 and CIFAR-100 image datasets to conduct comparative experiments of the activation function under the exact same model. Experimental results show that the proposed activation function showed superior performance beyond other functions. [ABSTRACT FROM AUTHOR]

The selection of weight initialization in an artificial neural network is one of the key aspects and affects the learning speed, convergence rate and correctness of classification by an artificial neural network. In this paper, we investigate the effects of weight initialization in an artificial neural network. Nguyen-Widrow weight initialization, random initialization, and Xavier initialization method are paired with five different activation functions. This paper deals with a feedforward neural network, consisting of an input layer, a hidden layer, and an output layer. The paired combination of weight initialization methods with activation functions are examined and tested and compared based on their best achieved loss rate in training. This work aims to better understand how weight initialization methods in neural networks, in combination with activation functions, affect the learning speed in comparison after a fixed number of training epochs. [ABSTRACT FROM AUTHOR]

Precise and rapid detection of seatbelts is an essential research field for intelligent traffic management. In order to improve the detection precision of seatbelts and speed up algorithm inference velocity, a lightweight seatbelt detection algorithm is proposed. Firstly, by adding the G-ELAN module designed in this paper to the YOLOv7-tiny network, the optimization of construction and reduction of parameters are accomplished, and the ResNet is compressed with the channel pruning approach to decrease computational overheads. Then, the Mish activation function is utilized to replace the Leaky Relu in the neck to enhance the non-linear competence of the network. Finally, the triplet attention module is integrated into the model after pruning to make up for the underlying performance reduction caused by the previous stage and upgrade overall detection precision. The experimental results based on the self-built seatbelt dataset showed that, compared to the initial network, the Mean Average Precision (mAP) achieved by the proposed GM-YOLOv7 was improved by 3.8%, while the volume and the computation amount were lowered by 20% and 24.6%, respectively. Compared with YOLOv3, YOLOX, and YOLOv5, the mAP of GM-YOLOv7 increased by 22.4%, 4.6%, and 4.2%, respectively, and the number of computational operations decreased by 25%, 63%, and 38%, respectively. In addition, the accuracy of the improved RST-Net increased to 98.25%, while the parameter value was reduced by 48% compared to the basic model, effectively improving the detection performance and realizing a lightweight structure. [ABSTRACT FROM AUTHOR]

Semantic segmentation of remote sensing (RS) images is a pivotal branch in the realm of RS image processing, which plays a significant role in urban planning, building extraction, vegetation extraction, etc. With the continuous advancement of remote sensing technology, the spatial resolution of remote sensing images is progressively improving. This escalation in resolution gives rise to challenges like imbalanced class distributions among ground objects in RS images, the significant variations of ground object scales, as well as the presence of redundant information and noise interference. In this paper, we propose a multi-scale context extraction network, ASPP+-LANet, based on the LANet for semantic segmentation of high-resolution RS images. Firstly, we design an ASPP+ module, expanding upon the ASPP module by incorporating an additional feature extraction channel, redesigning the dilation rates, and introducing the Coordinate Attention (CA) mechanism so that it can effectively improve the segmentation performance of ground object targets at different scales. Secondly, we introduce the Funnel ReLU (FReLU) activation function for enhancing the segmentation effect of slender ground object targets and refining the segmentation edges. The experimental results show that our network model demonstrates superior segmentation performance on both Potsdam and Vaihingen datasets, outperforming other state-of-the-art (SOTA) methods. [ABSTRACT FROM AUTHOR]

End-to-end disparity estimation algorithms based on cost volume deployed in edge-end neural network accelerators have the problem of structural adaptation and need to ensure accuracy under the condition of adaptation operator. Therefore, this paper proposes a novel disparity calculation algorithm that uses low-rank approximation to approximately replace 3D convolution and transposed 3D convolution, WReLU to reduce data compression caused by the activation function, and unimodal cost volume filtering and a confidence estimation network to regularize cost volume. It alleviates the problem of disparity-matching cost distribution being far away from the true distribution and greatly reduces the computational complexity and number of parameters of the algorithm while improving accuracy. Experimental results show that compared with a typical disparity estimation network, the absolute error of the proposed algorithm is reduced by 38.3%, the three-pixel error is reduced to 1.41%, and the number of parameters is reduced by 67.3%. The calculation accuracy is better than that of other algorithms, it is easier to deploy, and it has strong structural adaptability and better practicability. [ABSTRACT FROM AUTHOR]

Recent research has found that the activation function (AF) plays a significant role in introducing non-linearity to enhance the performance of deep learning networks. Researchers recently started developing activation functions that can be trained throughout the learning process, known as trainable, or adaptive activation functions (AAF). Research on AAF that enhances the outcomes is still in its early stages. In this paper, a novel activation function ‘ErfReLU’ has been developed based on the erf function and ReLU. This function leverages the advantages of both the Rectified Linear Unit (ReLU) and the error function (erf). A comprehensive overview of activation functions like Sigmoid, ReLU, Tanh, and their properties have been briefly explained. Adaptive activation functions like Tanhsoft1, Tanhsoft2, Tanhsoft3, TanhLU, SAAF, ErfAct, Pserf, Smish, and Serf is also presented. Lastly, comparative performance analysis of 9 trainable activation functions namely Tanhsoft1, Tanhsoft2, Tanhsoft3, TanhLU, SAAF, ErfAct, Pserf, Smish, and Serf with the proposed one has been performed. These activation functions are used in MobileNet, VGG16, and ResNet models and their performance is evaluated on benchmark datasets such as CIFAR-10, MNIST, and FMNIST. [ABSTRACT FROM AUTHOR]

We propose a neural-network-based watermarking method that introduces the quantized activation function that approximates the quantization of JPEG compression. Many neural-network-based watermarking methods have been proposed. Conventional methods have acquired robustness against various attacks by introducing an attack simulation layer between the embedding network and the extraction network. The quantization process of JPEG compression is replaced by the noise addition process in the attack layer of conventional methods. In this paper, we propose a quantized activation function that can simulate the JPEG quantization standard as it is in order to improve the robustness against the JPEG compression. Our quantized activation function consists of several hyperbolic tangent functions and is applied as an activation function for neural networks. Our network was introduced in the attack layer of ReDMark proposed by Ahmadi et al. to compare it with their method. That is, the embedding and extraction networks had the same structure. We compared the usual JPEG compressed images and the images applying the quantized activation function. The results showed that a network with quantized activation functions can approximate JPEG compression with high accuracy. We also compared the bit error rate (BER) of estimated watermarks generated by our network with those generated by ReDMark. We found that our network was able to produce estimated watermarks with lower BERs than those of ReDMark. Therefore, our network outperformed the conventional method with respect to image quality and BER. [ABSTRACT FROM AUTHOR]

As one of the important parts of Neural Network, activation function plays a very important role in model training in Neural Network. In this paper, the status quo, advantages and disadvantages of the existing common activation functions are analysed, and a new activation function is proposed and applied to target detection. To test the performance of the new activation function, this paper compares it with the ReLU activation functions on a variety of Neural Networks and data sets, and not only analyses the performance of the activation function itself but also verifies the effectiveness of the activation function in target detection. [ABSTRACT FROM AUTHOR]

Unsupervised learning based on restricted Boltzmann machine or autoencoders has become an important research domain in the area of neural networks. In this paper mathematical expressions to adaptive learning step calculation for RBM with ReLU transfer function are proposed. As a result, we can automatically estimate the step size that minimizes the loss function of the neural network and correspondingly update the learning step in every iteration. We give a theoretical justification for the proposed adaptive learning rate approach, which is based on the steepest descent method. The proposed technique for adaptive learning rate estimation is compared with the existing constant step and Adam methods in terms of generalization ability and loss function. We demonstrate that the proposed approach provides better performance. [ABSTRACT FROM AUTHOR]

Copyright of Automotive Digest is the property of Automotive Digest Editorial Office and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Generative Adversarial Networks (GANs) have surfaced as a transformative approach in the domain of cryptography, introducing a novel paradigm where two neural networks, the generator (akin to Alice) and the discriminator (akin to Bob), are pitted against each other in a cryptographic setting. A third network, representing Eve, attempts to decipher the encrypted information. The efficacy of this encryption–decryption process is deeply intertwined with the choice of activation functions employed within these networks. This study conducted a comparative analysis of four widely used activation functions within a standardized GAN framework. Our recent explorations underscore the superior performance achieved when utilizing the Rectified Linear Unit (ReLU) in the hidden layers combined with the Sigmoid activation function in the output layer. The non-linear nature introduced by the ReLU provides a sophisticated encryption pattern, rendering the deciphering process for Eve intricate. Simultaneously, the Sigmoid function in the output layer guarantees that the encrypted and decrypted messages are confined within a consistent range, facilitating a straightforward comparison with original messages. The amalgamation of these activation functions not only bolsters the encryption strength but also ensures the fidelity of the decrypted messages. These findings not only shed light on the optimal design considerations for GAN-based cryptographic systems but also underscore the potential of investigating hybrid activation functions for enhanced system optimization. In our exploration of cryptographic strength and training efficiency using various activation functions, we discovered that the "ReLU and Sigmoid" combination significantly outperforms the others, demonstrating superior security and a markedly efficient mean training time of 16.51 s per 2000 steps. This highlights the enduring effectiveness of established methodologies in cryptographic applications. This paper elucidates the implications of these choices, advocating for their adoption in GAN-based cryptographic models, given the superior results they yield in ensuring security and accuracy. [ABSTRACT FROM AUTHOR]

Image captioning extracts multiple semantic features from an image and integrates them into a sentence-level description. For efficient description of the captions, it becomes necessary to learn higher order interactions between detected objects and the relationship among them. Most of the existing systems take into account the first order interactions while ignoring the higher order ones. It is challenging to extract discriminant higher order semantics visual features in images with highly populated objects for caption generation. In this paper, an efficient higher order interaction learning framework is proposed using encoder-decoder based image captioning. A scaled version of Gaussian Error Linear Unit (GELU) activation function, x-GELU is introduced that controls the vanishing gradients and enhances the feature learning. To leverage higher order interactions among multiple objects, an efficient XGL Transformer (XGL-T) model is introduced that exploits both spatial and channel-wise attention by integrating four XGL attention modules in image encoder and one in Bilinear Long Short-Term Memory guided sentence decoder. The proposed model captures rich semantic concepts from objects, attributes, and their relationships. Extensive experiments are conducted on publicly available MSCOCO Karapathy test split and the best performance of the work is observed as 81.5 BLEU@1, 67.1 BLEU@2, 51.6 BLEU@3, 39.9 BLEU@4, 134 CIDEr, 59.9 ROUGE-L, 29.8 METEOR, 23.8 SPICE using CIDEr-D Score Optimization Strategy. The scores validate the significant improvements over state-of-the-art results. An ablation study is also carried out to support the experimental observations. [ABSTRACT FROM AUTHOR]

In this paper, our main aim is to present innovative fixed-point theorems that provide solutions to the fixed-circle problem with symmetric contractions. We accomplish this by employing operator enrichment techniques within the context of Banach spaces. Furthermore, we demonstrate the practical application of these theorems by showcasing their relevance to the rectified linear unit (ReLU) activation function. By exploring the connection between fixed points and activation functions, our work contributes to a deeper understanding of the behavior and properties of these fundamental mathematical concepts. [ABSTRACT FROM AUTHOR]

The main objective of this paper is to improve the efficiency and accuracy of convolutional neural network models for image classification and denoising tasks. The focus of the study is on enhancing the activation layer of these models, which is a critical component that determines the output of each neuron in the network. To achieve this goal, we propose a novel activation function based on Chebyshev polynomials, which is both data-driven and self-learnable. In addition to proposing the LIP model, the authors investigate its performance in approximating various nonlinearities and determine its Lipschitz bound. The study then evaluates the performance of the proposed activation function by conducting experiments on multiple datasets using different convolutional neural network models. The results show that the proposed activation function outperforms other activation layers and significantly enhances the accuracy of image classification and denoising tasks. [ABSTRACT FROM AUTHOR]

Detecting and classifying vehicles as objects from images and videos is challenging in appearance-based representation, yet plays a significant role in the substantial real-time applications of Intelligent Transportation Systems (ITSs). The rapid development of Deep Learning (DL) has resulted in the computer-vision community demanding efficient, robust, and outstanding services to be built in various fields. This paper covers a wide range of vehicle detection and classification approaches and the application of these in estimating traffic density, real-time targets, toll management and other areas using DL architectures. Moreover, the paper also presents a detailed analysis of DL techniques, benchmark datasets, and preliminaries. A survey of some vital detection and classification applications, namely, vehicle detection and classification and performance, is conducted, with a detailed investigation of the challenges faced. The paper also addresses the promising technological advancements of the last few years. [ABSTRACT FROM AUTHOR]

Recently, deep learning has achieved remarkable results in fields that require human cognitive ability, learning ability, and reasoning ability. Activation functions are very important because they provide the ability of artificial neural networks to learn complex patterns through nonlinearity. Various activation functions are being studied to solve problems such as vanishing gradients and dying nodes that may occur in the deep learning process. However, it takes a lot of time and effort for researchers to use the existing activation function in their research. Therefore, in this paper, we propose a universal activation function (UA) so that researchers can easily create and apply various activation functions and improve the performance of neural networks. UA can generate new types of activation functions as well as functions like traditional activation functions by properly adjusting three hyperparameters. The famous Convolutional Neural Network (CNN) and benchmark dataset were used to evaluate the experimental performance of the UA proposed in this study. We compared the performance of the artificial neural network to which the traditional activation function is applied and the artificial neural network to which the UA is applied. In addition, we evaluated the performance of the new activation function generated by adjusting the hyperparameters of the UA. The experimental performance evaluation results showed that the classification performance of CNNs improved by up to 5% through the UA, although most of them showed similar performance to the traditional activation function. [ABSTRACT FROM AUTHOR]

Since last decade, Speech Emotion recognition has attracted extensive research attention to identify emotions by user's pitch and voice. Many research has been done in this field to recognize emotions using different machine learning as well as deep learning approaches. In this paper, we tried three different machine learning algorithms named SVM, Logistic regression and Random forest which take four different features named MFCC, Chroma, Mel-scale spectrogram and tonnetz as an input on RAVDESS dataset where SVM is more accurate than others. As deep learning approaches are more capable to identify hidden patterns and classify the data more accurately, we tried popular algorithm like MLP, CNN and LSTM. In deep learning approach, activation function is one of the most dominant parameters which a designer can choose to make classification more accurate. In this paper, we tried to show the effect of different activation functions on the overall accuracy of the model and analyzed the results. [ABSTRACT FROM AUTHOR]

Comprehensive neural network applications led to the customization of a scheme to accelerate the computation on ASIC implementation. Hence, the determination of activation function in a neural network is an indispensable requisite. However, the specific design architecture of an activation function in a digital network encounters several difficulties as these activation functions demand additional hardware resources due to their non-linearity. This paper proposed an efficient hyperbolic tangent (tanh) function, wholly based on stochastic Computing methodology. The Hyperbolic tangent implementation is backed by the clock gating technique to curtail the dynamic power dissipation. The results are derived by implementing two different clock gating techniques on the proposed hardware. In this work, the proposed clock gating-based stochastic design for the implementation of activation function is efficient in terms of performance parameters such as area, power, and delay with negligible accuracy loss. MNIST dataset has been used for checking accuracy on LeNeT benchmark architecture. Furthermore, post-synthesis results show that the proposed clock gating design area is reduced by ≈ 70.62 % , power is reduced by ≈ 58.19 % , and delay is reduced by ≈ 98.87 % compared to the state of the art. [ABSTRACT FROM AUTHOR]

The prevalence of melanoma skin cancer has increased in recent decades. The greatest risk from melanoma is its ability to broadly spread throughout the body by means of lymphatic vessels and veins. Thus, the early diagnosis of melanoma is a key factor in improving the prognosis of the disease. Deep learning makes it possible to design and develop intelligent systems that can be used in detecting and classifying skin lesions from visible-light images. Such systems can provide early and accurate diagnoses of melanoma and other types of skin diseases. This paper proposes a new method which can be used for both skin lesion segmentation and classification problems. This solution makes use of Convolutional neural networks (CNN) with the architecture two-dimensional (Conv2D) using three phases: feature extraction, classification and detection. The proposed method is mainly designed for skin cancer detection and diagnosis. Using the public dataset International Skin Imaging Collaboration (ISIC), the impact of the proposed segmentation method on the performance of the classification accuracy was investigated. The obtained results showed that the proposed skin cancer detection and classification method had a good performance with an accuracy of 94%, sensitivity of 92% and specificity of 96%. Also comparing with the related work using the same dataset, i.e., ISIC, showed a better performance of the proposed method. [ABSTRACT FROM AUTHOR]

The current state of energy supply in Liberia is a combination of fossil fuel and hydroelectric power generation and the cost of generating, maintaining, and distributing energy is high. On the other hand, Liberia lies within a suitable zone for solar energy utilisation for photovoltaic applications, as its climate is relatively hot all year round. This paper investigates the use of the artificial neural network to model the reliability of solar radiation in a study area in Liberia, as a necessary prerequisite for alternative power generation. Seven variables (longitude, latitude, elevation, average temperature, precipitation, wind speed and relative humidity) were used as input data (causal variables) and one parameter/factor (solar radiation) was used as output (response variable) for 2000-2018. The obtained results showed that the employed model explains all the variabilities of the response data around the mean with an overall regression value of 0.93. It was found through visualised maps that the study area is in a suitable spot for the utilisation of solar energy potentials. [ABSTRACT FROM AUTHOR]

In this paper, the (2+1)-dimensional KdV equation is investigated by using the bilinear neural network method (BNNM). We construct six neural network models, extending beyond single hidden layer models to create deeper and broader network structures (e.g. [3-3-1], [3-4-1], [3-1-3-1], [3-4-1-1], [3-2-2-1] and [3-2-3-1-1] models). Introducing specific activation functions into the neural network model enables the generation of various test functions, resulting in novel solutions for equations that include rogue wave solutions, lump-kink solutions, periodic soliton solution, breather-like solutions and lump solutions. The physical properties of these novel solutions are vividly depicted through three-dimensional plots, density plots, and curve plots. The findings contribute to a better understanding of the propagation behavior of shallow water waves. [ABSTRACT FROM AUTHOR]