Search

Brain signal analysis from electroencephalogram (EEG) recordings is the gold standard for diagnosing various neural disorders especially epileptic seizure. Seizure signals are highly chaotic compared to normal brain signals and thus can be identified from EEG recordings. In the current seizure detection and classification landscape, most models primarily focus on binary classification—distinguishing between seizure and non-seizure states. While effective for basic detection, these models fail to address the nuanced stages of seizures and the intervals between them. Accurate identification of per-seizure or interictal stages and the timing between seizures is crucial for an effective seizure alert system. This granularity is essential for improving patient-specific interventions and developing proactive seizure management strategies. This study addresses this gap by proposing a novel AI-based approach for seizure stage classification using a Deep Convolutional Neural Network (DCNN). The developed model goes beyond traditional binary classification by categorizing EEG recordings into three distinct classes, thus providing a more detailed analysis of seizure stages. To enhance the model's performance, we have optimized the DCNN using two advanced techniques: the Stochastic Gradient Algorithm (SGA) and the evolutionary Genetic Algorithm (GA). These optimization strategies are designed to fine-tune the model's accuracy and robustness. Moreover, k-fold cross-validation ensures the model's reliability and generalizability across different data sets. Trained and validated on the Bonn EEG data sets, the proposed optimized DCNN model achieved a test accuracy of 93.2%, demonstrating its ability to accurately classify EEG signals. In summary, the key advancement of the present research lies in addressing the limitations of existing models by providing a more detailed seizure classification system, thus potentially enhancing the effectiveness of real-time seizure prediction and management systems in clinical settings. With its inherent classification performance, the proposed approach represents a significant step forward in improving patient outcomes through advanced AI techniques. [ABSTRACT FROM AUTHOR]

Lightweight convolutional neural network (CNN) models have proven effective in recognizing common pest species, yet challenges remain in enhancing their nonlinear learning capacity and reducing overfitting. This study introduces a grouped dropout strategy and modifies the CNN architecture to improve the accuracy of multi-class insect recognition. Specifically, we optimized the base model by selecting appropriate optimizers, fine-tuning the dropout probability, and adjusting the learning rate decay strategy. Additionally, we replaced ReLU with PReLU and added BatchNorm layers after each Inception layer, enhancing the model's nonlinear expression and training stability. Leveraging the Inception module's branching structure and the adaptive grouping properties of the WeDIV clustering algorithm, we developed two grouped dropout models, the iGDnet-IP and GDnet-IP. Experimental results on a dataset containing 20 insect species (15 pests and five beneficial insects) demonstrated an increase in cross-validation accuracy from 84.68% to 92.12%, with notable improvements in the recognition rates for difficult-to-classify species, such as Parnara guttatus Bremer and Grey (PGBG) and Papilio xuthus Linnaeus (PXLL), increasing from 38% and 47% to 62% and 93%, respectively. Furthermore, these models showed significant accuracy advantages over standard dropout methods on test sets, with faster training times compared to four conventional CNN models, highlighting their suitability for mobile applications. Theoretical analyses of model gradients and Fisher information provide further insight into the grouped dropout strategy's role in improving CNN interpretability for insect recognition tasks. [ABSTRACT FROM AUTHOR]

Fashion items are hard to classify since there are a million variations in style, texture, and pattern. Image classification is among the noted strengths of convolutional neural networks. This research introduces an improved CNN architecture for fashion classification, utilizing image augmentation and batch normalization to improve model performance and generalization. To make the model more robust, image augmentation techniques like rotation, width and height shift, zoom, and flips were employed. In addition, a Batch Normalization layer is added in the middle, which can help on stabilizing the learning process and accelerating convergence. The proposed model was trained on an augmented dataset, achieving a satisfactory improvement in test accuracy of 91.97% compared to a baseline CNN model, which obtained 88.5% accuracy. According to the results, the image augmentation with the application of Batch Normalization improves the CNN architecture for better effectiveness in fashion classification tasks. [ABSTRACT FROM AUTHOR]

Deep neural network-based machine learning algorithms are widely used within different sectors and produce excellent results. However, their use requires access to private, often confidential, and sensitive information (financial, medical, etc). This requires precise measures and particular attention to data security and confidentiality. In this paper, we propose a new solution to this problem by integrating a proposed Convolutional Neural Network (CNN) model on encrypted data within the constraints of homomorphic encryption techniques. Specifically, we focus on the approximate activation functions ReLU, Sigmoid, and Tanh, which appear to be the key functions of CNNs. We start by developing new low-degree polynomials, which are essential for successful Homomorphic Encryption (HE). The activation functions will be replaced by these polynomials, which are based on the Beta function and its primitive. To make certain that the data is contained within a given range, the next step is to build a new CNN model using batch normalization. Finally, our methodology and the effectiveness of the proposed strategy are evaluated using Mnist and Cifar10. The experimental results support the proposed approach's efficiency. [ABSTRACT FROM AUTHOR]

Background: Noonan syndrome (NS) is a rare genetic disease, and patients who suffer from it exhibit a facial morphology that is characterized by a high forehead, hypertelorism, ptosis, inner epicanthal folds, down-slanting palpebral fissures, a highly arched palate, a round nasal tip, and posteriorly rotated ears. Facial analysis technology has recently been applied to identify many genetic syndromes (GSs). However, few studies have investigated the identification of NS based on the facial features of the subjects. Objectives: This study develops advanced models to enhance the accuracy of diagnosis of NS. Methods: A total of 1,892 people were enrolled in this study, including 233 patients with NS, 863 patients with other GSs, and 796 healthy children. We took one to 10 frontal photos of each subject to build a dataset, and then applied the multi-task convolutional neural network (MTCNN) for data pre-processing to generate standardized outputs with five crucial facial landmarks. The ImageNet dataset was used to pre-train the network so that it could capture generalizable features and minimize data wastage. We subsequently constructed seven models for facial identification based on the VGG16, VGG19, VGG16-BN, VGG19-BN, ResNet50, MobileNet-V2, and squeeze-and-excitation network (SENet) architectures. The identification performance of seven models was evaluated and compared with that of six physicians. Results: All models exhibited a high accuracy, precision, and specificity in recognizing NS patients. The VGG19-BN model delivered the best overall performance, with an accuracy of 93.76%, precision of 91.40%, specificity of 98.73%, and F1 score of 78.34%. The VGG16-BN model achieved the highest AUC value of 0.9787, while all models based on VGG architectures were superior to the others on the whole. The highest scores of six physicians in terms of accuracy, precision, specificity, and the F1 score were 74.00%, 75.00%, 88.33%, and 61.76%, respectively. The performance of each model of facial recognition was superior to that of the best physician on all metrics. Conclusion: Models of computer-assisted facial recognition can improve the rate of diagnosis of NS. The models based on VGG19-BN and VGG16-BN can play an important role in diagnosing NS in clinical practice. [ABSTRACT FROM AUTHOR]

Human age and gender prediction from facial images has garnered significant attention due to its importance in various applications. Traditional models struggle with large-scale variations in unfiltered images. Convolutional Neural Networks (CNNs) have emerged as effective tools for facial analysis due to their robust performance. This paper presents a novel CNN approach for robust age and gender classification using unconstrained real-world images. The CNN architecture includes convolution, pooling, and fully connected layers for feature extraction, dimension reduction, and mapping to output classes. Adience and UTKFace datasets were utilized, with the best training and testing accuracies achieved using an 80% training and 20% testing data split. Robust image pre-processing and data augmentation techniques were applied to handle dataset variations. The proposed approach outperformed existing methods, achieving age prediction accuracies of 86.42% and 81.96%, and gender prediction accuracies of 97.65% and 96.32% on the Adience and UTKFace datasets, respectively. [ABSTRACT FROM AUTHOR]

Abnormalities commonly encountered in dental practice include tooth and supporting tissue issues such as caries, periapical abnormalities, resorption, and impacted third molars. Panoramic radiographs are frequently used for image scanning in dentistry and oral surgery. Diagnosing dental anomalies can be time-consuming due to the complexity of the orthodontic area, potentially leading to inaccuracies. This research proposes an end-to-end automated detection of dental and supporting tissue anomalies in patients, encompassing cavities, periapical lesions, resorption, and impacted third molars. This study evaluated the effectiveness of employing various pre-trained Convolutional Neural Network architectures, including ResNet-50, ResNeXt-50 32×4d, Inception-V3, and EfficientNet-V2. To enhance model performance, a batch normalization technique was integrated into the classification layer of these pre-trained models. Data pre-processing techniques, including horizontal and vertical flips, as well as random affine transformations, were applied to augment the dataset. Additionally, an image normalization procedure was implemented before the training and prediction phases. In the evaluation on 202 images, the integrated ResNeXt-50 32x4d model with batch normalization achieved the highest accuracy, precision, recall, and F1-score of 83.663%, 81.615%, 81.271%, and 81.066%, respectively. Based on the F1-score, this model demonstrates promising predictions of tooth and supporting tissue anomalies in an imbalanced dataset. [ABSTRACT FROM AUTHOR]

Failure to diagnose and treat retinal illnesses on time might lead to irreversible blindness. The focus is on three common retinal lesions associated with diabetic retinopathy (DR): microaneurysms (MAs), haemorrhages, and exudates. The proposed solution leverages deep learning, employing a customized residual network (ResNet) based classifier trained on real-time retinal images meticulously annotated and graded by ophthalmologists. Annotation noise was a significant obstacle addressed by downsampling and augmenting the data. Compared to cutting-edge techniques, this one performs better with test-set accuracy of 93.34% across all classes. This approach holds great promise for enhancing early detection and treatment of DR by automating the recognition of these vital retinal abnormalities. The ability to automatically classify these symptoms can aid clinicians in making more precise diagnosis and starting treatments sooner. This research shows that deep learning-based approaches are highly effective, especially when combined with a customised ResNet-based classifier and thorough preprocessing steps. We observed that this method provides the ability to better the lives of patients and lower the rate of permanent blindness resulting from retinal disorders. [ABSTRACT FROM AUTHOR]

Spiking neural networks (SNNs) have garnered significant attention due to their computational patterns resembling biological neural networks. However, when it comes to deep SNNs, how to focus on critical information effectively and achieve a balanced feature transformation both temporally and spatially becomes a critical challenge. To address these challenges, our research is centered around two aspects: structure and strategy. Structurally, we optimize the leaky integrate-and-fire (LIF) neuron to enable the leakage coefficient to be learnable, thus making it better suited for contemporary applications. Furthermore, the self-attention mechanism is introduced at the initial time step to ensure improved focus and processing. Strategically, we propose a new normalization method anchored on the learnable leakage coefficient (LLC) and introduce a local loss signal strategy to enhance the SNN's training efficiency and adaptability. The effectiveness and performance of our proposed methods are validated on the MNIST, FashionMNIST, and CIFAR-10 datasets. Experimental results show that our model presents a superior, high-accuracy performance in just eight time steps. In summary, our research provides fresh insights into the structure and strategy of SNNs, paving the way for their efficient and robust application in practical scenarios. [ABSTRACT FROM AUTHOR]

Brain-computer interface (BCI) is an emerging paradigm to achieve communication between external devices and the human brain. Due to the low signal-to-noise ratio of the original electroencephalograph (EEG) signals, it is different to achieve feature extraction and feature selection, and further high classification accuracy cannot be obtained. To address the above problems, this paper proposes a pattern recognition method that takes into account sample entropy combined with a batch-normalized convolutional neural network. In addition, the sample entropy is used to extract features from the EEG signal data processed by wavelet transform and independent component analysis, and then the extracted data are fed into the convolutional neural network structure to recognize the EEG signal. Based on the comparison of experimental results, it is found that the method proposed in this paper has a high recognition rate. [ABSTRACT FROM AUTHOR]

Context. The problem of automating of the edge detection on natural images in intelligent systems is considered. The subject of the research is the deep learning convolutional neural networks for edge detection on natural images. Objective. The objective of the research is to improve the edge detection performance of natural images by structural tuning the richer convolutional features network architecture. Method. In general, the edge detection performance is influenced by a neural network architecture. To automate the design of the network structure in the paper a structural tuning of a neural network is applied. Computational costs of a structural tuning are incomparably less compared with neural architecture search, but a higher qualification of the researcher is required, and the resulting solution will be suboptimal. In this research it is successively applied first a destructive approach and then a constructive approach to structural tuning of the based architecture of the RCF neural network. The constructive approach starts with a simple architecture network. Hidden layers, nodes, and connections are added to expand the network. The destructive approach starts with a complex architecture network. Hidden layers, nodes, and connections are then deleted to contract the network. The structural tuning of the richer convolutional features network includes: (1) reducing the number of convolutional layers; (2) reducing the number of convolutions in convolutional layers; (3) removing at each stage the sigmoid activation function with subsequent calculation of the loss function; (4) addition of the batch normalization layers after convolutional layers; (5) including the ReLU activation functions after the added batch normalization layers. The obtained neural network is named RCF-ST. The initial color images were scaled to the specified size and then inputted in the neural network. The advisability of each of the proposed stages of network structural tuning was reseached by estimating the edge detection performance using the confusion matrix elements and Figure of Merit. The advisability of a structural tuning of the neural network as a whole was estimated by comparing it with methods known from the literature using the Optimal Dataset Scale and Optimal Image Scale. Results. The proposed convolutional neural network has been implemented in software and researched for solving the problem of edge detection on natural images. The structural tuning technique may be used for informed design of the neural network architectures for other artificial intelligence problems. Conclusions. The obtained RCF-ST network allows to improve the performance of edge detection on natural images. RCF-ST network is characterized by a significantly fewer parameters compared to the RCF network, which makes it possible to reduce the resource consumption of the network. Besides, RCF-ST network ensures the enhancing of the robustness of edge detection on texture background. [ABSTRACT FROM AUTHOR]

Convolutional neural networks (CNNs) have been extensively utilized in medical image processing to automatically extract meaningful features and classify various medical conditions, enabling faster and more accurate diagnoses. In this paper, LeNet, a classic CNN architecture, has been successfully applied to breast cancer data analysis. It demonstrates its ability to extract discriminative features and classify malignant and benign tumors with high accuracy, thereby supporting early detection and diagnosis of breast cancer. LeNet with corrected Rectified Linear Unit (ReLU), a modification of the traditional ReLU activation function, has been found to improve the performance of LeNet in breast cancer data analysis tasks via addressing the "dying ReLU" problem and enhancing the discriminative power of the extracted features. This has led to more accurate, reliable breast cancer detection and diagnosis and improved patient outcomes. Batch normalization improves the performance and training stability of small and shallow CNN architecture like LeNet. It helps to mitigate the effects of internal covariate shift, which refers to the change in the distribution of network activations during training. This classifier will lessen the overfitting problem and reduce the running time. The designed classifier is evaluated against the benchmarking deep learning models, proving that this has produced a higher recognition rate. The accuracy of the breast image recognition rate is 89.91%. This model will achieve better performance in segmentation, feature extraction, classification, and breast cancer tumor detection. [ABSTRACT FROM AUTHOR]

Featured Application: Earthquake Detection, Earthquake Early Warning System (EEWS), Processing of Seismic data. Earthquakes threaten people, homes, and infrastructure. Earthquake detection is a complex task because it does not show any specific pattern, unlike object detection from images. Convolutional neural networks have been widely used for earthquake detection but have problems like vanishing gradients, exploding, and parameter optimization. The ensemble learning approach combines multiple models, each of which attempts to compensate for the shortcomings of the others to enhance performance. This article proposes an ensemble learning model based on a stacked normalized recurrent neural network (SNRNN) for earthquake detection. The proposed model uses three recurrent neural network models (RNN, GRU, and LSTM) with batch normalization and layer normalization. After preprocessing the waveform data, the RNN, GRU, and LSTM extract the feature map sequentially. Batch normalization and layer normalization methods take place in mini-batches and input layers for stable and faster training of the model and improving its performance. We trained and tested the proposed model on 6574 events from 2000 to 2018 (18 years) in Turkey, a highly targeted region. The SNRNN achieves RMSE values of 3.16 and 3.24 for magnitude and depth detection. The SNRNN model outperforms the three baseline models, as seen by their low RMSE values. [ABSTRACT FROM AUTHOR]

In view of the problem of weak diagnosis performance caused by insufficient training samples in industrial process fault diagnosis, a transfer BN-CNN framework is proposed based on transfer learning and deep learning in this study. In order to reduce the dependence of the network on the initialization method, a batch normalization layer is introduced into the convolution neural network to normalize the hidden layer of the model. To solve the problem of insufficient label data in the target domain, the sample-based transfer learning method is used to expand the labeled data volume of the target domain. By introducing the model based transfer learning method, the BN-CNN network is pre-trained with sufficient source domain data, and some parameters of the network are fine-tuned by using the expanded target domain. The difficulty of training the deep neural network with a small number of samples is reduced, and a fault diagnosis model suitable for target domain is obtained. The comparison experiments on TE industrial data set show that the proposed has good diagnostic performance for small samples of industrial process faults, and its average accuracy is 0.804. [ABSTRACT FROM AUTHOR]

Red, blue, white, pink, or black spots with irregular borders and small lesions on the skin are known as skin cancer that is categorized into two types: benign and malignant. Skin cancer can lead to death in advanced stages, however, early detection can increase the chances of survival of skin cancer patients. There exist several approaches developed by researchers to identify skin cancer at an early stage, however, they may fail to detect the tiniest tumours. Therefore, we propose a robust method for the diagnosis of skin cancer, namely SCDet, based on a convolutional neural network (CNN) having 32 layers for the detection of skin lesions. The images, having a size of 227 × 227, are fed to the image input layer, and then pair of convolution layers is utilized to withdraw the hidden patterns of the skin lesions for training. After that, batch normalization and ReLU layers are used. The performance of our proposed SCDet is computed using the evaluation matrices: precision 99.2%; recall 100%; sensitivity 100%; specificity 99.20%; and accuracy 99.6%. Moreover, the proposed technique is compared with the pre-trained models, i.e., VGG16, AlexNet, and SqueezeNet and it is observed that SCDet provides higher accuracy than these pre-trained models and identifies the tiniest skin tumours with maximum precision. Furthermore, our proposed model is faster than the pre-trained model as the depth of its architecture is not too high as compared to pre-trained models such as ResNet50. Additionally, our proposed model consumes fewer resources during training; therefore, it is better in terms of computational cost than the pre-trained models for the detection of skin lesions. [ABSTRACT FROM AUTHOR]

The wide application of image classification has given rise to many intelligent systems, such as face recognition systems, which makes our life more convenient. However, the ensuing privacy leakage problem has become increasingly serious. The training of a deep neural network requires lots of data, which may contain sensitive information of users and may be exploited by data collectors. A perturbation algorithm named RRN is proposed for image data based on local differential privacy, which provides a rigorous privacy guarantee. Existing solutions have low accuracy due to the high sensitivity of an image; the authors' method combines the Randomized Response mechanism with the Laplace mechanism to solve this problem. Experiments were conducted on the MNIST and CIFAR‐10 datasets to show the effectiveness of the algorithm. Experimental results shows that the model is better than baseline models. The algorithm was also implemented on the commonly used model in deep learning, the VGG model, which can achieve 96.4% accuracy in the non‐private version on the CIFAR‐10 dataset. The accuracy of the differential private VGG model based on the RRN algorithm is 83% when ε=0.5$\varepsilon =0.5$, which is still excellent. The experimental results show that the RRN algorithm can both preserve privacy and data utility. [ABSTRACT FROM AUTHOR]

The removal of mixed noise is a challenging task because the attenuation of the noise distribution cannot be described precisely. The coupling of additive white Gaussian noise and impulse noise (IN) is a typical case. At present, most methods use a two‐phase strategy, that is, IN detection coupled with additive white Gaussian noise removal, often leading to poor denoising results with an increase in the ratio of IN. In this paper, an effective convolutional neural network (CNN) model is proposed, namely a serial attention module‐based CNN (SACNN), for mixed noise removal. In contrast to the existing two‐phase methods, SACNN unifies the denoising process into a single CNN framework. In SACNN, residual learning and batch normalization are used to train the model, which speeds up the convergence and improves the mixed noise removal performance. Meanwhile, the serial attention module is applied to better preserve the texture details. The experimental results reveal that SACNN achieves superior quality metrics and visual appearance when compared to several leading approaches. [ABSTRACT FROM AUTHOR]

Abstract Background Circular RNAs (CircRNAs) play critical roles in gene expression regulation and disease development. Understanding the regulation mechanism of CircRNAs formation can help reveal the role of CircRNAs in various biological processes mentioned above. Back-splicing is important for CircRNAs formation. Back-splicing sites prediction helps uncover the mysteries of CircRNAs formation. Several methods were proposed for back-splicing sites prediction or circRNA-realted prediction tasks. Model performance was constrained by poor feature learning and using ability. Results In this study, CircCNN was proposed to predict pre-mRNA back-splicing sites. Convolution neural network and batch normalization are the main parts of CircCNN. Experimental results on three datasets show that CircCNN outperforms other baseline models. Moreover, PPM (Position Probability Matrix) features extract by CircCNN were converted as motifs. Further analysis reveals that some of motifs found by CircCNN match known motifs involved in gene expression regulation, the distribution of motif and special short sequence is important for pre-mRNA back-splicing. Conclusions In general, the findings in this study provide a new direction for exploring CircRNA-related gene expression regulatory mechanism and identifying potential targets for complex malignant diseases. The datasets and source code of this study are freely available at: https://github.com/szhh521/CircCNN .

India has the largest deaf population in the world and sign language is the principal medium for such persons to share information with normal people and among themselves. Yet, normal people do not have any knowledge of such language. As a result, there is a huge communication barrier between normal and deaf-dumb persons. Again, sign language interpreters are not easily available and it is a very costly solution for a long period. The sign language recognition system reduces the communication gaps between normal and deaf-dumb persons. The methodologies to recognize Indian sign language are recently in the developing stage and there is no approach to recognize signs in real-time. Here, we have proposed a fingerspelling recognition system of static signs for the Indian sign language alphabet using convolutional neural networks combined with data augmentation, batch normalization, dropout, stochastic pooling, and diffGrad optimizer. To continue the research, a total of 62,400 images of 26 static signs have been taken from various users. The proposed method achieves the highest training and validation accuracy of 99.76% and 99.64%, respectively , that outperforms other examined systems. [ABSTRACT FROM AUTHOR]

In the biometric field, vein identification is a vital process that is constrained by the invisibility of veins as well as other unique features. Moreover, users generally do not wish to have their personal information uploaded to the cloud, so edge computing has become popular for the sake of protecting user privacy. In this paper, we propose a low-complexity and lightweight convolutional neural network (CNN) and we design intellectual property (IP) for shortening the inference time in finger vein recognition. This neural network system can operate independently in client mode. After fetching the user's finger vein image via a near-infrared (NIR) camera mounted on an embedded system, vein features can be efficiently extracted by vein curving algorithms and user identification can be completed quickly. Better image quality and higher recognition accuracy can be obtained by combining several preprocessing techniques and the modified CNN. Experimental data were collected by the finger vein image capture equipment developed in our laboratory based on the specifications of similar products currently on the market. Extensive experiments demonstrated the practicality and robustness of the proposed finger vein identification system. [ABSTRACT FROM AUTHOR]

In order to realize the intelligent diagnosis of maize diseases with complicated backgrounds and similar disease spot characteristics in the real field environment, MFaster R-CNN is proposed by improving the Faster R-CNN algorithm. Firstly, a batch normalization processing layer is added to the convolution layer to speed up the convergence speed of the network and improve the generalization ability of the model; secondly, a central cost function is proposed to construct a mixed loss function to improve the detection accuracy of similar lesions; then, four kinds of pre-trained convolution structures are selected as the basic feature extraction network of Faster R-CNN for training, and the random gradient descent algorithm is used to optimize the training model to test the optimal feature extraction network; finally, the trained model is used to select test sets under different weather conditions for comparison, and MFaster R-CNN is compared with Faster R-CNN and SSD. The experimental results show that in MFaster R-CNN disease detection framework, VGG16 convolution layer structure as feature extraction network has better performance, the average recall rate is 0.9719, F1 is 0.9718, the overall average accuracy rate can reach 97.23%; compared with Faster R-CNN, MFaster R-CNN has an average accuracy of 0.0886 higher and a single image detection time of 0.139 s less; compared with the SSD, the average accuracy is 0.0425 higher, and the single image detection time is reduced by 0.018 s. Our method also provides a basis for timely and accurate prevention and control of maize diseases in the field. [ABSTRACT FROM AUTHOR]

The overlapped handwritten digit classification is a global challenge and a significant measure to assess the network recognition ability ratio. Most efficient models have been designed based on convolutional neural networks (CNN) for effective image classification and digit identification. Subsequently, multiple CNN models have inadequate accuracy because of high degree parameter dimensions that lead to abnormal digit detection error rates and computation complexity. We propose a Deep Digit Recognition Network (DDRNet) based on Deep ConvNets to minimize the number of parameters and features to keep the model light while maximizing the accuracy with an adaptive voting (AV) scheme for digit recognition. The individual digit is identified by CNN, and uncertain digits or strings are identified by Deep Convolutional Network (DCN) with AV scheme through Voting-Weight Conditional Random Field (VWCRF) strategy. These methods originated with the YOLO algorithm. The simulations show that our DDRNet approach achieves an accuracy of 99.4% without error fluctuations, in a stable state with less than 15 epochs contrast with state-of-art approaches. Additionally, specific convolution techniques (SqueezeNet, batch normalization) and image augmentation techniques (dropout, back-propagation, and an optimum learning rate) were examined to assess the system performance based on MNIST dataset (available at: http://yann.lecun.com/exdb/mnist/). [ABSTRACT FROM AUTHOR]

Deep learning is the most frequently used tool in the classification of tumors in medical applications. In recent decades, many research works have been done on the Breast Imaging Reporting & Data System (BI-RADS) atlas based classification of Breast cancer. As reported in the existing research works, training the larger datasets is a challenging task. Therefore, a customized ResNet based Convolution Neural Network (cRN-CNN) with batch normalization is proposed in this manuscript for addressing the above mentioned issue. The proposed cRN-CNN method has the advantage of faster training and computationally effective for the classification of BIRADS atlas based MRI breast cancer records, where the proposed model’s performance is superior compared to the conventional CNN model. The extensive experiments performed on the Dynamic Contrast Enhanced-Magnetic Resonance Imaging (DCE-MRI) dataset confirmed that the proposed cRN-CNN method achieved better classification results than the existing methods. In the proposed model, the deformation technique based on elastic deformation is also applied to increase the training size of data that helps to improve the outcomes of prediction up-to 99.80%, because of the efficient strategy of batch normalization as customization and elastic deformation. [ABSTRACT FROM AUTHOR]

Computed tomography (CT) is used in medical applications to produce digital medical imaging of the human body and is acquired by the reconstruction process, where X-rays are the key component of CT imaging. The present coronavirus outbreak has spawned new medical device and technology research fields. COVID-19 most severely affects people with poor immunity; children and pregnant women are more susceptible. A CT scan will be required to assess the infection's severity. As a result, to reduce the radiation levels significantly there is a need to minimize the CT scan noise. The quality of CT images may degrade in the form of noisy images due to low radiation levels. Hence, this study proposes a novel denoising methodology for COVID-19 CT images with a low dose, where a convolution neural network (CNN) and batch normalization were utilized for denoising. From different output metrics such as peak signal-to-noise ratio (PSNR) and image quality index (IQI), the accuracy of the resulting CT images was checked and evaluated, where IQI obtained the best results in terms of 99% accuracy. The findings were also compared with the outcomes of related recent research in the domain. After a detailed review of the findings, it was noted that the proposed algorithm in the present study performed better in comparision to the existing literature. [ABSTRACT FROM AUTHOR]

The prevalence of melanoma skin cancer has increased in recent decades. The greatest risk from melanoma is its ability to broadly spread throughout the body by means of lymphatic vessels and veins. Thus, the early diagnosis of melanoma is a key factor in improving the prognosis of the disease. Deep learning makes it possible to design and develop intelligent systems that can be used in detecting and classifying skin lesions from visible-light images. Such systems can provide early and accurate diagnoses of melanoma and other types of skin diseases. This paper proposes a new method which can be used for both skin lesion segmentation and classification problems. This solution makes use of Convolutional neural networks (CNN) with the architecture two-dimensional (Conv2D) using three phases: feature extraction, classification and detection. The proposed method is mainly designed for skin cancer detection and diagnosis. Using the public dataset International Skin Imaging Collaboration (ISIC), the impact of the proposed segmentation method on the performance of the classification accuracy was investigated. The obtained results showed that the proposed skin cancer detection and classification method had a good performance with an accuracy of 94%, sensitivity of 92% and specificity of 96%. Also comparing with the related work using the same dataset, i.e., ISIC, showed a better performance of the proposed method. [ABSTRACT FROM AUTHOR]

Earthquakes threaten people, homes, and infrastructure. Early warning systems provide prior warning of oncoming significant shaking to decrease seismic risk by providing location, magnitude, and depth information of the event. Their usefulness depends on how soon a strong shake begins after the warning. In this article, the authors implement a deep learning model for predicting earthquakes. This model is based on a graph convolutional neural network with batch normalization and attention mechanism techniques that can successfully predict the depth and magnitude of an earthquake event at any number of seismic stations in any number of locations. After preprocessing the waveform data, CNN extracts the feature map. Attention mechanism is used to focus on important features. The batch normalization technique takes place in batches for stable and faster training of the model by adding an extra layer. GNN with extracted features and event location information predicts the event information accurately. We test the proposed model on two datasets from Japan and Alaska, which have different seismic dynamics. The proposed model achieves 2.8 and 4.0 RMSE values in Alaska and Japan for magnitude prediction, and 2.87 and 2.66 RMSE values for depth prediction. Low RMSE values show that the proposed model significantly outperforms the three baseline models on both datasets to provide an accurate estimation of the depth and magnitude of small, medium, and large-magnitude events. [ABSTRACT FROM AUTHOR]

Alzheimer's disease is the most common cause of dementia and is a generic term for memory and other cognitive abilities that are severe enough to interfere with daily life. In this paper, we propose an improved prediction method for Alzheimer's disease using a quantization method that transforms the MRI data set using a VGG-C Transform model and a convolutional neural network (CNN) consisting of batch normalization. MRI image data of Alzheimer's disease are not fully disclosed to general research because it is data from real patients. So, we had to find a solution that could maximize the core functionality in a limited image. In other words, since it is necessary to adjust the interval, which is an important feature of MRI color information, rather than expressing the brain shape, the brain texture dataset was modified in the quantized pixel intensity method. We also use the VGG family, where the VGG-C Transform model with bundle normalization added to the VGG-C model performed the best with a test accuracy of about 0.9800. However, since MRI images are 208 × 176 pixels, conversion to 224 × 224 pixels may result in distortion and loss of pixel information. To address this, the proposed VGG model-based architecture can be trained while maintaining the original MRI size. As a result, we were able to obtain a prediction accuracy of 98% and the AUC score increased by up to 1.19%, compared to the normal MRI image data set. It is expected that our study will be helpful in predicting Alzheimer's disease using the MRI dataset. [ABSTRACT FROM AUTHOR]

Background: Circular RNAs (CircRNAs) play critical roles in gene expression regulation and disease development. Understanding the regulation mechanism of CircRNAs formation can help reveal the role of CircRNAs in various biological processes mentioned above. Back-splicing is important for CircRNAs formation. Back-splicing sites prediction helps uncover the mysteries of CircRNAs formation. Several methods were proposed for back-splicing sites prediction or circRNA-realted prediction tasks. Model performance was constrained by poor feature learning and using ability. Results: In this study, CircCNN was proposed to predict pre-mRNA back-splicing sites. Convolution neural network and batch normalization are the main parts of CircCNN. Experimental results on three datasets show that CircCNN outperforms other baseline models. Moreover, PPM (Position Probability Matrix) features extract by CircCNN were converted as motifs. Further analysis reveals that some of motifs found by CircCNN match known motifs involved in gene expression regulation, the distribution of motif and special short sequence is important for pre-mRNA back-splicing. Conclusions: In general, the findings in this study provide a new direction for exploring CircRNA-related gene expression regulatory mechanism and identifying potential targets for complex malignant diseases. The datasets and source code of this study are freely available at: https://github.com/szhh521/CircCNN. [ABSTRACT FROM AUTHOR]

Featured Application: Earthquake Detection, Earthquake Early Warning System (EEWS), Processing of Seismic data. Earthquake is a major hazard to humans, buildings, and infrastructure. Early warning systems should detect an earthquake and issue a warning with earthquake information such as location, magnitude, and depth. Earthquake detection from raw waveform data using deep learning models such as graph neural networks (GNN) is becoming an important research area. The multilayered structure of the GNN with a number of epochs takes more training time. It is also hard to train the model with saturating nonlinearities. The batch normalization technique is applied to each mini-batch to reduce epochs in training and obtain a steady distribution of activation values. It improves model training and prediction accuracy. This study proposes a deep learning model batch normalization graph convolutional neural network (BNGCNN) for early earthquake detection. It consists of two main components: CNN and GNN. Input to the CNN model is multi-station and three-component waveform data with magnitude ≥ 3.0 were collected from January 2000 to January 2015 for Southern California. The extracted features of CNN are appended with location information and input to GNN model for earthquake detection. After hyperparameter tuning of the BNGCNN, when testing and evaluating the model on the Southern California dataset, our method shows promising results to the baseline model GNN by obtaining a low error rate to predict the magnitude, depth, and location of an earthquake. [ABSTRACT FROM AUTHOR]

Facial beauty prediction (FBP) aims to develop a system to assess facial attractiveness automatically. Through prior research and our own observations, it has become evident that attribute information, such as gender and race, is a key factor leading to the distribution discrepancy in the FBP data. Such distribution discrepancy hinders current conventional FBP models from generalizing effectively to unseen attribute domain data, thereby discounting further performance improvement. To address this problem, in this paper, we exploit the attribute information to guide the training of convolutional neural networks (CNNs), with the final purpose of implicit feature alignment across various attribute domain data. To this end, we introduce the attribute information into convolution layer and batch normalization (BN) layer, respectively, as they are the most crucial parts for representation learning in CNNs. Specifically, our method includes: 1) Attribute-guided convolution (AgConv) that dynamically updates convolutional filters based on attributes by parameter tuning or parameter rebirth; 2) Attribute-guided batch normalization (AgBN) is developed to compute the attribute-specific statistics through an attribute guided batch sampling strategy; 3) To benefit from both approaches, we construct an integrated framework by combining AgConv and AgBN to achieve a more thorough feature alignment across different attribute domains. Extensive qualitative and quantitative experiments have been conducted on the SCUT-FBP, SCUT-FBP5500 and HotOrNot benchmark datasets. The results show that AgConv significantly improves the attribute-guided representation learning capacity and AgBN provides more stable optimization. Owing to the combination of AgConv and AgBN, the proposed framework (Ag-Net) achieves further performance improvement and is superior to other state-of-the-art approaches for FBP. • Deep learning techniques for facial beauty prediction. • Learning feature alignment across multiple attribute domain data. • Learning attribute-guided feature representation for better feature alignment. • Attribute-guided convolution to dynamically adjusts convolutional parameters. • Attribute-guided batch normalization to compute attribute-specific statistics. [ABSTRACT FROM AUTHOR]

In this paper problem of offline signature verification has been discussed with a novel high-performance convolution Siamese network. The paper proposes modifications in the already existing convolution Siamese network. The proposed method makes use of the Batch Normalization technique instead of Local Response Normalization to achieve better accuracy. The regularization factor has been added in the fully connected layers of the convolution neural network to deal with the problem of overfitting. Apart from this, a wide range of learning rates are provided during the training of the model and optimal one having the least validation loss is used. To evaluate the proposed changes and compare the results with the existing solution, our model is validated on three benchmarks datasets viz. CEDAR, BHSig260, and GPDS Synthetic Signature Corpus. The evaluation is done via two methods firstly by Test-Train validation and then by K-fold cross-validation (K = 5), to test the skill of our model. We show that the proposed modified Siamese network outperforms all the prior results for offline signature verification. One of the major advantages of our system is its capability of handling an unlimited number of new users which is the drawback of many research works done in the past. [ABSTRACT FROM AUTHOR]

Normalization Layers (NLs) are widely used in modern deep-learning architectures. Despite their apparent simplicity, their effect on optimization is not yet fully understood. This paper introduces a spherical framework to study the optimization of neural networks with NLs from a geometric perspective. Concretely, the radial invariance of groups of parameters, such as filters for convolutional neural networks, allows to translate the optimization steps on the L 2 unit hypersphere. This formulation and the associated geometric interpretation shed new light on the training dynamics. Firstly, the first effective learning rate expression of Adam is derived. Then the demonstration that, in the presence of NLs, performing Stochastic Gradient Descent (SGD) alone is actually equivalent to a variant of Adam constrained to the unit hypersphere, stems from the framework. Finally, this analysis outlines phenomena that previous variants of Adam act on and their importance in the optimization process are experimentally validated. [ABSTRACT FROM AUTHOR]

In recent years, Convolution Neural Networks (CNN) has showed dominant performance over real world applications due to their ability to find good solution and deal with noisy and uncertain data. However their performance is highly dependent on the network architecture and methods for optimizing their hyper parameters. Designing a good CNN architecture requires human expertise and domain knowledge. So, it is difficult for users who have no extended knowledge in CNN to design good network architecture for their own classification problems. Keeping this in mind, a technique for finding optimum CNN architecture using methods based on Genetic Algorithm (GA) and Grey Wolf Optimization (GWO) is implemented in this paper. This paper presents optimization of hyper parameters (number and size of filters in convolution layer) of CNN using GA and GWO as well as optimizes AlexNet model hyper-parameters to improve the performance of the model. The performance of the implemented methods is evaluated by classifying kidney ultrasound images dataset. Experimental results showed that optimization of CNN with both algorithms outperform CNN optimized with PSO and conventional CNN yielding accuracies 84.2% and 94.2% respectively. Also the modified AlexNet model achieved 93.1% accuracy which is more than accuracy achieved by standard AlexNet model. [ABSTRACT FROM AUTHOR]

A sudden blow or jolt to the human brain called traumatic brain injury (TBI) is one of the most common injuries recorded in the health insurance claim. Generally, computed tomography (CT) or magnetic resonance imaging (MRI) is required to identify the trauma's severity. Unfortunately, CT and MRI equipment are bulky, expensive, and not always available, limiting their use in TBI detection. Therefore, as an alternative, this study presents a novel classification architecture that can classify non-severe TBI patients from healthy subjects by using resting-state electroencephalogram (EEG) as the input. The proposed architecture employs a convolutional neural network (CNN), and error-correcting output codes support vector machine (ECOC-SVM) to perform automated feature extraction and multi-class classification. In this architecture, complex feature selection and extraction steps are avoided. The proposed architecture attained a high-performance classification accuracy of 99.76%, potentially being used as a classification approach to preventing healthcare insurance fraud. The proposed method is compared to existing studies in the literature. The outcome from the comparisons indicates that the proposed method has outperformed the benchmarked methods by presenting the highest classification accuracy and precision.

Many people around the world have lost their lives due to COVID-19. The symptoms of most COVID-19 patients are fever, tiredness and dry cough, and the disease can easily spread to those around them. If the infected people can be detected early, this will help local authorities control the speed of the virus, and the infected can also be treated in time. We proposed a six-layer convolutional neural network combined with max pooling, batch normalization and Adam algorithm to improve the detection effect of COVID-19 patients. In the 10-fold cross-validation methods, our method is superior to several state-of-the-art methods. In addition, we use Grad-CAM technology to realize heat map visualization to observe the process of model training and detection. [ABSTRACT FROM AUTHOR]

Batch normalization (BN) is a popular and ubiquitous method in deep learning that has been shown to decrease training time and improve generalization performance of neural networks. Despite its success, BN is not theoretically well understood. It is not suitable for use with very small mini-batch sizes or online learning. In this paper, we propose a new method called Batch Normalization Preconditioning (BNP). Instead of applying normalization explicitly through a batch normalization layer as is done in BN, BNP applies normalization by conditioning the parameter gradients directly during training. This is designed to improve the Hessian matrix of the loss function and hence convergence during training. One benefit is that BNP is not constrained on the mini-batch size and works in the online learning setting. Furthermore, its connection to BN provides theoretical insights on how BN improves training and how BN is applied to special architectures such as convolutional neural networks. For a theoretical foundation, we also present a novel Hessian condition number based convergence theory for a locally convex but not strongconvex loss, which is applicable to networks with a scale-invariant property. [ABSTRACT FROM AUTHOR]

OBJECTIVES: Alzheimer's disease (AD) is a progressive neurodegenerative disease with insidious onset and one of the biggest challenges in geriatrics. Because the cause of the disease is unknown and there is currently no cure, AD Early diagnosis is particularly important. METHODS: In this paper, we built a 5-layer convolutional neural network based on deep learning technology. We used six data augmentation methods to increase the size of the training set. Batch normalization and dropout techniques are also used, which are respectively associated with the convolutional layer and the fully connected layer, Form convolution batch normalization (CB) and dropout fully connected (DOFC) block respectively. RESULTS: Our 5-layer CNN has shown excellent results on the training set, a sensitivity of 94.80%, a specificity of 93.98%, a precision of 94.04% and an accuracy of 94.39%, and has good performance compared with several other state-of-the-art methods. CONCLUSION: In terms of classification performance, our method performs better than 8 state-of-the-art approaches and the performance of human observers. Therefore, this proposed method is effective in the detection of Alzheimer’s disease. [ABSTRACT FROM AUTHOR]

Aiming at solving the problem of difficulty in extracting features of radar emitter signals and low recognition accuracy under the condition of low signal to noise ratio, a radar emitter signal based on sliced recurrent neural networks (SRNN), attention mechanism and convolutional neural networks (CNN) is proposed. Batch normalization layer is introduced into CNN to further improve the recognition ability of the network. Taking the amplitude sequence of radar emitter signal as input, the signal characteristic is extracted automatically and the recognition result of radar emitter signal is output. Compared with gated recurrent unit (GRU), the experimental results show that the training speed of SRNN is greatly improved, and the attention mechanism and batch normalization layer can effectively improve the recognition accuracy. In the experiments with eight common radar emitter signals, the proposed method still has a high recognition accuracy under the condition of low signal to noise ratio. [ABSTRACT FROM AUTHOR]

Recently, the introduction of Convolutional Neural Network (CNNs) has advanced the way of solving image segmentation tasks. Semantic image segmentation has considerably benefited from employing various CNN models. The most widely used network in this field is U-Net and its different variations. However, these models require significant number of trainable parameters, floating-point operations per second, and great computational power to be trained. These factors make real-time semantic segmentation in low powered devices very hard. Therefore, in the present paper, we aim to modify particular aspects of the U-Net model to improve its performance through developing a fast U-Net (FU-Net) relying on bottleneck convolution layers in the contraction and expansion paths of the model. The proposed model can be utilized in semantic segmentation applications even on the devices with limited computational power and memory by ensuring the state-of-the-art performance. The amount of memory required by the proposed model is reduced by 23 times when compared with the original U-Net. Moreover, the modifications allowed achieving better performance. In conducted experiments, we assessed the performance of the proposed model on two biomedical image segmentation datasets, namely 2018 Data Science Bowl and ICIS 2018: Skin Lesion Analysis Towards Melanoma Detection. FU-Net demonstrated the state-of-the-art results in biomedical image segmentation, requiring the number of trainable parameters reduced by eight times compared with the original U-Net model. In addition, using bottleneck layers decreased the number of computations, resulting in nearly 30% speed-up at the training, validation and test stages. Furthermore, despite relying on fewer parameters FU-Net achieved a slight improvement of the performance in terms of pixel accuracy, Jaccard index, and dice coefficient evaluation metrics. [ABSTRACT FROM AUTHOR]

Batch Normalization's (BN) unique property of depending on other samples in a batch is known to cause problems in several tasks, including sequence modeling. Yet, BN-related issues are hardly studied for long video understanding, despite the ubiquitous use of BN in CNNs (Convolutional Neural Networks) for feature extraction. Especially in surgical workflow analysis, where the lack of pretrained feature extractors has led to complex, multi-stage training pipelines, limited awareness of BN issues may have hidden the benefits of training CNNs and temporal models end to end. In this paper, we analyze pitfalls of BN in video learning, including issues specific to online tasks such as a 'cheating' effect in anticipation. We observe that BN's properties create major obstacles for end-to-end learning. However, using BN-free backbones, even simple CNN–LSTMs beat the state of the art on three surgical workflow benchmarks by utilizing adequate end-to-end training strategies which maximize temporal context. We conclude that awareness of BN's pitfalls is crucial for effective end-to-end learning in surgical tasks. By reproducing results on natural-video datasets, we hope our insights will benefit other areas of video learning as well. Code is available at: https://gitlab.com/nct_tso_public/pitfalls_bn. [Display omitted] • Most video-based surgical workflow methods use CNNs with Batch Normalization. • Batch Normalization has several pitfalls when applied to video data. • Previous work resorts to complex multi-stage training strategies to avoid issues. • Awareness of pitfalls enables effective training of simpler end-to-end approaches. • Simple CNN-LSTMs beat the state of the art on 3 surgical workflow benchmarks. [ABSTRACT FROM AUTHOR]

INTRODUCTION: Lip language is one of the most important communication methods in social life for people with hearing impairment and impaired expression ability. This communication method relies on visual recognition to understand the meaning expressed in communication. OBJECTIVES: In order to improve the accuracy of this natural language recognition, we propose six-layer optimized convolutional neural network for lip recognition. METHODS: The calculation method of the convolutional layer in the CNN model is used, and two pooling methods are compared: the maximum pooling operation and the average pooling operation to analyse the most important feature data in the picture. In order to reduce the simulation in the model training process, the closing rate has been optimized by introducing Dropout technology. RESULTS: It shows that the recognition accuracy rate based on the six-layer convolutional neural network can reach 85.74% on average. This method can effectively recognize lip language. CONCLUSION: We propose a six-layer optimized convolutional neural network method for lip language recognition, and the identification of lip language features of this method is better than 3D+ DenseNet +1 × 1 Conv +resBi-LSTM, 3D+CNN, ConvNet+2 -256-LSTM+VGG-16 three advanced methods. [ABSTRACT FROM AUTHOR]

Sleep stage classification is important to accurately predict and diagnose patients with sleep disorders. Though various deep learning approaches have been implemented to classify sleep classes, these consist limitations that impact the accuracy and processing time of the classification model. The aim of this research is to enhance the accuracy and minimize the training time of the deep learning classification model. The proposed system consists of One Dimensional Convolutional Neural Network (CNN) with enhanced loss function to improve the accuracy of scoring of five different sleep classes. Preprocessing, Feature Extraction and Classification are the main components of the proposed system. Initially, EEG signals are fed to an adaptive filter for preprocessing, in order to remove any noise in signal. Thereafter, feature is extracted through multiple convolutional and pooling layers, and finally the classification is done by fully connected layer using softmax activation with enhanced loss function. The proposed solution is tested on data samples from multiple datasets with five classes of Sleep classification. Based on the obtained results, the proposed solution has found to achieve an accuracy of 96.26% which is almost 4.2% higher than the state-of-the-art solution which is 92.76%. Furthermore, the processing time has been reduced by 11 milliseconds against the state-of-the-art solution. The proposed system focused on classifying sleep stages in five classes using EEG signals with deep learning approach. It enhances the loss function in order to minimize errors in the prediction of sleep classes and improves the accuracy of the model. Furthermore, the training speed of the model has also been reduced by applying batch normalization techniques inside the model. In the future, larger datasets of different sleep disorder patients with varying features can be used for training and implementing the proposed solution. The datasets can also be pre-processed using additional techniques to refine the data before feeding to the neural network model. [ABSTRACT FROM AUTHOR]