Search

In recent years, with the rapid development of deep learning technology, a large number of excellent convolutional neural networks (CNNs) have been proposed, many of which are based on improvements to classical methods. Based on the Inception family of methods, depthwise separable convolution was applied to Xception to achieve lightweighting, and Inception-ResNet introduces residual connections to accelerate model convergence. However, existing improvements for the Inception module often neglect further enhancement of its receptive field, while increasing the receptive field of CNNs has been widely studied and proven to be effective in improving classification performance. Motivated by this fact, three effective expansion modules are proposed in this paper. The first expansion module, Inception expand (Inception-e) module, is proposed to improve the classification accuracy by concatenating more and deeper convolutional branches. To reduce the number of parameters for Inception e, this paper proposes a second expansion module—Equivalent Inception-e (Eception) module, which is equivalent to Inception-e in terms of feature extraction capability, but which suppresses the growth of the parameter quantity brought by the expansion by effectively reducing the redundant convolutional layers; on the basis of Eception, this paper proposes a third expansion module—Lightweight Eception (Lception) module, which crosses depthwise convolution with ordinary convolution to further effectively reduce the number of parameters. The three proposed modules have been validated on the Cifar10 dataset. The experimental results show that all these extensions are effective in improving the classification accuracy of the models, and the most significant effect is the Lception module, where Lception (rank = 4) on the Cifar10 dataset improves the accuracy by 1.5% compared to the baseline model (Inception module A) by using only 0.15 M more parameters. [ABSTRACT FROM AUTHOR]

Since plant diseases occurring during the growth process are a significant factor leading to the decline in both yield and quality, the classification and detection of plant leaf diseases, followed by timely prevention and control measures, are crucial for safeguarding plant productivity and quality. As the traditional convolutional neural network structure cannot effectively recognize similar plant leaf diseases, in order to more accurately identify the diseases on plant leaves, this paper proposes an effective plant disease image recognition method aECA-ResNet34. This method is based on ResNet34, and in the first and the last layers of this network, respectively, we add this paper's improved aECAnet with the symmetric structure. aECA-ResNet34 is compared with different plant disease classification models on the peanut dataset constructed in this paper and the open-source PlantVillage dataset. The experimental results show that the aECA-ResNet34 model proposed in this paper has higher accuracy, better performance, and better robustness. The results show that the aECA-ResNet34 model proposed in this paper is able to recognize diseases of multiple plant leaves very accurately. [ABSTRACT FROM AUTHOR]

Recently, transfer learning has gained popularity in the machine learning community. Transfer Learning (TL) has emerged as a promising paradigm that leverages knowledge learned from one or more related domains to improve prediction accuracy in a target domain with limited data. However, for time series forecasting (TSF) applications, transfer learning is relatively new. This paper addresses the need for empirical studies as identified in recent reviews advocating the need for practical guidelines for Transfer Learning approaches and method designs for time series forecasting. The main contribution of this paper is the suggestion of a comprehensive framework for Transfer Learning Sensitivity Analysis (SA) for time series forecasting. We achieve this by identifying various parameters seen from various angles of transfer learning applied to time series, aiming to uncover factors and insights that influence the performance of transfer learning in time series forecasting. Undoubtedly, symmetry appears to be a core aspect in the consideration of these factors and insights. A further contribution is the introduction of four TL performance metrics encompassed in our framework. These TL performance metrics provide insight into the extent of the transferability between the source and the target domains. Analyzing whether the benefits of transferred knowledge are equally or unequally accessible and applicable across different domains or tasks speaks to the requirement of symmetry or asymmetry in transfer learning. Moreover, these TL performance metrics inform on the possibility of the occurrence of negative transfers and also provide insight into the possible vulnerability of the network to catastrophic forgetting. Finally, we discuss a sensitivity analysis of an Ensemble TL technique use case (with Multilayer Perceptron models) as a proof of concept to validate the suggested framework. While the results from the experiments offer empirical insights into various parameters that impact the transfer learning gain, they also raise the question of network dimensioning requirements when designing, specifically, a neural network for transfer learning. [ABSTRACT FROM AUTHOR]

This paper introduces an enhanced variant of the Adam optimizer—the BGE-Adam optimization algorithm—that integrates three innovative technologies to augment the adaptability, convergence, and robustness of the original algorithm under various training conditions. Firstly, the BGE-Adam algorithm incorporates a dynamic β parameter adjustment mechanism that utilizes the rate of gradient variations to dynamically adjust the exponential decay rates of the first and second moment estimates ( β 1 and β 2 ), the adjustment of β 1 and β 2 is symmetrical, which means that the rules that the algorithm considers when adjusting β 1 and β 2 are the same. This design helps to maintain the consistency and balance of the algorithm, allowing the optimization algorithm to adaptively capture the trending movements of gradients. Secondly, it estimates the direction of future gradients by a simple gradient prediction model, combining historic gradient information with the current gradient. Lastly, entropy weighting is integrated into the gradient update step. This strategy enhances the model's exploratory nature by introducing a certain amount of noise, thereby improving its adaptability to complex loss surfaces. Experimental results on classical datasets, MNIST and CIFAR10, and gastrointestinal disease medical datasets demonstrate that the BGE-Adam algorithm has improved convergence and generalization capabilities. In particular, on the specific medical image gastrointestinal disease test dataset, the BGE-Adam optimization algorithm achieved an accuracy of 69.36%, a significant improvement over the 67.66% accuracy attained using the standard Adam algorithm; on the CIFAR10 test dataset, the accuracy of the BGE-Adam algorithm reached 71.4%, which is higher than the 70.65% accuracy of the Adam optimization algorithm; and on the MNIST dataset, the BGE-Adam algorithm's accuracy was 99.34%, surpassing the Adam optimization algorithm's accuracy of 99.23%. The BGE-Adam optimization algorithm exhibits better convergence and robustness. This research not only demonstrates the effectiveness of the combination of these three technologies but also provides new perspectives for the future development of deep learning optimization algorithms. [ABSTRACT FROM AUTHOR]

Accurate and fast traffic prediction is the data-based foundation for achieving traffic control and management, and the accuracy of prediction results will directly affect the effectiveness of traffic control and management. This paper proposes a new spatiotemporal periodic polynomial model for road traffic, which integrates the temporal, spatial, and periodic features of speed time series and can effectively handle the nonlinear mapping relationship from input to output. In terms of the model, we establish a road traffic speed prediction model based on polynomial regression. In terms of spatial feature extraction methods, we introduce a maximum mutual information coefficient spatial feature extraction method. In terms of periodic feature extraction methods, we introduce a periodic trend modeling method into the prediction of speed time series, and effective fusion is carried out. Four strategies are evaluated based on the Guangzhou road speed dataset: a univariate polynomial model, a spatiotemporal polynomial model, a periodic polynomial model, and a spatiotemporal periodic polynomial model. The test results show that the three methods proposed in this article can effectively improve prediction accuracy. Comparing the spatiotemporal periodic polynomial model with multiple machine learning models and deep learning models, the prediction accuracy is improved by 5.94% compared to the best feedforward neural network. The research in this article can effectively deal with the temporal, spatial, periodic, and nonlinear characteristics of speed prediction, and to a certain extent, improve the accuracy of speed prediction. [ABSTRACT FROM AUTHOR]

Autonomy of breast cancer classification is a challenging problem, and early diagnosis is highly important. Histopathology images provide microscopic-level details of tissue samples and play a crucial role in the accurate diagnosis and classification of breast cancer. Moreover, advancements in deep learning play an essential role in early cancer diagnosis. However, existing techniques involve unique models for each classification based on the magnification factor and require training numerous models or using a hierarchical approach combining multiple models irrespective of the focus of the cell features. This may lead to lower performance for multiclass categorization. This paper adopts the DenseNet161 network by adding a learnable residual layer. The learnable residual layer enhances the features, providing low-level information. In addition, residual features are obtained from the convolution features of the preceding layer, which ensures that the future size is consistent with the number of channels in DenseNet's layer. The concatenation of spatial features with residual features helps better learn texture classification without the need for an additional texture feature extraction module. The model was validated for both binary and multiclass categorization of malignant images. The proposed model's classification accuracy ranges from 94.65% to 100% for binary and multiclass classification, and the error rate is 2.78%. Overall, the suggested model has the potential to improve the survival of breast cancer patients by allowing precise diagnosis and therapy. [ABSTRACT FROM AUTHOR]

It is crucial to evaluate and anticipate stability under various conditions, as the ability to stabilize a smart grid (SG) is one of its key features for assessing the effectiveness of its design. Intelligent approaches to stability forecasting are necessary to mitigate inadvertent instability in SG design. This is particularly crucial with the expansion of residential and commercial infrastructures, along with the growing integration of renewable energies into these grids. Predicting the stability of SGs is currently a major challenge. The concept of an SG encompasses a broad range of emerging technologies in which artificial intelligence (AI) plays a crucial role and is increasingly being utilized in light of the limitations of conventional methods. It empowers informed decision-making and adaptable responses to fluctuations in customer energy needs, unexpected power outages, rapid changes in renewable energy generation, or any unforeseen crises within an SG system. In this paper, we propose a symmetric approach to enhance SG stability by integrating various machine learning (ML) and deep learning (DL) algorithms, where symmetry is observed in the balanced application of these diverse computational techniques to predict and ensure the grid's stability. These algorithms utilized a dataset containing the simulation results of the SG stability. The learning phase of these algorithms is based on imprecise and unreliable data. To overcome this limitation, the fusion of classifiers can be a powerful approach to modeling inaccurate and uncertain data, providing more robust and reliable predictions than individual classifiers. Voting and Dempster–Shafer (DS) methods, two commonly used techniques in ensemble learning, were employed and compared. The results show that the use of the fusion of distinct classifiers with voting theory achieves an accuracy of 99.8% and outperforms several other methods including the DS method. [ABSTRACT FROM AUTHOR]

In the field of direction of arrival (DOA) estimation for coherent sources, subspace-based model-driven methods exhibit increased computational complexity due to the requirement for eigenvalue decomposition. In this paper, we propose a new neural network, i.e., the signal space deep convolution (SSDC) network, which employs the signal space covariance matrix as the input and performs independent two-dimensional convolution operations on the symmetric real and imaginary parts of the input signal space covariance matrix. The proposed SSDC network is designed to address the challenging task of DOA estimation for coherent sources. Furthermore, we leverage the spatial sparsity of the output from the proposed SSDC network to conduct a spectral peak search for obtaining the associated DOAs. Simulations demonstrate that, compared to existing state-of-the-art deep learning-based DOA estimation methods for coherent sources, the proposed SSDC network achieves excellent results in both matching and mismatching scenarios between the training and test sets. [ABSTRACT FROM AUTHOR]

Printed circuit board (PCB) manufacturing processes are becoming increasingly complex, where even minor defects can impair product performance and yield rates. Precisely identifying PCB defects is critical but remains challenging. Traditional PCB defect detection methods, such as visual inspection and automated technologies, have limitations. While defects can be readily identified based on symmetry, the operational aspect proves to be quite challenging. Deep learning has shown promise in defect detection; however, current deep learning models for PCB defect detection still face issues like large model size, slow detection speed, and suboptimal accuracy. This paper proposes a lightweight YOLOv8 (You Only Look Once version 8)-based model called LW-YOLO (Lightweight You Only Look Once) to address these limitations. Specifically, LW-YOLO incorporates a bidirectional feature pyramid network for multiscale feature fusion, a Partial Convolution module to reduce redundant calculations, and a Minimum Point Distance Intersection over Union loss function to simplify optimization and improve accuracy. Based on the experimental data, LW-YOLO achieved an mAP0.5 of 96.4%, which is 2.2 percentage points higher than YOLOv8; the precision reached 97.1%, surpassing YOLOv8 by 1.7 percentage points; and at the same time, LW-YOLO achieved an FPS of 141.5. The proposed strategies effectively enhance efficiency and accuracy for deep-learning-based PCB defect detection. [ABSTRACT FROM AUTHOR]

Nowadays, deep representations have gained significant attention due to their outstanding performance in a wide range of tasks. However, the interpretability of deep representations in specific applications poses a significant challenge. For instances where the generated quantity matrices exhibit symmetry, this paper introduces a variant of deep matrix factorization (deep MF) called deep L p smooth symmetric non-negative matrix factorization (DSSNMF), which aims to improve the extraction of clustering structures inherent in complex hierarchical and graphical representations in high-dimensional datasets by improving the sparsity of the factor matrices. We successfully applied DSSNMF to synthetic datasets as well as datasets related to post-traumatic stress disorder (PTSD) to extract several hierarchical communities. Specifically, we identified non-disjoint communities within the partial correlation networks of PTSD psychiatric symptoms, resulting in highly meaningful clinical interpretations. Numerical experiments demonstrate the promising applications of DSSNMF in fields like network analysis and medicine. [ABSTRACT FROM AUTHOR]

Given the substantial value and considerable training costs associated with deep neural network models, the field of deep neural network model watermarking has come to the forefront. While black-box model watermarking has made commendable strides, the current methodology for constructing poisoned images in the existing literature is simplistic and susceptible to forgery. Notably, there is a scarcity of black-box model watermarking techniques capable of discerning a unique user in a multi-user model distribution setting. For this reason, this paper proposes a novel black-box model watermarking method for unique identity identification, which is denoted as the ID watermarking of neural networks (IDwNet). Specifically, to enhance the distinguishability of deep neural network models in multi-user scenarios and mitigate the likelihood of poisoned image counterfeiting, this study develops a discrete cosine transform (DCT) and singular value decomposition (SVD)-based symmetrical embedding method to form the poisoned image. As this ID embedding method leads to indistinguishable deep features, the study constructs a poisoned adversary training strategy by simultaneously inputting clean images, poisoned images with the correct ID, and poisoned adversary images with incorrect IDs to train a deep neural network. Extensive simulation experiments show that the proposed scheme achieves excellent invisibility for the concealed ID, surpassing remarkably the state-of-the-art. In addition, the proposed scheme obtains a validation success rate exceeding 99% for the poisoned images at the cost of a marginal classification accuracy reduction of less than 0.5%. Moreover, even though there is only a 1-bit discrepancy between IDs, the proposed scheme still results in an accurate validation of user copyright. These results indicate that the proposed scheme is promising. [ABSTRACT FROM AUTHOR]

Thermoacoustic oscillation is indeed a phenomenon characterized by the symmetric coupling of thermal and acoustic waves. This paper introduces a novel approach for monitoring and predicting thermoacoustic combustion instability using a combination of recurrence quantification analysis (RQA) and an optimized deep belief network (DBN). Six samples of combustion state data were collected using two distinct types of burners to facilitate the training and validation of GA-DBN. The proposed methodology leverages RQA to extract intricate patterns and dynamic features from time series data representing combustion behavior. By quantifying the recurrence plot of specific patterns, the analysis provides valuable insights into the underlying thermoacoustic dynamics. Among three different feature extraction methods, RQA stands out remarkably in performance. These RQA-derived features serve as input to a carefully tuned DBN, which is trained to learn the complex relationships within the combustion process. The classification accuracy of deep belief network optimized by genetic algorithm (GA-DBN) reached an impressive 99.8%. Subsequent multiple comparisons were conducted between GA-DBN, DBN, and support vector machine (SVM), revealing that GA-DBN consistently demonstrated satisfactory classification results. This method holds significant importance in monitoring intricate combustion states. [ABSTRACT FROM AUTHOR]

The escalating reliance on revolutionary online web services has introduced heightened security risks, with persistent challenges posed by phishing despite extensive security measures. Traditional phishing systems, reliant on machine learning and manual features, struggle with evolving tactics. Recent advances in deep learning offer promising avenues for tackling novel phishing challenges and malicious URLs. This paper introduces a two-phase stack generalized model named AntiPhishStack, designed to detect phishing sites. The model leverages the learning of URLs and character-level TF-IDF features symmetrically, enhancing its ability to combat emerging phishing threats. In Phase I, features are trained on a base machine learning classifier, employing K-fold cross-validation for robust mean prediction. Phase II employs a two-layered stacked-based LSTM network with five adaptive optimizers for dynamic compilation, ensuring premier prediction on these features. Additionally, the symmetrical predictions from both phases are optimized and integrated to train a meta-XGBoost classifier, contributing to a final robust prediction. The significance of this work lies in advancing phishing detection with AntiPhishStack, operating without prior phishing-specific feature knowledge. Experimental validation on two benchmark datasets, comprising benign and phishing or malicious URLs, demonstrates the model's exceptional performance, achieving a notable 96.04% accuracy compared to existing studies. This research adds value to the ongoing discourse on symmetry and asymmetry in information security and provides a forward-thinking solution for enhancing network security in the face of evolving cyber threats. [ABSTRACT FROM AUTHOR]

Ear recognition has made good progress as an emerging biometric technology. However, the recognition performance, generalization ability, and feature robustness of ear recognition systems based on hand-crafted features are relatively poor. With the development of deep learning, these problems have been partly overcome. However, the recognition performance of existing ear recognition systems still needs to be improved when facing unconstrained ear databases in realistic scenarios. Another critical problem is that most systems with ear feature template databases are vulnerable to software attacks that disclose users' privacy and even bring down the system. This paper proposes a software-attack-proof ear recognition system using deep feature learning and blockchain protection to address the problem that the recognition performance of existing systems is generally poor in the face of unconstrained ear databases in realistic scenarios. First, we propose an accommodative DropBlock (AccDrop) to generate drop masks with adaptive shapes. It has an advantage over DropBlock in coping with unconstrained ear databases. Second, we introduce a simple and parameterless attention module that uses 3D weights to refine the ear features output from the convolutional layer. To protect the security of the ear feature template database and the user's privacy, we use Merkle tree nodes to store the ear feature templates, ensuring the determinism of the root node in the smart contract. We achieve Rank-1 (R1) recognition accuracies of 83.87% and 96.52% on the AWE and EARVN1.0 ear databases, which outperform most advanced ear recognition systems. [ABSTRACT FROM AUTHOR]

A large amount of sensitive information is generated in today's evolving network environment. Some hackers utilize low-frequency attacks to steal sensitive information from users. This generates minority attack samples in real network traffic. As a result, the data distribution in real network traffic is asymmetric, with a large number of normal traffic and a rare number of attack traffic. To address the data imbalance problem, intrusion detection systems mainly rely on machine-learning-based methods to detect minority attacks. Although this approach can detect minority attacks, the performance is not satisfactory. To solve the above-mentioned problems, this paper proposes a novel high-performance multimodal deep learning method. The method is based on deep learning. It captures the features of minority class attacks based on generative adversarial networks, which in turn generate high-quality minority class sample attacks. Meanwhile, it uses the designed multimodal deep learning model to learn the features of minority attacks. The integrated classifier then utilizes the learned features for multi-class classification. This approach achieves 99.55% and 99.95% F-measure, 99.56% and 99.96% accuracy on the CICIDS2017 dataset and the NSL-KDD dataset, respectively, with the highest false positives at only 3.4%. This exceeds the performance of current state-of-the-art methods. [ABSTRACT FROM AUTHOR]