Search

Semantic relationships between words provide relevant information about the whole idea in the texts. Existing embedding representation models characterize each word as a vector of numbers with a fixed length. These models have been used in tasks involving text classification, such as recommendation and question–answer systems. However, the embedded information provided by semantic relationships has been neglected. Therefore, this paper proposes an approach that involves semantic relationships in embedding models for text classification, which is evaluated. Three embedding models based on semantic relations extracted from Wikipedia are presented and compared with existing word-based models. Our approach considers the following relationships: synonymy, hyponymy, and hyperonymy. They were considered since previous experiments have shown that they provide semantic knowledge. The relationships are extracted from Wikipedia using lexical-syntactic patterns identified in the literature. The extracted relationships are embedded as a vector: synonymy, hyponymy–hyperonymy, and a combination of all relationships. A Convolutional Neural Network using semantic relationship embeddings was trained for text classification. An evaluation was carried out for the proposed relationship embedding configurations and existing word-based models to compare them based on two corpora. The results were obtained with the metrics of precision, accuracy, recall, and F 1 -measure. The best results for the 20-Newsgroup corpus were obtained with the hyponymy–hyperonymy embeddings, achieving an accuracy of 0.79. For the Reuters corpus, F 1 -measure and recall of 0.87 were obtained using synonymy–hyponymy–hyperonymy. [ABSTRACT FROM AUTHOR]

Lung ultrasound images have shown great promise to be an operative point-of-care test for the diagnosis of COVID-19 because of the ease of procedure with negligible individual protection equipment, together with relaxed disinfection. Deep learning (DL) is a robust tool for modeling infection patterns from medical images; however, the existing COVID-19 detection models are complex and thereby are hard to deploy in frequently used mobile platforms in point-of-care testing. Moreover, most of the COVID-19 detection models in the existing literature on DL are implemented as a black box, hence, they are hard to be interpreted or trusted by the healthcare community. This paper presents a novel interpretable DL framework discriminating COVID-19 infection from other cases of pneumonia and normal cases using ultrasound data of patients. In the proposed framework, novel transformer modules are introduced to model the pathological information from ultrasound frames using an improved window-based multi-head self-attention layer. A convolutional patching module is introduced to transform input frames into latent space rather than partitioning input into patches. A weighted pooling module is presented to score the embeddings of the disease representations obtained from the transformer modules to attend to information that is most valuable for the screening decision. Experimental analysis of the public three-class lung ultrasound dataset (PCUS dataset) demonstrates the discriminative power (Accuracy: 93.4%, F1-score: 93.1%, AUC: 97.5%) of the proposed solution overcoming the competing approaches while maintaining low complexity. The proposed model obtained very promising results in comparison with the rival models. More importantly, it gives explainable outputs therefore, it can serve as a candidate tool for empowering the sustainable diagnosis of COVID-19-like diseases in smart healthcare. [ABSTRACT FROM AUTHOR]

Accurate prediction of the carbon trading price (CTP) is crucial to the decision-making of relevant stakeholders, and can also provide a reference for policy makers. However, the time interval for the CTP is one day, resulting in a relatively small sample size of data available for predictions. When dealing with small sample data, deep learning algorithms can trade only a small improvement in prediction accuracy at the expense of efficiency and computing time. In contrast, fine-grained configurations of traditional model inputs and parameters often perform no less well than deep learning algorithms. In this context, this paper proposes a novel hybrid CTP prediction model based on the complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN) and a windowed-based XGBoost approach. First, the initial CTP data is decomposed into multiple subsequences with relatively low volatility and randomness based on the CEEMDAN algorithm. Then, the decomposed carbon valence series and covariates are subject to windowed processing to become the inputs of the XGBoost model. Finally, the universality of the proposed model is verified through case studies of four carbon emission trading markets with different modal characteristics, and the superiority of the proposed model is verified by comparing with seven other models. The results show that the prediction error of the proposed XGBoost(W-b) algorithm is reduced by 4.72%~81.47% compared to other prediction algorithms. In addition, the introduction of CEEMDAN further reduces the prediction error by 25.24%~89.28% on the basis of XGBoost(W-b). [ABSTRACT FROM AUTHOR]

Solving variational image segmentation problems with hidden physics is often expensive and requires different algorithms and manually tuned model parameters. The deep learning methods based on the UNet structure have obtained outstanding performances in many different medical image segmentation tasks, but designing such networks requires many parameters and training data, which are not always available for practical problems. In this paper, inspired by the traditional multiphase convexity Mumford–Shah variational model and full approximation scheme (FAS) solving the nonlinear systems, we propose a novel variational-model-informed network (FAS-UNet), which exploits the model and algorithm priors to extract the multiscale features. The proposed model-informed network integrates image data and mathematical models and implements them through learning a few convolution kernels. Based on the variational theory and FAS algorithm, we first design a feature extraction sub-network (FAS-Solution module) to solve the model-driven nonlinear systems, where a skip-connection is employed to fuse the multiscale features. Secondly, we further design a convolutional block to fuse the extracted features from the previous stage, resulting in the final segmentation possibility. Experimental results on three different medical image segmentation tasks show that the proposed FAS-UNet is very competitive with other state-of-the-art methods in the qualitative, quantitative, and model complexity evaluations. Moreover, it may also be possible to train specialized network architectures that automatically satisfy some of the mathematical and physical laws in other image problems for better accuracy, faster training, and improved generalization. [ABSTRACT FROM AUTHOR]

Speaker embeddings represent a means to extract representative vectorial representations from a speech signal such that the representation pertains to the speaker identity alone. The embeddings are commonly used to classify and discriminate between different speakers. However, there is no objective measure to evaluate the ability of a speaker embedding to disentangle the speaker identity from the other speech characteristics. This means that the embeddings are far from ideal, highly dependent on the training corpus and still include a degree of residual information pertaining to factors such as linguistic content, recording conditions or speaking style of the utterance. This paper introduces an analysis over six sets of speaker embeddings extracted with some of the most recent and high-performing deep neural network (DNN) architectures, and in particular, the degree to which they are able to truly disentangle the speaker identity from the speech signal. To correctly evaluate the architectures, a large multi-speaker parallel speech dataset is used. The dataset includes 46 speakers uttering the same set of prompts, recorded in either a professional studio or their home environments. The analysis looks into the intra- and inter-speaker similarity measures computed over the different embedding sets, as well as if simple classification and regression methods are able to extract several residual information factors from the speaker embeddings. The results show that the discriminative power of the analyzed embeddings is very high, yet across all the analyzed architectures, residual information is still present in the representations in the form of a high correlation to the recording conditions, linguistic contents and utterance duration. However, we show that this correlation, although not ideal, could still be useful in downstream tasks. The low-dimensional projections of the speaker embeddings show similar behavior patterns across the embedding sets with respect to intra-speaker data clustering and utterance outlier detection. [ABSTRACT FROM AUTHOR]

Recent technologies such as artificial intelligence, machine learning, and big data are essential for supporting healthcare monitoring systems, particularly for monitoring Monkeypox confirmed cases. Infected and uninfected cases around the world have contributed to a growing dataset, which is publicly available and can be used by artificial intelligence and machine learning to predict the confirmed cases of Monkeypox at an early stage. Motivated by this, we propose in this paper a new approach for accurate prediction of the Monkeypox confirmed cases based on an optimized Long Short-Term Memory (LSTM) deep network. To fine-tune the hyper-parameters of the LSTM-based deep network, we employed the Al-Biruni Earth Radius (BER) optimization algorithm; thus, the proposed approach is denoted by BER-LSTM. Experimental results show the effectiveness of the proposed approach when assessed using various evaluation criteria, such as Mean Bias Error, which is recorded as (0.06) using BER-LSTM. To prove the superiority of the proposed approach, six different machine learning models are included in the conducted experiments. In addition, four different optimization algorithms are considered for comparison purposes. The results of this comparison confirmed the superiority of the proposed approach. On the other hand, several statistical tests are applied to analyze the stability and significance of the proposed approach. These tests include one-way Analysis of Variance (ANOVA), Wilcoxon, and regression tests. The results of these tests emphasize the robustness, significance, and efficiency of the proposed approach. [ABSTRACT FROM AUTHOR]

Credit card transactions may contain some categorical attributes with large domains, involving up to hundreds of possible values, also known as high-cardinality attributes. The inclusion of such attributes makes analysis harder, due to results with poorer generalization and higher resource usage. A common practice is, therefore, to ignore such attributes, removing them, albeit wasting the information they provided. Contrariwise, this paper reports our findings on the positive impacts of using high-cardinality attributes on credit card fraud detection. Thus, we present a new algorithm for domain reduction that preserves the fraud-detection capabilities. Experiments applying a deep feedforward neural network on real datasets from a major Brazilian financial institution have shown that, when measured by the F-1 metric, the inclusion of such attributes does improve fraud-detection quality. As a main contribution, this proposed algorithm was able to reduce attribute cardinality, improving the training times of a model while preserving its predictive capabilities. [ABSTRACT FROM AUTHOR]

Low-rank matrix/tensor decompositions are promising methods for reducing the inference time, computation, and memory consumption of deep neural networks (DNNs). This group of methods decomposes the pre-trained neural network weights through low-rank matrix/tensor decomposition and replaces the original layers with lightweight factorized layers. A main drawback of the technique is that it demands a great amount of time and effort to select the best ranks of tensor decomposition for each layer in a DNN. This paper proposes a Proxy-based Automatic tensor Rank Selection method (PARS) that utilizes a Bayesian optimization approach to find the best combination of ranks for neural network (NN) compression. We observe that the decomposition of weight tensors adversely influences the feature distribution inside the neural network and impairs the predictability of the post-compression DNN performance. Based on this finding, a novel proxy metric is proposed to deal with the abovementioned issue and to increase the quality of the rank search procedure. Experimental results show that PARS improves the results of existing decomposition methods on several representative NNs, including ResNet-18, ResNet-56, VGG-16, and AlexNet. We obtain a 3 × FLOP reduction with almost no loss of accuracy for ILSVRC-2012ResNet-18 and a 5.5 × FLOP reduction with an accuracy improvement for ILSVRC-2012 VGG-16. [ABSTRACT FROM AUTHOR]

Climate disasters such as floods and droughts often bring heavy losses to human life, national economy, and public safety. El Niño/Southern Oscillation (ENSO) is one of the most important inter-annual climate signals in the tropics and has a global impact on atmospheric circulation and precipitation. To address the impact of climate change, accurate ENSO forecasts can help prevent related climate disasters. Traditional prediction methods mainly include statistical methods and dynamic methods. However, due to the variability and diversity of the temporal and spatial evolution of ENSO, traditional methods still have great uncertainty in predicting ENSO. In recent years, with the rapid development of artificial intelligence technology, it has gradually penetrated into all aspects of people's lives, and the climate field has also benefited. For example, deep learning methods in artificial intelligence can automatically learn and train from a large amount of sample data, obtain excellent feature representation, and effectively improve the performance of various learning tasks. It is widely used in computer vision, natural language processing, and other fields. In 2019, Ham et al. used a convolutional neural network (CNN) model in ENSO forecasting 18 months in advance, and the winter ENSO forecasting skill could reach 0.64, far exceeding the dynamic model with a forecasting skill of 0.5. The research results were regarded as the pioneering work of deep learning in the field of weather forecasting. This paper introduces the traditional ENSO forecasting methods and focuses on summarizing the various latest artificial intelligence methods and their forecasting effects for ENSO forecasting, so as to provide useful reference for future research by researchers. [ABSTRACT FROM AUTHOR]

Tracking of hypersonic glide vehicles (HGVs) by a constellation tracking and observation system is an important part of the space-based early warning system. The uncertainty in the maneuver intentions of HGVs has a non-negligible impact on the tracking and observation process. The cooperative scheduling of multiple satellites in an environment of uncertainty in the maneuver intentions of HGVs is the main problem researched in this paper. For this problem, a satellite constellation tracking decision method that considers the HGVs' maneuver intentions is proposed. This method is based on building an HGV maneuver intention model, developing a maneuver intention recognition and prediction algorithm, and designing a sensor-switching strategy to improve the local consensus-based bundle algorithm (LCBBA). Firstly, a recognizable maneuver intention model that can describe the maneuver types and directions of the HGVs in both the longitudinal and lateral directions was designed. Secondly, a maneuver intention recognition and prediction algorithm based on parallel, stacked long short-term memory neural networks (PSLSTM) was developed to obtain maneuver directions of the HGV. On the basis of that, a satellite constellation tracking decision method (referred to as SS-LCBBA in the following) considering the HGVs' maneuver intentions was designed. Finally, the maneuver intention prediction capability of the PSLSTM network and two currently popular network structures: the multilayer LSTM (M-LSTM) and the dual-channel and bidirectional neural network (DCBNN) were tested for comparison. The simulation results show that the PSLSTM can recognize and predict the maneuver directions of HGVs with high accuracy. In the simulation of a satellite constellation tracking HGVs, the SS-LCBBA improved the cumulative tracking score compared to the LCBBA, the blackboard algorithm (BM), and the variable-center contract network algorithm (ICNP). Thus, it is concluded that SS-LCBBA has better adaptability to environments with uncertain intentions in solving multi-satellite collaborative scheduling problems. [ABSTRACT FROM AUTHOR]

With the booming development of bridge construction, bridge operation and maintenance have always been major issues to ensure the safety of the community. Affected by the long-term service of bridges and natural factors, the safety and durability of cables can be threatened. Cables are critical stress-bearing elements of large bridges such as cable-stayed bridges. Realizing the health monitoring of bridge cables is the key to ensuring the normal operation of bridges. Acoustic emission (AE) is a dynamic nondestructive testing method that is increasingly used in the local monitoring of bridge cables. In this paper, a testbed is described for generating the acoustic emission signals for signal identification testing with machine learning (ML) models. Owing to the limited number of measured signals being available, an algorithm is proposed to simulate acoustic emission signals for model training. A multi-angle feature extraction method is proposed to extract the acoustic emission signals and construct a comprehensive feature vector to characterize the acoustic emission signals. Seven ML models are trained with the simulated acoustic emission signals. Long short-term memory (LSTM) has been specially applied for deep learning demonstration which requires a large amount of training data. As all machine learning models (including LSTM) provide desired performance, it shows that the proposed approach of simulating acoustic emission signals can be effective. [ABSTRACT FROM AUTHOR]

Automatic vessel structure segmentation is essential for an automatic disease diagnosis system. The task is challenging due to vessels' different shapes and sizes across populations. This paper proposes a multiscale network with dual attention to segment various retinal blood vessels. The network injects a spatial attention module and channel attention module on a feature map, whose size is one-eighth of the input size. The network also uses multiscale input to receive multi-level information, and the network uses the multiscale output to gain more supervision. The proposed method is tested on two publicly available datasets: DRIVE and CHASEDB1. The accuracy, AUC, sensitivity, and specificity on the DRIVE dataset are 0.9615, 0.9866, 0.7709, and 0.9847, respectively. On the CHASEDB1 dataset, the metrics are 0.9800, 0.9892, 0.8215, and 0.9877, respectively. The ablative study further shows effectiveness for each part of the network. Multiscale and dual attention mechanism both improve performance. The proposed architecture is simple and effective. The inference time is 12 ms on a GPU and has potential for real-world applications. The code will be made publicly available. [ABSTRACT FROM AUTHOR]

Predicting short-term passenger flow accurately is of great significance for daily management and for a timely emergency response of rail transit networks. In this paper, we propose an attention-based Graph–Temporal Fused Neural Network (GTFNN) that can make online predictions of origin–destination (OD) flows in a large-scale urban transit network. In order to solve the key issue of the passenger hysteresis in online flow forecasting, the proposed GTFNN takes finished OD flow and a series of features, which are known or observable, as the input and performs multi-step prediction. The model is constructed from capturing both spatial and temporal characteristics. For learning spatial characteristics, a multi-layer graph neural network is proposed based on hidden relationships in the rail transit network. Then, we embedded the graph convolution into a Gated Recurrent Unit to learn spatial–temporal features. For learning temporal characteristics, a sequence-to-sequence structure embedded with the attention mechanism is proposed to enhance its ability to capture both local and global dependencies. Experiments based on real-world data collected from Chongqing's rail transit system show that the metrics of GTFNN are better than other methods, e.g., the SMAPE (Symmetric Mean Absolute Percentage Error) score is about 14.16%, with a range from 5% to 20% higher compared to other methods. [ABSTRACT FROM AUTHOR]

Pulmonary diseases are life-threatening diseases commonly observed worldwide, and timely diagnosis of these diseases is essential. Meanwhile, increased use of Convolution Neural Networks has promoted the advancement of computer-assisted clinical recommendation systems for diagnosing diseases using chest radiographs. The texture and shape of the tissues in the diagnostic images are essential aspects of prognosis. Therefore, in the latest studies, the vast set of images with a larger resolution is paired with deep learning techniques to enhance the performance of the disease diagnosis in chest radiographs. Moreover, pulmonary diseases have irregular and different sizes; therefore, several studies sought to add new components to existing deep learning techniques for acquiring multi-scale imaging features from diagnostic chest X-rays. However, most of the attempts do not consider the computation overhead and lose the spatial details in an effort to capture the larger receptive field for obtaining the discriminative features from high-resolution chest X-rays. In this paper, we propose an explainable and lightweight Multi-Scale Chest X-ray Network (MS-CheXNet) to predict abnormal diseases from the diagnostic chest X-rays. The MS-CheXNet consists of four following main subnetworks: (1) Multi-Scale Dilation Layer (MSDL), which includes multiple and stacked dilation convolution channels that consider the larger receptive field and captures the variable sizes of pulmonary diseases by obtaining more discriminative spatial features from the input chest X-rays; (2) Depthwise Separable Convolution Neural Network (DS-CNN) is used to learn imaging features by adjusting lesser parameters compared to the conventional CNN, making the overall network lightweight and computationally inexpensive, making it suitable for mobile vision tasks; (3) a fully connected Deep Neural Network module is used for predicting abnormalities from the chest X-rays; and (4) Gradient-weighted Class Activation Mapping (Grad-CAM) technique is employed to check the decision models' transparency and understand their ability to arrive at a decision by visualizing the discriminative image regions and localizing the chest diseases. The proposed work is compared with existing disease prediction models on chest X-rays and state-of-the-art deep learning strategies to assess the effectiveness of the proposed model. The proposed model is tested with a publicly available Open-I Dataset and data collected from a private hospital. After the comprehensive assessment, it is observed that the performance of the designed approach showcased a 7% to 18% increase in accuracy compared to the existing method. [ABSTRACT FROM AUTHOR]

In the Forex market, trend trading, where trend traders identify trends and attempt to capture gains through the analysis of an asset's momentum in a particular direction, is a great way to profit from market movement. When the price of currency is moving in one either of the direction such as; up or down, it is known as trends. This trend analysis helps traders and investors find low risk entry points or exit points until the trend reverses. In this paper, empirical trade and trend analysis results are suggested by two-phase experimentations. First, considering the blended learning paradigm and wide use of deep-learning methodologies, the variants of long-short-term-memory (LSTM) networks such as Vanilla-LSTM, Stacked-LSTM, Bidirectional-LSTM, CNN-LSTM, and Conv-LSTM are used to build effective investing trading systems for both short-term and long-term timeframes. Then, a deep network-based system used to obtain the trends (up trends and down trends) of the predicted closing price of the currency pairs is proposed based on the best fit predictive networks measured using a few performance measures and Friedman's non-parametric tests. The observed trends are compared and validated with a few readily available technical indicators such as average directional index (ADX), rate of change (ROC), momentum, commodity channel index (CCI), and moving average convergence divergence (MACD). The predictive ability of the proposed strategy for trend analysis can be summarized as follows: (a) with respect to the previous day for short-term predictions, AUD:INR achieves 99.7265% and GBP:INR achieves 99.6582% for long-term predictions; (b) considering the trend analysis strategy with respect to the determinant day, AUD:INR achieves 98.2906% for short-term predictive days and USD:INR achieves an accuracy of trend forecasting with 96.0342%. The significant outcome of this article is the proposed trend forecasting methodology. An attempt has been made to provide an environment to understand the average, maximum, and minimum unit up and/or downs observed during trend forecasting. In turn, this deep learning-based strategy will help investors and traders to comprehend the entry and exit points of this financial market. [ABSTRACT FROM AUTHOR]

With the help of machine learning, many of the problems that have plagued mammography in the past have been solved. Effective prediction models need many normal and tumor samples. For medical applications such as breast cancer diagnosis framework, it is difficult to gather labeled training data and construct effective learning frameworks. Transfer learning is an emerging strategy that has recently been used to tackle the scarcity of medical data by transferring pre-trained convolutional network knowledge into the medical domain. Despite the well reputation of the transfer learning based on the pre-trained Convolutional Neural Networks (CNN) for medical imaging, several hurdles still exist to achieve a prominent breast cancer classification performance. In this paper, we attempt to solve the Feature Dimensionality Curse (FDC) problem of the deep features that are derived from the transfer learning pre-trained CNNs. Such a problem is raised due to the high space dimensionality of the extracted deep features with respect to the small size of the available medical data samples. Therefore, a novel deep learning cascaded feature selection framework is proposed based on the pre-trained deep convolutional networks as well as the univariate-based paradigm. Deep learning models of AlexNet, VGG, and GoogleNet are randomly selected and used to extract the shallow and deep features from the INbreast mammograms, whereas the univariate strategy helps to overcome the dimensionality curse and multicollinearity issues for the extracted features. The optimized key features via the univariate approach are statistically significant (p-value ≤ 0.05) and have good capability to efficiently train the classification models. Using such optimal features, the proposed framework could achieve a promising evaluation performance in terms of 98.50% accuracy, 98.06% sensitivity, 98.99% specificity, and 98.98% precision. Such performance seems to be beneficial to develop a practical and reliable computer-aided diagnosis (CAD) framework for breast cancer classification. [ABSTRACT FROM AUTHOR]

The world is still trying to recover from the devastation caused by the wide spread of COVID-19, and now the monkeypox virus threatens becoming a worldwide pandemic. Although the monkeypox virus is not as lethal or infectious as COVID-19, numerous countries report new cases daily. Thus, it is not surprising that necessary precautions have not been taken, and it will not be surprising if another worldwide pandemic occurs. Machine learning has recently shown tremendous promise in image-based diagnosis, including cancer detection, tumor cell identification, and COVID-19 patient detection. Therefore, a similar application may be implemented to diagnose monkeypox as it invades the human skin. An image can be acquired and utilized to further diagnose the condition. In this paper, two algorithms are proposed for improving the classification accuracy of monkeypox images. The proposed algorithms are based on transfer learning for feature extraction and meta-heuristic optimization for feature selection and optimization of the parameters of a multi-layer neural network. The GoogleNet deep network is adopted for feature extraction, and the utilized meta-heuristic optimization algorithms are the Al-Biruni Earth radius algorithm, the sine cosine algorithm, and the particle swarm optimization algorithm. Based on these algorithms, a new binary hybrid algorithm is proposed for feature selection, along with a new hybrid algorithm for optimizing the parameters of the neural network. To evaluate the proposed algorithms, a publicly available dataset is employed. The assessment of the proposed optimization of feature selection for monkeypox classification was performed in terms of ten evaluation criteria. In addition, a set of statistical tests was conducted to measure the effectiveness, significance, and robustness of the proposed algorithms. The results achieved confirm the superiority and effectiveness of the proposed methods compared to other optimization methods. The average classification accuracy was 98.8%. [ABSTRACT FROM AUTHOR]

Person re-identification has become an essential application within computer vision due to its ability to match the same person over non-overlapping cameras. However, it is a challenging task because of the broad view of cameras with a large number of pedestrians appearing with various poses. As a result, various approaches of supervised model learning have been utilized to locate and identify a person based on the given input. Nevertheless, several of these approaches perform worse than expected in retrieving the right person in real-time over multiple CCTVs/camera views. This is due to inaccurate segmentation of the person, leading to incorrect classification. This paper proposes an efficient and real-time person re-identification system, named ReID-DeePNet system. It is based on fusing the matching scores generated by two different deep learning models, convolutional neural network and deep belief network, to extract discriminative feature representations from the pedestrian image. Initially, a segmentation procedure was developed based on merging the advantages of the Mask R-CNN and GrabCut algorithm to tackle the adverse effects caused by background clutter. Afterward, the two different deep learning models extracted discriminative feature representations from the pedestrian segmented image, and their matching scores were fused to make the final decision. Several extensive experiments were conducted, using three large-scale and challenging person re-identification datasets: Market-1501, CUHK03, and P-DESTRE. The ReID-DeePNet system achieved new state-of-the-art Rank-1 and mAP values on these three challenging ReID datasets. [ABSTRACT FROM AUTHOR]

Image data play an important role in our daily lives, and scholars have recently leveraged deep learning to design steganography networks to conceal and protect image data. However, the complexity of computation and the running speed have been neglected in their model designs, and steganography security still has much room for improvement. For this purpose, this paper proposes an RDA-based network, which can achieve higher security with lower computation complexity and faster running speed. To improve the hidden image's quality and ensure that the hidden image and cover image are as similar as possible, a residual dense attention (RDA) module was designed to extract significant information from the cover image, thus assisting in reconstructing the salient target of the hidden image. In addition, we propose an activation removal strategy (ARS) to avoid undermining the fidelity of low-level features and to preserve more of the raw information from the input cover image and the secret image, which significantly boosts the concealing and revealing performance. Furthermore, to enable comprehensive supervision for the concealing and revealing processes, a mixed loss function was designed, which effectively improved the hidden image's visual quality and enhanced the imperceptibility of secret content. Extensive experiments were conducted to verify the effectiveness and superiority of the proposed approach. [ABSTRACT FROM AUTHOR]

In recent years, with the rapid development of the Internet and information technology, video websites, shopping websites, and other portals have grown rapidly. However, malicious webpages can disguise themselves as benign websites and steal users' private information, which seriously threatens network security. Current detection methods for malicious webpages do not fully utilize the syntactic and semantic information in the web source code. In this paper, we propose a GCN-based malicious webpage detection method (GMWD), which constructs a text graph to describe and then a GCN model to learn the syntactic and semantic correlations within and between webpage source codes. We replace word nodes in the text graph with phrase nodes to better maintain the syntactic and semantic integrity of the webpage source code. In addition, we use the URL links appearing in the source code as auxiliary detection information to further improve the detection accuracy. The experiments showed that the proposed method can achieve 99.86% accuracy and a 0.137% false negative rate, achieving a better performance than other related malicious webpage detection methods. [ABSTRACT FROM AUTHOR]

In recent years, with the rapid development of Internet services in all walks of life, a large number of malicious acts such as network attacks, data leakage, and information theft have become major challenges for network security. Due to the difficulty of malicious traffic collection and labeling, the distribution of various samples in the existing dataset is seriously imbalanced, resulting in low accuracy of malicious traffic classification based on machine learning and deep learning, and poor model generalization ability. In this paper, a feature image representation method and Adversarial Generative Network with Filter (Filter-GAN) are proposed to solve these problems. First, the feature image representation method divides the original session traffic into three parts. The Markov matrix is extracted from each part to form a three-channel feature image. This method can transform the original session traffic format into a uniform-length matrix and fully characterize the network traffic. Then, Filter-GAN uses the feature images to generate few attack samples. Compared with general methods, Filter-GAN can generate more efficient samples. Experiments were conducted on public datasets. The results show that the feature image representation method can effectively characterize the original session traffic. When the number of samples is sufficient, the classification accuracy can reach 99%. Compared with unbalanced datasets, Filter-GAN has significantly improved the recognition accuracy of small-sample datasets, with a maximum improvement of 6%. [ABSTRACT FROM AUTHOR]

Deep learning has been widely used in different fields such as computer vision and speech processing. The performance of deep learning algorithms is greatly affected by their hyperparameters. For complex machine learning models such as deep neural networks, it is difficult to determine their hyperparameters. In addition, existing hyperparameter optimization algorithms easily converge to a local optimal solution. This paper proposes a method for hyperparameter optimization that combines the Sparrow Search Algorithm and Particle Swarm Optimization, called the Hybrid Sparrow Search Algorithm. This method takes advantages of avoiding the local optimal solution in the Sparrow Search Algorithm and the search efficiency of Particle Swarm Optimization to achieve global optimization. Experiments verified the proposed algorithm in simple and complex networks. The results show that the Hybrid Sparrow Search Algorithm has the strong global search capability to avoid local optimal solutions and satisfactory search efficiency in both low and high-dimensional spaces. The proposed method provides a new solution for hyperparameter optimization problems in deep learning models. [ABSTRACT FROM AUTHOR]

The unidirectional consolidation theory of soils is widely used in certain conditions and approximate calculations. The multidirectional theory of soil consolidation is more reasonable than the unidirectional theory in practical applications but is much more complicated in terms of index determination and solution. To address the above problem, in this paper, we propose a deep learning method using physics-informed neural networks (PINN) to predict the excess pore water pressure of two-dimensional soil consolidation. In the proposed method, (1) a fully connected neural network is constructed; (2) the computational domain, partial differential equation (PDE), and constraints are defined to generate data for model training; and (3) the PDE of two-dimensional soil consolidation and the model of the neural network are connected to reduce the loss of the model. The effectiveness of the proposed method is verified by comparison with the numerical solution of PDE for two-dimensional consolidation. Moreover, the FEM and the proposed PINN-based method are applied to predict the consolidation of foundation soils in a real case of Sichuan Railway in China, and the results are quite consistent. The proposed deep learning approach can be used to investigate large and complex multidirectional soil consolidation. [ABSTRACT FROM AUTHOR]

In the context of the new era of high digitization and informatization, the emergence of the internet and artificial intelligence technologies has profoundly changed people's lifestyles. The traditional cyber attack detection has become increasingly weak in the context of the increasingly complex network environment in the new era, and deep learning technology has begun to play a significant role in the field of network security. There are many kinds of attacks against web applications, which are very harmful, including SQL (Structured Query Language) injection, XSS (Cross-Site Scripting), and command injection. Based on the detection of SQL injection and XSS attacks, this paper combines the detection of command injection attacks, which are also very harmful, and proposes a multi-classification detection method for web injection attacks. We extract features in the URL (Uniform Resource Locator) and request body of HTTP (Hyper Text Transfer Protocol) requests and combine deep learning technology to build a multi-classification model for injection attacks. Firstly, aiming at the problem of imbalanced distribution of training samples and low detection accuracy of command injection attack, a sample generation method is proposed. The experimental results show that the proposed method ensures a higher detection rate of command injection attacks and lower false alarms. Secondly, we propose a more expressive feature fusion model, which effectively combines the features extracted by deep learning with the discrete features extracted manually. The experimental results show that the feature fusion model proposed in this work is more effective compared with a single deep learning model. The accuracy of the model is improved by about 1%. [ABSTRACT FROM AUTHOR]

With the rapid development of network technologies, the network security of industrial control systems has aroused widespread concern. As a defense mechanism, an ideal intrusion detection system (IDS) can effectively detect abnormal behaviors in a system without affecting the performance of the industrial control system (ICS). Many deep learning methods are used to build an IDS, which rely on massive numbers of variously labeled samples for model training. However, network traffic is imbalanced, and it is difficult for researchers to obtain sufficient attack samples. In addition, the attack variants are rich, and constructing all possible attack types in advance is impossible. In order to overcome these challenges and improve the performance of an IDS, this paper presents a novel intrusion detection approach which integrates a one-dimensional convolutional autoencoder (1DCAE) and support vector data description (SVDD) for the first time. For the two-stage training process, 1DCAE fails to retain the key features of intrusion detection and SVDD has to add restrictions, so a joint optimization solution is introduced. A three-stage optimization process is proposed to obtain better performance. Experiments on the benchmark intrusion detection dataset NSL-KDD show that the proposed method can effectively detect various unknown attacks, learning with only normal traffic. Compared with the recent state-of-art intrusion detection baselines, the proposed method is improved in most metrics. [ABSTRACT FROM AUTHOR]

Cloud storage has become eminent, with an increasing amount of data being produced daily; this has led to substantial concerns related to privacy and unauthorized access. To secure privacy, users can protect their private data by uploading encrypted data to the cloud. Data encryption allows computations to be performed on encrypted data without the data being decrypted in the cloud, which requires enormous computation resources and prevents unauthorized access to private data. Data analysis such as classification, and image query and retrieval can preserve data privacy if the analysis is performed using encrypted data. This paper proposes an image-captioning method that generates captions over encrypted images using an encoder–decoder framework with attention and a double random phase encoding (DRPE) encryption scheme. The images are encrypted with DRPE to protect them and then fed to an encoder that adopts the ResNet architectures to generate a fixed-length vector of representations or features. The decoder is designed with long short-term memory to process the features and embeddings to generate descriptive captions for the images. We evaluate the predicted captions with BLEU, METEOR, ROUGE, and CIDEr metrics. The experimental results demonstrate the feasibility of our privacy-preserving image captioning on the popular benchmark Flickr8k dataset. [ABSTRACT FROM AUTHOR]

Cavity under urban roads has increasingly become a huge threat to traffic safety. This paper aims to study cavity morphology characteristics and proposes a deep learning (DL)-based morphology classification method using the 3D ground-penetrating radar (GPR) data. Fine-tuning technology in DL can be used in some cases with relatively few samples, but in the case of only one or very few samples, there will still be overfitting problems. To address this issue, a simple and general framework, few-shot learning (FSL), is first employed for the cavity classification tasks, based on which a classifier learns to identify new classes given only very few examples. We adopt a relation network (RelationNet) as the FSL framework, which consists of an embedding module and a relation module. Furthermore, the proposed method is simpler and faster because it does not require pre-training or fine-tuning. The experimental results are validated using the 3D GPR road modeling data obtained from the gprMax3D system. The proposed method is compared with other FSL networks such as ProtoNet, R2D2, and BaseLine relative to different benchmarks. The experimental results demonstrate that this method outperforms other prior approaches, and its average accuracy reaches 97.328% in a four-way five-shot problem using few support samples. [ABSTRACT FROM AUTHOR]

In different kinds of sports, the balance control ability plays an important role for every athlete. Therefore, coaches and athletes need accurate and efficient assessments of the balance control ability to improve the athletes' training performance scientifically. With the fast growth of sport technology and training devices, intelligent and automatic assessment methods have been in high demand in the past years. This paper proposes a deep-learning-based method for a balance control ability assessment involving an analysis of the time-series signals from the athletes. The proposed method directly processes the raw data and provides the assessment results, with an end-to-end structure. This straight-forward structure facilitates its practical application. A deep learning model is employed to explore the target features with a multi-headed self-attention mechanism, which is a new approach to sports assessments. In the experiments, the real athletes' balance control ability assessment data are utilized for the validation of the proposed method. Through comparisons with different existing methods, the accuracy rate of the proposed method is shown to be more than 95% for all four tasks, which is higher than the other compared methods for tasks containing more than one athlete of each level. The results show that the proposed method works effectively and efficiently in real scenarios for athlete balance control ability evaluations. However, reducing the proposed method's calculation costs is an important task for future studies. [ABSTRACT FROM AUTHOR]

It is significant for anesthesiologists to have a precise grasp of the recovery time of the patient after anesthesia. Accurate prediction of anesthesia recovery time can support anesthesiologist decision-making during surgery to help reduce the risk of surgery in patients. However, effective models are not proposed to solve this problem for anesthesiologists. In this paper, we seek to find effective forecasting methods. First, we collect 1824 patient anesthesia data from the eye center and then performed data preprocessing. We extracted 85 variables to predict recovery time from anesthesia. Second, we extract anesthesia information between variables for prediction using machine learning methods, including Bayesian ridge, lightGBM, random forest, support vector regression, and extreme gradient boosting. We also design simple deep learning models as prediction models, including linear residual neural networks and jumping knowledge linear neural networks. Lastly, we perform a comparative experiment of the above methods on the dataset. The experiment demonstrates that the machine learning method performs better than the deep learning model mentioned above on a small number of samples. We find random forest and XGBoost are more efficient than other methods to extract information between variables on postoperative anesthesia recovery time. [ABSTRACT FROM AUTHOR]

With developments in artificial intelligence (AI), it is possible for novel applications to utilize deep learning to compose music by the format of musical instrument digital interface (MIDI) even without any knowledge of musical theory. The composed music is generally evaluated by human-based Turing test, which is a subjective approach and does not provide any quantitative criteria. Therefore, objective evaluation approaches with many general descriptive parameters are applied to the evaluation of MIDI data while considering MIDI features such as pitch distances, chord rates, tone spans, drum patterns, etc. However, setting several general descriptive parameters manually on large datasets is difficult and has considerable generalization limitations. In this paper, an enhanced evaluation method based on random masking and sequence-to-sequence (Seq2Seq) model is proposed to evaluate MIDI data. An experiment was conducted on real MIDI data, generated MIDI data, and random MIDI data. The bilingual evaluation understudy (BLEU) is a common MIDI data evaluation approach and is used here to evaluate the performance of the proposed method in a comparative study. In the proposed method, the ratio of the average evaluation score of the generated MIDI data to that of the real MIDI data was 31%, while that of BLEU was 79%. The lesser the ratio, the greater the difference between the real MIDI data and generated MIDI data. This implies that the proposed method quantified the gap while accurately identifying real and generated MIDI data. [ABSTRACT FROM AUTHOR]

Along with the wide use of deep learning technology, its security issues have drawn much attention over the years. Adversarial examples expose the inherent vulnerability of deep learning models and make it a challenging task to improve their robustness. Model robustness is related not only to its parameters but also to its architecture. This paper proposes a novel robustness enhanced approach for neural networks based on a neural architecture search. First, we randomly sample multiple neural networks to construct the initial population. Second, we utilize the individual networks in the population to fit and update the surrogate models. Third, the population of neural networks is evolved through a multi-objective evolutionary algorithm, where the surrogate models accelerate the performance evaluation of networks. Finally, the second and third steps are performed alternately until a network architecture with high accuracy and robustness is achieved. Experimental results show that the proposed method outperforms some classical artificially designed neural networks and other architecture search algorithms in terms of robustness. [ABSTRACT FROM AUTHOR]

Cloud detection is an essential step in optical remote sensing data processing. With the development of deep learning technology, cloud detection methods have made remarkable progress. Among them, researchers have started to try to introduce Transformer into cloud detection tasks due to its excellent performance in image semantic segmentation tasks. However, the current Transformer-based methods suffer from training difficulty and low detection accuracy of small clouds. To solve these problems, this paper proposes Cloudformer V2 based on the previously proposed Cloudformer. For the training difficulty, Cloudformer V2 uses Set Attention Block to extract intermediate features as Set Prior Prediction to participate in supervision, which enables the model to converge faster. For the detection of small clouds, Cloudformer V2 decodes the features by a multi-scale Transformer decoder, which uses multi-resolution features to improve the modeling accuracy. In addition, a binary mask weighted loss function (BW Loss) is designed to construct weights by counting pixels classified as clouds; thus, guiding the network to focus on features of small clouds and improving the overall detection accuracy. Cloudformer V2 is experimented on the dataset from GF-1 satellite and has excellent performance. [ABSTRACT FROM AUTHOR]

Metastasis detection in lymph nodes via microscopic examination of histopathological images is one of the most crucial diagnostic procedures for breast cancer staging. The manual analysis is extremely labor-intensive and time-consuming because of complexities and diversities of histopathology images. Deep learning has been utilized in automatic cancer metastasis detection in recent years. Due to the huge size of whole-slide images, most existing approaches split each image into smaller patches and simply treat these patches independently, which ignores the spatial correlations among them. To solve this problem, this paper proposes an effective spatially sensitive learning framework for cancer metastasis detection in whole-slide images. Moreover, a novel spatial loss function is designed to ensure the consistency of prediction over neighboring patches. Specifically, through incorporating long short-term memory and spatial loss constraint on top of a convolutional neural network feature extractor, the proposed method can effectively learn both the appearance of each patch and spatial relationships between adjacent image patches. With the standard back-propagation algorithm, the whole framework can be trained in an end-to-end way. Finally, the regions with high tumor probability in the resulting probability map are the metastasis locations. Extensive experiments on the benchmark Camelyon 2016 Grand Challenge dataset show the effectiveness of the proposed approach with respect to state-of-the-art competitors. The obtained precision, recall, and balanced accuracy are 0.9565, 0.9167, and 0.9458, respectively. It is also demonstrated that the proposed approach can provide more accurate detection results and is helpful for early diagnosis of cancer metastasis. [ABSTRACT FROM AUTHOR]

Software authorship attribution, defined as the problem of software authentication and resolution of source code ownership, is of major relevance in the software engineering field. Authorship analysis of source code is more difficult than the classic task on literature, but it would be of great use in various software development activities such as software maintenance, software quality analysis or project management. This paper addresses the problem of code authorship attribution and introduces, as a proof of concept, a new supervised classification model AutoSoft for identifying the developer of a certain piece of code. The proposed model is composed of an ensemble of autoencoders that are trained to encode and recognize the programming style of software developers. An extension of the AutoSoft classifier, able to recognize an unknown developer (a developer that was not seen during the training), is also discussed and evaluated. Experiments conducted on software programs collected from the Google Code Jam data set highlight the performance of the proposed model in various test settings. A comparison to existing similar solutions for code authorship attribution indicates that AutoSoft outperforms most of them. Moreover, AutoSoft provides the advantage of adaptability, illustrated through a series of extensions such as the definition of class membership probabilities and the re-framing of the AutoSoft system to address one-class classification. [ABSTRACT FROM AUTHOR]

Artificial intelligence (AI) is an evolving set of technologies used for solving a wide range of applied issues. The core of AI is machine learning (ML)—a complex of algorithms and methods that address the problems of classification, clustering, and forecasting. The practical application of AI&ML holds promising prospects. Therefore, the researches in this area are intensive. However, the industrial applications of AI and its more intensive use in society are not widespread at the present time. The challenges of widespread AI applications need to be considered from both the AI (internal problems) and the societal (external problems) perspective. This consideration will identify the priority steps for more intensive practical application of AI technologies, their introduction, and involvement in industry and society. The article presents the identification and discussion of the challenges of the employment of AI technologies in the economy and society of resource-based countries. The systematization of AI&ML technologies is implemented based on publications in these areas. This systematization allows for the specification of the organizational, personnel, social and technological limitations. This paper outlines the directions of studies in AI and ML, which will allow us to overcome some of the limitations and achieve expansion of the scope of AI&ML applications. [ABSTRACT FROM AUTHOR]

In the paper by Jovanovic et al. [[11]], the authors propose a hybrid ML and swarm intelligence approach to address credit card fraud detection. 10.3390/math10081232 11 Jovanovic D., Antonijevic M., Stankovic M., Zivkovic M., Tanaskovic M., Bacanin N. Tuning Machine Learning Models Using a Group Search Firefly Algorithm for Credit Card Fraud Detection. Swarm and evolutionary computation (SEC) [[1]] is a broad and growing area of modern computer sciences, dealing with nature-inspired systems that are capable of displaying intelligent behaviour, thus optimising a vast range of challenging real-world scenarios that cannot be addressed via the direct application of purely theoretical exact approaches (e.g., [[2]]). [Extracted from the article]

According to the World Energy Investment 2018 report, the global annual investment in renewable energy exceeded USD 200 billion for eight consecutive years until 2017. In this paper, a deep-learning-based time-series prediction method, namely a gated recurrent unit (GRU)-based prediction method, is proposed to predict energy generation in Taiwan. Data on thermal power (coal, oil, and gas power), renewable energy (conventional hydropower, solar power, and wind power), pumped hydropower, and nuclear power generation for 1991 to 2020 were obtained from the Bureau of Energy, Ministry of Economic Affairs, Taiwan, and the Taiwan Power Company. The proposed GRU-based method was compared with six common forecasting methods: autoregressive integrated moving average, exponential smoothing (ETS), Holt–Winters ETS, support vector regression (SVR), whale-optimization-algorithm-based SVR, and long short-term memory. Among the methods compared, the proposed method had the lowest mean absolute percentage error and root mean square error and thus the highest accuracy. Government agencies and power companies in Taiwan can use the predictions of accurate energy forecasting models as references to formulate energy policies and design plans for the development of alternative energy sources. [ABSTRACT FROM AUTHOR]

Domain discrepancy is a key research problem in the field of deep domain adaptation. Two main strategies are used to reduce the discrepancy: the parametric method and the nonparametric method. Both methods have achieved good results in practical applications. However, research on whether the combination of the two can further reduce domain discrepancy has not been conducted. Therefore, in this paper, a deep transfer learning method based on automatic domain alignment and moment matching (DA-MM) is proposed. First, an automatic domain alignment layer is embedded in the front of each domain-specific layer of a neural network structure to preliminarily align the source and target domains. Then, a moment matching measure (such as MMD distance) is added between every domain-specific layer to map the source and target domain features output by the alignment layer to a common reproduced Hilbert space. The results of an extensive experimental analysis over several public benchmarks show that DA-MM can reduce the distribution discrepancy between the two domains and improve the domain adaptation performance. [ABSTRACT FROM AUTHOR]

In recent years, reinforcement learning algorithms based on automatic curriculum learning have been increasingly applied to multi-agent system problems. However, in the sparse reward environment, the reinforcement learning agents get almost no feedback from the environment during the whole training process, which leads to a decrease in the convergence speed and learning efficiency of the curriculum reinforcement learning algorithm. Based on the automatic curriculum learning algorithm, this paper proposes a curriculum reinforcement learning method based on the curiosity model (CMCL). The method divides the curriculum sorting criteria into temporal-difference error and curiosity reward, uses the K-fold cross validation method to evaluate the difficulty priority of task samples, uses the Intrinsic Curiosity Module (ICM) to evaluate the curiosity priority of the task samples, and uses the curriculum factor to adjust the learning probability of the task samples. This study compares the CMCL algorithm with other baseline algorithms in cooperative-competitive environments, and the experimental simulation results show that the CMCL method can improve the training performance and robustness of multi-agent deep reinforcement learning algorithms. [ABSTRACT FROM AUTHOR]

Clustering algorithms and deep learning methods have been widely applied in the multimode process monitoring. However, for the process data with unknown mode, traditional clustering methods can hardly identify the number of modes automatically. Further, deep learning methods can learn effective features from nonlinear process data, while the extracted features cannot follow the Gaussian distribution, which may lead to incorrect control limit for fault detection. In this paper, a comprehensive monitoring method based on modified density peak clustering and parallel variational autoencoder (MDPC-PVAE) is proposed for multimode processes. Firstly, a novel clustering algorithm, named MDPC, is presented for the mode identification and division. MDPC can identify the number of modes without prior knowledge of mode information and divide the whole process data into multiple modes. Then, the PVAE is established based on distinguished multimode data to generate the deep nonlinear features, in which the generated features in each VAE follow the Gaussian distribution. Finally, the Gaussian feature representations obtained by PVAE are provided to construct the statistics H 2 , and the control limits are determined by the kernel density estimation (KDE) method. The effectiveness of the proposed method is evaluated by the Tennessee Eastman process and semiconductor etching process. [ABSTRACT FROM AUTHOR]

Cerebrospinal fluid (CSF) exists in the surrounding spaces of mammalian central nervous systems (CNS); therefore, there are numerous potential protein biomarkers associated with CNS disease in CSF. Currently, approximately 4300 proteins have been identified in CSF by protein profiling. However, due to the diverse modifications, as well as the existing technical limits, large-scale protein identification in CSF is still considered a challenge. Inspired by computational methods, this paper proposes a deep learning framework, named DenSec, for secreted protein prediction in CSF. In the first phase of DenSec, all input proteins are encoded as a matrix with a fixed size of 1000 × 20 by calculating a position-specific score matrix (PSSM) of protein sequences. In the second phase, a dense convolutional network (DenseNet) is adopted to extract the feature from these PSSMs automatically. After that, Transformer with a fully connected dense layer acts as classifier to perform a binary classification in terms of secretion into CSF or not. According to the experiment results, DenSec achieves a mean accuracy of 86.00% in the test dataset and outperforms the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

The metastasis detection in lymph nodes via microscopic examination of H&E stained histopathological images is one of the most crucial diagnostic procedures for breast cancer staging. The manual analysis is extremely labor-intensive and time-consuming because of complexities and diversities of histopathological images. Deep learning has been utilized in automatic cancer metastasis detection in recent years. The success of supervised deep learning is credited to a large labeled dataset, which is hard to obtain in medical image analysis. Contrastive learning, a branch of self-supervised learning, can help in this aspect through introducing an advanced strategy to learn discriminative feature representations from unlabeled images. In this paper, we propose to improve breast cancer metastasis detection through self-supervised contrastive learning, which is used as an accessional task in the detection pipeline, allowing a feature extractor to learn more valuable representations, even if there are fewer annotation images. Furthermore, we extend the proposed approach to exploit unlabeled images in a semi-supervised manner, as self-supervision does not need labeled data at all. Extensive experiments on the benchmark Camelyon2016 Grand Challenge dataset demonstrate that self-supervision can improve cancer metastasis detection performance leading to state-of-the-art results. [ABSTRACT FROM AUTHOR]

In this paper, we propose an augmented barrier certificate-based method for formally verifying the approximate initial-state opacity property of discrete time control systems. The opacity verification problem is formulated as the safety verification of an augmented system and is then addressed by searching for augmented barrier certificates. A set of well-defined verification conditions is a prerequisite for successfully identifying augmented barrier certificates of a specific type. We first suggest a new type of augmented barrier certificate which produces a weaker sufficient condition for approximate initial-state opacity. Furthermore, we develop an algorithmic framework where a learner and a verifier interact to synthesize augmented barrier certificates in the form of neural networks. The learner trains neural certificates via the deep learning method, and the verifier solves several mixed integer linear programs to either ensure the validity of the candidate certificates or yield counterexamples, which are passed back to further guide the learner. The experimental results demonstrate that our approach is more scalable and effective than the existing sum of squares programming method. [ABSTRACT FROM AUTHOR]

Automated fruit classification is a stimulating problem in the fruit growing and retail industrial chain as it assists fruit growers and supermarket owners to recognize variety of fruits and the status of the container or stock to increase business profit and production efficacy. As a result, intelligent systems using machine learning and computer vision approaches were explored for ripeness grading, fruit defect categorization, and identification over the last few years. Recently, deep learning (DL) methods for classifying fruits led to promising performance that effectively extracts the feature and carries out an end-to-end image classification. This paper introduces an Automated Fruit Classification using Hyperparameter Optimized Deep Transfer Learning (AFC-HPODTL) model. The presented AFC-HPODTL model employs contrast enhancement as a pre-processing step which helps to enhance the quality of images. For feature extraction, the Adam optimizer with deep transfer learning-based DenseNet169 model is used in which the Adam optimizer fine-tunes the initial values of the DenseNet169 model. Moreover, a recurrent neural network (RNN) model is utilized for the identification and classification of fruits. At last, the Aquila optimization algorithm (AOA) is exploited for optimal hyperparameter tuning of the RNN model in such a way that the classification performance gets improved. The design of Adam optimizer and AOA-based hyperparameter optimizers for DenseNet and RNN models show the novelty of the work. The performance validation of the presented AFC-HPODTL model is carried out utilizing a benchmark dataset and the outcomes report the promising performance over its recent state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

Experimental studies were carried out to analyze the impact of optimizers and learning rate on the performance of deep learning-based algorithms for rock thin-section image classification. A total of 2634 rock thin-section images including three rock types—metamorphic, sedimentary, and volcanic rocks—were acquired from an online open-source science data bank. Four CNNs using three different optimizer algorithms (Adam, SGD, RMSprop) under two learning-rate decay schedules (lambda and cosine decay modes) were trained and validated. Then, a systematic comparison was conducted based on the performance of the trained model. Precision, f1-scores, and confusion matrix were adopted as the evaluation indicators. Trials revealed that deep learning-based approaches for rock thin-section image classification were highly effective and stable. Meanwhile, the experimental results showed that the cosine learning-rate decay mode was the better option for learning-rate adjustment during the training process. In addition, the performance of the four neural networks was confirmed and ranked as VGG16, GoogLeNet, MobileNetV2, and ShuffleNetV2. In the last step, the influence of optimization algorithms was evaluated based on VGG16 and GoogLeNet, and the results demonstrated that the capabilities of the model using Adam and RMSprop optimizers were more robust than that of SGD. The experimental study in this paper provides important practical value for training a high-precision rock thin-section image classification model, which can also be transferred to other similar image classification tasks. [ABSTRACT FROM AUTHOR]

The composition of self-compacting concrete (SCC) contains 60–70% coarse and fine aggregates, which are replaced by construction waste, such as recycled aggregates (RA). However, the complexity of its structure requires a time-consuming mixed design. Currently, many researchers are studying the prediction of concrete properties using soft computing techniques, which will eventually reduce environmental degradation and other material waste. There have been very limited and contradicting studies regarding prediction using different ANN algorithms. This paper aimed to predict the 28-day splitting tensile strength of SCC with RA using the artificial neural network technique by comparing the following algorithms: Levenberg–Marquardt (LM), Bayesian regularization (BR), and Scaled Conjugate Gradient Backpropagation (SCGB). There have been very limited and contradicting studies regarding prediction by using and comparing different ANN algorithms, so a total of 381 samples were collected from various published journals. The input variables were cement, admixture, water, fine and coarse aggregates, and superplasticizer; the data were randomly divided into three sets—training (60%), validation (10%), and testing (30%)—with 10 neurons in the hidden layer. The models were evaluated by the mean squared error (MSE) and correlation coefficient (R). The results indicated that all three models have optimal accuracy; still, BR gave the best performance (R = 0.91 and MSE = 0.2087) compared with LM and SCG. BR was the best model for predicting TS at 28 days for SCC with RA. The sensitivity analysis indicated that cement (30.07%) was the variable that contributed the most to the prediction of TS at 28 days for SCC with RA, and water (2.39%) contributed the least. [ABSTRACT FROM AUTHOR]

With the development of Internet of Vehicles (IoV) technology, the car is no longer a closed individual. It exchanges information with an external network, communicating through the vehicle-mounted network (VMN), which, inevitably, gives rise to security problems. Attackers can intrude on the VMN, using a wireless network or vehicle-mounted interface devices. To prevent such attacks, various intrusion-detection methods have been proposed, including convolutional neural network (CNN) ones. However, the existing CNN method was not able to best use the CNN's capability, of extracting two-dimensional graph-like data, and, at the same time, to reflect the time connections among the sequential data. Therefore, this paper proposed a novel CNN model, based on two-dimensional Mosaic pattern coding, for anomaly detection. It can not only make full use of the ability of a CNN to extract grid data but also maintain the sequential time relationship of it. Simulations showed that this method could, effectively, distinguish attacks from the normal information on the vehicular network, improve the reliability of the system's discrimination, and, at the same time, meet the real-time requirement of detection. [ABSTRACT FROM AUTHOR]

Homomorphic encryption with the ability to compute over encrypted data without access to the secret key provides benefits for the secure and powerful computation, storage, and communication of resources in the cloud. One of its important applications is fast-growing robot control systems for building lightweight, low-cost, smarter robots with intelligent brains consisting of data centers, knowledge bases, task planners, deep learning, information processing, environment models, communication support, synchronous map construction and positioning, etc. It enables robots to be endowed with secure, powerful capabilities while reducing sizes and costs. Processing encrypted information using homomorphic ciphers uses the sign function polynomial approximation, which is a widely studied research field with many practical results. State-of-the-art works are mainly focused on finding the polynomial of best approximation of the sign function (PBAS) with the improved errors on the union of the intervals [ − 1 , − ϵ ] ∪ [ ϵ , 1 ] . However, even though the existence of the single PBAS with the minimum deviation is well known, its construction method on the complete interval [ − 1 , 1 ] is still an open problem. In this paper, we provide the PBAS construction method on the interval [ − 1 , 1 ] , using as a norm the area between the sign function and the polynomial and showing that for a polynomial degree n ≥ 1 , there is (1) unique PBAS of the odd sign function, (2) no PBAS of the general form sign function if n is odd, and (3) an uncountable set of PBAS, if n is even. [ABSTRACT FROM AUTHOR]

With the advent of big data and the information age, the data magnitude of various complex networks is growing rapidly. Many real-life situations cannot be portrayed by ordinary networks, while hypergraphs have the ability to describe and characterize higher order relationships, which have attracted extensive attention from academia and industry in recent years. Firstly, this paper described the development process, the application areas, and the existing review research of hypergraphs; secondly, introduced the theory of hypergraphs briefly; then, compared the learning methods of ordinary graphs and hypergraphs from three aspects: matrix decomposition, random walk, and deep learning; next, introduced the structural optimization of hypergraphs from three perspectives: dynamic hypergraphs, hyperedge weight optimization, and multimodal hypergraph generation; after that, the applicability of three uncertain hypergraph models were analyzed based on three uncertainty theories: probability theory, fuzzy set, and rough set; finally, the future research directions of hypergraphs and uncertain hypergraphs were prospected. [ABSTRACT FROM AUTHOR]

The 12 leads of electrocardiogram (ECG) signals show the heart activities from different angles of coronal and axial planes; hence, the signals of these 12 leads have functional dependence on each other. This paper proposes a novel method for fusing the data of 12-lead ECG signals to diagnose heart problems. In the first phase of the proposed method, the time-frequency transform is employed to fuse the functional data of leads and extract the frequency data of ECG signals in 12 leads. After that, their dependence is evaluated through the correlation analysis. In the second phase, a structural learning method is adopted to extract the structural data from these 12 leads. Moreover, deep convolutional neural network (CNN) models are coded in this phase through genetic programming. These trees are responsible for learning deep structural features from functional data extracted from 12 leads. These trees are upgraded through the execution of the genetic programming (GP) algorithm to extract the optimal features. These two phases are used together to fuse the leads of ECG signals to diagnose various heart problems. According to the test results on ChapmanECG, including the signals of 10,646 patients, the proposed method enjoys the mean accuracy of 97.60% in the diagnosis of various types of arrhythmias in the Chapman dataset. It also outperformed the state-of-the-art methods. [ABSTRACT FROM AUTHOR]