Search

Deep learning has achieved remarkable success in a wide range of domains. However, it has not been comprehensively evaluated as a solution for the task of Chinese biomedical named entity recognition (Bio-NER). The traditional deep-learning approach for the Bio-NER task is usually based on the structure of recurrent neural networks (RNN) and only takes word embeddings into consideration, ignoring the value of character-level embeddings to encode the morphological and shape information. We propose an RNN-based approach, WCP-RNN, for the Chinese Bio-NER problem. Our method combines word embeddings and character embeddings to capture orthographic and lexicosemantic features. In addition, POS tags are involved as a priori word information to improve the final performance. The experimental results show our proposed approach outperforms the baseline method; the highest F-scores for subject and lesion detection tasks reach 90.36 and 90.48% with an increase of 3.10 and 2.60% compared with the baseline methods, respectively. [ABSTRACT FROM AUTHOR]

Leukemia is a severe blood disorder that poses a threat to the life and well-being of patients. Accurate diagnosis of leukemia typing is crucial for treatment and prognosis. However, manual diagnosis requires significant resources and is subject to observer variation. Computer-aided diagnostic (CAD) technologies have emerged as a promising approach to complement manual diagnosis and alleviate the workload of specialized physicians. With the recent advances in deep learning, the performance of CAD has been substantially enhanced. In this paper, we propose a novel classification model, AMLNet, for acute myeloid leukemia (AML) cells. AMLNet is a lightweight model based on the ShuffleNet V2 architecture. We first validate the performance of the convolutional neural network and Transformer architecture models on the largest AML cell dataset to date. Then, we enhance the data augmentation approach and model structure of the ShuffleNet V2 model to develop AMLNet. The AUC, sensitivity, and accuracy of AMLNet are up to 0.9935%, 63.91%, and 96.43%, respectively, which achieves the optimal classification performance. Finally, the classification performance of AMLNet is verified on other public datasets, which is superior to other methods. AMLNet presents a promising solution for practical diagnosis scenarios since the end devices of assisted diagnosis often have limited computational power. The proposed AMLNet model effectively improves the classification performance of ShuffleNet V2 while offering the advantages of high performance and low computational consumption. Overall, our study demonstrates the potential of lightweight deep learning models in improving the accuracy of leukemia typing and reducing the workload of specialized physicians. [ABSTRACT FROM AUTHOR]

Fraud detection and prevention has received a lot of attention from the research community due to its high impact on financial institutions' revenues and reputation. The increased use of the web and the provision of online services open up the pathway for exposing these systems to numerous threats and jeopardizing their effective functioning. Naturally, financial frauds are increased in number and form imposing various requirements for their efficient and immediate detection. These requirements are related to the performance of the adopted models as well as the timely response of the decision-making mechanism. Machine learning and data mining are two research domains that can provide a number of techniques/algorithms for fraud detection and setup the road for mitigation actions. However, these methods still need to be improved with respect to the detection of unknown fraud patterns and the incorporation of big data processing mechanisms. This paper presents our attempt to build a hybrid system, i.e., a sequential scheme for combining two deep learning models and efficiently detecting potential financial frauds. We elaborate on the combination of an autoencoder and a Long Short-Term Memory Recurrent Neural Network trained upon datasets which are processed through the use of an oversampling technique. Oversampling is adopted to handle heavily imbalanced datasets which is the 'natural' scenario due to the limited number of frauds compared to the humongous volumes of transactions. The proposed approach tends to capture much more fraud events in comparison with other conventional ML techniques. Our experimental evaluation exposes that our model exhibits a good performance in terms of recall and precision. [ABSTRACT FROM AUTHOR]

In the era of heterogeneous computing, a new paradigm called accelerator level parallelism (ALP) has emerged. In ALP, accelerators are used concurrently to provide unprecedented levels of performance and energy efficiency. To reach that there are many problems to be solved, one of the most challenging being co-execution. In this paper, we present a new scheduling framework called POAS, a general method for providing co-execution to applications. Our proposal consists of four steps: predict, optimize, adapt and schedule. With POAS, an unseen application can be executed concurrently in ALP with little effort. We evaluate POAS on a heterogeneous environment consisting of CPUs, GPUs (CUDA cores), and XPUs (Tensor cores) on two different fields, namely linear algebra (matrix multiplication benchmark) and deep learning (convolution benchmark). Our experiments prove that POAS provides excellent performance and completes the tasks within a time very close to the optimal time for the hardware and applications used, with a negligible execution time overhead. Moreover, the POAS predictor performed exceptionally well, achieving very low RMSE values for both use cases. Therefore, POAS can be a valuable tool for fully exploiting ALP and improving overall performance over offloading in heterogeneous settings. [ABSTRACT FROM AUTHOR]

The term "Internet of Things" (IoT) encompasses an entire group of gadgets that are capable of connecting to the Internet in order to gather and share data. The IoT paradigm is being pushed into computer networks by numerous highly advanced intrusions. Cloud computing greatly enhances the success of the IoT by enabling users to perform computing tasks using Internet-based services accessed through connected devices. This seamless integration of cloud technology and the IoT has become a powerful catalyst, revolutionizing the way we operate. The adoption of a distributed architecture, such as cloud computing, exposes the system to potential threats like Distributed Denial of Service (DDoS) and Denial of Service (DoS) attacks. To mitigate these risks, the concept of an intrusion detection system (IDS) has been introduced within the cloud environment. Various machine learning (ML) and deep learning (DL) algorithms have been proposed and implemented to effectively detect and respond to such malicious traffic in the cloud system. For dimension reduction during the training process of those algorithms, multiple independent and hybrid techniques have been proposed. This study presents an efficient ML-based real-time IDS framework with proposed hybrid feature selection techniques. Additionally, in this study, a concise comparative analysis has been conducted using five well-known public datasets. The findings presented in this paper reveal that our proposed IDS achieved a maximum accuracy of 99.98% in identifying malicious traffic. [ABSTRACT FROM AUTHOR]

The accurate and automated segmentation of neurons in electron microscopy (EM) images presents a challenging problem in neuroscience. In this paper, we propose a deep learning (DL)-based neuron segmentation approach using a relatively shallow residual deconvolutional neural network (DNN) as the core. Our proposed segmentation framework differs from most EM image segmentation networks that use original images as input, as we suggest using modified morphological features with attention characteristics as input instead. We calculate Bhattacharyya distances in morphology feature space to provide attention features, which are then concatenated with the original image and used as input to the segmentation network. By attending to membrane pixels, our network efficiently highlights the discrimination between membrane boundaries and background pixels. To remove noise pixels in the segmentation map while preserving membrane boundaries in the final result, the dilated version of the output is then processed using a guided filtering scenario. Our proposed model outperforms several competing DL methods such as DeepLabv3 as well as different versions of deconvolutional neural networks on the ISBI 2012 dataset in terms of various image segmentation metrics. The proposed attention-based morphological guided framework can help advance research in neuroscience by providing an automated and accurate segmentation of neurons in EM images. [ABSTRACT FROM AUTHOR]

The rapid growth of Internet of Things (IoT) devices has changed human interactions with the environment. IoT networks require specialized defense strategies distinct from traditional corporate contexts. Security measures such as anti-malware software, firewalls, authentication protocols, and encryption techniques are established but face limitations against evolving attack strategies. Therefore, this study proposes an intrusion detection approach for a realistic IoT environment, employing various variants of deep learning models such as deep neural networks (DNNs), convolutional neural networks (CNNs), and recurrent neural networks (RNNs). The research tested three variants for each model: DNN1, DNN2, DNN3, CNN1, CNN2, CNN3, RNN1, RNN2, RNN3. All these variants are customized and tuned differently to analyze the efficacy of the suggested methodology. Likewise, notable observation highlights the significance of aligning training and validation accuracy curves that indicate controlled overfitting, validating the model's reliability in accurately predicting intrusion and benign network traffic in IoT settings. Results reveal that RNN1 achieves the best results: accuracy of 98.61%, precision of 98.55%, recall of 98.61%, and F1-score of 98.57% compared with other DNN and CNN architectures and benchmark papers. This study advances intrusion detection within IoT networks through a comprehensive evaluation of deep learning models and inspires ongoing research to enhance intrusion detection systems' resilience in dynamic IoT environments. [ABSTRACT FROM AUTHOR]

General matrix multiplication (gemm) is a fundamental kernel in scientific computing and current frameworks for deep learning. Modern realisations of gemm are mostly written in C, on top of a small, highly tuned micro-kernel that is usually encoded in assembly. The high performance realisation of gemm in linear algebra libraries in general include a single micro-kernel per architecture, usually implemented by an expert. In this paper, we explore a couple of paths to automatically generate gemm micro-kernels, either using C++ templates with vector intrinsics or high-level Python scripts that directly produce assembly code. Both solutions can integrate high performance software techniques, such as loop unrolling and software pipelining, accommodate any data type, and easily generate micro-kernels of any requested dimension. The performance of this solution is tested on three ARM-based cores and compared with state-of-the-art libraries for these processors: BLIS, OpenBLAS and ArmPL. The experimental results show that the auto-generation approach is highly competitive, mainly due to the possibility of adapting the micro-kernel to the problem dimensions. [ABSTRACT FROM AUTHOR]

Lung cancer is an uncontrolled growth of tissue causing a lump in the human lung. If lung cancer can be detected early, it can increase the survival rate. Therefore, a multi-classification approach of lung nodule detection with high computational effectiveness is required. In this paper, a multi-classification approach of lung nodule detection and classification is proposed using artificial intelligence on computed tomography (CT) scan images. Different preprocessing steps are applied for resizing, smoothing, and enhancement of the CT images. Then, two different approaches for feature extraction using VGG16 transfer learning and morphological segmentation are proposed. Morphological segmentation and feature extraction are applied for the segmentation of the region of interest and to extract the distinct features. Finally, the proposed deep learning architecture and seven different machine learning algorithms are applied on the preprocessed data and the extracted features for the classification of lung nodules into three classes: malignant, benign, and normal. It is observed that the stacked ensemble model of deep learning convolutional neural network (CNN) and VGG16 transfer learning models (CNN+VGG16) can achieve 99.55% accuracy using preprocessed data. It is also observed that all the ML algorithms perform with reasonably high accuracy using the low-dimensional morphological features. It is observed from the fivefold cross-validation results that logistic regression performs with 99.36% accuracy in 23.71 s time using the preprocessed data. Whereas, using the morphological features, k-nearest neighbor, and the support vector machine perform with the highest accuracy of 99.76% with very reduced computational time of 0.017 and 0.008 s, respectively. [ABSTRACT FROM AUTHOR]

In aerial image detection, difficulties in feature extraction and low detection accuracy arise due to background interference, occlusion, and the presence of multiple small objects. This paper proposes a DM-YOLOX aerial object target detection method with intensive attention mechanism. Firstly, the proposed approach incorporates coordinate attention (CA) and a dense connection method into the backbone network architecture, enabling adaptive channel weighting throughout the feature extraction process. This facilitates the enhancement of significant features while suppressing less relevant ones, thereby augmenting the network's capacity to represent object features and ensuring retention and reinforcement of key features. Secondly, the multibranch extraction module (MBE) is incorporated into the feature fusion network to enhance the network's ability in extracting multi-scale feature information from images with extensive coverage, thereby enhancing the detection accuracy and efficiency of small- and medium-sized objects in complex scenes. Finally, the utilization of SIoU instead of IoU as the bounding box loss function effectively addresses the issue of mismatch between real and predicted boxes, leading to accelerated network convergence and improved performance during model training. After training and testing on the VisDrone 2019 dataset, this method effectively detects small objects in complex environments. The DM-YOLOX model shows a significant improvement of 2.7% in mAP compared to the baseline network, while achieving an 8% increase in frames per second (FPS). [ABSTRACT FROM AUTHOR]

Recurrent neural networks (RNNs) are widely utilized in neural network research to capture spatiotemporal features in video data. However, their effectiveness heavily relies on the spatial features upon which they trained. This paper introduces innovative ensembles of features for constructing frame-wise structures by employing impactful neural network models with innovative training pipelines. These features are designed to enhance the recognition of hand gesture videos using RNN by leveraging temporal information. Recognizing hand gestures from videos is a complex task that presents considerable challenges. One notable challenge is the overlap in gesture motion, where different gesture categories exhibit similar hand poses within a single video clip. To overcome this issue, we were motivated to develop extensive and diverse features that offer a more comprehensive description of the gesture video clips, thereby mitigating recognition problems caused by images overlapping. Overall, our efforts to generate diverse features have yielded promising results in enhancing the recognition of hand gestures from videos, particularly in scenarios where overlap poses a significant challenge. We have combined the extracted features from a deep neural network trained from scratch with features obtained from various standard neural networks (Self-Organizing Map, Radial Base Function) that are trained to enhance the deep-trained features. The mutual arrangement for combining the shared features has configured new frame-wise image features. Furthermore, we have provided a performance comparison of the newly constructed frame-wise features through time-sharing to train RNN for recognition. The proposed models have been evaluated on two-hand gesture video datasets, where a preserving gesture sequence is crucial due to overlapping motions. Our work demonstrates a significant improvement in performance for both datasets. [ABSTRACT FROM AUTHOR]

With the massive amount of data available on the Internet, many users face problems when they need access to the information and goods they need. Adapting information to user needs has become a complex and time-consuming process. Recommender systems are powerful tools for guiding users in electronic settings to information, services, and goods of interest. With the ability to identify users and predict their preferences, recommender systems can extract information likely to be of interest to users from vast amounts of data and save time and energy by providing them with relevant recommendations. Therefore, in this paper, a framework for a recommender system is proposed, which consists of four main phases. In the first phase, the preprocessing operation is performed. The purpose of this phase is to create an ontology model for products based on the convolutional neural network method. Data collection operations are carried out in the second phase. The purpose of this phase is to receive and store user information based on behavior and different criteria. The third phase is related to creating an ontology. The purpose of this phase is to create an ontology model for user behavior using OWL libraries. Finally, in the last phase, finding similarities and presented a proposal. The purpose of this phase is the necessary calculations of the proposed framework in the similarity finding section using ontology models. Simulation results show that the proposed framework measures MAE (more than 10 and 15%), RMSE (more than 12 and 16%), precision (less than 10 and 13%), recall (more than 25 and 28%), F1-score (higher than 25 and 30%) and Score of User (higher than 15 and 17%) outperform two related approaches, namely CF and CF+ Ontology, respectively. [ABSTRACT FROM AUTHOR]

One of the most important strategies used to mitigate the adverse impacts of traffic growth on mobile networks is caching. By caching at the edge, the backhaul traffic load is reduced, and the quality of service for the user is increased. Developing an effective caching algorithm requires accurate prediction of the future popularity of the content, which is a challenging issue. In recent years, deep learning models have achieved high predictive accuracy due to advancements in data availability and increased computing power. In this paper, we present a caching algorithm called the user preference-aware content caching algorithm (UPACA). This algorithm is specifically designed for an edge content delivery platform where users can access content services provided by a remote content provider. UPACA operates in two steps. In the first step, the proposed collaborative filtering-based popularity prediction algorithm (CFPA) is used to predict future content popularities. CFPA utilizes a gated residual variational autoencoder collaborative filtering model to predict users' future preferences and calculate the future popularity of content. This algorithm considers the popularity of the content as well as the number and timing of content requests. Experimental results demonstrate that UPACA outperforms previous methods in terms of cache hit rates and user utilities. [ABSTRACT FROM AUTHOR]

Monitoring battery health is critical for electric vehicle maintenance and safety. However, existing research has limited focus on predicting capacity degradation paths for entire battery packs, representing a gap between literature and application. This paper proposes a multi-horizon time series forecasting model (MMRNet, which consists of MOSUM, flash-MUSE attention, and RNN core modules) to predict the capacity degradation paths of battery packs. First, domain knowledge (DK) extracts the features from extensive battery aging datasets. The moving sum (MOSUM) and improved flash multi-scale attention (MUSE) methods are proposed to capture capacity curve mutations and multi-scale trends. Dynamic dropout training, transposition linear architecture, residual connections, and module stacking improve model generalization and accuracy. Experiments on battery pack and cell datasets demonstrate the superior performance of MMRNet over six baseline time series models. The proposed data-driven approach effectively predicts battery degradation trajectories, with implications for condition monitoring and the safety of electric vehicles. [ABSTRACT FROM AUTHOR]

Transportation mode recognition is an important and challenging problem in intelligent transportation systems. For decades, many data mining and deep learning methods have been proposed, but all these methods have some deficiencies and do not fully use temporal and spatial information hidden in raw GPS trajectory data. In this paper, we presented a new deep learning model called TSANET, which designs an attention mechanism to combine global and local spatiotemporal features. The input of the model is seven kinematic features, derived from the GPS trajectories and covers all information hidden in the GPS points. The model adopts TCN and ST-Block to extract global and local spatiotemporal features, respectively. In ST-Block, BiGRU is used to capture temporal features and DenseNet is applied to capture spatial features. In TCN, convolution operations are applied to obtain temporal and spatial information. Moreover, a dual-layer attention mechanism is developed to integrate and reconstruct these features by assigning different weights to global and local spatiotemporal features caught by TCN and ST-Block. By comprehensively considering global and local features, the model can greatly improve classification performance and recognition granularity, and with the further introduction of the attention mechanism, it can identify all transportation modes in a fine-grained manner. At last, the experiments have been conducted on the Geo1 and Geo2 datasets used by many researchers. The experimental results show that our model cannot only achieve the highest accuracy of 94.34 % among all recent methods but also identify all seven transportation modes clearly, thus verifying the advantages and effectiveness of the model. [ABSTRACT FROM AUTHOR]

The increasing number of network attacks has led to the development of intrusion detection systems. However, these methods often face limitations such as high traffic flow data dimensions, which can reduce attack detection rates and noise sensitivity, affecting anomaly detection performance. This paper introduces a new model based on recurrent deep learning and instance-level horizontal reduction to detect anomalies and network attacks. The model uses nested sliding windows, which move with a specific step in the data and generate a different number of histogram outputs based on the type of anomaly in the data. Evaluation results on five databases show that the proposed model achieves a high accuracy of 99% in detecting different attacks, demonstrating the success of this new approach combined with deep recurrent neural networks in detecting anomalies. [ABSTRACT FROM AUTHOR]

Broad Learning System (BLS) is proposed as an alternative to deep learning. It has a fast adaptive model selection and online incremental learning capability, which has been successfully applied in many fields. In this paper, the BLS model is introduced into intrusion detection, and considering the weakness of the BLS model in mining the internal structural information of samples, this paper proposes a Manifold Regularized Broad Learning System based on LU decomposition (LU-MRBLS) intrusion detection. Based on the manifold hypothesis, the LU-MRBLS model firstly constructs the graph Laplacian operator in the data input space to mine the potential information of the data. Then, under the manifold regularized framework, the feature nodes, enhancement nodes, and Laplacian matrix are combined to construct the objective function to regularize and optimize the BLS model to avoid the model falling into local optimization. Finally, the LU decomposition method is used to solve the output weight matrix of the MRBLS model, shorten the training time of the MRBLS model, avoid singular value problems of the solution process, and improve the intrusion detection performance of the model. In this paper, we use the KDD Cup99 dataset for parameter selection and apply it to other network models. Through rigorous experiments, the LU-MRBLS model is applied to KDD Cup99, NSL-KDD, UNSW-NB15, and CIDDS-001 datasets with better detection results than the classical machine learning models and the latest intrusion detection models. [ABSTRACT FROM AUTHOR]

The accurate identification of bird image is of great significance to protecting the ecological environment and bird species diversity. To address the issue of low recognition accuracy arising from the similarities in features among different bird species and the susceptibility of shallow edge features to loss, this paper proposes a fine-grained bird image classification model that incorporates hierarchical feature fusion and counterfactual feature selection. The model is based on vision transformer and builds a hierarchical feature fusion module and a counterfactual feature enhancement module. The hierarchical feature fusion module superimposes shallow features rich in fine-grained information into deep features to improve the problem of lack of edge detail information in key features. The counterfactual feature enhancement module selects distinguishing features through counterfactual intervention to reduce classification errors caused by highly similar features of different species of birds. The experimental results show that the method can achieve 91.9 % and 91.4 % accuracy on two available datasets, CUB-200-2011 and NABirds, respectively, which is higher than the current mainstream fine-grained bird recognition algorithms and shows excellent classification performance. [ABSTRACT FROM AUTHOR]

Microelectronics production failure analysis is an important step in improving product quality and development. In fact, the understanding of the failure mechanisms and therefore the implementation of corrective actions on the cause of the failure depend on the results of this analysis. These analyses are saved under textual features format. Then such data need first to be preprocessed and vectorized (converted to numeric). Second, to overcome the curse of dimensionality caused by the vectorisation process, a dimension reduction is applied. A two-stage variable selection and feature extraction is used to reduce the high dimensionality of a feature space. We are first interested in studying the potential of using an unsupervised variable selection technique, the genetic algorithm, to identify the variables that best demonstrate discrimination in the separation and compactness of groups of textual data. The genetic algorithm uses a combination of the K-means or Gaussian Mixture Model clustering and validity indices as a fitness function for optimization. Such a function improves both compactness and class separation. The second work looks into the feasibility of applying a feature extraction technique. The adopted methodology is a Deep learning algorithm based on variational autoencoder (VAE) for latent space disentanglement and Gaussian Mixture Model for clustering of the latent space for cluster identification. The last objective of this paper is to propose a new methodology based on the combination between variational autoencoder (VAE) for the latent space disentanglement, and genetic algorithm (GA) to find, in an unsupervised way, the latent variables allowing the best discrimination of clusters of failure analysis data. This methodology is called VAE-GA. Experiments on textual datasets of failure analysis demonstrate the effectiveness of the VAE-GA proposed method which allows better discrimination of textual classes compared to the use of GA or VAE separately or the combination of PCA with GA (PCA-GA) or a simple Auto-encoders with GA (AE-GA). [ABSTRACT FROM AUTHOR]

In recent years, the combination of deep learning and side-channel analysis has received extensive attention. Previous research has shown that the key recovery problem can be transformed into a classification problem. The performance of these models strongly depends on the size of the dataset and the number of instances in each target class. The training time is very long. In this paper, the key recovery problem is transformed into a similarity measurement problem in Siamese neural networks. We use simulated power traces and true power traces to form power pairs to augment data and simplify key recovery steps. The trace pairs are selected based on labels and added to the training to improve model performance. The model adopts a Siamese, CNN-based architecture, and it can evaluate the similarity between the inputs. The correct key is revealed by the similarity of different trace pairs. In experiments, three datasets are used to evaluate our method. The results show that the proposed method can be successfully trained with 1000 power traces and has excellent attack efficiency and training speed. [ABSTRACT FROM AUTHOR]

Business process compliance is an essential part of business process management, which saves organizations from penalties caused by non-compliant processes. However, current researches on business process compliance mainly focus on checking using general constraint rules that have been formalized without in-depth analysis of related regulatory documents and mostly involve extensive human efforts. In this paper, we aim to propose an automatic and interpretable compliance checking approach for design-time business processes. By combining deep learning and a natural language processing approach based on rule templates, we extract semantic information from regulatory documents at different granularities for subsequent compliance checking. In addition, we match appropriate rules to the process model and detect the degree of violation of the business process from three control-flow perspectives. The effectiveness of this method is validated on two real-world datasets. [ABSTRACT FROM AUTHOR]

Effective means for transportation form a critical city infrastructure, particularly for resource-constrained smart cities. Rapid advancements in information and communication technologies have paved the path for intelligent transportation system (ITS), specifically designed for optimal effectiveness and safety with existing transportation infrastructure. A key function of ITS is its ability to aggregate large volumes of data across various sources for event detection. However, prediction accuracy remains a challenge since ITS event detection is characterized by very stringent latency requirements necessitating the use of lightweight detection schemes, thus seriously compromising the efficiency of ITS. This paper attempts to tackle this problem by introducing an IoT-integrated distributed context-aware fog-cloud ensemble that intelligently manages context instances at fog nodes ensuring availability of context instances for ITS. This system enhances prediction accuracy by utilizing a hybrid convolutional neural network (CNN) where each vehicle within the system retains only local information, while adjacent fog nodes gain access to global events via continual federated learning, updating regularly between fog and cloud models. Experiments presented herein illustrate the superiority of the CNN model, yielding an accuracy of more than 95%, which is an improvement of around 3% compared to the LeNet with same RGB input images. [ABSTRACT FROM AUTHOR]

Agriculture is one of the most crucial aspects of a nation's growth. However, the quality and quantity of crop yield are severely affected by various plant diseases. Plant diseases must be identified and prevented at an early stage to improve food quality and production rate. The emerging deep learning network of convolutional neural networks (CNNs) achieved excellent results in plant disease classification. However, the classification potential of the network depends largely on the configuration of hyperparameters. Finding the optimal set of hyperparameters is a tedious, time-consuming, and challenging task. To tackle such an issue, this paper proposes an optimized CNN integrated with a novel modified whale optimization algorithm (MWOA) to achieve plant disease classification. Here, the novel method of optimizing CNN hyperparameters facilitates quicker implementation. To boost the proposed model's efficiency, several data augmentation methods are used such as rotation, scaling, and flipping. The proposed MWOA-CNN model is implemented using a plant village database that includes fourteen plant species, 38 disease classes, and healthy leaves as well. The experiential findings showed that the proposed model outperforms the existing models by attaining the highest classification accuracy of 99.92%, resulting in an effective model for plant disease classification. [ABSTRACT FROM AUTHOR]

The ability to identify human actions in uncontrolled surroundings is important for Human–Computer Interaction (HCI), especially in the sports areas to offer athletes, coaches, and analysts valuable knowledge about movement techniques and aid referees in making well-informed decisions regarding sports movements. Noteworthy, recognizing human actions in the context of basketball sports remains a difficult task due to issues like intricate backgrounds, obstructed actions, and inconsistent lighting conditions. Accordingly, a method based on the combination of YOLO and deep fuzzy LSTM network is proposed in this paper. YOLO is utilized for detecting players in the frame and the combination of LSTM and Fuzzy layer is used to perform the final classification. The reason behind using LSTM along with fuzzy logic refers to its inability in coping with uncertainty which led to the creation of a more transparent, interpretable, and accurate predictive system. The proposed model was validated on SpaceJam and Basketball-51 datasets. Based on the empirical results, the proposed model outperformed all baseline models on both datasets which obviously confirms the priority of our combinational model for basketball action recognition. [ABSTRACT FROM AUTHOR]

Semantic relatedness is one of the most significant aspects of natural language processing. It has been identified as a critical technology for developing intelligent systems like Siri, Microsoft Ice, Cortana, and Xiaoai. In 2014, SemEval ranked SR as the top task. While many existing studies have focused on analyzing the entailment of single phrases, advancements in deep learning have made it possible to analyze complete sentences or texts. While the natural parallelism of membrane computing has shown promise for data processing, harnessing this potential to advance semantic relatedness remains an open problem yet to be tackled. This paper proposes a novel Siamese Manhattan LSTM-SNP approach (SiMaLSTM-SNP) for the SR problem. The approach uses a collaborative Word2vec and 10-Layer Attention strategy to represent and extract sentence pairs and a Siamese LSTM-SNP structure to calculate the hidden states of sentences. The multi-head self-attention layer identifies text associations and redistributes hidden state weights. The last hidden state is extracted, and the relatedness score is calculated using the Manhattan distance. The experiments demonstrate that SiMaLSTM-SNP outperforms 17 classical SR baselines and 7 novel approaches on the standard datasets SICK and STS in terms of mean square error performance. This indicates that SiMaLSTM-SNP can accurately capture the semantic distinction between two sentences and effectively preserve their semantic information. [ABSTRACT FROM AUTHOR]

Routability estimation identifies potentially congested areas in advance to achieve high-quality routing solutions. To improve the routing quality, this paper presents a deep learning-based congestion estimation algorithm, which serves to guide the initial pattern routing of a global router to reduce unexpected overflows. Unlike existing methods based on traditional compressed 2D features for model training and prediction, our algorithm extracts appropriate 3D features from the placed netlists. Furthermore, an improved RUDY (Rectangular Uniform wire DensitY) method is developed to estimate 3D routing demands. For the model selection, we employ the U-net model with good image prediction ability as a routability estimator, and the prediction results are used for guiding the initial routing process. Since our training set is formed by cropping training designs, the structural information between two adjacent cropped designs may be missed. Hence, we divide experiments into two parts, experiments based on normal designs and experiments based on big designs. Compared with the state-of-the-art method, experimental results show that our routability estimator can significantly improve the Pearson Correlation Coefficient (PCC) index and reduce the Mean Absolute Normalized Error (MANE) and the Standard Deviation in the Normalized Error (SDNE). Furthermore, our congestion-guided global routing can reduce the routing overflows, wirelength, and via count both on normal designs and big designs, compared to CUGR. [ABSTRACT FROM AUTHOR]

Mission-critical applications refer to the real-time applications, which require fast and secure service provisioning, such as defense sector and disaster management. This paper proposes a delay-aware and secure service provisioning model for such types of applications. As a use-case, we have considered the defense sector, which is a vital sector for a country's all-round well-being including security, safety, society, and economy. In the conventional sensor-cloud model, the sensor data is stored and processed in the cloud. However, the sensor nodes have small coverage and the use of the long distant cloud servers increases the delay. Therefore, the conventional sensor-cloud model may not be efficient for defense application. Moreover, data hiding for security purposes is another important aspect of this field. To address these challenges, this paper proposes a mobility-aware sensor-fog paradigm for mission-critical applications based on network coding and steganography, referred to as Mobi-Sense. In Mobi-Sense, steganography is used for hiding the data during transmission. The theoretical results demonstrate that Mobi-Sense outperforms the existing frameworks with respect to delay and power consumption by ∼ (40 - 80) %. The simulation results present that Mobi-Sense reduces the delay by ∼ (18 - 40) % than the conventional sensor-cloud framework for mission-critical applications. An optimal path finding algorithm based on deep learning has been deployed in the context of disaster scenario. The experimental analysis shows that the proposed optimal path finding method achieves precision and accuracy above 90%. This is observed that our proposed modules have outperformed existing baselines in terms of accuracy, delay, and power consumption. [ABSTRACT FROM AUTHOR]

With the advancements in mobile technology, a large amount of data is generated by end devices, which has created a renewed interest in developing new AI-based applications for gaining insights into these data. However, most of these distributed applications need data aggregated at a central server, which has posed severe bandwidth, latency, security, and privacy issues. This paper presents OSGAN (One-Shot distributed learning algorithm using Generative Adversarial Networks), a generic framework that trains a generative adversarial network (GAN) at each client and uses GAN's generative capabilities to create sample data at the server. The server aggregates these data from various clients, builds a deep learning model and sends its parameters back to the clients in one communication round, i.e. the exchange of information between the clients and the server happens only once. In this paper, we present the design and implementation of OSGAN and evaluate its performance by comparing it with the state-of-the-art federated learning (FL) and central training algorithms for both IID and non-IID distribution of data. Our experiments on multiple datasets show that our proposed approach achieves a similar accuracy when compared with both FL and central training algorithms. Specifically, the accuracy drop with OSGAN is maintained within 2% for multiple datasets and multiple numbers of clients. Our results show that the proposed approach reduces the amount of data transfer by almost 98% when compared with federated learning and close to 80% when compared with the central learning approach thereby providing substantial benefit in terms of saving the bandwidth. [ABSTRACT FROM AUTHOR]

Electronic health records provide a vast amount of text health data written by physicians as patient clinical notes. The world health organization released the international classification of diseases version 10 (ICD-10) system to monitor and analyze clinical notes. ICD-10 is system physicians and other healthcare providers use to classify and code all diagnoses and symptom records in conjunction with hospital care. Therefore, the data can be easily stored, retrieved, and analyzed for decision-making. In order to address the problem, this paper introduces a system to classify the clinical notes to ICD-10 codes. This paper examines 7541 clinical notes collected from a health institute in Jordan and annotated by ICD-10's coders. In addition, the research uses another outsource dataset to augment the actual dataset. The research presented many approaches, such as the baseline and pipeline models. The Baseline model employed several methods like Word2vec embedding for representing the text. The model structure also involves long-short-term memory a convolutional neural network, and two fully-connected layers. The second Pipeline approach adopts the transformer model, such as Bidirectional Encoder Representations from Transformers (BERT), which is pre-trained on a similar health domain. The Pipeline model builds on two BERT models. The first model classifies the category codes representing the first three characters of ICD-10. The second BERT model uses the outputs from the general BERT model (first model) as input for the special BERT (second model) to classify the clinical notes into total codes of ICD-10. Moreover, Baseline and Pipeline models applied the Focal loss function to eliminate the imbalanced classes. However, The Pipeline model demonstrates a significant performance by evaluating it over the F1 score, recall, precision, and accuracy metric, which are 92.5%, 84.9%, 91.8%, and 84.97%, respectively. [ABSTRACT FROM AUTHOR]

In industry, robots are widely used to solve repetitive or dangerous actions in product production, so that product production can be more efficient. However, the problem that robots are often challenged is the convenience and the efficiency of introducing the production line. Therefore, the intuitive robot guidance method is an important issue; this paper will introduce the concept of human–robot interactions (HRI) and use deep learning methods on the machine vision system to complete the robot-guided assembly operation analysis, and the assembly operation analysis requires semantic segmentation as pre-processing. Therefore, we propose a novel semantic template correlation model architecture based on zero-shot learning (ZSL) to achieve the effect of rapid deployment. The semantic template correlation model is to search for the object area offline learning through the semantic template generated by the physics engine, and when inferring online, we can directly enter the semantic template to obtain the relevant object region. Finally, this paper verifies that the MIoU can be increased by more than 5% through the verification of the general database VOC2012. [ABSTRACT FROM AUTHOR]

Remaining useful life (RUL) prediction is critical for industrial equipment status detection, and the accurate prediction results provide decision-makers with actionable information. Preventive maintenance can be carried out to prevent the sudden failure of the equipment effectively based on the predicted results. However, as Industry 4.0 technologies develop, the amount of data collected by sensors is also rapidly increasing. The existing RUL prediction methods are gradually unable to cope with complex industrial equipment data, and deep learning methods gradually come to the fore. In this background, this paper proposes a transformer-based model with a multi-layer encoder–decoder structure to extract domain-invariant features. The decoder in the traditional transformer structure only obtains a single piece of information from the last layer of the encoder, and this paper uses an integrated layer-cross decoding strategy to compensate. Based on the encoder–decoder cross-connection, each decoder layer is provided with global view information from the final encoder layer simultaneously, improving the model's performance. The validity and superiorities of the proposed method are evaluated through several experiments on the publicly available C-MAPSS dataset provided by NASA. As can be seen from the results, the proposed method gets higher prediction accuracy than other network architectures and state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

Traditional deep learning models prefer large data sets, and in reality small data sets are easier to obtain. It is more practical to build models suitable for small data sets. Based on CNN, this paper proposes the concept of union convolution to build a deep learning model Union-net that is suitable for small data sets. The Union-net has small model size and superior performance. In this paper, the model is tested based on multiple commonly used data sets. The experimental results show that Union-net outperforms most models when dealing with small datasets, and Union-net outperforms other models when dealing with complex classification tasks or dealing with few-shot datasets. The codes for this paper have been uploaded to https://github.com/yeaso/union-net. [ABSTRACT FROM AUTHOR]

Facial expression recognition (FER) is a technology that recognizes human emotions based on biometric markers. Over the past decade, FER has been a popular research area, particularly in the computer vision community. With deep learning (DL) development, FER can achieve impressive recognition accuracy. In addition, favorable Internet-of-Things (IoT) advancements generate massive amounts of visual data needed to enable reliable DL-based emotion analysis. However, training DL models can suffer from significant memory consumption and computational costs, complicating many vision tasks. Additionally, the direct use of RGB images during the training and inference stages might raise privacy concerns in various FER applications. On the other hand, adopting large deep networks hampers quick and accurate recognition on resource-constrained end devices such as smartphones. As a viable solution, edge computing can be employed to bring data storage and computation closer to end devices rather than relying on a central cloud server. As a result, it can potentially facilitate the deployment of real-time FER applications since the latency and efficiency problems are well addressed by utilizing the computing resources at the edge. In this paper, we develop an efficient FER framework that integrates DL with edge computing. Our framework relies on facial landmarks to enable privacy-preserving and low-latency FER. Accordingly, various landmark detection models and feature types are studied empirically to investigate their capabilities in capturing the dynamic information of facial expressions in videos. Then, using the extracted landmark-based features, we design lightweight DL models to classify human emotions on IoT devices. Extensive experiments performed on benchmark datasets further validate the efficiency and robustness of our framework. [ABSTRACT FROM AUTHOR]

Fast and accurate detection of traffic signs is crucial for the development of intelligent transportation systems. To address the issue of false detection and missing detection of small traffic signs in complex scenes, this paper proposes a YOLO-SG model based on YOLOv5. The YOLO-SG approach employs SPD-Conv as a down-sampling structure to mitigate the loss of feature information during the down-sampling process. This enhances the detection performance of small objects in complex scenes and improves the generalization and robustness of the model. The feature extraction architecture uses GhostNet, which effectively reduces the number of model parameters and weight, enhancing the feasibility of practical model deployment. Furthermore, this study optimizes the output feature structure by introducing a small object detection layer and removing the large object detection layer, enabling the detection of small objects. Extensive experiments conducted on the GTSDB and TT100K datasets demonstrate that YOLO-SG exhibits excellent detection performance. On the GTSDB dataset, YOLO-SG achieved a 2.3% increase in mAP compared to the baseline network, while reducing the number of parameters by 42%. Similarly, on the TT100K dataset, YOLO-SG increased mAP by 6.3% and reduced the number of parameters by 43%. These experimental results showcase the effectiveness and accuracy of YOLO-SG, particularly in detecting small traffic signs. [ABSTRACT FROM AUTHOR]

This paper proposed a fusion model of the deep long- and short-term memory network named as deep LSTM and the stochastic dominance named as SD filter method to construct an index-tracking portfolio. We present a practical model that provides investors for portfolio construction using the deep LSTM model on extracting stock features and integrating the dimension deduction ability of the SD approach to the fusion model. Three main conclusions are drawn. First, deep learning in a supervised framework can work in portfolio management with our tuned model. Second, our deep LSTM model with an SD selection filter has enhanced the feature extraction ability to construct an index-tracking portfolio. Third, the linear activator, rectified linear unit, cooperated with SD methods can better reduce the estimation errors than the nonlinear activator in our deep LSTM model. These findings can implement on the portfolio construction in the neural network field. Additionally, entropy suggests to evaluate the learning effect of forecasting. The SD methods also are indicated for prosecuting in other neural network models to extract features of time series data, like transformer-based models. [ABSTRACT FROM AUTHOR]

Cancer is a devastating and deadly disease, and liver cancer is one of the leading causes of cancer deaths. Early detection of liver tumor is important to choose a treatment plan, get an accurate prognosis, and gain a deep understanding of the tumor to determine its severity. Despite a lot of research, performing automatic segmentation and classification liver tumor is still a challenging task due to the low tissue contrast between the surrounding organs and the deformable shape of the CT image. Therefore, this paper introduces the unified learning multi-task model network for combined automatic liver tumor segmentation and classification. The first step is to build a multi-task deformable attention U-Net (MDAUnet) technique to segment the liver tumor and capture the features for classification. Here, an attention-based deformable module is used instead of convolution to learn the irregular and inconspicuous appearance of tumors by combining context attention and deformable convolution. Further, a residual skip connection is used to avoid duplicate transmission of low-resolution data by introducing a residual path. In the second step, the segmented liver tumor features from MDAUnet are fed into the deep DenseNet (DDNet) model and concatenation layer. Based on the segmented liver tumor features, DDNet learns distinguishable features for classification. The concatenation layer combines the learned features of the MDAUnet and DDNet models for liver tumor classification. Finally, a fully connected layer classifies primary and secondary liver tumors. Therefore, our proposed ULM-net model outperforms single models in terms of precision, F-1 measure, recall, classification accuracy, and kappa coefficient. [ABSTRACT FROM AUTHOR]

The management of diabetes involves much compliance, disease awareness, and patient empowerment. Blood glucose levels in people with diabetes are abnormal, resulting in various health problems that affect their kidneys, hearts, and vision. Monitoring the patient's health has the benefit of using the Internet of Things (IoT) and diabetes patient monitoring systems. Because this system uses machine learning to classify data, its value goes beyond patient monitoring. So, this paper proposes a novel deep learning model with an effective feature selection mechanism for diabetes mellitus (DM) prediction in IoT-based healthcare environment. The proposed work starts by collecting the real-time IoT data of the patients from the Pima Indian Diabetes database. The data imbalance problem of the collected dataset is rectified by applying the synthetic minority oversampling technique. Afterward, the balanced dataset is preprocessed by applying missing value imputation and data standardization to improve the classification performance. Then, the hybrid algorithm called k-means clustering-based sailfish optimization is utilized, which performs clustering and optimization to select the essential features from the preprocessed dataset. Finally, the selected features are fed into the kaiming and switan included bidirectional long short-term memory for DM prediction. The proposed system achieves better results than the existing state-of-the-art techniques, which shows its effectiveness in predicting DM in real-time IoT datasets. [ABSTRACT FROM AUTHOR]

Sensor nodes, typically small and low-power devices, are components of wireless sensor networks (WSNs). Each node monitors its surroundings for relevant environmental changes and sends all detected events to the data collector for analysis. If the sensor nodes are not placed correctly, there may be areas that are not within the detection zone of any sensor node. Coverage holes in WSNs are usually caused by random deployment and node failure. Energy holes and dead nodes are the main problems caused by detection and recovery of coverage holes in WSNs. The size of coverage holes increases the time complexity and power of recent protocols. However, there is a high computational complexity associated with distributed methods proposed in recent years to solve the coverage hole detection problem. In this paper, we propose optimal cluster-based node position estimation and coverage hole detection in WSNs using a hybrid deep learning approach. First, a modified Lyapunov optimization (MLO) algorithm to compute the node position is presented, which ensures edge nodes in the network. Next, we design optimal clustering technique by using improved sand cat swarm optimization (ISCSO) algorithm to formulate efficient balanced clusters which computes coverage hole area in the network. Afterward, we develop a hybrid deep reinforcement learning (Hyb-DRL) technique for hole shape detection and hole size judgment within clusters, among clusters and along edges. The results show that the proposed approach achieves significant improvements compared to existing benchmark approaches. Specifically, the average energy consumption of CG-DCHD approach is 43.835%, 32.674% and 26.164% lower for node density, hole density and simulation rounds, respectively. The hole detection time is 18.4%, 16.802% and 15.462% lower, while the coverage is 16.885%, 14.977% and 12.219% higher for node density, hole density and simulation rounds, respectively. Additionally, the network lifetime of CG-DCHD approach is 15.58%, 17.702% and 20.492% higher, while the control packet overhead is 0.83%, 1.907% and 1.466% lower for node density, hole density and simulation rounds, respectively. [ABSTRACT FROM AUTHOR]

Pedestrian trajectory prediction has many real-world applications, such as crowd video surveillance and self-driving cars. However, this is a challenging problem due to the complexity of modeling social interactions between road agents (pedestrians and vehicles). Previous studies that addressed trajectory prediction applied traditional deep learning approaches, including CNN and RNN. Meanwhile, the applications of graph neural networks have drawn increasing interest recently, and significant progress has been made in extracting features from complex and unstructured data. In this paper, we propose STM-GCN, a spatiotemporal multi-graph convolutional network for pedestrian trajectory prediction. Our approach consists of collecting information about the social interactions between pedestrians in a crowd that are position-based and velocity-based interactions integrated into a multi-graph. Then we apply Graph Neural Network learning on the obtained multi-graph for predictions. Through experiments on the ETH and UCY benchmarking datasets, our proposed method outperforms the state of the art by 6% average displacement error (ADE) and 10% final displacement error (FDE). Our implementation is available at https://github.com/YoussefTaki/STMGCN. [ABSTRACT FROM AUTHOR]

The powerful parallel computing capability of GPU and the development of matrix processing unit in recent years provide more possibilities to improve the performance of convolutional neural network (CNN) on GPU. For the Winograd convolution algorithm, which is the most widely used in CNN and has the best performance, there are already some tuning results, but they all ignore the utilization of the matrix operation unit and fail to make full use of the computing resources of GPU. This paper introduces a single precision accelerated solution on GPU for CNN. According to the indicators of architecture, the optimal data layout, grid division and block division methods are derived. In order to adapt to a variety of padding in practical application, an efficient dynamic scheme for filling is designed, and by the use of matrix cores, a pipeline algorithm with operator fusion is implemented. The deep learning accelerated library MIOpen in AMD is used as the baseline. Taking several convolutional layers of ResNet50 as the experimental input, the evaluation shows that our approach outperforms MIOpen with the speedup of 1.41x on MI210, and reaches 74% of the peak value of single precision calculations. Applying this method to the training and inference of ResNet50, the speedup of 1.68x is obtained. [ABSTRACT FROM AUTHOR]

In recent years, surveillance systems have significantly increased in various environments. The Internet of Things (IoT), which interconnects objects through the Internet, enhances the capabilities of these systems. Typically, in IoT-enabled surveillance systems, the recorded videos should be processed and stored in cloud servers, which increases the system response time. Sending the videos to a cloud data center through the infrastructure network may also cause performance issues such as delay, jitter, and packet loss. Usually, edge computing is a viable solution to overcome these problems. This paper proposes a deep learning-based network traffic management policy for IoT-enabled surveillance systems in which the user's quality of experience (QoE) is a performance metric. For this purpose, first, a host in the edge data center preprocesses the video to reduce the video file size. Next, we designed a QoE-aware routing algorithm that uses a minimum delay-maximum mean opinion score heuristic algorithm to create a dataset that contains optimal routes with high QoE values. Then, it trains a sequential deep recurrent neural network (deep-RNN) on this dataset, and the trained model predicts the optimal routes between a source edge data center and a destination cloud center. Because software-defined networks (SDN) facilitate the execution of intelligent algorithms in networks, we implemented the proposed policy in the SDN controller of the infrastructure network. The simulation results showed that the proposed method could reduce the end-to-end packet loss ratio and increase the QoE values in IoT-enabled surveillance systems. Furthermore, converting the video codec on the edge layer reduces the video size, reducing the network latency by 50%. The experimental results also showed 94% accuracy using the proposed sequential deep RNN model. [ABSTRACT FROM AUTHOR]

Several machine learning-based methods are available in the literature to find anomalies in large log sequences. Recently, deep learning based solutions demonstrated promising performance in this domain. The majority of these methods formulate the problem as a classification task, where the final layer of the neural network model is a softmax layer. This paper presents a different, regression-based approach. The input of the model is a sequence of numeric vectors representing the semantic information of the log lines, and the output is also a numeric vector corresponding to the expected log line. To cope with the inherent uncertainty of log sequences, we introduce "Top-K" layers, allowing the model to emit multiple predictions, from which the best one is chosen. This feature, together with the "Top-K" loss function, makes it possible to develop an unsupervised, fully template-less model that can also handle log lines it has never seen during the training phase, which is essential for online applications. On benchmark data sets we demonstrate that the model achieves competitive performance. [ABSTRACT FROM AUTHOR]

Cellular networks have been widely deployed and are under ever-growing pressure from increasing energy consumption. Identifying the spatio-temporal properties of network traffic is crucial to effectively manage large-scale cellular networks and to optimize energy control strategies. However, traditional methods need to construct complex mathematical models to describe the properties of traffic, making the process inefficient and not automatic. Recent methods based on deep learning techniques are unexplainable and untraceable. In this paper, we propose a novel modeling and analysis approach by applying the spatio-temporal model checking technique to the identification of network traffic properties. First, we model the spatial structure of the cellular network by a closure space and the temporal structure of network traffic by the Kripke structure. Second, we provide logical characterizations of the spatio-temporal properties of network traffic by suitable spatio-temporal logic of closure space (STLCS) formulas. Third, we use model checking algorithms to detect the spatio-temporal properties of network traffic and to visualize the results. The experiments are illustrated with the Milan network traffic dataset and indicate that our approach can automatically and effectively detect desirable spatio-temporal properties of cellular network traffic. [ABSTRACT FROM AUTHOR]

Metaverse, which is anticipated to be the future of the internet, is a 3D virtual world in which users interact via highly customizable computer avatars. It is considerably promising for several industries, including gaming, education, and business. However, it still has drawbacks, particularly in the privacy and identity threads. When a person joins the metaverse via a virtual reality (VR) human-robot equipment, their avatar, digital assets, and private information may be compromised by cybercriminals. This paper introduces a specific Finger Vein Recognition approach for the virtual reality (VR) human-robot equipment of the metaverse of the Metaverse to prevent others from misappropriating it. Finger vein is a is a biometric feature hidden beneath our skin. It is considerably more secure in person verification than other hand-based biometric characteristics such as finger print and palm print since it is difficult to imitate. Most conventional finger vein recognition systems that use hand-crafted features are ineffective, especially for images with low quality, low contrast, scale variation, translation, and rotation. Deep learning methods have been demonstrated to be more successful than traditional methods in computer vision. This paper develops a finger vein recognition system based on a convolution neural network and anti-aliasing technique. We employ/ utilize a contrast image enhancement algorithm in the preprocessing step to improve performance of the system. The proposed approach is evaluated on three publicly available finger vein datasets. Experimental results show that our proposed method outperforms the current state-of-the-art methods, improvement of 97.66% accuracy on FVUSM dataset, 99.94% accuracy on SDUMLA dataset, and 88.19% accuracy on THUFV2 dataset. [ABSTRACT FROM AUTHOR]

Adaptive algorithms are widely used in deep learning because of their fast convergence. Among them, Adam is the most widely used algorithm. However, studies have shown that Adam's generalization ability is weak. AdaX is a variant of Adam, which introduces a novel second-order momentum, modifies the second-order moment of Adam, and has good generalization ability. However, these algorithms may fail to converge due to instability and extreme learning rates during training. In this paper, we propose a new adaptive and momental bound algorithm, called AdaXod, which characterizes of exponentially averaging the learning rate and is particularly useful for training deep neural networks. By setting an adaptively limited learning rate in the AdaX algorithm, the resultant AdaXod can effectively eliminate the problem of excessive learning rate in the later stage of neural networks training and thus results in stable training. We conduct extensive experiments on different datasets and verify the advantages of the AdaXod algorithm by comparing with other advanced adaptive optimization algorithms. AdaXod eliminates large learning rates during neural networks training and outperforms other optimizers, especially for some neural networks with complex structures, such as DenseNet. [ABSTRACT FROM AUTHOR]

The existing deep learning compilers are unable to perform efficient hardware performance-related graph fusion when both time and power consumption are considered. In addition, the compilers optimize the computational graph of deep neural networks (DNNs) by performing static graph transformation based on the greedy algorithm, only considering the runtime performance, and ignoring the cost of the tuning process. To solve these problems, this paper proposes a DNN computational graph optimization compiler (PCGC). Through the performance feedback at runtime, PCGC designs a computational graph fusion and splitting optimization strategy based on multilevel operator layer fusion-splitting rules. First, PCGC uses a rule-guided graph segmentation algorithm to recursively segment the computational graph into smaller subgraph to achieve an efficient and detailed search. Then, PCGC uses the cost model to receive feedback from hardware performance information, proposes cost model and operator fusion rules to synthetically guide the partial fusion and partitioning of nodes and edges of the computational graph, and generates optimal subgraphs flexibly according to different hardware to optimize the search space for partial fusion. Finally, we make the cost model converge quickly to the loss value we set by manually adjusting the parameters. Compared with other advanced compilers, PCGC optimizes the overall power consumption on an embedded GPU by an average of 130.5% when the time consumption on each hardware is not lower than the average time consumption. On domain-specific architecture, PCGC optimizes power consumption by an average of 66.5%. On FPGA, PCGC optimizes power consumption by 66.1%. In a sense, PCGC can achieve high-speed inference in specific power supply scenarios, reducing the carbon emissions of edge computing. [ABSTRACT FROM AUTHOR]

Improving and enhancing the effectiveness of software vulnerability detection methods is urgently needed today. In this study, we propose a new source code vulnerability detection method based on intelligent and advanced computational algorithms. It's a combination of four main processing techniques including (i) Source Embedding, (ii) Feature Learning, (iii) Resampling Data, and (iv) Classification. The Source Embedding method will perform the task of analyzing and standardizing the source code based on the Joern tool and the data mining algorithm. The Feature Learning model has the function of aggregating and extracting source code attribute based on node using machine learning and deep learning methods. The Resampling Data technique will perform equalization of the experimental dataset. Finally, the Classification model has the function of detecting source code vulnerabilities. The novelty and uniqueness of the new intelligent cognitive computing method is the combination and synchronous use of many different data extracting techniques to compute, represent, and extract the properties of the source code. With this new calculation method, many significant unusual properties and features of the vulnerability have been synthesized and extracted. To prove the superiority of the proposed method, we experiment to detect source code vulnerabilities based on the Verum dataset, details of this part are presented in the experimental section. The experimental results show that the method proposed in the paper has brought good results on all measures. These results have shown to be the best research results for the source code vulnerability detection task using the Verum dataset according to our survey to date. With such results, the proposal in this study is not only meaningful in terms of science but also in practical terms when the method of using intelligent cognitive computing techniques to analyze and evaluate source code has helped to improve the efficiency of the source code analysis and vulnerability detection process. [ABSTRACT FROM AUTHOR]

The traffic flow prediction task is essential to the urban intelligent transportation system. Due to the complex correlation of traffic flow data, insufficient use of spatiotemporal features will often lead to significant deviations in prediction results. This paper proposes an adaptive traffic flow prediction model AD-GNN based on spatiotemporal graph neural network. The gated temporal convolutional network captures the temporal dependence between layers. Moreover, the diffusion graph convolutional network simulates the spatial relationship between nodes. Then, the parameterized adjacency matrix is used to construct an adaptive convolutional network to adaptively mine the implicit global deep spatial dependence. The experimental results show that the model has good prediction performance on three real public datasets and can sufficiently meet real needs. [ABSTRACT FROM AUTHOR]

The paper presents RelFL, a Reliable Federated Learning system for collaborative and decentralized training of a deep learning model in a cloud-fog-Internet of Things (IoT) environment. Data generated by IoT devices is used at fog nodes for locally train a global deep learning model received from a cloud server. Further, a subset of reliable fog nodes is selected as the dominating set (DS) to act as local aggregators (LAs). A LA is responsible for aggregating its own locally trained model's weights with the weights shared by non-LA nodes in its vicinity. The locally aggregated weights are transferred by the LAs to the cloud server for updating the global model. The updated global model is then pushed back to the LAs, which transfer this model to non-LA nodes to start the next round of training. The selection of reliable fog nodes as LAs alleviates the risk of losing model updates due to fog nodes' failures. Results show that RelFL outperforms FedAvg, a widely established FL method, and its variant, FedProx in the presence of fog nodes' failures. RelFL also achieves the results of a centralized convolutional neural network (CNN) while preserving data privacy. [ABSTRACT FROM AUTHOR]

Recent years have witnessed significant achievements in deep learning (DL) technologies. In the meantime, an increasing number of online service operators take advantage of deep learning to provide intelligent and personalized services. Although significant efforts have been put into optimizing the inference efficiency, our investigation shows that for many DL models that process data-intensive requests, the network I/O subsystem also plays an essential role in determining responsiveness. Furthermore, under the latency constraint, uncontrolled network flow processing will impact request batching. Based on the above observation, this paper proposes CoFB, an inference service system that optimizes performance in a holistic way. CoFB improves the load imbalance in the network I/O subsystem with a lightweight flow scheduling scheme that collaborates the network interface card with a dispatcher thread. In addition, CoFB introduces a request reordering and batching policy and an interference-aware concurrent batch throttling strategy for enforcing inference concerning the deadline. We evaluate CoFB on four DL inference services and compare it to two state-of-the-art inference systems: NVIDIA Triton and DVABatch. Experimental results show that CoFB outperforms these two baselines by serving up to 2.69 × and 1.96 × higher load under preset tail latency objectives, respectively. [ABSTRACT FROM AUTHOR]