Search

Intelligent Fault-Tolerant Control (IFTC) refers to the applications of machine learning algorithms for fault diagnosis and design of Fault-Tolerant Control (FTC). The overall goal of the FTC is to accommodate defects in the system components while they are in use and maintain stability with little to no performance reduction. These systems are crucial for mission-critical and safety-related applications where the safety of people is at stake and service continuity is crucial. In this review paper, a systematic study has been done for the development of FTC with machine learning, deep learning, and transfer learning algorithms. The challenges and limitations faced with their possible solutions through machine learning theories for the IFTC model are lined up. This paper guides researchers on the different possible types of machine learning algorithms and their advanced forms like deep learning and transfer learning. The differences among these are highlighted by the challenges and limitations of each. The paper is significant such that most of the important literature references from the Scopus database particularly related to important electrical and mechanical industrial problems have been discussed to guide the researchers who want to apply IFTC for specific industrial problems, being the research gap. Finally, future research directions for the development of IFTC are highlighted. [ABSTRACT FROM AUTHOR]

This paper proposes a new model for creating fuzzy time series and conducting future forecasts using the Bi-directional Long Short-term Memory model (Bi-LSTM). This study contributes by using both classical machine learning and deep learning techniques. Firstly, a self-updating clustering algorithm is used to determine the optimal number of clusters for time series. Secondly, the study describes a method for determining the fuzzy relationship between elements of time series and the identified clusters. Thirdly, the developed model creates the fuzzy time series based on the established rules. Finally, forecasting for the future value using the Bi-LSTM model with some improvement in optimizing the parameters and increasing the accuracy of forecasting result. The steps of the proposed model are clearly illustrated by a numerical example. Moreover, this model also tested on various datasets, including M3, M4, and M5 datasets. By evaluating its performance using parameters such as sMAPE and RMSSE, the proposed model has shown significant enhancements when compared to existing models. Additionally, the model's effectiveness is further demonstrated through its successful application in forecasting the VN index stock in Vietnam. [ABSTRACT FROM AUTHOR]

The continuous malicious software (malware) attacks on smartphones pose a serious threat to the security of users, especially the dominant platform Android. Data-driven methods based on machine learning algorithms are a promising way to defend against that. In this paper, we explore the limitations of this kind of methods in improving the performance of malware detection, e.g., considering each feature in isolation and relying on balanced class data, and propose a multi-model ensemble framework MEFDroid by combining individual predictors, where hybrid deep learning based feature extraction methods are adopted to learn meaningful features from raw data. Besides, a novel fusion scheme is exploited to fuse multi-level representations and mine their correlations to maximize the utilization of original information. To evaluate the effectiveness of the MEFDroid framework, we conduct a step-by-step model justification experiment to investigate how the proposed algorithms (i.e., ESAES, EDAES and EDAFS) enhance the malware detection performance gradually. We also compare the proposed algorithms with classical machine learning methods, conventional sampling solutions for the imbalance problem. Besides, we investigate the reliability of our proposed methods using another public dataset. Our extensive experimental results demonstrate that the target model EDAFS in MEFDroid outperforms others in terms of most metrics, which means that it is an effective alternative solution to detect Android malware. [ABSTRACT FROM AUTHOR]

Non-contact identification of cattle is the basis for cattle management, insurance claims, and traceability of milk sources and meat products. In order to lower the threshold of deep learning technology in practical application and improve the recognition accuracy, this paper uses the localization and classification paradigm, fills the image and proposes the LKA (Locate Key Area) module, which further improves the identification accuracy of cattle without using any labeling information. To avoid the impact of different collection devices, we use three brands of mobile phones and two cameras to collect data. Due to the fact that CNN features do not have scale invariance, the image is filled as a square, and the filling value is discussed. Finally, (0,0,0) is selected for filling. In order to reduce the labeling of new data, the face, torso, and body of the cattle are first detected, and then automatically trimmed to build a classification dataset. In order to further improve the accuracy of classification, LKA module is proposed. This module locates key areas based on the results of class activation map and uses shallow features for monitoring. Thereby completing fine-grained feature extraction without supervision information. Finally, the features of different parts are adjusted for fusion recognition. Experiments were conducted on the head, torso, and whole of cattle. Based on the ResNet50 network architecture, after filling the image into a square, the recognition accuracy has been improved by 0.30%, 0.21%, and 0.43%, respectively. After further adding LKA modules, the accuracy rates were improved by 0.59%, 0.59%, and 0.45%, respectively. In the fusion experiment, ResNet50_LKA and MobileNetV3_LKA achieves 100% accuracy. Experiments have realized dataset construction and fine-grained feature extraction without labeled information, and achieved high recognition accuracy, proving the effectiveness of our method. [ABSTRACT FROM AUTHOR]

Financial markets play an important role on the economical and social organization of modern society. In these kinds of markets, information is an invaluable asset. However, with the modernization of the financial transactions and the information systems, the large amount of information available for a trader can make prohibitive the analysis of a financial asset. In the last decades, many researchers have attempted to develop computational intelligent methods and algorithms to support the decision-making in different financial market segments. In the literature, there is a huge number of scientific papers that investigate the use of computational intelligence techniques to solve financial market problems. However, only few studies have focused on review the literature of this topic. Most of the existing review articles have a limited scope, either by focusing on a specific financial market application or by focusing on a family of machine learning algorithms. This paper presents a review of the application of several computational intelligent methods in several financial applications. This paper gives an overview of the most important primary studies published from 2009 to 2015, which cover techniques for preprocessing and clustering of financial data, for forecasting future market movements, for mining financial text information, among others. The main contributions of this paper are: (i) a comprehensive review of the literature of this field, (ii) the definition of a systematic procedure for guiding the task of building an intelligent trading system and (iii) a discussion about the main challenges and open problems in this scientific field. [ABSTRACT FROM AUTHOR]

This aims to propose a collaborative filtering algorithm based on embedding representation and word embedding techniques, namely UI2vec. According to the joint feature extraction network designed in this paper, UI2vec embeds users and items on the potential space at the same time, and uses the item similarity between them to predict the user's content of interest. Then a generative model VUI2vec with more stable performance is proposed based on UI2vec, which maps users and items as independent Gaussian distributions and obtains the approximate posterior distribution of both by variational inference. The recommendation performance of UI2vec and VUI2vec is evaluated on TaFeng, Movielens, and Netflix datasets. The impact of important superparameters within the model on performance is investigated. The experimental results show that compared with the baseline model, the proposed methods performs consistently well. • UI2vec and VUI2vec as collaborative filtering-based recommendation systems. • It is based on embedding representation of users or items. • Performance is competitive with the state of the art on real-world datasets. • Results encourage integrating collaborative filtering with deep learning. [ABSTRACT FROM AUTHOR]

In fingerprint-based positioning methods, the received signal strength (RSS) vectors from access points are measured at reference points and saved in a database. Then, this dataset is used for the training phase of a pattern recognition algorithm. Several noise types impact the signals in radio channels, and RSS values are corrupted correspondingly. These noises can be mitigated by averaging the RSS samples. In real-time applications, the users cannot wait to collect uncorrelated RSS samples to calculate their average in the online phase of the positioning process. In this paper, we propose a solution for this problem by leveraging the distribution of RSS samples in the offline phase and the preceding state of the user in the online phase. In the first step, we propose a fast and accurate positioning algorithm using a deep neural network (DNN) to learn the distribution of available RSS samples instead of averaging them at the offline phase. Then, the similarity of an online RSS sample to the RPs' fingerprints is obtained to estimate the user's location. Next, the proposed DNN model is combined with a novel state-based positioning method to more accurately estimate the user's location. Extensive experiments on both benchmark and our collected datasets in two different scenarios (single RSS sample and many RSS samples for each user in the online phase) verify the superiority of the proposed algorithm compared with traditional regression algorithms such as deep neural network regression, Gaussian process regression, random forest, and weighted KNN. • Real-time positioning by considering the limitations of smartphones. • Extracting the RSS distribution in the offline phase. • Leveraging previous states of users in positioning. • Showing the robustness by extensive experiments on benchmark dataset. [ABSTRACT FROM AUTHOR]

• 3 layer neural networks outperform other ML algorithms in automated assessments. • In subjective assignments, deep neural networks can improve score prediction. • Word embeddings and linguistic feature results in more precise text mining models. • Extraction and selection of appropriate features can improve results. Machine learning (ML) has recently gained popularity in a variety of domains for automating tasks. This has also been used in the evaluation of student responses/ assignments. This paper presents a qualitatively enhanced methodology for automated score prediction of subjective assignments. To consider all the major aspects of a human grader, 21 linguistic features related to syntactical, grammatical, sentimental, and readability are analyzed from the assignments. This study makes use of the ASAP competition dataset from Kaggle. The evaluation metric used is Quadratic Weighted Kappa (QWK), which measures the agreement between the human graded score and the predicted score. The effect of appropriate feature selection has been observed using Mutual Information Regression. Four ML algorithms are investigated on identified linguistic features. 3 Layer Neural Network with feature selection performed well among all chosen ML algorithms with average QWK of 0.678. To include the benefits of deep learning, a new hybrid model (LF-BiLSTM-att-FS) is proposed that combines a higher level deep neural network (DNN) with the selected features. Pre-trained Glove embedding is used to include contextual information of the text. The proposed model results demonstrated an improvement in overall accuracy, with an average QWK value of 0.768. [ABSTRACT FROM AUTHOR]

Research about recommender systems emerges over the last decade and comprises valuable services to increase different companies' revenue. While most existing recommender systems rely either on a content-based approach or a collaborative approach, there are hybrid approaches that can improve recommendation accuracy using a combination of both approaches. Even though many algorithms are proposed using such methods, it is still necessary for further improvement. This paper proposes a recommender system method using a graph-based model associated with the similarity of users' ratings in combination with users' demographic and location information. By utilizing the advantages of Autoencoder feature extraction, we extract new features based on all combined attributes. Using the new set of features for clustering users, our proposed approach (GHRS) outperformed many existing recommendation algorithms on recommendation accuracy. Also, the method achieved significant results in the cold-start problem. All experiments have been performed on the MovieLens dataset due to the existence of users' side information. • We propose a recommendation system using graph-based features and Autoencoders. • We investigate and analyze each step of the method to find the best setup. • We examine the method with the most popular evaluation metrics and datasets. • Results show an improvement in comparison with basic and state of the art methods. [ABSTRACT FROM AUTHOR]

• We proposed a deep learning model for electrocardiogram signals delineation. • An end-to-end model using CNN and LSTM were designed to extract high level features. • The model was trained and validated on two different publicly available databases. • DENS-ECG model shows a high performance in electrocardiogram signals delineation. With the technological advancements in the field of tele-health monitoring, it is now possible to gather huge amount of electro-physiological signals such as the electrocardiogram (ECG). It is therefore necessary to develop models/algorithms that are capable of analysing these massive amount of data in real-time. This paper proposes a deep learning model for real-time segmentation of heartbeats. The proposed DENS-ECG algorithm, combines convolutional neural network (CNN) and long short-term memory (LSTM) model to detect onset, peak, and offset of different heartbeat waveforms such as the P-waves, QRS complexes, T-waves, and No waves (NW). Using ECG as the inputs, the model learns to extract high level features through the training process, which, unlike other classical machine learning based methods, eliminates the feature engineering step. The proposed DENS-ECG model was trained and validated on a dataset with 105 ECG records of length 15 min each and achieved an average sensitivity and precision of 97.95% and 95.68%, respectively, using a stratified 5-fold cross validation. Additionally, the model was evaluated on an unseen dataset to examine its robustness in QRS detection, which resulted in a sensitivity of 99.61% and precision of 99.52%. The empirical results show the flexibility and accuracy of the combined CNN-LSTM model for ECG signal delineation. This paper proposes an efficient and easy to use approach using deep learning for heartbeat segmentation, which could potentially be used in real-time tele-health monitoring systems. [ABSTRACT FROM AUTHOR]

• The appropriate features of news are analyzed for training models. • The ensemble learning model for fake news detection is proposed in the paper. • The weights of the ensemble learning model are optimized in the paper. • The cross-domain intractability issue is investigated in the paper. In general, the features of fake news are almost the same as those of real news, so it is not easy to identify them. In this paper, we propose a fake news detection system using a deep learning model. First, news articles are preprocessed and analyzed based on different training models. Then, an ensemble learning model combining four different models called embedding LSTM, depth LSTM, LIWC CNN, and N-gram CNN is proposed for fake news detection. Besides, to achieve higher accuracy in fake news detection, the optimized weights of the ensemble learning model are determined using the Self-Adaptive Harmony Search (SAHS) algorithm. In the experiments, we verify that the proposed model is superior to the state-of-the-art methods, with the highest accuracy of 99.4%. Furthermore, we also investigate the cross-domain intractability issue and achieve the highest accuracy of 72.3%. Finally, we believe there is still room for improving the ensemble learning model in addressing the cross-domain intractability issue. [ABSTRACT FROM AUTHOR]

• An ability to reduce dimensions in the new proposed clustering algorithm. • A new clustering objective function based on multi-kernels and auto-encoder. • First time to apply a clustering algorithm based on auto-encoder in the brain network. • An assistance of doctors in diagnosing brain diseases. • A new chance for unsupervised algorithms in fMRI brain networks. The existing clustering algorithms based on auto-encoder only use one layer of information. In this paper, we propose a new subspace clustering algorithm that uses multiple hidden layers of information from the stacked auto-encoder to construct different kernels. The proposed fuzzy multi-kernel clustering method based on auto-encoder is realized by updating the membership matrix and coefficients of these kernels until the value of objective function is iterated to the minimum error. Also, the proposed method combining auto-encoder achieves high dimension reduction of input data. In order to test the effectiveness of the algorithm, we first performed experiments on the published brain network dataset. Compared with MKFC, RMKKM and other algorithms, the proposed method can significantly improve accuracy. Specifically, the brain network has large dimensions when the functional magnetic resonance imaging (fMRI) data is preprocessed. Therefore, the method in this paper is applied to the Autism Spectrum Disorder (ASD) of the Autism Brain Imaging Data Exchange (ABIDE) database. The experiment results on the network dataset we constructed with high dimension are better than the present several clustering algorithms. The results show that the subspace information after dimensionality reduction is more conducive to clustering. [ABSTRACT FROM AUTHOR]

Face sketch synthesis plays a crucial role in face recognition for law enforcement applications. However, the current face sketch synthesis approaches generate sketches from photos based on a model trained by a certain database that is usually collected from individuals of the same ethnicity, and therefore such sketches merely inherit distinct facial distributions (shape and texture) of this database. This also makes such models inapplicable for real-world applications which mainly include multiple photo variations such as pose, lighting, skin color, and ethnic origin. In this paper, a unified face sketch synthesis model considering ethnicity issue as well as photo variations is proposed. A new deep learning scheme is designed to handle the generic visual representation and global structure of the face. Towards the final objective, the recent success of deep residual blocks is exploited and incorporated into a plain feedforward network, termed as DResNet, to learn a regression model for face sketch synthesis. A heterogeneous database containing photos with lighting, ethnicity, hair and skin variations is utilized for training the DResNet model. Extensive subjective and objective evaluations showed the superiority of the proposed DResNet method on state-of-the-art face sketch synthesis methods. Experimental results also demonstrated that the proposed DResNet method can be generalized to face sketch synthesis for real-world applications. • A face sketch synthesis algorithm on the basis of Deep Residual Network is proposed. • A plain feedforward convolutional network is introduced to enhance the feature maps. • Generic visual representation and global structure of the face are clearly presented. • A heterogeneous database is used to evaluate the proposed algorithm. • The proposed algorithm is performed favorably with real-world face photos. [ABSTRACT FROM AUTHOR]

Development of region proposal algorithms has rapidly become one of the most critical research areas over recent years. The perfect accuracy of region-based recognition techniques has led to the use of proposal algorithms as an imperative core in various recognition problems. The main purpose of these algorithms is to extract effective regions of an image with an appropriate number that will reduce the search space and increase detection accuracy. The early development of these algorithms was based on a set of hand-crafted features. Recently, with advances in deep learning techniques, they have been widely and successfully applied to the region proposals. This paper reviews region proposal algorithms, theory, and evaluation metrics and also addresses the existing challenges. In addition, we present a classification for generating proposals, including classical and advanced methods based on hand-crafted features and deep learning, respectively. Both categories are described in details, and an extensive review of recent works is presented. The proposal improvement methods, including ranking algorithms, are also described. In total, more than 60 different algorithms have been studied and classified, and we also point out several applications based on region proposals. • A comprehensive review of recent works of region proposal algorithms is presented. • A taxonomy of region proposals is presented. • Advantages and disadvantages of different categories in this area are discussed. • Applications in different areas including natural and medical images are reviewed. • Different challenges and open-questions are pointed out. [ABSTRACT FROM AUTHOR]

Male infertility is a disease that affects approximately 7% of men. Sperm morphology analysis (SMA) is one of the main diagnosis methods for this problem. However, manual SMA is an inexact, subjective, non-reproducible, and hard to teach process. Therefore, in this paper, we introduce a novel automatic SMA technique that is based on the neural architecture search algorithm, named Genetic Neural Architecture Search (GeNAS). For this purpose, we used a collection of images termed MHSMA dataset, which contains 1540 sperm images that have been collected from 235 patients with infertility problems. In detail, GeNAS consists of a special genetic algorithm that acts as a meta-controller which explores the constrained search space of plain convolutional neural network architectures. Every individual of this genetic algorithm is a convolutional neural network trained to predict morphological deformities in different segments of human sperm (head, vacuole, and acrosome). The fitness of each individual is calculated by a novel proposed method, named GeNAS Weighting Factor (GeNAS-WF). This technique is specially designed to evaluate the fitness of neural networks which, during their learning process, validation accuracy highly fluctuates. To speed up the algorithm, a hashing method is practiced to save each trained neural architecture fitness, so we could reuse them during fitness evaluation. In terms of running time and computational power, our proposed architecture search method is far more efficient than most of the other existing neural architecture search algorithms. Moreover, whereas most of the existing neural architecture search algorithms are designed to work well with well-prepared benchmark datasets, the overall paradigm of GeNAS is specially designed to address the challenges of real-world datasets, particularly shortage of data and class imbalance. In our experiments, the best neural architecture found by GeNAS has reached an accuracy of 91.66%, 77.33%, and 77.66% in the vacuole, head, and acrosome abnormality detection, respectively. In comparison to other proposed algorithms for MHSMA dataset, GeNAS achieved state-of-the-art results. • A novel neural architecture search based on Genetic Algorithm. • A novel neural architecture encoding for exploration of search space. • A specific Crossover and Mutation operations for neural architecture search. • Work on imbalanced datasets, low resolution images, and non-stained sperms. • State-of-the-art accuracy & Real-time sperm morphological analysis. [ABSTRACT FROM AUTHOR]

• Deep learning algorithm for identification of type and flight mode of detected drones. • Novel deep learning algorithm for detection of multiple drones in radio frequency domain. • Results of three indoor experiments concerning collecting and analyzing radio signals from drones. • Public available database with radio signals of drones from two different radio frequency bands. Unmanned aerial systems, especially drones have gone through remarkable improvement and expansion in recent years. Drones have been widely utilized in many applications and scenarios, due to their low price and ease of use. However, in some applications drones can pose a malicious threat. To diminish risks to public security and personal privacy, it is necessary to deploy an effective and affordable anti-drone system in sensitive areas to detect, localize, identify, and defend against intruding malicious drones. This research article presents a new publicly available radio frequency drone dataset and investigates detection and identification methodologies to detect single or multiple drones and identify a single detected drone's type. Moreover, special attention in this paper has been underlined to examine the possibility of using deep learning algorithms, particularly fully connected deep neural networks as an anti-drone solution within two different radio frequency bands. We proposed a supervised deep learning algorithm with fully-connected deep neural network models that use raw drone signals rather than features. Regarding the research results, the proposed algorithm shows a lot of potentials. The probability of detecting a single drone is 99.8%, and the probability of type identification is 96.1%. Moreover, the results of multiple drones detection demonstrate an average accuracy of 97.3%. There have not been such comprehensive publications, to this time, in the open literature that have presented and enlightened the problem of multiple drones detection in the radio frequency domain. [ABSTRACT FROM AUTHOR]

[Display omitted] • A contribution to drug repurposing using deep learning and optimization is proposed. • A novel deep learning model is developed to predict Drug-Target binding affinities. • The proposed model integrates a Convolutional Neural Network on the top of a BiLSTM. • The Hyperparameters of the model are set using a Differential Evolution algorithm. • The Differential Evolution-based learning framework outperformed baseline methods. The identification of drug-target interactions (DTIs) is an important process in drug repositioning and drug discovery. However, it is very expensive and time-consuming to determine all possible DTIs with experimental approaches. Most existing machine learning-based methods formulate the DTIs prediction problem as a binary classification problem. Nevertheless, the lack of experimentally validated negative samples results in imbalanced class distribution within the datasets, which may have a negative influence on the DTI prediction performance. Casting DTI prediction task as a regression problem seems an interesting alternative to avoid this issue especially with the recent increase in protein structural data and DTI datasets. Within this context, a twofold contribution is described in this paper. First, we propose a novel deep learning model for predicting drug-target binding affinities called "Convolution Neural Network with Attention-based bidirectional Long Short-Term Memory network" (CNN-AbiLSTM), which combines a CNN with an attention-based biLSTM. Second, building a powerful hybrid CNN-AbiLSTM model can be highly complicated and requires a suitable setting of the model's hyperparameters. To handle this problem, we propose an evolutionary algorithm-based framework more specifically a Differential Evolution (DE) algorithm to find the optimal configuration of the proposed model. Experimental results show that the proposed DE-based CNN-AbiLSTM model achieves better performance compared with baseline methods. [ABSTRACT FROM AUTHOR]

Automatic liver and tumour segmentation in CT images are crucial in numerous clinical applications, such as postoperative assessment, surgical planning, and pathological diagnosis of hepatic diseases. However, there are still a considerable number of difficulties to overcome due to the fuzzy boundary, irregular shapes, and complex tissues of the liver. In this paper, for liver and tumor segmentation and to overcome the mentioned challenges a simple but powerful strategy is presented based on a cascade convolutional neural network. At the first, the input image is normalized using the Z-Score algorithm. This normalized image provides more information about the boundary of tumor and liver. Also, the Local Direction of Gradient (LDOG) which is a novel encoding algorithm is proposed to demonstrate some key features inside the image. The proposed encoding image is highly effective in recognizing the border of liver, even in the regions close to the touching organs. Then, a cascade CNN structure for extracting both local and semi-global features is used which utilized the original image and two other obtained images as the input data. Rather than using a complex deep CNN model with a lot of hyperparameters, we employ a simple but effective model to decrease the train and testing time. Our technique outperforms the state-of-the-art works in terms of segmentation accuracy and efficiency. [ABSTRACT FROM AUTHOR]

• A deep learning algorithm predicts the latency and availability of cloud services. • Experiments on 15 servers from 3 cloud service providers in 3 regions for 6 weeks. • The prediction is usually above 90% accurate as compared with the data obtained. Low latency and high availability in cloud services give users satisfactory response time and guarantee stability to request they make to services that are hosted in the cloud, thus increasing the usability and reliability of cloud services. On the other hand, high latencies and poor availability will cost businesses their customers due customers dissatisfaction, thus losing customers to competitors. This situation noticeable in e-commerce businesses where real-time response and decision making within seconds are critical for business service delivery. Therefore, latency and availability are important parameters in the Service Level Agreement for cloud users when choosing Cloud Services Providers (CSPs). But the challenge for businesses is that they do not have their in–house mechanism that can accurately predict the required latency and availability for their requirements. Companies only rely on the CSPs tools for estimating resource requirements, which is biased towards the CSPs business model. In this paper, we developed a deep learning algorithm that predicts the latency and availability of cloud services using real-time live data from three CSPs. We designed and implemented experiments on Amazon Web Services, Alibaba Cloud and Tencent Cloud in Beijing University of Posts and Telecommunications to run compute instances across the United States, Europe and Asia Pacific regions. In each cloud platform, five servers were used that resulted in 30,815,100 invocations of http and ping operations for 6 weeks. The algorithm used the data on hourly, daily and weekly basis as historical network data to predict latency and availability. We used MATLABs deep learning toolbox for the implementation of our algorithm and the results showed that the prediction is usually above 90% accurate as compared with the data obtained. The results also revealed that latency performance depends on the locations of users and the availability depends on number of availability zones used. [ABSTRACT FROM AUTHOR]

• Genetic-State Grouping Algorithm guarantees enhancing the performance of RL agents. • The genetic algorithm has successfully been combined with Monte Carlo Tree Search. • Video game provides a valid proving ground for testing AI's capability. Although Reinforcement learning has already been considered one of the most important and well-known techniques of machine learning, its applicability remains limited in the real-world problems due to its long initial learning time and unstable learning. Especially, the problem of an overwhelming number of the branching factors under real-time constraint still stays unconquered, demanding a new method for the next generation of reinforcement learning. In this paper, we propose Genetic State-Grouping Algorithm based on deep reinforcement learning. The core idea is to divide the entire set of states into a few state groups. Each group consists of states that are mutually similar, thus representing their common features. The state groups are then processed with the Genetic Optimizer, which finds outstanding actions. These steps help the Deep Q Network avoid excessive exploration, thereby contributing to the significant reduction of initial learning time. The experiment on the real-time fighting video game (FightingICE) shows the effectiveness of our proposed approach. [ABSTRACT FROM AUTHOR]

• Multi-hybrid neural network with EWT extraction and q -DSCID evaluation is pro-posed. • Novel algorithm based on random inheritance formula is advanced and engrafted. • Ameliorated DBLSTMNN, integrates deep bidirectional training configuration. • Variational learning rate arithmetic is put forward to optimize parameter selection. • Results show the high accuracy of the proposed model for crude oil futures. Machine learning algorithms provide feasibility for crude oil price prediction. In this paper, a novel multi-hybrid predictive neural network model is proposed based on complex deep learning algorithm, which integrates empirical wavelet transform, random inheritance formula error correction algorithm, deep bidirectional LSTM neural network and Elman recurrent neural network with variational learning rate. The prediction model is selected according to the sequence frequency after EWT feature extraction, and the prediction results are obtained by separately predicting and reintegrating. On the basis of individual model, the structure of deep bidirectional training, random inheritance formula and variational learning rate are proposed, which further ameliorate the performance of the model and achieve more effective data information capture. Simultaneously, the examination of variational learning rate provides us with a feasible parameter selection. The proposed model achieves high-precision prediction of crude oil futures price, and stands out in the multi-model comparison analysis and q -DSCID synchronous evaluation, with superior prediction accuracy. [ABSTRACT FROM AUTHOR]

• Apply a state-of-the-art deep reinforcement learning algorithm to the PM problem. • Formulate the PM problem as an MDP with guidance of domain knowledge. • Use a data-driven modeling method to build a fast simulator for learning. • The DRL agent learns to perform group maintenance and opportunistic maintenance. • Proposed method outperforms age-dependent policy and opportunistic policy. In the manufacturing industry, the preventive maintenance (PM) is a common practice to reduce random machine failures by replacing/repairing the aged machines or parts. The decision on when and where the preventive maintenance needs to be carried out is nontrivial due to the complex and stochastic nature of a serial production line with intermediate buffers. In order to improve the cost efficiency of the serial production lines, a deep reinforcement learning based approach is proposed to obtain PM policy. A novel modeling method for the serial production line is adopted during the learning process. A reward function is proposed based on the system production loss evaluation. The algorithm based on the Double Deep Q-Network is applied to learn the PM policy. Using the simulation study, the learning algorithm is proved effective in delivering PM policy that leads to an increased throughput and reduced cost. Interestingly, the learned policy is found to frequently conduct "group maintenance" and "opportunistic maintenance", although their concepts and rules are not provided during the learning process. This finding further demonstrates that the problem formulation, the proposed algorithm and the reward function setting in this paper are effective. [ABSTRACT FROM AUTHOR]