Search

This paper develops a data-driven remaining useful life prediction model for solenoid pumps. The model extracts high-level features using stacked autoencoders from decomposed pressure signals (using complementary ensemble empirical mode decomposition with adaptive noise (CEEMDAN) algorithm). These high-level features are then received by a recurrent neural network-gated recurrent units (GRUs) for the RUL estimation. The case study presented demonstrates the robustness of the proposed RUL estimation model with extensive empirical validations. Results support the validity of using the CEEMDAN for non-stationary signal decomposition and the accuracy, ease-of-use, and superiority of the proposed DL-based model for solenoid pump failure prognostics.

Data based intelligent fault diagnosis is a critical tool for the healthy development of industry process. In actual industrial production, there are often a few or even no labeled samples for target monitoring problem, while large amounts of training data come from different but related diagnosis task under variable working conditions. To utilize the labeled data on related issue for better monitoring performance, the cross-task fault diagnosis based on deep domain adaptation with local feature leaning is proposed. In our strategy, the two-stream stacked autoencoders based deep architecture is used to extract transferable features of collected data across the target diagnosis task domain and the related data-rich monitoring task domain. Then, the maximum mean discrepancy is introduced to establish a deep transfer diagnosis model. Moreover, to further optimize the model, we propose the local feature learning, which can make test data with better intra-class compactness and inter-class separability. Eventually, the proposed method is verified on the Tennessee Eastman process and the rolling bearing data, the results show that our approach achieves positive performance for cross-task fault diagnosis problems.

Evaluating operator mental workload (MW) in human-machine systems via neurophysiological signals is crucial for preventing unpredicted operator performance degradation. However, the feature of physiological signals is associated with the historical values at the previous time steps and its statistical properties vary across individuals and types of mental tasks. In this study, we propose a new transfer dynamical autoencoder (TDAE) to capture the dynamical properties of electroencephalograph (EEG) features and the individual differences. The TDAE consists of three consecutively-connected modules, which are termed as feature filter, abstraction filter, and transferred MW classifier. The feature and abstraction filters introduce dynamical deep network to abstract the EEG features across adjacent time steps to salient MW indicators. Transferred MW classifier exploits large volume EEG data from an source-domain EEG database recorded under emotional stimuli to improve the model training stability. We tested our algorithms on two target EEG databases. The classification performance shows TDAE significantly outperforms existing shallow and deep MW classification models. We also investigated how to select TDAE hyper-parameters and found its superiority in accuracy can be achieved with proper filter orders. This work is sponsored by the National Natural Science Foun- dation of China under Grant No. 61703277 and the Shanghai Sail- ing Program ( 17YF14270 0 0 ). This work is partially supported by the National Natural Science Foundation of China under Grant No. 61673276 , No. 11502145 and theShanghaiPhilosophy andSocial Sciences Fund ( 2017EZX008 ).

Soft sensors based on deep learning have been growing in industrial process applications, inferring hard-to-measure but crucial quality-related variables. However, applications may present strong non-linearity, dynamicity, and a lack of labeled data. To deal with the above-cited problems, the extraction of relevant features is becoming a field of interest in soft-sensing. A novel deep representative learning soft-sensor modeling approach is proposed based on stacked autoencoder (SAE), mutual information (MI), and long-short term memory (LSTM). SAE is trained layer by layer with MI evaluation performed between extracted features and targeted output to evaluate the relevance of learned representation in each layer. This approach highlights relevant information and eliminates irrelevant information from the current layer. Thus, deep output-related representative features are retrieved. In the supervised fine-tuning stage, an LSTM is coupled to the tail of the SAE to address system inherent dynamic behavior. Also, a k-fold cross-validation ensemble strategy is applied to enhance the soft-sensor reliability. Two real-world industrial non-linear processes are employed to evaluate the proposed method performance. The obtained results show improved prediction performance in comparison to other traditional and state-of-art methods. Compared to the other methods, the proposed model can generate more than 38.6% and 39.4% improvement of RMSE for the two analyzed industrial cases.

In-line anomaly detection (AD) not only identifies the needs for semiconductor equipment maintenance but also indicates potential line yield problems. Prompt AD based on available equipment sensory data (ESD) facilitates proactive yield and operations management. However, ESD items are highly diversified and drastically scale up along with the increased use of sensors. Even veteran engineers lack knowledge about ESD items for automated AD. This paper presents a novel Spectral and Time Autoencoder Learning for Anomaly Detection (STALAD) framework. The design consists of four innovations: (1) identification of cycle series and spectral transformation (CSST) from ESD, (2) unsupervised learning from CSST of ESD by exploiting Stacked AutoEncoders, (3) hypothesis test for AD based on the difference between the learned normal data and the tested sample data, (4) dynamic procedure control enabling periodic and parallel learning and testing. Applications to ESD of an HDP-CVD tool demonstrate that STALAD learns normality without engineers&rsquo, prior knowledge, is tolerant to some abnormal data in training input, performs correct AD, and is efficient and adaptive for fab applications. Complementary to the current practice of using control wafer monitoring for AD, STALAD may facilitate early detection of equipment anomaly and assessment of impacts to process quality.

Electroencephalography (EEG)-based brain-computer interfaces (BCIs) that decode cortical activity in reaching and grasping movements can enable natural upper limb motor control. In this paper, we studied the performance of stacked autoencoders in decoding asynchronous reaching movements in the dominant upper limb using EEG. Five individuals without any motor disabilities performed three self-paced reaching tasks while the endpoints of the arm movements were recorded with a motion tracker. Power spectral densities of the relevant cortical signals were extracted among eight bandwidths in the range of 1-45Hz to train a stacked autoencoder. For comparison, convolutional neural network (CNN) and traditional linear decoding using principal component analysis (PCA) for feature selection and linear discriminant analysis (LDA) for classification were also used. An average classification accuracy of 79±5.5% (best up to 88±6%) was achieved from all subjects on wide frequency band (1-45Hz) in offline analysis with stacked autoencoders while average classification accuracies of 68±9.1% (best up to 74±9.1%) with PCA-LDA and 49±13.8% (best up to 56±7.2%) with CNN were achieved. The simultaneous dimensionality reduction and feature extraction capabilities of stacked autoencoders can have significant advantages in BCI applications.

Driving modes play vital roles in understanding the stochastic nature of a railway system and can support studies of automatic driving and capacity utilization optimization. Integrated trajectory data containing information such as GPS trajectories and gear changes can be good proxies in the study of driving modes. However, in the absence of labeled data, discovering driving modes is challenging. In this paper, instead of classical models (railway-specified feature extraction and classical clustering), we used five deep unsupervised learning models to overcome this difficulty. In these models, adversarial autoencoders and stacked autoencoders are used as feature extractors, along with generative adversarial network-based and Kullback&ndash, Leibler (KL) divergence-based networks as clustering models. An experiment based on real and artificial datasets showed the following: (i) The proposed deep learning models outperform the classical models by 27.64% on average. (ii) Integrated trajectory data can improve the accuracy of unsupervised learning by approximately 13.78%. (iii) The different performance rankings of models based on indices with labeled data and indices without labeled data demonstrate the insufficiency of people&rsquo, s understanding of the existing modes. This study also analyzes the relationship between the discovered modes and railway carrying capacity.

Automated analysis of broadcast soccer game video is a challenging computer vision problem. Prior to performing high-level analysis (such as event detection), accurate classification of shot views and play-break segmentation are required to analyze the structure of soccer video. A novel deep network called parallel feature fusion network (PFF-Net) combines local and full-scene features to produce accurate shot view classification based on camera zoom and out-of-field status. Then, a novel hidden-to-observable Markov model (H2O-MM) is introduced to determine play/break status of the shots. Testing is performed using a variety of professional broadcast soccer videos. Variations of the PFF-Net are considered and compared with four existing methods where the PFF-Net demonstrates superior performance (92.6%). The H2O-MM has the accuracy of 98.7% for play-break segmentation, which is an improvement over two existing hidden Markov models. The new methods provide improved temporal labeling of broadcast soccer videos, which can be used to further improve overall automated event detection.

In today’s world, where all records are carried into an electronic environment, cyber security represents a very broad scope, with the primary objective of preventing the loss of financial and/or emotional loss of people, institutions, organizations through the security of data in the digital environment. Today, the most common cyber security threat is phishing attacks. With the phishing attack, the attacker aims to capture the data which are very important for the individuals like identification number, social security number, bank account information, and so on. In this study, using deep learning, it was checked whether the web sites are real or not by using neural networks and support vector machine, decision tree and stacked autoencoders as classification methods. As a result of the study, 86% success rate was reached by using stacked autoencoders which are a part of deep learning techniques.

The use of a deep neural network scheme is proposed to help clinicians solve a difficult diagnosis problem in neurology. The proposed multilayer architecture includes a feature engineering step (from time-frequency transformation), a double compressing stage trained by unsupervised learning, and a classification stage trained by supervised learning. After fine-tuning, the deep network is able to discriminate well the class of patients from controls with around 90% sensitivity and specificity. This deep model gives better classification performance than some other standard discriminative learning algorithms. As in clinical problems there is a need for explaining decisions, an effort has been carried out to qualitatively justify the classification results. The main novelty of this paper is indeed to give an entropic interpretation of how the deep scheme works and reach the final decision.

Traditional methods for early detection of melanoma rely on the visual analysis of the skin lesions performed by a dermatologist. The analysis is based on the so-called ABCDE (Asymmetry, Border irregularity, Colour variegation, Diameter, Evolution) criteria, although confirmation is obtained through biopsy performed by a pathologist. The proposed method exploits an automatic pipeline based on morphological analysis and evaluation of skin lesion dermoscopy images. Preliminary segmentation and pre-processing of dermoscopy image by SC-cellular neural networks is performed, in order to obtain ad-hoc grey-level skin lesion image that is further exploited to extract analytic innovative hand-crafted image features for oncological risks assessment. In the end, a pre-trained Levenberg–Marquardt neural network is used to perform ad-hoc clustering of such features in order to achieve an efficient nevus discrimination (benign against melanoma), as well as a numerical array to be used for follow-up rate definition and assessment. Moreover, the authors further evaluated a combination of stacked autoencoders in lieu of the Levenberg–Marquardt neural network for the clustering step.

Bearing fault diagnosis is imperative for the maintenance, reliability, and durability of rotary machines. It can reduce economical losses by eliminating unexpected downtime in industry due to failure of rotary machines. Though widely investigated in the past couple of decades, continued advancement is still desirable to improve upon existing fault diagnosis techniques. Vibration acceleration signals collected from machine bearings exhibit nonstationary behavior due to variable working conditions and multiple fault severities. In the current work, a two-layered bearing fault diagnosis scheme is proposed for the identification of fault pattern and crack size for a given fault type. A hybrid feature pool is used in combination with sparse stacked autoencoder (SAE)-based deep neural networks (DNNs) to perform effective diagnosis of bearing faults of multiple severities. The hybrid feature pool can extract more discriminating information from the raw vibration signals, to overcome the nonstationary behavior of the signals caused by multiple crack sizes. More discriminating information helps the subsequent classifier to effectively classify data into the respective classes. The results indicate that the proposed scheme provides satisfactory performance in diagnosing bearing defects of multiple severities. Moreover, the results also demonstrate that the proposed model outperforms other state-of-the-art algorithms, i.e., support vector machines (SVMs) and backpropagation neural networks (BPNNs).

Novel neural network training methods (commonly referred to as deep learning) have emerged in recent years. Using a combination of unsupervised pre-training and subsequent fine-tuning, deep neural networks have become one of the most reliable classification methods. Since deep neural networks are especially powerful for high-dimensional and non-linear feature vectors, electroencephalography (EEG) and event-related potentials (ERPs) are one of the promising applications. Furthermore, to the authors' best knowledge, there are very few papers that study deep neural networks for EEG/ERP data. The aim of the experiments subsequently presented was to verify if deep learning-based models can also perform well for single trial P300 classification with possible application to P300-based brain-computer interfaces. The P300 data used were recorded in the EEG/ERP laboratory at the Department of Computer Science and Engineering, University of West Bohemia, and are publicly available. Stacked autoencoders (SAEs) were implemented and compared with some of the currently most reliable state-of-the-art methods, such as LDA and multi-layer perceptron (MLP). The parameters of stacked autoencoders were optimized empirically. The layers were inserted one by one and at the end, the last layer was replaced by a supervised softmax classifier. Subsequently, fine-tuning using backpropagation was performed. The architecture of the neural network was 209-130-100-50-20-2. The classifiers were trained on a dataset merged from four subjects and subsequently tested on different 11 subjects without further training. The trained SAE achieved 69.2 % accuracy that was higher (p < 0.01) than the accuracy of MLP (64.9 %) and LDA (65.9 %). The recall of 58.8 % was slightly higher when compared with MLP (56.2 %) and LDA (58.4 %). Therefore, SAEs could be preferable to other state-of-the-art classifiers for high-dimensional event-related potential feature vectors.

Building energy consumption prediction plays an important role in improving the energy utilization rate through helping building managers to make better decisions. However, as a result of randomness and noisy disturbance, it is not an easy task to realize accurate prediction of the building energy consumption. In order to obtain better building energy consumption prediction accuracy, an extreme deep learning approach is presented in this paper. The proposed approach combines stacked autoencoders (SAEs) with the extreme learning machine (ELM) to take advantage of their respective characteristics. In this proposed approach, the SAE is used to extract the building energy consumption features, while the ELM is utilized as a predictor to obtain accurate prediction results. To determine the input variables of the extreme deep learning model, the partial autocorrelation analysis method is adopted. Additionally, in order to examine the performances of the proposed approach, it is compared with some popular machine learning methods, such as the backward propagation neural network (BPNN), support vector regression (SVR), the generalized radial basis function neural network (GRBFNN) and multiple linear regression (MLR). Experimental results demonstrate that the proposed method has the best prediction performance in different cases of the building energy consumption.

A Brain-Computer Interface (BCI) provides an alternative communication interface between the human brain and a computer. The Electroencephalogram (EEG) signals are acquired, processed and machine learning algorithms are further applied to extract useful information. During EEG acquisition, artifacts are induced due to involuntary eye movements or eye blink, casting adverse effects on system performance. The aim of this research is to predict eye states from EEG signals using Deep learning architectures and present improved classifier models. Recent studies reflect that Deep Neural Networks are trending state of the art Machine learning approaches. Therefore, the current work presents the implementation of Deep Belief Network (DBN) and Stacked AutoEncoders (SAE) as Classifiers with encouraging performance accuracy. One of the designed SAE models outperforms the performance of DBN and the models presented in existing research by an impressive error rate of 1.1% on the test set bearing accuracy of 98.9%. The findings in this study, may provide a contribution towards the state of the art performance on the problem of EEG based eye state classification.