Search

Abstract Sleep is a physiological activity and human body restores itself from various diseases during sleep. It is necessary to get sufficient amount of sleep to have sound physiological and mental health. Nowadays, due to our present hectic lifestyle, the amount of sound sleep is reduced. It is very difficult to decipher the various stages of sleep manually. Hence, an automated system may be useful to detect the different stages of sleep. This paper presents a novel method for the classification of sleep stages based on RR-time series and electroencephalogram (EEG) signal. The method uses iterative filtering (IF) based multiresolution analysis approach for the decomposition of RR-time series into intrinsic mode functions (IMFs). The delta (δ), theta (θ), alpha (α), beta (β) and gamma (γ) waves are evaluated from EEG signal using band-pass filtering. The recurrence quantification analysis (RQA) and dispersion entropy (DE) based features are evaluated from the IMFs of RR-time series. The dispersion entropy and the variance features are evaluated from the different bands of EEG signal. The RR-time series features and the EEG features coupled with the deep neural network (DNN) are used for the classification of sleep stages. The simulation results demonstrate that our proposed method has achieved an average accuracy of 85.51%, 94.03% and 95.71% for the classification of 'sleep vs wake', 'light sleep vs deep sleep' and 'rapid eye movement (REM) vs non-rapid eye movement (NREM)' sleep stages. [ABSTRACT FROM AUTHOR]

In this paper, we propose a self-adaptive deep neural network architecture suitable for object tracking and labelling. In particular, an adaptation mechanism is introduced that automatically evaluates the performance of the network and then updates its weights to fit the current environmental conditions. The retraining algorithm trusts as much as possible the current conditions (discriminative constraints), while simultaneously providing a minimal degradation of the already obtained knowledge of the network (generative constraints). The underline assumption is that a small weight perturbation is adequate to modify the performance of the network to the new situations, without this constraint implying a small modification of the network output due to the highly non-linear surface that the network models. Under this assumption, we propose a computationally efficient re-training algorithm to tackle the variations of the visual environment, requiring a small number of labelled samples for the adaptation. Weight updating is combined with an unsupervised learning paradigm, implemented through stacked autoencoders, in order to improve convergence, stability and performance of the object tracking and labeling process by propagating the sensory inputs into deep level of hierarchies and therefore structuring the inputs from low representations to more abstract forms. Approximates of current visual environment are provided through a dynamic tracker that combines motion and learning features to automatically create few confident labeled data. The proposed retraining scheme is computationally efficient and able to model non-stationary environments, like the ones appeared in real-life computer vision application scenarios. Experimental results and comparisons are provided on video datasets of very complicated visual content, monitoring industrial workflows of car assembly within a manufactory. The results indicates that our self-adaptive deep neural network architecture is able to correctly label and separate foreground objects from background even under severe visual changes, such as occlusion, illumination variations and change of camera views, and within real time computational constraints. [ABSTRACT FROM AUTHOR]

In this paper, we present a deep learning framework for efficient 3D shape classification using geodesic moments. Our approach inherits many useful properties from the geodesic distance, most notably the capture of the intrinsic geometric structure of 3D shapes and the invariance to isometric deformations. Moreover, we show the similarity between the convergent series of the geodesic moments and the inverse-square eigenvalues of the Laplace–Beltrami operator in the continuous setting. The proposed algorithm uses a two-layer stacked sparse autoencoder to learn deep features from geodesic moments by training the hidden layers individually in an unsupervised fashion, followed by a softmax classifier. Experimental results on three standard 3D shape benchmarks demonstrate superior performance of the proposed approach compared to existing methods. [ABSTRACT FROM AUTHOR]

Deep learning algorithms are the hottest topics in machine learning area lately. Although deep learning has made great progress in many domains, the robustness of learning systems with deep architectures is still rarely studied and needs further investigation. For instance, the impulsive noises (or outliers) are pervasive in real world data and can badly influence the mean square error (MSE) cost function based deep learning algorithms. Correntropy based loss function, which uses Gaussian kernel, is widely utilized to reject the above noises, however, the effect is not satisfactory. Therefore, generalized Correntropy (GC) is put forward to further improve the robustness, which uses generalized Gaussian density (GGD) function as kernel. GC can achieve extra flexibility through the GC parameters, which control the behavior of the induced metric, and shows a markedly better robustness than Correntropy. Motivated by the enhanced robustness of GC, we propose a new robust algorithm named generalized Correntropy based stacked autoencoder (GC-SAE), which is developed by combining the GC and stacked autoencoder (SAE). The new algorithms can extract useful features from the data corrupted by impulsive noises (or outliers) in a more effective way. The good robustness of the proposed method is confirmed by the experimental results on MNIST benchmark dataset. Furthermore, we show how our model can be applied for robust network classification, based on Moore network data of 377,526 samples with 12 classes. [ABSTRACT FROM AUTHOR]

Recurrent neural network (RNN) and Long Short-term Memory (LSTM) networks are the common go-to architecture for exploiting sequential information where the output is dependent on a sequence of inputs. However, in most considered problems, the dependencies typically lie in the latent domain which may not be suitable for applications involving the prediction of a step-wise transformation sequence that is dependent on the previous states only in the visible domain with a known terminal state. We propose a hybrid architecture of convolution neural networks (CNN) and stacked autoencoders (SAE) to learn a sequence of causal actions that nonlinearly transform an input visual pattern or distribution into a target visual pattern or distribution with the same support and demonstrated its practicality in a real-world engineering problem involving the physics of fluids. We solved a high-dimensional one-to-many inverse mapping problem concerning microfluidic flow sculpting, where the use of deep learning methods as an inverse map is very seldom explored. This work serves as a fruitful use-case to applied scientists and engineers in how deep learning can be beneficial as a solution for high-dimensional physical problems, and potentially opening doors to impactful advance in fields such as material sciences and medical biology where multistep topological transformations is a key element. [ABSTRACT FROM AUTHOR]

The use of a deep neural network scheme is proposed to help clinicians solve a difficult diagnosis problem in neurology. The proposed multilayer architecture includes a feature engineering step (from time-frequency transformation), a double compressing stage trained by unsupervised learning, and a classification stage trained by supervised learning. After fine-tuning, the deep network is able to discriminate well the class of patients from controls with around 90% sensitivity and specificity. This deep model gives better classification performance than some other standard discriminative learning algorithms. As in clinical problems there is a need for explaining decisions, an effort has been carried out to qualitatively justify the classification results. The main novelty of this paper is indeed to give an entropic interpretation of how the deep scheme works and reach the final decision. [ABSTRACT FROM AUTHOR]

Novel neural network training methods (commonly referred to as deep learning) have emerged in recent years. Using a combination of unsupervised pre-training and subsequent fine-tuning, deep neural networks have become one of the most reliable classification methods. Since deep neural networks are especially powerful for high-dimensional and non-linear feature vectors, electroencephalography (EEG) and event-related potentials (ERPs) are one of the promising applications. Furthermore, to the authors' best knowledge, there are very few papers that study deep neural networks for EEG/ERP data. The aim of the experiments subsequently presented was to verify if deep learning-based models can also perform well for single trial P300 classification with possible application to P300-based brain-computer interfaces. The P300 data used were recorded in the EEG/ERP laboratory at the Department of Computer Science and Engineering, University of West Bohemia, and are publicly available. Stacked autoencoders (SAEs) were implemented and compared with some of the currently most reliable state-of-the-art methods, such as LDA and multi-layer perceptron (MLP). The parameters of stacked autoencoders were optimized empirically. The layers were inserted one by one and at the end, the last layer was replaced by a supervised softmax classifier. Subsequently, fine-tuning using backpropagation was performed. The architecture of the neural network was 209-130-100-50-20-2. The classifiers were trained on a dataset merged from four subjects and subsequently tested on different 11 subjects without further training. The trained SAE achieved 69.2% accuracy that was higher (p < 0.01) than the accuracy of MLP (64.9%) and LDA (65.9%). The recall of 58.8% was slightly higher when compared with MLP (56.2%) and LDA (58.4%). Therefore, SAEs could be preferable to other state-of-the-art classifiers for high-dimensional event-related potential feature vectors.

Driving modes play vital roles in understanding the stochastic nature of a railway system and can support studies of automatic driving and capacity utilization optimization. Integrated trajectory data containing information such as GPS trajectories and gear changes can be good proxies in the study of driving modes. However, in the absence of labeled data, discovering driving modes is challenging. In this paper, instead of classical models (railway-specified feature extraction and classical clustering), we used five deep unsupervised learning models to overcome this difficulty. In these models, adversarial autoencoders and stacked autoencoders are used as feature extractors, along with generative adversarial network-based and Kullback−Leibler (KL) divergence-based networks as clustering models. An experiment based on real and artificial datasets showed the following: (i) The proposed deep learning models outperform the classical models by 27.64% on average. (ii) Integrated trajectory data can improve the accuracy of unsupervised learning by approximately 13.78%. (iii) The different performance rankings of models based on indices with labeled data and indices without labeled data demonstrate the insufficiency of people’s understanding of the existing modes. This study also analyzes the relationship between the discovered modes and railway carrying capacity.

Bearing fault diagnosis is imperative for the maintenance, reliability, and durability of rotary machines. It can reduce economical losses by eliminating unexpected downtime in industry due to failure of rotary machines. Though widely investigated in the past couple of decades, continued advancement is still desirable to improve upon existing fault diagnosis techniques. Vibration acceleration signals collected from machine bearings exhibit nonstationary behavior due to variable working conditions and multiple fault severities. In the current work, a two-layered bearing fault diagnosis scheme is proposed for the identification of fault pattern and crack size for a given fault type. A hybrid feature pool is used in combination with sparse stacked autoencoder (SAE)-based deep neural networks (DNNs) to perform effective diagnosis of bearing faults of multiple severities. The hybrid feature pool can extract more discriminating information from the raw vibration signals, to overcome the nonstationary behavior of the signals caused by multiple crack sizes. More discriminating information helps the subsequent classifier to effectively classify data into the respective classes. The results indicate that the proposed scheme provides satisfactory performance in diagnosing bearing defects of multiple severities. Moreover, the results also demonstrate that the proposed model outperforms other state-of-the-art algorithms, i.e., support vector machines (SVMs) and backpropagation neural networks (BPNNs). [ABSTRACT FROM AUTHOR]

Road traffic status is significant for trip decision and traffic management, and thus should be predicted accurately. A contribution is that we consider multi-modal data for traffic status prediction than only using single source data. With the substantial data from Ningbo Passenger Transport Management Sector (NPTMS), we wished to determine whether it was possible to develop Stacked Autoencoders (SAEs) for accurately predicting road traffic status from taxicab operating data and bus smart card data. We show that SAE performed better than linear regression model and Back Propagation (BP) neural network for determining the relationship between road traffic status and those factors. In a 26-month data experiment using SAE, we show that it is possible to develop highly accurate predictions (91% test accuracy) of road traffic status from daily taxicab operating data and bus smart card data. [ABSTRACT FROM AUTHOR]

Building energy consumption prediction plays an important role in improving the energy utilization rate through helping building managers to make better decisions. However, as a result of randomness and noisy disturbance, it is not an easy task to realize accurate prediction of the building energy consumption. In order to obtain better building energy consumption prediction accuracy, an extreme deep learning approach is presented in this paper. The proposed approach combines stacked autoencoders (SAEs) with the extreme learning machine (ELM) to take advantage of their respective characteristics. In this proposed approach, the SAE is used to extract the building energy consumption features, while the ELM is utilized as a predictor to obtain accurate prediction results. To determine the input variables of the extreme deep learning model, the partial autocorrelation analysis method is adopted. Additionally, in order to examine the performances of the proposed approach, it is compared with some popular machine learning methods, such as the backward propagation neural network (BPNN), support vector regression (SVR), the generalized radial basis function neural network (GRBFNN) and multiple linear regression (MLR). Experimental results demonstrate that the proposed method has the best prediction performance in different cases of the building energy consumption. [ABSTRACT FROM AUTHOR]

Novel neural network training methods (commonly referred to as deep learning) have emerged in recent years. Using a combination of unsupervised pre-training and subsequent fine-tuning, deep neural networks have become one of the most reliable classification methods. Since deep neural networks are especially powerful for high-dimensional and non-linear feature vectors, electroencephalography (EEG) and event-related potentials (ERPs) are one of the promising applications. Furthermore, to the authors' best knowledge, there are very few papers that study deep neural networks for EEG/ERP data. The aim of the experiments subsequently presented was to verify if deep learning-based models can also perform well for single trial P300 classification with possible application to P300-based brain-computer interfaces. The P300 data used were recorded in the EEG/ERP laboratory at the Department of Computer Science and Engineering, University of West Bohemia, and are publicly available. Stacked autoencoders (SAEs) were implemented and compared with some of the currently most reliable state-of-the-art methods, such as LDA and multi-layer perceptron (MLP). The parameters of stacked autoencoders were optimized empirically. The layers were inserted one by one and at the end, the last layer was replaced by a supervised softmax classifier. Subsequently, fine-tuning using backpropagation was performed. The architecture of the neural network was 209-130-100-50-20-2. The classifiers were trained on a dataset merged from four subjects and subsequently tested on different 11 subjects without further training. The trained SAE achieved 69.2% accuracy that was higher (p < 0.01) than the accuracy of MLP (64.9%) and LDA (65.9%). The recall of 58.8% was slightly higher when compared with MLP (56.2%) and LDA (58.4%). Therefore, SAEs could be preferable to other state-of-the-art classifiers for high-dimensional event-related potential feature vectors. [ABSTRACT FROM AUTHOR]

A good skin detector that is capable of capturing skin tones under different conditions is important for human–machine interaction applications. In a general situation, skin detectors, such as skin probability maps or Gaussian mixture models, achieve acceptable skin segmentation results. However, the false positive rate increases significantly when the skin tones are in shadow or when skin-like background objects are under similar illumination. In this paper, we propose a novel skin feature learning algorithm based on stacked autoencoders, which are deep neural networks. To overcome the problems encountered in skin segmentation that are caused by different ethnicities and varying illumination conditions, the stacked autoencoders are utilized to learn more discriminative representations of the skin area in both the RGB color space and the HSV color space. Unlike traditional machine learning methods, instead of predicting each pixel individually, our algorithm utilizes blocks to learn the representations and detect the skin areas. The algorithm exploits the learning ability of deep neural networks to learn high-level representations of skin tones. Experiments on test images show that the proposed algorithm achieves acceptable results on several publicly available data sets. To reduce the difficulty of detecting skin pixels in these data sets, the ground truths of these data sets are commonly focused on foreground skin area detection. Our skin detector is also able to detect background areas, as shown in our experiments. [ABSTRACT FROM PUBLISHER]

A Brain-Computer Interface (BCI) provides an alternative communication interface between the human brain and a computer. The Electroencephalogram (EEG) signals are acquired, processed and machine learning algorithms are further applied to extract useful information. During EEG acquisition, artifacts are induced due to involuntary eye movements or eye blink, casting adverse effects on system performance. The aim of this research is to predict eye states from EEG signals using Deep learning architectures and present improved classifier models. Recent studies reflect that Deep Neural Networks are trending state of the art Machine learning approaches. Therefore, the current work presents the implementation of Deep Belief Network (DBN) and Stacked AutoEncoders (SAE) as Classifiers with encouraging performance accuracy. One of the designed SAE models outperforms the performance of DBN and the models presented in existing research by an impressive error rate of 1.1% on the test set bearing accuracy of 98.9%. The findings in this study, may provide a contribution towards the state of the art performance on the problem of EEG based eye state classification. [ABSTRACT FROM AUTHOR]

Protein-protein interactions play essential roles in many biological processes. Acquiring knowledge of the residue-residue contact information of two interacting proteins is not only helpful in annotating functions for proteins, but also critical for structure-based drug design. The prediction of the protein residue-residue contact matrix of the interfacial regions is challenging. In this work, we introduced deep learning techniques (specifically, stacked autoencoders) to build deep neural network models to tackled the residue-residue contact prediction problem. In tandem with interaction profile Hidden Markov Models, which was used first to extract Fisher score features from protein sequences, stacked autoencoders were deployed to extract and learn hidden abstract features. The deep learning model showed significant improvement over the traditional machine learning model, Support Vector Machines (SVM), with the overall accuracy increased by 15% from 65.40% to 80.82%. We showed that the stacked autoencoders could extract novel features, which can be utilized by deep neural networks and other classifiers to enhance learning, out of the Fisher score features. It is further shown that deep neural networks have significant advantages over SVM in making use of the newly extracted features. [ABSTRACT FROM AUTHOR]

Face recognition is a widely used biometric technology due to its convenience but it is vulnerable to spoofing attacks made by nonreal faces such as photographs or videos of valid users. The antispoof problem must be well resolved before widely applying face recognition in our daily life. Face liveness detection is a core technology to make sure that the input face is a live person. However, this is still very challenging using conventional liveness detection approaches of texture analysis and motion detection. The aim of this paper is to propose a feature descriptor and an efficient framework that can be used to effectively deal with the face liveness detection problem. In this framework, new feature descriptors are defined using a multiscale directional transform (shearlet transform). Then, stacked autoencoders and a softmax classifier are concatenated to detect face liveness. We evaluated this approach using the CASIA Face antispoofing database and replay-attack database. The experimental results show that our approach performs better than the state-of-the-art techniques following the provided protocols of these databases, and it is possible to significantly enhance the security of the face recognition biometric system. In addition, the experimental results also demonstrate that this framework can be easily extended to classify different spoofing attacks. ©2016 SPIE and IS&T[DO!: 10.1117/1.JEI.25.4.043014] [ABSTRACT FROM AUTHOR]

The use of a deep neural network scheme is proposed to help clinicians solve a difficult diagnosis problem in neurology. The proposed multilayer architecture includes a feature engineering step (from time-frequency transformation), a double compressing stage trained by unsupervised learning, and a classification stage trained by supervised learning. After fine-tuning, the deep network is able to discriminate well the class of patients from controls with around 90% sensitivity and specificity. This deep model gives better classification performance than some other standard discriminative learning algorithms. As in clinical problems there is a need for explaining decisions, an effort has been carried out to qualitatively justify the classification results. The main novelty of this paper is indeed to give an entropic interpretation of how the deep scheme works and reach the final decision.

Bearing fault diagnosis is imperative for the maintenance, reliability, and durability of rotary machines. It can reduce economical losses by eliminating unexpected downtime in industry due to failure of rotary machines. Though widely investigated in the past couple of decades, continued advancement is still desirable to improve upon existing fault diagnosis techniques. Vibration acceleration signals collected from machine bearings exhibit nonstationary behavior due to variable working conditions and multiple fault severities. In the current work, a two-layered bearing fault diagnosis scheme is proposed for the identification of fault pattern and crack size for a given fault type. A hybrid feature pool is used in combination with sparse stacked autoencoder (SAE)-based deep neural networks (DNNs) to perform effective diagnosis of bearing faults of multiple severities. The hybrid feature pool can extract more discriminating information from the raw vibration signals, to overcome the nonstationary behavior of the signals caused by multiple crack sizes. More discriminating information helps the subsequent classifier to effectively classify data into the respective classes. The results indicate that the proposed scheme provides satisfactory performance in diagnosing bearing defects of multiple severities. Moreover, the results also demonstrate that the proposed model outperforms other state-of-the-art algorithms, i.e., support vector machines (SVMs) and backpropagation neural networks (BPNNs).

Nowadays, Deep Neural Networks have been applied to many applications (such as classification, denoising and inpainting) and achieved impressive performance. However, most of these works pay much attention to describe how to construct the relative framework but ignore to provide a clear and intuitive understanding of why their framework performs so well. In this paper, we present a general-purpose no-reference (NR) image quality assessment (IQA) framework based on deep neural network and give insight into the operation of this network. In this NR-IQA framework, simple features are extracted by a new multiscale directional transform (shearlet transform) and the sum of subband coefficient amplitudes (SSCA) is utilized as primary features to describe the behavior of natural images and distorted images. Then, stacked autoencoders are applied as ‘evolution process’ to ‘amplify’ the primary features and make them more discriminative. Finally, by translating the NR-IQA problem into classification problem, the differences of evolved features are identified by softmax classifier. Moreover, we have also incorporated some visualization techniques to analysis and visualize this NR-IQA framework. The resulting algorithm, which we name SESANIA (ShEarlet and Stacked Autoencoders based No-reference Image quality Assessment) is tested on several database (LIVE, Multiply Distorted LIVE and TID2008) individually and combined together. Experimental results demonstrate the excellent performance of SESANIA, and we also give intuitive explanations of how it works and why it works well. In addition, SESANIA is extended to estimate quality in local regions. Further experiments demonstrate the local quality estimation ability of SESANIA on images with local distortions. [ABSTRACT FROM AUTHOR]

With the increase in sophistication and connectedness of the healthcare networks, their attack surfaces and vulnerabilities increase significantly. Malicious agents threaten patients' health and life by stealing or altering data as it flows among the multiple domains of healthcare networks. The problem is likely to exacerbate with the increasing use of IoT devices, edge, and core clouds in the next generation healthcare networks. Presented in this paper is MUSE, a system of deep hierarchical stacked neural networks for timely and accurate detection of malicious activity that leads to alteration of meta-information or payload of the dataflow between the IoT gateway, edge and core clouds. Smaller models at the edge clouds take substantially less time to train as compared to the large models in the core cloud. To improve the speed of training and accuracy of detection of large core cloud models, the MUSE system uses a novel method of merging and aggregating layers of trained edge cloud models to construct a partly pre-trained core cloud model. As a result, the model in the core cloud takes substantially smaller number of epochs (6 to 8) and, consequently, less time, compared to those in the edge clouds, training of which take 35 to 40 epochs to converge. With the help of extensive evaluations, it is shown that with the MUSE system, large, merged models can be trained in significantly less time than the unmerged models that are created independently in the core cloud. Through several runs it is seen that the merged models give on an average 26.2% reduction in training times. From the experimental evaluation we demonstrate that along with fast training speeds the merged MUSE model gives high training and test accuracies, ranging from 95% to 100%, in detection of unknown attacks on dataflows. The merged model thus generalizes very well on the test data. This is a marked improvement when compared with the accuracy given by un-merged model as well as accuracy reported by other researchers with newer datasets. [ABSTRACT FROM AUTHOR]

• Predicts the imposed workload of patients to providers using supervised and unsupervised methods. • Generates robust features by using deep learning stacked auto-encoders. • Proposes a deep multi-task learning approach to improve prediction performance. • Uses a two-stage stochastic resource allocation and workload optimization model. • Generates scenarios based on uncertain demand and simulation. The aging of the global population and the increasing number of patients with chronic diseases necessitate an efficient healthcare operations mechanism to enable provision of appropriate services to patients in a timely and cost-efficient manner. This research provides a solution for two unanswered and critical challenges in healthcare team-based resource planning by employing machine learning and stochastic optimization. The first challenge is how the required workload of a patient should be measured and predicted. The second challenge is how decision-makers should plan and optimize resources in a healthcare team and eventually allocate patients to the available resources to efficiently satisfy needs and minimize costs. In this research, we develop a novel integrated model that provides a mathematical and systematic solution for predicting healthcare providers' total workload and balancing their workload when the required workload is unknown. The proposed approach consists of predictive and prescriptive phases. First, we predict the required workload for different patient types by proposing a deep multi-task learning approach. Then, we use the result of the prediction stage as input for assigning every patient to one of the available healthcare teams, determining the number of required teams, and balancing the teams' workloads in the prescriptive decision-making stage. The outcome of this study suggests that using multi-task learning on represented data outperforms other conventional prediction methods. Moreover, the results of using the proposed stochastic optimization model for resource planning indicate that consideration of randomness and stochastic variables in modeling team-based resource allocation reduces the total cost of healthcare operations considerably, and as a result, leads to enhanced access to healthcare. [ABSTRACT FROM AUTHOR]

This paper develops a data-driven remaining useful life prediction model for solenoid pumps. The model extracts high-level features using stacked autoencoders from decomposed pressure signals (using complementary ensemble empirical mode decomposition with adaptive noise (CEEMDAN) algorithm). These high-level features are then received by a recurrent neural network-gated recurrent units (GRUs) for the RUL estimation. The case study presented demonstrates the robustness of the proposed RUL estimation model with extensive empirical validations. Results support the validity of using the CEEMDAN for non-stationary signal decomposition and the accuracy, ease-of-use, and superiority of the proposed DL-based model for solenoid pump failure prognostics. [ABSTRACT FROM AUTHOR]

Soft sensors based on deep learning have been growing in industrial process applications, inferring hard-to-measure but crucial quality-related variables. However, applications may present strong non-linearity, dynamicity, and a lack of labeled data. To deal with the above-cited problems, the extraction of relevant features is becoming a field of interest in soft-sensing. A novel deep representative learning soft-sensor modeling approach is proposed based on stacked autoencoder (SAE), mutual information (MI), and long-short term memory (LSTM). SAE is trained layer by layer with MI evaluation performed between extracted features and targeted output to evaluate the relevance of learned representation in each layer. This approach highlights relevant information and eliminates irrelevant information from the current layer. Thus, deep output-related representative features are retrieved. In the supervised fine-tuning stage, an LSTM is coupled to the tail of the SAE to address system inherent dynamic behavior. Also, a k-fold cross-validation ensemble strategy is applied to enhance the soft-sensor reliability. Two real-world industrial non-linear processes are employed to evaluate the proposed method performance. The obtained results show improved prediction performance in comparison to other traditional and state-of-art methods. Compared to the other methods, the proposed model can generate more than 38.6% and 39.4% improvement of RMSE for the two analyzed industrial cases. [ABSTRACT FROM AUTHOR]

In-line anomaly detection (AD) not only identifies the needs for semiconductor equipment maintenance but also indicates potential line yield problems. Prompt AD based on available equipment sensory data (ESD) facilitates proactive yield and operations management. However, ESD items are highly diversified and drastically scale up along with the increased use of sensors. Even veteran engineers lack knowledge about ESD items for automated AD. This paper presents a novel Spectral and Time Autoencoder Learning for Anomaly Detection (STALAD) framework. The design consists of four innovations: (1) identification of cycle series and spectral transformation (CSST) from ESD, (2) unsupervised learning from CSST of ESD by exploiting Stacked AutoEncoders, (3) hypothesis test for AD based on the difference between the learned normal data and the tested sample data, (4) dynamic procedure control enabling periodic and parallel learning and testing. Applications to ESD of an HDP-CVD tool demonstrate that STALAD learns normality without engineers' prior knowledge, is tolerant to some abnormal data in training input, performs correct AD, and is efficient and adaptive for fab applications. Complementary to the current practice of using control wafer monitoring for AD, STALAD may facilitate early detection of equipment anomaly and assessment of impacts to process quality. [ABSTRACT FROM AUTHOR]

Driving modes play vital roles in understanding the stochastic nature of a railway system and can support studies of automatic driving and capacity utilization optimization. Integrated trajectory data containing information such as GPS trajectories and gear changes can be good proxies in the study of driving modes. However, in the absence of labeled data, discovering driving modes is challenging. In this paper, instead of classical models (railway-specified feature extraction and classical clustering), we used five deep unsupervised learning models to overcome this difficulty. In these models, adversarial autoencoders and stacked autoencoders are used as feature extractors, along with generative adversarial network-based and Kullback–Leibler (KL) divergence-based networks as clustering models. An experiment based on real and artificial datasets showed the following: (i) The proposed deep learning models outperform the classical models by 27.64% on average. (ii) Integrated trajectory data can improve the accuracy of unsupervised learning by approximately 13.78%. (iii) The different performance rankings of models based on indices with labeled data and indices without labeled data demonstrate the insufficiency of people's understanding of the existing modes. This study also analyzes the relationship between the discovered modes and railway carrying capacity. [ABSTRACT FROM AUTHOR]

The amount of digital data in the universe is growing at an exponential rate, doubling every 2 years, and changing how we live in the world. The information storage capacity and data requirement crossed the zettabytes. With this level of bombardment of data on machine learning techniques, it becomes very difficult to carry out parallel computations. Deep learning is broadening its scope and gaining more popularity in natural language processing, feature extraction and visualization, and almost in every machine learning trend. The purpose of this study is to provide a brief review of deep learning architectures and their working. Research papers and proceedings of conferences from various authentic resources (Institute of Electrical and Electronics Engineers, Wiley, Nature, and Elsevier) are studied and analyzed. Different architectures and their effectiveness to solve domain specific problems are evaluated. Various limitations and open problems of current architectures are discussed to provide better insights to help researchers and student to resume their research on these issues. One hundred one articles were reviewed for this meta‐analysis of deep learning. From this analysis, it is concluded that advanced deep learning architectures are combinations of few conventional architectures. For example, deep belief network and convolutional neural network are used to build convolutional deep belief network, which has higher capabilities than the parent architectures. These combined architectures are more robust to explore the problem space and thus can be the answer to build a general‐purpose architecture. [ABSTRACT FROM AUTHOR]