Search

Supervised machine learning methods for geological mapping via remote sensing face limitations due to the scarcity of accurately labelled training data that can be addressed by unsupervised learning, such as dimensionality reduction and clustering. Dimensionality reduction methods have the potential to play a crucial role in improving the accuracy of geological maps. Although conventional dimensionality reduction methods may struggle with nonlinear data, unsupervised deep learning models such as autoencoders can model non-linear relationships. Stacked autoencoders feature multiple interconnected layers to capture hierarchical data representations useful for remote sensing data. We present an unsupervised machine learning-based framework for processing remote sensing data using stacked autoencoders for dimensionality reduction and k -means clustering for mapping geological units. We use Landsat 8, ASTER, and Sentinel-2 datasets to evaluate the framework for geological mapping of the Mutawintji region in Western New South Wales, Australia. We also compare stacked autoencoders with principal component analysis (PCA) and canonical autoencoders. Our results reveal that the framework produces accurate and interpretable geological maps, efficiently discriminating rock units. The results reveal that the combination of stacked autoencoders with Sentinel-2 data yields the best performance accuracy when compared to other combinations. We find that stacked autoencoders enable better extraction of complex and hierarchical representation of the input data when compared to canonical autoencoders and PCA. We also find that the generated maps align with prior geological knowledge of the study area while providing novel insights into geological structures. [ABSTRACT FROM AUTHOR]

In order to evaluate the effects of forest fires on the dynamics of the function and structure of ecosystems, it is necessary to determine burned forest areas with high accuracy, effectively, economically, and practically using satellite images. Extraction of burned forest areas utilizing high-resolution satellite images and image classification algorithms and assessing the successfulness of varied classification algorithms has become a prominent research field. This study aims to indicate on the capability of the deep learning-based Stacked Autoencoders method for the burned forest areas mapping from Sentinel-2 satellite images. The Stacked Autoencoders, used in this study as an unsupervised learning method, were compared qualitatively and quantitatively with frequently used supervised learning algorithms (k-Nearest Neighbors (k-NN), Subspaced k-NN, Support Vector Machines, Random Forest, Bagged Decision Tree, Naive Bayes, Linear Discriminant Analysis) on two distinct burnt forest zones. By selecting burned forest zones with contrasting structural characteristics from one another, an objective assessment was achieved. Manually digitized burned areas from Sentinel-2 satellite images were utilized for accuracy assessment. For comparison, different classification performance and quality metrics (Overall Accuracy, Mean Squared Error, Correlation Coefficient, Structural Similarity Index Measure, Peak Signal-to-Noise Ratio, Universal Image Quality Index, and KAPPA metrics) were used. In addition, whether the Stacked Autoencoders method produces consistent results was examined through boxplots. In terms of both quantitative and qualitative analysis, the Stacked Autoencoders method showed the highest accuracy values. [ABSTRACT FROM AUTHOR]

In most batch processes, the correlations of process variables present multi‐stage characteristic as the process progress and operating conditions change. The methods building a local model at each stage ignore the potential correlations among stages, resulting in poor quality prediction of batch process. To solve this problem, a batch process quality prediction method based on multi‐stage fusion regression network (MSFRN) is proposed. First, the affine propagation clustering (AP) algorithm is used to automatically divide the stages for batch process without relying on prior knowledge. Second, the input reconstruction error and quality prediction error are organically combined to develop a stacked isomorphic and quality‐driven autoencoder (SIQAE) for each stage, which fully extracts the quality‐related features for each stage while reducing the input cumulative loss. Then, the self‐attention mechanism is used to integrate the quality‐related features of each stage so as to obtain global features which consider the correlations among stages. Finally, the global features are input into the fully connected regression layer to predict the quality variables of batch process. The effectiveness of the proposed method was verified by applying on penicillin fermentation process. [ABSTRACT FROM AUTHOR]

In the face of climate change and rising energy prices, lowering energy usage of industrial machines is gaining widespread attention. Αpropriate machine settings could lead to reduced production costs and lower environmental impact, while simultaneously maintaining products' quality. However, defining the complex, nonlinear dependencies between these settings and energy usage or quality in manufacturing is often a challenging task. In the presented work, a method for optimized machine settings recommendation is proposed using inverse classification via autoencoders. The algorithm can suggest operation parameters, based on predefined intervals of energy consumption and product properties. The performance is evaluated on data generated by a digital twin of an extrusion process. [ABSTRACT FROM AUTHOR]

Epistemic uncertainty is widespread in reliability analysis of practical engineering products. Evidence theory is regarded as a powerful model for quantifying and analyzing epistemic uncertainty. However, the heavy computational burden has severely hindered its application in practical engineering problems, which is essentially caused by the repeated extreme analysis of limit-state function (LSF). In order to address the issue, this paper proposes a novel method to solve the evidence-theory-based reliability analysis (ETRA). It transforms the conventional ETRA problem into the classification of three classes of joint focal elements (JFEs) and then solves the classification problem effectively through a deep learning approach. The core of solving an ETRA problem is to determine whether the joint focal element is located in the reliable region, failure region, or intersected with the LSF. A spatial position feature reduction and arrangement method is proposed to classify the JFEs, which can effectively reduce the feature dimension and take into account the integrity and correlation of features. The stacked autoencoders model is then constructed and updated by extracting the spatial position features of the sampled JFEs to achieve high-accuracy classification of the remaining JFEs, and the reliability interval is calculated efficiently according to the classification results. Finally, the effectiveness of the proposed method is demonstrated using several numerical examples. [ABSTRACT FROM AUTHOR]

In recent years, heart disease has superseded several other contributory death factors. It is challenging to predict an individual's risk of acquiring heart disease since it requires both expert knowledge and real-world experience. Developing an effective method for the prognosis of heart disease using Internet of Medical Things (IoMT) technology in healthcare organizations by collecting sensor data from patients' bodies, utilizing robust expert systems, and incorporating vast healthcare data on cardiac disorders to alert physicians in critical situations is a challenging task. Several machine learning-based techniques for predicting and diagnosing cardiac disease have recently been demonstrated. However, these algorithms cannot effectively handle high-dimensional information due to the need for an intelligent framework incorporating multiple sources to predict cardiac illness. This work proposes a unique model for heart disease prediction based on deep learning, Deep Gated Recurrent Units (D-GRU), which combines with Stacked Auto Encoders. A novel optimization algorithm, the Logistic Chaos Honey Badger Algorithm, is proposed for optimal feature selection. Publicly available heart disease-related datasets collected from UCI Repository, Cleveland Database, are used for training the proposed D-GRU model. The trained model is further tested on the data gathered from the sensors in the IoMT framework. The performance of the proposed model is compared against several deep learning models and existing works in the literature. The proposed D-GRU model outperforms the other models taken for comparison and exhibits performance supremacy with an accuracy of 95.15% in predicting heart diseases. [ABSTRACT FROM AUTHOR]

Tourism volume forecasting is the hot topic in tourism management, and deep learning techniques as the promising tool are becoming popular for capturing the characteristics of tourism volume data, which is reflected in two aspects: data dimension reduction (i.e., stacked autoencoders [SAE]) and model forecasting (i.e., bi‐directional gated recurrent unit neural network [Bi‐GRU]). With Hong Kong inbound arrivals as a case, this study has empirically verified that deep learning techniques can improve forecasting accuracy. Furthermore, the proposed approach (i.e., SAE‐Bi‐GRU) is significantly superior to benchmark models (i.e., PCA‐Bi‐GRU with Baidu index and Google trends). [ABSTRACT FROM AUTHOR]

The presence of community structures in complex networks reveals meaningful insights about such networks and their constituent entities. Finding groups of related nodes based on mutual interests, common features, objectives, or interactions in a network is known as community detection. In this paper, we propose a novel Stacked Autoencoder-based deep learning approach augmented by the Crow Search Algorithm (CSA)-based k-means clustering algorithm to uncover community structure in complex networks. As per our approach, firstly, we generate a modularity matrix for the input graph. The modularity matrix is then passed through a series of stacked autoencoders to reduce the dimensionality of the matrix while preserving the topology of the network and improving the computational time of the proposed algorithm. The obtained matrix is then provided as an input to a modified k-means clustering algorithm augmented with the crow search optimization to detect the communities. We use Crow Search Algorithm-based optimization to generate the initial centroids for the k-means algorithm instead of generating them randomly. We perform extensive experimental analysis on several real-world and synthetic datasets and evaluate various performance metrics. We compare the results obtained by our algorithm with several traditional and contemporary community detection algorithms. The obtained results reveal that our proposed method achieves commendable results. [ABSTRACT FROM AUTHOR]

Soft sensors based on deep learning approaches are growing in popularity due to their ability to extract high-level features from training, improving soft sensors' performance. In the training process of such a deep model, the set of hyperparameters is critical to archive generalization and reliability. However, choosing the training hyperparameters is a complex task. Usually, a random approach defines the set of hyperparameters, which may not be adequate regarding the high number of sets and the soft sensing purposes. This work proposes the RB-PSOSAE, a Representation-Based Particle Swarm Optimization with a modified evaluation function to optimize the hyperparameter set of a Stacked AutoEncoder-based soft sensor. The evaluation function considers the mean square error (MSE) of validation and the representation of the features extracted through mutual information (MI) analysis in the pre-training step. By doing this, the RB-PSOSAE computes hyperparameters capable of supporting the training process to generate models with improved generalization and relevant hidden features. As a result, the proposed method can generate more than 16.4% improvement in RMSE compared to another standard PSO-based method and, in some cases, more than 50% improvement compared to traditional methods applied to the same real-world nonlinear industrial process. Thus, the results demonstrate better prediction performance than traditional and state-of-the-art methods. [ABSTRACT FROM AUTHOR]

• The proposed method detects the onset of ductile fracture using acoustic emission signals. • Energy allocation maps show short-term and relative energies of acoustic emission signals. • Accumulative relative energy shows the global energy distribution across resolutions. • Normalized short-term energy captures the local energy distribution of resolutions. • Partial experiments are conducted to evaluate the accuracy and efficacy of the proposed method. The quantitative assessment of the formability of bulk metallic materials remains challenging, primarily due to considerable difficulties in accurately identifying the initiation of fractures. This study introduces a novel methodology for detecting the onset of ductile fracture by using acoustic emission signals. Unlike traditional methods that rely on standard characteristics, this approach utilizes acoustic emission signals that are depicted through vivid and direct feature images. These images represent short-term and relative energies, which are termed as energy allocation maps. These maps are generated utilizing a novel signal processing technique that employs the maximal overlap discrete wavelet transform for multi-resolution analysis. In this study, a parameter known as accumulative relative energy is developed to illustrate the global energy distribution across resolutions, while another parameter, normalized short-term energy, is designed to capture the local energy distribution of resolutions. The natural neighbor interpolation technique is subsequently employed to derive the energy allocation map. A set of stacked autoencoders is accordingly configured to transform this map into a singular value that represents the condition of the material. Consequently, a ductile fracture indicator is proposed to identify the onset of fracture. The accuracy and efficacy of the proposed method are validated through partial experiments. The study examines four bulk metallic materials, providing insights into their deformation capabilities under compression. [Display omitted] [ABSTRACT FROM AUTHOR]

The financial markets have been influenced by the emerging spread of Coronavirus disease, COVID-19. The oil, and gold as well have experienced a downward trend due to the increased rate in the number of confirmed COVID-19 cases. Lately, the published COVID data comprised new variables that may influence the accuracy of the oil/gold prices forecasting models including the Stringency index, Reproduction rate, Positive Rate, and Vaccinations. In this study, Deep Autoencoders are introduced and combined with the well-known approach: Pearson Correlation Coefficient, PCC, analysis in selecting the key features that affect the accuracy of the forecasting models of gold and oil prices with respect to COVID-19 pandemic. We have utilized a hybrid approach of PCC along with a 2-Stage Stacked Autoencoder, SA, to extract the latent features which are then submitted to Neural Network, NN, regression model. The NN regressor has been trained using the Bayesian Regularization-backpropagation algorithm which provides a good generalization for small noisy datasets. The hybrid approach has yielded the minimum MSE values of 8.97 × 10−3 and 5.356 × 10−2 on the oil/gold validation set, respectively. Compared to the existing approaches, the proposed approach has outperformed the ARIMA, ML based regression models in forecasting the oil/gold prices. In addition, the introduced framework has yielded lower Mean Absolute Error, MAE, than the Recurrent Neural Network, RNN, and the Principal Component Analysis, PCA, for dimension reduction. The retrieved results showed that the hybrid method produced more robust features by considering the relationship between the input features. [ABSTRACT FROM AUTHOR]

Data based intelligent fault diagnosis is a critical tool for the healthy development of industry process. In actual industrial production, there are often a few or even no labeled samples for target monitoring problem, while large amounts of training data come from different but related diagnosis task under variable working conditions. To utilize the labeled data on related issue for better monitoring performance, the cross-task fault diagnosis based on deep domain adaptation with local feature leaning is proposed. In our strategy, the two-stream stacked autoencoders based deep architecture is used to extract transferable features of collected data across the target diagnosis task domain and the related data-rich monitoring task domain. Then, the maximum mean discrepancy is introduced to establish a deep transfer diagnosis model. Moreover, to further optimize the model, we propose the local feature learning, which can make test data with better intra-class compactness and inter-class separability. Eventually, the proposed method is verified on the Tennessee Eastman process and the rolling bearing data, the results show that our approach achieves positive performance for cross-task fault diagnosis problems.

Soft sensors based on deep learning approaches are growing in popularity due to their ability to extract high-level features from training, improving soft sensors’ performance. In the training process of such a deep model, the set of hyperparameters is critical to archive generalization and reliability. However, choosing the training hyperparameters is a complex task. Usually, a random approach defines the set of hyperparameters, which may not be adequate regarding the high number of sets and the soft sensing purposes. This work proposes the RB-PSOSAE, a Representation-Based Particle Swarm Optimization with a modified evaluation function to optimize the hyperparameter set of a Stacked AutoEncoder-based soft sensor. The evaluation function considers the mean square error (MSE) of validation and the representation of the features extracted through mutual information (MI) analysis in the pre-training step. By doing this, the RB-PSOSAE computes hyperparameters capable of supporting the training process to generate models with improved generalization and relevant hidden features. As a result, the proposed method can generate more than 16.4% improvement in RMSE compared to another standard PSO-based method and, in some cases, more than 50% improvement compared to traditional methods applied to the same real-world nonlinear industrial process. Thus, the results demonstrate better prediction performance than traditional and state-of-the-art methods.

Summary: In most real‐world monitoring scenarios, the lack of measurements from damaged conditions requires the application of unsupervised approaches, mainly the ones based on modal features estimated from raw vibration data through traditional system identification methods. Although numerous successful applications using modal parameters have been reported, they have demonstrated to be insufficient to estimate a robust set of damage‐sensitive features. Inspired by the idea of compressed sensing and deep learning, an intelligent two‐level feature extraction approach using stacked autoencoders over pre‐processed vibration data is proposed. This procedure can improve the performance of traditional damage detection classifiers by compressing modal parameters into a smaller set of highly informative features when considering information entropy metrics. The proposed technique demonstrates significant improvement in the performance of damage detection and classification approaches when evaluated on the well‐known monitoring data sets from the Z‐24 Bridge, where several damage scenarios were carried out under rigorous operational and environmental effects. [ABSTRACT FROM AUTHOR]

This paper presents a new method of predicting the P300 component of an electroencephalography (EEG) signal to recognize the characters in a P300 brain-computer interface (BCI) speller accurately. This method consists of a deep learning model and the nonlinear time-frequency features. It is believed that the combination of the deep model network and extracting the nonlinear features of the EEG led this research to a better prediction of the P300 and, therefore, character recognition. Cohen's class distribution is used in order to extract the nonlinear features of the EEG. Evaluating all of the kernels, Butterworth found to be more informative and it produced better results. Based on the differences observed between time-frequency responses of target and nontarget signals, specific subbands are selected to extract seven features. A deep-structured neural network, namely stacked sparse autoencoders, is applied for BCI character recognition. This deep network reduces the dimension of feature space by extracting unsupervised features. Then, the features are fed to a Softmax classifier. Afterward, the whole network passes a fine-tuning phase by a supervised backpropagation algorithm. For evaluating the work, Dataset II of BCI Competition III is utilized. Based on the results, this approach would improve the accuracy in both P300 detection and character recognition. This research results in 82.7% and 93.5% accuracy for P300 classification and character recognition, respectively. [ABSTRACT FROM AUTHOR]

• A ZSL model named Domain-aware Stacked AutoEncoders (DaSAE) is proposed to build the tight relationships of the different spaces. • Our model learns the domain-aware projections under the encoder-decoder framework, which tackles the projection domain shift effectively. • The objective function of DaSAE takes a simple formulation which can be cast into a min-min optimization problem. • The extensive experiments show its super performance for zero-shot learning under the different settings. Zero-shot learning (ZSL), which focuses on transferring the knowledge from the seen (source) classes to unseen (target) ones, is getting more and more attention in the computer vision community. However, there often has a large domain gap between the source and target classes, resulting in the projection domain shift problem. To this end, we propose a novel model, named Domain-aware Stacked AutoEncoders (DaSAE), that consists of two interactive stacked auto-encoders to learn the domain-aware projections for adapting source and target domains respectively. In each of them, the first-layer encoder aims to project a visual feature vector into the semantic space, and the second-layer encoder connects the semantic description of a sample with its label directly. Meanwhile, the two-layer decoders seek to reconstruct the visual representation from the label information and semantic description successively. Moreover, the manifold regularization that explores the manifold structure residing in the target data is integrated to the basic DaAE, which further improves the generalization ability of our model. Extensive experiments on the benchmark datasets clearly demonstrate that our DaSAE outperforms the state-of-the-art alternatives by the significant margins. [ABSTRACT FROM AUTHOR]

Traditional methods for early detection of melanoma rely on the visual analysis of the skin lesions performed by a dermatologist. The analysis is based on the so‐called ABCDE (Asymmetry, Border irregularity, Colour variegation, Diameter, Evolution) criteria, although confirmation is obtained through biopsy performed by a pathologist. The proposed method exploits an automatic pipeline based on morphological analysis and evaluation of skin lesion dermoscopy images. Preliminary segmentation and pre‐processing of dermoscopy image by SC‐cellular neural networks is performed, in order to obtain ad‐hoc grey‐level skin lesion image that is further exploited to extract analytic innovative hand‐crafted image features for oncological risks assessment. In the end, a pre‐trained Levenberg–Marquardt neural network is used to perform ad‐hoc clustering of such features in order to achieve an efficient nevus discrimination (benign against melanoma), as well as a numerical array to be used for follow‐up rate definition and assessment. Moreover, the authors further evaluated a combination of stacked autoencoders in lieu of the Levenberg–Marquardt neural network for the clustering step.

Electroencephalography (EEG)-based brain-computer interfaces (BCIs) that decode cortical activity in reaching and grasping movements can enable natural upper limb motor control. In this paper, we studied the performance of stacked autoencoders in decoding asynchronous reaching movements in the dominant upper limb using EEG. Five individuals without any motor disabilities performed three self-paced reaching tasks while the endpoints of the arm movements were recorded with a motion tracker. Power spectral densities of the relevant cortical signals were extracted among eight bandwidths in the range of 1-45Hz to train a stacked autoencoder. For comparison, convolutional neural network (CNN) and traditional linear decoding using principal component analysis (PCA) for feature selection and linear discriminant analysis (LDA) for classification were also used. An average classification accuracy of 79±5.5% (best up to 88±6%) was achieved from all subjects on wide frequency band (1-45Hz) in offline analysis with stacked autoencoders while average classification accuracies of 68±9.1% (best up to 74±9.1%) with PCA-LDA and 49±13.8% (best up to 56±7.2%) with CNN were achieved. The simultaneous dimensionality reduction and feature extraction capabilities of stacked autoencoders can have significant advantages in BCI applications.

Human action recognition (HAR) is a popular subject for academic society and other stakeholders. Nowadays it has a wide-spread use for lots of practical applications such as for health, assistive living, elderly care, and so on. Both visual and sensor-based data can be used for HAR. Visual data includes video images, still images, skeleton images, etc., whilst sensor-based data is acquired as numerical data from devices such as accelerometer, gyroscope, and so on. Employed classification methods and data types are of crucial importance on HAR performance. In this paper, sensor data based activity recognition is performed using stacked autoencoders (SAE). Finding near optimal accuracy results with SAEs is a challenging process if structural optimization is left to user experience. The purpose of this study is to improve the accuracy of HAR classifying methods such as SAEs using heuristic optimization algorithms. Hence the structural parameters of SAEs have been optimized using artificial bee colony optimization algorithm (ABC), genetic algorithm, differential evolution algorithm, particle swarm optimization algorithm (PSO) and an afresh developed hybrid algorithm (hABCPSO) which includes PSO and ABC in its internal structure. Leave-one-out cross-validation (LOOCV) test method is used for validating the results. Each algorithm is performed for 30 runs and the results of these runs are analyzed by statistical methods in detail. According to experimental results, hABCPSO supported SAE gives the minimum error and is the most robustness algorithm among the others. Obtained success rates show that the proposed SAE achieved the best accuracy rate ever on UCI human activity recognition dataset and a close result to best on wireless sensor data mining dataset with regard to LOOCV test technique. [ABSTRACT FROM AUTHOR]

As a useful data augmentation technique, generative adversarial networks have been successfully applied in fault diagnosis field. But traditional generative adversarial networks can only generate one category fault signals in one time, which is time-consuming and costly. To overcome this weakness, we develop a novel fault diagnosis method which combines conditional generative adversarial networks and stacked autoencoders, and both of them are built by stacking one-dimensional full connection layers. First, conditional generative adversarial networks is used to generate artificial samples based on the frequency samples, and category labels are adopted as the conditional information to simultaneously generate different category signals. Meanwhile, spectrum normalization is added to the discriminator of conditional generative adversarial networks to enhance the model training. Then, the augmented training samples are transferred to stacked autoencoders for feature extraction and fault classification. Finally, two datasets of bearing and gearbox are employed to investigate the effectiveness of the proposed conditional generative adversarial network–stacked autoencoder method. [ABSTRACT FROM AUTHOR]

Time-series forecasting is applied to many areas of smart factories, including machine health monitoring, predictive maintenance, and production scheduling. In smart factories, machine speed prediction can be used to dynamically adjust production processes based on different system conditions, optimize production throughput, and minimize energy consumption. However, making accurate data-driven machine speed forecasts is challenging. Given the complex nature of industrial manufacturing process data, predictive models that are robust to noise and can capture the temporal and spatial distributions of input time-series signals are prerequisites for accurate forecasting. Motivated by recent deep learning studies in smart manufacturing, in this article, we propose an end-to-end model for multistep machine speed prediction. The model comprises a deep convolutional LSTM encoder–decoder architecture. Extensive empirical analyses using real-world data obtained from a metal packaging plant in the United Kingdom demonstrate the value of the proposed method when compared with the state-of-the-art predictive models. [ABSTRACT FROM AUTHOR]

Multivariate statistical process control (MSPC) has been widely employed for process fault detection. Recently, deep neural networks (DNNs), i.e., stacked autoencoder (SAE) enjoys its popularization in process fault detection. SAE shows good performance in extracting representative features from the process data based on unsupervised learning, which provides a new monitoring method without large amount of labeled data. However, the extraction of the intrinsic geometrical information from process signals is not considered by these regular SAEs This paper proposes a new DNN, manifold regularized stacked autoencoders (MRSAE) for fault detection in complex industrial processes. The local/global information preservation is incorporated into the encoding phase of SAE to capture intrinsic structure of the process data. MRSAE is used to describe distribution of the nonlinear process data and learn effective features for process fault detection. Two typical statistics (i.e., Hotelling's T-squared (T 2) and squared prediction error (SPE)) based on the extracted features by MRSAE are developed for process fault detection. The comparison between MRSAE and other typical DNNs on a complex numerical process and two benchmark processes, i.e., Tennessee Eastman process (TEP) and Fed-Batch fermentation penicillin process (FBFP), indicates the effectiveness of the proposed method for process fault detection. The manifold regularization-based DNN technique provides a new way for feature learning from high-dimensional and nonlinear process signals. • A new DNN (MRSAE) is proposed to learn features from process variables. • MRSAE is embedded manifold regularization for feature learning. • MRSAE-based feature learning is effective for process fault detection. • MRSAE provides an effective way for fault detection due to powerful feature learning. [ABSTRACT FROM AUTHOR]

The ability to proactively monitor business processes is a main competitive differentiator for firms. Process execution logs generated by process aware information systems help to make process specific predictions for enabling a proactive situational awareness. The goal of the proposed approach is to predict the next process event from the completed activities of the running process instance, based on the execution log data from previously completed process instances. By predicting process events, companies can initiate timely interventions to address undesired deviations from the desired workflow. The paper proposes a multi-stage deep learning approach that formulates the next event prediction problem as a classification problem. Following a feature pre-processing stage with n-grams and feature hashing, a deep learning model consisting of an unsupervised pre-training component with stacked autoencoders and a supervised fine-tuning component is applied. Experiments on a variety of business process log datasets show that the multi-stage deep learning approach provides promising results. The study also compared the results to existing deep recurrent neural networks and conventional classification approaches. Furthermore, the paper addresses the identification of suitable hyperparameters for the proposed approach, and the handling of the imbalanced nature of business process event datasets. [ABSTRACT FROM AUTHOR]

Genome-Wide Association Studies (GWAS) are used to identify statistically significant genetic variants in case-control studies. The main objective is to find single nucleotide polymorphisms (SNPs) that influence a particular phenotype (i.e., disease trait). GWAS typically use a p-value threshold of 5 * 10-8 to identify highly ranked SNPs. While this approach has proven useful for detecting disease-susceptible SNPs, evidence has shown that many of these are, in fact, false positives. Consequently, there is some ambiguity about the most suitable threshold for claiming genome-wide significance. Many believe that using lower p-values will allow us to investigate the joint epistatic interactions between SNPs and provide better insights into phenotype expression. One example that uses this approach is multifactor dimensionality reduction (MDR), which identifies combinations of SNPs that interact to influence a particular outcome. However, computational complexity is increased exponentially as a function of higher-order combinations making approaches like MDR difficult to implement. Even so, understanding epistatic interactions in complex diseases is a fundamental component for robust genotype-phenotype mapping. In this paper, we propose a novel framework that combines GWAS quality control and logistic regression with deep learning stacked autoencoders to abstract higher-order SNP interactions from large, complex genotyped data for case-control classification tasks in GWAS analysis. We focus on the challenging problem of classifying preterm births which has a strong genetic component with unexplained heritability reportedly between 20-40 percent. A GWAS data set, obtained from dbGap is utilised, which contains predominantly urban low-income African-American women who had normal and preterm deliveries. Epistatic interactions from original SNP sequences were extracted through a deep learning stacked autoencoder model and used to fine-tune a classifier for discriminating between term and preterm births observations. All models are evaluated using standard binary classifier performance metrics. The findings show that important information pertaining to SNPs and epistasis can be extracted from 4,666 raw SNPs generated using logistic regression (p-value = 5 * 10-3) and used to fit a highly accurate classifier model. The following results (Sen = 0.9562, Spec = 0.8780, Gini = 0.9490, Logloss = 0.5901, AUC = 0.9745, and MSE = 0.2010) were obtained using 50 hidden nodes and (Sen = 0.9289, Spec = 0.9591, Gini = 0.9651, Logloss = 0.3080, AUC = 0.9825, and MSE = 0.0942) using 500 hidden nodes. The results were compared with a Support Vector Machine (SVM), a Random Forest (RF), and a Fishers Linear Discriminant Analysis classifier, which all failed to improve on the deep learning approach. [ABSTRACT FROM AUTHOR]

This paper develops a novel hyperspectral image (HSI) classification framework by exploiting the spectral-spatial features of multiscale superpixels via recurrent neural networks with stacked autoencoders. The superpixels can be used to segment an HSI into shape-adaptive regions, and multiscale superpixels can capture the object information more accurately. Therefore, the superpixel-based classification methods have been studied by many researchers. In this paper, we propose a multiscale superpixel-based classification method. In contrast to current research, the proposed method not only captures the features of each scale but also considers the correlation among different scales via recurrent neural networks. In this way, the spectral-spatial information within a superpixel is more efficiently exploited. In this paper, we first segment the HSI from coarse to fine scales using the superpixels. Then, the spatial features within each superpixel and among superpixels are sufficiently exploited by the local and nonlocal similarity measure. Finally, recurrent neural networks with stacked autoencoders are proposed to learn the high-level multiscale spectral-spatial features. Experiments are conducted on real HSI datasets. The results demonstrate the superiority of the proposed method over several well-known methods in both visual appearance and classification accuracy. [ABSTRACT FROM AUTHOR]

Automated analysis of broadcast soccer game video is a challenging computer vision problem. Prior to performing high-level analysis (such as event detection), accurate classification of shot views and play-break segmentation are required to analyze the structure of soccer video. A novel deep network called parallel feature fusion network (PFF-Net) combines local and full-scene features to produce accurate shot view classification based on camera zoom and out-of-field status. Then, a novel hidden-to-observable Markov model (H2O-MM) is introduced to determine play/break status of the shots. Testing is performed using a variety of professional broadcast soccer videos. Variations of the PFF-Net are considered and compared with four existing methods where the PFF-Net demonstrates superior performance (92.6%). The H2O-MM has the accuracy of 98.7% for play-break segmentation, which is an improvement over two existing hidden Markov models. The new methods provide improved temporal labeling of broadcast soccer videos, which can be used to further improve overall automated event detection.

Soft sensors based on deep learning have been growing in industrial process applications, inferring hard-to-measure but crucial quality-related variables. However, applications may present strong non-linearity, dynamicity, and a lack of labeled data. To deal with the above-cited problems, the extraction of relevant features is becoming a field of interest in soft-sensing. A novel deep representative learning soft-sensor modeling approach is proposed based on stacked autoencoder (SAE), mutual information (MI), and long-short term memory (LSTM). SAE is trained layer by layer with MI evaluation performed between extracted features and targeted output to evaluate the relevance of learned representation in each layer. This approach highlights relevant information and eliminates irrelevant information from the current layer. Thus, deep output-related representative features are retrieved. In the supervised fine-tuning stage, an LSTM is coupled to the tail of the SAE to address system inherent dynamic behavior. Also, a k-fold cross-validation ensemble strategy is applied to enhance the soft-sensor reliability. Two real-world industrial non-linear processes are employed to evaluate the proposed method performance. The obtained results show improved prediction performance in comparison to other traditional and state-of-art methods. Compared to the other methods, the proposed model can generate more than 38.6% and 39.4% improvement of RMSE for the two analyzed industrial cases.

Currently, many types of traffic data from different advanced data collection techniques are available. Plenty of effort has been spent to take full advantage of the heterogeneous data to enhance the prediction accuracy of the model in the advanced travel information system. The objective of this study is to build a deep feature fusion model to predict space–mean–speed using heterogeneous data. The temporal and spatial features are defined as the raw input which are extracted and trained by stacked autoencoders in the first step. Then, the extracted representative features from the data are fused. Finally, prediction models can be developed to capture the correlations. Therefore, the prediction model can consider both temporal–spatial correlation and correlation of heterogeneous data. Using the real-world data, some machine learning models including artificial neural network, support vector regression, regression tree and k-nearest neighbor are implemented and compared. The best result can be obtained when deep feature fusion model and support vector regression are jointly applied. Moreover, we compare the new proposed deep feature–level fusion method with the widely used data–level fusion method. The results indicate that proposed deep feature fusion model can achieve a better performance. [ABSTRACT FROM AUTHOR]

• A deep-learning framework to localize and characterize acoustic emissions. • A single sensor approach for metallic panels with rivet-connected doublers. • A procedure to determine the optimal number of deep learning layers and nodes. • Sensitivity analyses to the number of sensors, input patterns, and output layers. • Experimental validations, comprehensive comparisons, and generalization assessments. This paper introduces a deep learning-based framework to localize and characterize acoustic emission (AE) sources in plate-like structures that have complex geometric features, such as doublers and rivet connections. Specifically, stacked autoencoders are pre-trained and utilized in a two-step approach that first localizes AE sources and then characterizes them. To achieve these tasks with only one AE sensor, the paper leverages the reverberation patterns, multimodal characteristics, and dispersive behavior of AE waveforms. The considered waveforms include AE sources near rivet connections, on the surface of the plate-like structure, and on its edges. After identifying AE sources that occur near rivet connections, the proposed framework classifies them into four source-to-rivet distance categories. In addition, the paper investigates the sensitivity of localization results to the number of sensors and compares their localization accuracy with the triangulation method as well as machine learning algorithms, including support vector machine (SVM) and shallow neural network. Moreover, the generalization of the deep learning approach is evaluated for typical scenarios in which the training and testing conditions are not identical. To train and test the performance of the proposed approach, Hsu-Nielsen pencil lead break tests were carried out on two identical aluminum panels with a riveted stiffener. The results demonstrate the effectiveness of the deep learning-based framework for single-sensor, AE-based structural health monitoring of plate-like structures. [ABSTRACT FROM AUTHOR]

This paper strives to construct the relevance between smartphone sensor data and personal health via a deep learning method. Firstly, the data captured by smartphone sensors is categorized into groups by deep stacked autoencoders (SAE), which is a traditional deep learning architecture with multi-layer sparse autoencoders for feature extraction and a softmax layer for classification. For instance, accelerometer data from smartphones can be classified in accordance with different user motion states such as sitting, standing, walking, and running. Secondly, the quantitative correlation between divided sensor data and individual health is built. For example, walking and running for a suitable amount of time are beneficial to human health. Finally, a simulation experiment is devised to verify the performance of the proposed method. A 0.15 percent gain on the overall classification accuracy for automatic feature extraction on the human action recognition (HAR) dataset has been achieved compared with the state of the art methods and 0.5 percent gain for manual feature extraction. It is also demonstrated by the experimental results that smartphone sensor data can reveal the health status of the user to some degree and smartphone can serve as an assistant for human health. [ABSTRACT FROM AUTHOR]

Hyperspectral remote sensing image includes hundreds of narrow contiguous spectral bands with high spectral resolution, and can provides a contiguous spectral curve for each pixel, which is an important tool for the cropland monitoring in rapid and large-scale way. However, due to the contradiction between spectral resolution and spatial resolution, hyperspectral remote sensing image usually possesses relative low spatial resolution. Therefore, it is very important to mix various vegetation and soil at one pixel point for spectral decomposition in hyperspectral image and spectral unmixing of farmland. The first step in spectral unmixing is usually endmember extraction. Endmember extraction from a hyperspectral remote sensing image is to find some pixels (endmembers) and regard them as pure spectral reflectance of the vegetation and soil in the image to get the best accuracy of spectral unmixing results. So endmember extraction from hyperspectral remote sensing image can be regarded as a typical discrete optimization problem, which can be solved by swarm intelligence optimization algorithm. Before optimization, the spectral dimension of the image should be reduced by deep learning. In order to solve the problem of spectral unmixing of hyperspectral images, a method of extracting farmland endmember based on deep learning and shuffled frog leaping algorithm (SFLA) is proposed in this paper. Firstly, deep learning model named stacked auto encoders (SAE) was used to extract spectral features . SAE performs a non-linear transfer from the original spectral signals to a form with significant features and less dimensions. In the low-dimension space, the candidate endmembers were selected as the input of the SFLA. The purpose of extracting the candidate endmembers is to simplified the computational complexity in the next step. Secondly, the endmember extraction of hyperspectral image was transferred into the combinatorial optimization and the objective function was constructed, then the SFLA was used to optimize the objective functionto get the best combination of endmembers. The objective function in study was designed as the RMSE (root mean square error) between the real hyperspectral remote sensing image and the simulated hyperspectral remote sensing image using the endmembers and abundance after endmember extraction and spectral unmixing. Thirdly, 2 groups experiments were carried out on the synthetic hyperspectral datasets with 3 different SNR (signal to noise ratio, 20 , 30 and 40 dB) and the real AVIRIS hyperspectral remote sensing dataset in Salinas region, respectively. In experiments, the experimental results of the proposed method was compared with that of 3 traditional methods for endmember extraction including the sequential maximum angle convex cone (SMACC), N-FINDR, Vertex Component Analysis (VCA). The results were evaluated by RMSE and spectral angle. The results showed that the RMSE was 0.050 8, 0.015 9, 0.005 1, 0.006 7 for 20, 30 and 40 dB dataset, and the real dataset, respectively. The average spectral angle was 0.106 88, 0.030 32, 0.009 94 for 20, 30 and 40 dB dataset respectively. The proposed method was better than traditional methods in terms of extraction accuracy, which had wide potential applications on cropland monitoring using hyperspectral remote sensing. The method proposed in this paper reduced the influence of the non-linear factors and the noise, better endmember extraction and spectral unmixing results (both less spectral angle and less RMSE) could be obtained, and the proposed method was robust when the noise of the image increased sharply. In conclusion, the endmember extraction method proposed by this study is of significant importance for the cropland monitoring using hyperspectral remote sensing and has a prosperous future for the application on the remote sensing of agriculture. [ABSTRACT FROM AUTHOR]

Abstract In recent years, some adaptive subspace models perform well for hyperspectral anomaly detection (AD). In this paper, a stacked autoencoders-based adaptive subspace model (SAEASM) is proposed. First, three windows, namely, inner, outer and dictionary window, centered at the test point are used to obtain the local background pixel points and dictionary in the hyperspectral image (HSI). Second, the deep features of differences between the test point and the local dictionary pixels are first acquired by the use of SAE architectures. Then, the deep features of differences between the local background pixels and the local dictionary pixels are also acquired by the use of SAE architectures. Finally, the detection result is obtained by the stacked autoencoders-based adaptive subspace model that is based on the 2-norm of the above two deep features. The experimental results carried out on real and synthetic HSI demonstrate that the proposed SAEASM generally performs better than the comparison algorithms. [ABSTRACT FROM AUTHOR]

In-line anomaly detection (AD) not only identifies the needs for semiconductor equipment maintenance but also indicates potential line yield problems. Prompt AD based on available equipment sensory data (ESD) facilitates proactive yield and operations management. However, ESD items are highly diversified and drastically scale up along with the increased use of sensors. Even veteran engineers lack knowledge about ESD items for automated AD. This paper presents a novel Spectral and Time Autoencoder Learning for Anomaly Detection (STALAD) framework. The design consists of four innovations: (1) identification of cycle series and spectral transformation (CSST) from ESD, (2) unsupervised learning from CSST of ESD by exploiting Stacked AutoEncoders, (3) hypothesis test for AD based on the difference between the learned normal data and the tested sample data, (4) dynamic procedure control enabling periodic and parallel learning and testing. Applications to ESD of an HDP-CVD tool demonstrate that STALAD learns normality without engineers’ prior knowledge, is tolerant to some abnormal data in training input, performs correct AD, and is efficient and adaptive for fab applications. Complementary to the current practice of using control wafer monitoring for AD, STALAD may facilitate early detection of equipment anomaly and assessment of impacts to process quality.

Relational aggregated search (RAS) is defined as a complementary set of approaches in which the relations between information nuggets are taken into account. From this viewpoint, the relational aggregated search should retrieve information nuggets and their relations, that are used to coherently assemble the final search result. Traditional approaches used for RAS are based on Information Extraction (IE) techniques and knowledge bases (ontologies, linked data) as the major sources for identifying and extracting useful relations between different results. However, with the big data collections stored on the web, the different results obtained from the different verticals for a given query are not homogeneous. Therefore, the challenge is to extract the different features related to each vertical result, and to recognize the various relationships between these results. In this paper, we propose a new solution based on a Deep Learning architecture, specifically, stacked autoencoders. This approach enables us to exploit the advantages of deep neural networks. Experimental results show that our model achieves good accuracy for 30 queries, and demonstrate that the use of stacked autoencoders for representation learning is more beneficial for clustering tasks and can considerably improve aggregated search results. [ABSTRACT FROM AUTHOR]

Learning results depend on the representation of data, so how to efficiently represent data has been a research hot spot in machine learning and artificial intelligence. With the deepening of the deep learning research, studying how to train the deep networks to express high dimensional data efficiently also has been a research frontier. In order to present data more efficiently and study how to express data through deep networks, we propose a novel stacked denoising sparse autoencoder in this paper. Firstly, we construct denoising sparse autoencoder through introducing both corrupting operation and sparsity constraint into traditional autoencoder. Then, we build stacked denoising sparse autoencoders which has multi-hidden layers by layer-wisely stacking denoising sparse autoencoders. Experiments are designed to explore the influences of corrupting operation and sparsity constraint on different datasets, using the networks with various depth and hidden units. The comparative experiments reveal that test accuracy of stacked denoising sparse autoencoder is much higher than other stacked models, no matter what dataset is used and how many layers the model has. We also find that the deeper the network is, the less activated neurons in every layer will have. More importantly, we find that the strengthening of sparsity constraint is to some extent equal to the increase in corrupted level. [ABSTRACT FROM AUTHOR]

Abstract Sleep is a physiological activity and human body restores itself from various diseases during sleep. It is necessary to get sufficient amount of sleep to have sound physiological and mental health. Nowadays, due to our present hectic lifestyle, the amount of sound sleep is reduced. It is very difficult to decipher the various stages of sleep manually. Hence, an automated system may be useful to detect the different stages of sleep. This paper presents a novel method for the classification of sleep stages based on RR-time series and electroencephalogram (EEG) signal. The method uses iterative filtering (IF) based multiresolution analysis approach for the decomposition of RR-time series into intrinsic mode functions (IMFs). The delta (δ), theta (θ), alpha (α), beta (β) and gamma (γ) waves are evaluated from EEG signal using band-pass filtering. The recurrence quantification analysis (RQA) and dispersion entropy (DE) based features are evaluated from the IMFs of RR-time series. The dispersion entropy and the variance features are evaluated from the different bands of EEG signal. The RR-time series features and the EEG features coupled with the deep neural network (DNN) are used for the classification of sleep stages. The simulation results demonstrate that our proposed method has achieved an average accuracy of 85.51%, 94.03% and 95.71% for the classification of 'sleep vs wake', 'light sleep vs deep sleep' and 'rapid eye movement (REM) vs non-rapid eye movement (NREM)' sleep stages. [ABSTRACT FROM AUTHOR]