Search

Estimating the full state of physical systems, including thermal and flow status, from sparse measurements of limited sensors is a critical technology for perception and control. Neural networks have been used in recent studies to reconstruct the global field in a supervised learning paradigm. However, these studies encounter two major challenges: the lack of interpretability of black-box models and performance bottleneck caused by network structure and parameter optimization. This paper proposes a hybrid method based on proper orthogonal decomposition (POD) and deep neural networks (DNNs) to further enhance the interpretability and accuracy of flow and heat field reconstruction. The key idea is to leverage the inherent data modes extracted by POD that capture essential features in physical fields, and formulate the reconstruction problem as finding an optimal linear combination of dominant POD modes. To reduce the error introduced by underfitting and model structure, this paper estimates the coefficients of POD modes by establishing and solving a linear optimization problem that minimizes the gap between the recovered field and the exact measurements, rather than employing regression models. However, the underdetermined issue cased by the sparse measurements restricts the optimization problem to obtain a proper solution. To alleviate this problem, this paper presents to utilize the powerful non-linear approximation ability of DNNs to produce a reference field as auxiliary observations, which combines exact measurements to jointly constrain the optimization problem solving. Finally, the global physical field is reconstructed by superposing dominant POD modes weighted with the solved coefficients. By combining with POD technology, the proposed method can also improve the performance of neural networks on reconstruction problems with large-scale and irregular domains. The experiments conducted on the fluid and thermal benchmarks demonstrate that the proposed method can significantly boost neural network reconstruction performance and outperform existing POD-based methods. • A hybrid method based on POD and deep neural networks is developed for flow and heat field reconstruction from sparse measurements. • The data modes extracted by POD are explicitly used to alleviate black-box problems of neural networks. • The field reconstruction problem is converted to a linear optimization jointly constrained by exact measurements and DNN predictions. • The underdetermined issues with sparse measurements can be addressed. [ABSTRACT FROM AUTHOR]

• A scheme to recognize the distraction level was proposed. • MLP-AutoEncoder and DPC algorithms were used to define distraction-level labels. • Factors affecting driver distraction level were explored with mixed-model analyses. • A distraction-level prediction model with stacking ensemble learning was made. Driver distraction-level recognition while performing secondary tasks in full-touch in-vehicle information systems (FTIVISs) is essential for the harmonious co-driving of human and intelligent vehicle systems. However, there has been little research on this topic. To respond to this issue, this paper proposes a distraction-level recognition framework with a combination of semi-supervised learning, unsupervised learning, and supervised learning. First, unsupervised learning and semi-supervised is used to divide the collected unlabeled samples of driving distraction behavior into three categories of distraction levels: high, medium, and low. Second, the factors influencing the distraction level are explored through a mixed model analysis. Finally, a stacking-based ensemble learning model is proposed to recognize the driver distraction level by supervised learning, with the influencing factors of the distraction level used as model input parameters. To improve the recognition performance of the stacking model, four heterogeneous classifiers selected as the base classifiers, and principal component analysis (PCA) is introduced in the base classifier layer, which improves the ability of the meta -classifier to process raw features based on overfitting resistance. We conducted a real road experiment under different road and FTIVIS task conditions, and the proposed model performed better than traditional machine learning models. In addition, the model exhibited the greatest advantage when the deep neural network (DNN) algorithm was used as the meta -classifier of the model, with a recognition accuracy of 92.5%. The study findings are significant for developing a human–machine co-driving control strategy and improving vehicle driving safety. [ABSTRACT FROM AUTHOR]

Lithology identification is a crucial task for reservoir characterization and evaluation. There exists an intricate non-linear response between formation lithology and logging data. However, it is difficult to avoid lithology mislabeling due to human error and interpretation coarsening, and label quality can seriously affect the effectiveness of supervised learning. The presence of noisy labels makes it essential to learn with noisy labels. Noise-filtering methods and noise-robust algorithms only concentrate on a singular aspect of data or algorithm. In this paper, hybrid noise label filtering and correction framework for lithology identification (HNFCL) is proposed. Isolation forest is utilized to detect suspicious data, as it is efficient and fast. Baseline classifiers are built by ensemble tree models. In particular, the labels of abnormal data are removed and Tri-training semi-supervised method is introduced to relabel these data, which minimizes the loss of valid training data. Comprehensive experiments of the HNFCL framework, noise filtering methods and deep neural network methods with optimized loss functions were carried out in the industrial application of logging lithology identification. HNFCL achieved average accuracy of 87.94% and 94.93% in two study wells. These results outperformed the noise filtering methods and showed no significant difference from the state-of-the-art method. The correction of noise by HNFCL will provide a prospect for lithology identification applications. [ABSTRACT FROM AUTHOR]

3D point clouds are a crucial type of data collected by LiDAR sensors and widely used in transportation applications due to its concise descriptions and accurate localization. Deep neural networks (DNNs) have achieved remarkable success in processing large amount of disordered and sparse 3D point clouds, especially in various computer vision tasks, such as pedestrian detection and vehicle recognition. Among all the learning paradigms, Self-Supervised Learning (SSL), an unsupervised training paradigm that mines effective information from the data itself, is considered as an essential solution to solve the time-consuming and labor-intensive data labeling problems via smart pre-training task design. This paper provides a comprehensive survey of recent advances on SSL for point clouds. We first present an innovative taxonomy, categorizing the existing SSL methods into four broad categories based on the pretexts' characteristics. Under each category, we then further categorize the methods into more fine-grained groups and summarize the strength and limitations of the representative methods. We also compare the performance of the notable SSL methods in literature on multiple downstream tasks on benchmark datasets both quantitatively and qualitatively. Finally, we propose a number of future research directions based on the identified limitations of existing SSL research on point clouds. [ABSTRACT FROM AUTHOR]

Illegal practices that cause stock prices to vary from their fair values are known as stock price manipulations. Our prior study used unsupervised deep learning to detect these unlawful activities, in which the autoencoder model could catch five out of six cases in the Stock Exchange of Thailand (SET). Trading actions of good governance equities in a limit order book format were used to train the model, and abnormalities were viewed as manipulation. This unsupervised technique has the strength of discovering never-before-seen patterns, although it has a low recall rate. In this work, the findings were improved by incorporating knowledge about the popular pump-and-dump pattern into another supervised deep learning model, which has a strength in catching known patterns. To combine the advantages of both models, an ensemble of supervised and unsupervised deep neural networks was proposed. The main contribution of this paper is to demonstrate that the new ensemble model achieved a higher detection rate than the unsupervised model alone. The supervised learning model used was a long short-term memory (LSTM) network, the unsupervised learning model used was an LSTM-based autoencoder, and the stack classifier in the ensemble model was a support vector machine (SVM). The proposed method was evaluated on six real manipulation cases from the SET. The empirical results showed that our ensemble model could improve the detection strength, achieved high accuracy on pump-and-dump unseen data, and could identify six out of six real manipulation cases with low false alarm rates. [ABSTRACT FROM AUTHOR]

Semi-supervised learning (SSL) has drawn much attention since it can alleviate the predicament in which only limited labels can be accessed, with the help of numerous unlabeled data. Many deep neural networks (NNs) based SSL methods have been proposed recently, which mostly focus on image classification tasks, while fewer efforts have been taken to tabular data related regression scenarios. In this paper, to handle the semi-supervised regression problem for tabular data with NNs based methods, we empirically observe the demerits of previous methods and present a novel framework, to explore the combination of self teaching and mutual teaching. For one thing, self teaching is employed to each base learner by using consistency regularization so that the model is driven to be more stable and also robust to local perturbations. For another, the knowledge of other base learners is extracted and filtered to perform mutual teaching to avoid the confirmation bias problem and boost the training of each base learner. Here, we employ an uncertainty-based strategy for the filtering of knowledge in mutual teaching. Extensive experiments are performed on multiple real-world datasets to demonstrate the effectiveness of previous methods and the proposed framework, and further analyses are conducted to understand the influence of each factor in the proposed framework. • Data augmentation tricks may be unapplicable for tabular data. • The combination of self and mutual teaching for SSL regression tasks is explored. • Self teaching improves performance by enhancing robustness and stability. • Mutual teaching provides help by avoiding confirmation bias. • Experiments verify the effectiveness of the method. [ABSTRACT FROM AUTHOR]

High-accuracy temperature sensors usually come at a high cost, making them cost-ineffective solutions for mass-produced commercial products. The overall product cost can be one of the decisive factors for its success (e.g., commercial freezers). On the other hand, low-cost temperature sensors are viable options, but they suffer from accuracy issues. This paper presents a system aimed at improving the accuracy of a low-cost temperature sensor using a hybrid machine learning algorithm. The system utilizes a low-cost single-board computer connected to a series of sensors to collect data. These data are then used as an input vector to the proposed linear regression heterogeneous hybrid extreme learning machine (LR-HE-HyELM), a supervised learning algorithm responsible for increasing the sensors' accuracy. The proposed linear regression heterogeneous extreme learning machine (LR-HE-HyELM) combines linear regression with the self-adaptive heterogeneous hybrid extreme learning machine (SA-He-HyELM) algorithm. SA-He-HyELM is an extreme learning machine (ELM) based algorithm that utilizes a genetic algorithm (GA) for the creation of an optimized single-layer neural network (SLNN) having different combinations of custom-created neurons at its hidden layer. The proposed training algorithm has all the advantages of SA-He-HyELM. These advantages include utilizing low-order and higher-order neurons in the same network while retaining ELM's simplicity by making all GA parameters self-adaptive. The utilization of low-order and higher-order unit types in the same network resulted in optimized neural networks with better generalization ability than those with the same neuron types in their hidden layer. The creation of SA-He-HyELM was nature-inspired since the human brain contains different neuron types. The proposed system is tested with eight existing methods, with most of them having different neuron types in the hidden layer. The experimental results involved using a custom-created regression dataset having measurements taken from a commercial fridge. The proposed method managed to get the lowest mean square error (MSE) compared to the other eight methods. • The proposed approach retains extreme learning machine's simplicity. • The use of software means improves temperature sensors' accuracy in fridges. • The proposed algorithm makes all genetic algorithm parameters self-adaptive. [ABSTRACT FROM AUTHOR]

• First attempt using image-based methods for iron ore load estimation. • The problem is modelled as a weakly supervised learning problem. • A new two-stage modelling for training neural networks are proposed. • Empirical evidence of experiments shows good performance and economic boost. Iron ore feed-load control is one of the most critical settings in a mineral grinding process. It has direct impact on the quality of final iron products. The setting of the feed load heavily replies the characteristics of the ore pellets. However, such characteristics are challenging to acquire in many production environments, requiring speical equipments and complicated modelling process with a high cost. To provide an low-cost and easier-to-implement solution, in this paper, we present our work on using deep learning models for direct ore feed load estimation from ore pellet images. To address the challenges caused by the large size of ore images and the shortage of accurately annotated data, we proposed to use a weakly supervised learning apporach with a two-stage model training algorithm and two neural network architectures developed. The experiment results show competitive model performance, and the trained models can be used for real-time feed load estimation for grind process optimisation. [ABSTRACT FROM AUTHOR]

Open Relation Extraction (ORE) task remains a challenge to obtain a semantic representation by discovering arbitrary relations from the unstructured text. Conventional methods heavily depend on feature engineering or syntactic parsing, which are inefficient or error-cascading. Recently, leveraging supervised deep learning methods to address the ORE task is a promising way. However, there are two main challenges: (1) The lack of enough labeled corpus to support supervised training; (2) The exploration of specific neural architecture that adapts to the characteristics of open relation extracting. In this paper, we build a large-scale, high-quality training corpus in a fully automated way. And wedesign a tagging scheme to assist in transforming the ORE task into a sequence tagging processing. Furthermore, we propose a hybrid neural network model (HNN4ORT) for open relation tagging. The model employs the Ordered Neurons LSTM to encode potential syntactic information to capture the associations among the arguments and relations. It also emerges a novel Dual Aware Mechanism, including Local-aware Attention and Global-aware Convolution. The dual awarenesses complement each other. Takes the sentence-level semantics as a global perspective, and at the same time, the model implements salient local features to achieve sparse annotation. Experiment results on various testing sets show that our model achieves state-of-the-art performance compared toconventional methods or other neural models. • We complete Open relation extraction task by the overlap-aware sequence tagging. • We propose a Hybrid neural network model with the Dual Aware Mechanism. • We automatically build a large-scale, high-quality corpus of open relation. • Experiments show that our model achieves state-of-the-art performance. [ABSTRACT FROM AUTHOR]

• Deep learning algorithm for identification of type and flight mode of detected drones. • Novel deep learning algorithm for detection of multiple drones in radio frequency domain. • Results of three indoor experiments concerning collecting and analyzing radio signals from drones. • Public available database with radio signals of drones from two different radio frequency bands. Unmanned aerial systems, especially drones have gone through remarkable improvement and expansion in recent years. Drones have been widely utilized in many applications and scenarios, due to their low price and ease of use. However, in some applications drones can pose a malicious threat. To diminish risks to public security and personal privacy, it is necessary to deploy an effective and affordable anti-drone system in sensitive areas to detect, localize, identify, and defend against intruding malicious drones. This research article presents a new publicly available radio frequency drone dataset and investigates detection and identification methodologies to detect single or multiple drones and identify a single detected drone's type. Moreover, special attention in this paper has been underlined to examine the possibility of using deep learning algorithms, particularly fully connected deep neural networks as an anti-drone solution within two different radio frequency bands. We proposed a supervised deep learning algorithm with fully-connected deep neural network models that use raw drone signals rather than features. Regarding the research results, the proposed algorithm shows a lot of potentials. The probability of detecting a single drone is 99.8%, and the probability of type identification is 96.1%. Moreover, the results of multiple drones detection demonstrate an average accuracy of 97.3%. There have not been such comprehensive publications, to this time, in the open literature that have presented and enlightened the problem of multiple drones detection in the radio frequency domain. [ABSTRACT FROM AUTHOR]