Search

Art has become an important new investment vehicle. Thus, interest is growing in art price prediction as a tool for assessing the returns and risks of art investments. Both traditional statistical methods and machine learning methods have been used to predict art prices. However, both methods incur substantial human costs for data preprocessing for the construction of prediction models, necessitating a reduction in the workload. In this study, we propose the zero-shot classification method to perform automatic annotation in data processing for art price prediction by leveraging large language models (LLMs). The proposed method can perform annotation without new training data. Thus, it minimizes human costs. Our experiments demonstrated that the 4-bit quantized Llama-3 70B model, which can run on a local server, achieved the most accurate (over 0.9) automatic annotation of different art forms using LLMs, performing slightly better than the GPT-4o model from OpenAI. These results are practical for data preprocessing and comparable with the results of previous machine learning methods.

In contemporary society, the rapid progression of urbanization and technological advancements has led to a substantial increase in the number of vehicles, consequently elevating the rate of traffic accidents. To minimize the human and economic losses resulting from these accidents, extensive research has been conducted over the past decades. Recent studies utilizing Grid Maps and time-series methods have shown promising results in identifying factors related to traffic accidents and predicting accident occurrence rates. However, most existing research employs data without thoroughly analyzing its definitive correlation with traffic accidents, focusing solely on an overarching integration process or exclusively considering road conditions, while neglecting environmental factors surrounding the roads. Therefore, this paper focuses on a detailed analysis of factors contributing to traffic accidents in Seoul, the capital city of South Korea, limiting the study period from January 2020 to December 2021. The data encompasses various aspects such as traffic accidents, weather conditions, standard node links, traffic speed, Points of Interest (POI), solar altitude and azimuth, speed bumps, and traffic surveillance cameras. Most of this data is available from national public institutions, with some being computed through specific formulas. Since weather and POI data possess a wide range of features, a Pearson correlation analysis is conducted to extract features relevant to traffic accidents. The extracted features are then compiled and normalized to facilitate the deep learning model’s training, thereby constructing a comprehensive dataset.

Electromyography (EMG) is used to recognize user finger gestures for applications in real-time interfaces. Finger movements are classified by preprocessing to extract the features from the collected EMG data, which are then used for machine learning. The data were extracted using the overlapped segmentation method to ensure sufficient training data. The preprocessing of EMG data uses standard formulae, such as integrated EMG (IEMG) and mean absolute value (MAV). Furthermore, preprocessing involves using original data, simple moving average (SMA), and Fast Fourier transform (FFT) for feature extraction. Subsequently, these preprocessed data sets are used to train machine learning models, facilitating a comparative analysis. Four machine learning models were used: eXtreme Gradient Boost, Random Forest, k-Nearest Neighbors, and Logistic Regression. The experimental results revealed the best accuracy from preprocessing using a simple moving average followed by a Fourier transform, but classification was not possible using all nine finger movements. On the other hand, it showed more than 90% accuracy because the model learned by reducing it to a specific finger gesture. Rest movements, index finger taps, and force-taps movements achieved the highest accuracy, approximately 95%.

The prediction of loan defaults is crucial for banks and financial institutions due to its impact on earnings, and it also plays a significant role in shaping credit scores. This task is a challenging one, and as the demand for loans increases, so does the number of applications. Traditional methods of checking eligibility are time-consuming and laborious, and they may not always accurately identify suitable loan recipients. As a result, some applicants may default on their loans, causing financial losses for banks. Artificial Intelligence, using Machine Learning and Deep Learning techniques, can provide a more efficient solution. These techniques can use various classification algorithms to predict which applicants will likely be eligible for loans. This study uses five Machine Learning classification algorithms (Gaussian Naive Bayes, AdaBoost, Gradient Boosting, K Neighbors Classifier, Decision Trees, Random Forest, and Logistic Regression) and eight Deep Learning algorithms (MLP, CNN, LSTM, Transformer, GRU, Autoencoder, ResNet, and DenseNet). The use of Ensemble Methods and SMOTE with SMOTE-TOMEK Techniques also has a positive impact on the results. Four metrics are used to evaluate the effectiveness of these algorithms: accuracy, precision, recall, and F1-measure. The study found that DenseNet and ResNet were the most accurate predictive models. These findings highlight the potential of predictive modeling in identifying credit disapproval among vulnerable consumers in a sea of loan applications.

The Internet of Things (IoT) is transforming how we live and work, and its applications are widespread, spanning smart homes, industrial monitoring, smart cities, healthcare, agriculture, and retail. Considering its wide range of applications, addressing the security challenges arising from IoT devices’ massive collection and transmission of user data is vital. Intrusion detection systems (IDS) based on deep learning techniques offer new means and research directions for resolving IoT security issues. Deep learning models can process large volumes of data and extract complex patterns, making them generally more effective than traditional rule based IDSs. While deep learning techniques are gradually gaining popularity in IDS applications, current research needs a comprehensive summary of deep learning-based IDS in IoT. This paper introduces intrusion detection technologies, followed by a detailed comparison, analysis, and discussion of deep learning models, datasets, feature extraction and classifiers, data preprocessing techniques, and experimental design of the models. It also highlights the challenges and issues associated with deep learning models and relevant techniques for IDS. Finally, it concludes by providing recommendations to assist researchers in this domain.

Random splitting strategy is a common approach for training, testing, and validating object detection algorithms based on deep learning. Is common for datasets to have images extracted from video sources, in which there are frames with high spatial correlation, i.e., frames with rotated positions or different view angles of the same object. These highly correlated frames may lead to information leakage in training, if these frames are not well-distributed. In this work, it is shown that datasets created with highly spatial correlation frames from the same video have information leakage if using the random splitting strategy to distribute the image into the sub-datasets. It proposed a clustering dataset split algorithm in which images are distributed randomly in the sub-datasets in a pack or clusters instead of a single image at the time. The clusters are created by extracting the image features from a video of the dataset using an image-text pre-trained model, CLIP, and reducing the feature vector dimensionality with t-Distributed Stochastic Neighbor embedding (t-SNE). In this reduced dimensional representation, images are separated into clusters using a clustering algorithms like DBSCAN, OPTICS, and Agglomerative Clustering. These clusters are distributed into the train, test, and validation datasets randomly to avoiding information leakage by highly spatial correlation frames. YOLOv8 is used as the object detector algorithm to test the dataset splitting.

Realistic anomalous data noise is usually locally distributed in nature. To address the challenge posed by clustered anomalous noise in time series data, which not only reduces the accuracy of forecasting models but also presents a unique challenge due to its clustered nature and existing methods’ limitations in handling such noise, this study introduces a novel denoising algorithm leveraging an optimized Long Short-Term Memory (LSTM) network, named ‘Denoising Time Window Optimized LSTM’ (DTW-LSTM), which innovatively identifies and rectifies clustered anomalous noise in time series data, significantly enhancing prediction accuracy by adeptly balancing the denoising process and data integrity preservation, as evidenced by comprehensive experimental validations. The algorithm employs a sliding time window mechanism to analyze the original series and its Fourier approximation, identifying clustered anomalies through a clustering model. It then uses a symmetrical data mixing and crossover technique to enhance the data quality in noisy segments. This process preserves the integrity of time series data by maintaining a balance between the original and fitted series, leading to a significant improvement in prediction accuracy. Experimental results indicate that the algorithm can adeptly discern the presence of clustered anomalous noise within the current time window and effectively rectify sections of data containing such noise. By surpassing the performance benchmarks set by contemporary forecasting models, this algorithm has established itself as the new SOTA method in the field of time series forecasting. Each sub-model contributes to the superior accuracy and reliability of the algorithm.

Optimizing machine learning (ML) model performance relies heavily on appropriate data preprocessing techniques. Despite the widespread use of standardization and normalization, empirical comparisons across different models, dataset sizes, and domains remain sparse. This study bridges this gap by evaluating five machine learning algorithms- Support Vector Machine (SVM), Logistic Regression (LR), Random Forest (RF), Extreme Gradient Boosting (XGBoost), and Adaptive Boosting (AdaBoost)- on datasets of varying sizes from the business, health, and agriculture domains. This study assessed the models without scaling, with standardized data, and with normalized data. The comparative analysis reveals that while standardization consistently improves the performance of linear models like SVM and LR for large and medium datasets, normalization enhances the performance of linear models for small datasets. Moreover, this study employs SHapley Additive exPlanations (SHAP) summary plots to understand how each feature contributes to the model’s performance interpretability with unscaled and scaled datasets. This study provides practical guidelines for selecting appropriate scaling techniques based on the characteristics of datasets and compatibility with various algorithms. Finally, this investigation laid the foundation for data preprocessing and feature engineering across diverse models and domains which offers actionable insights for practitioners.

In the era of digital transformation, customer feedback has become crucial for improving service quality. This study aims to enhance customer satisfaction through sentiment analysis utilizing machine learning techniques, with additional case studies conducted to ensure comprehensive method validation. Traditional sentiment analysis methods frequently fail to manage the complexity and volume of feedback data, yielding to less accurate insights. To address this challenge, we analyzed six machine learning models: Naïve Bayes, Support Vector Machine (SVM), Long Short-Term Memory (LSTM), Random Forest, AdaBoost, and BERT, with a particular focus on BERT. Our results demonstrate that BERT outperformed the other models in terms of both accuracy and processing speed, achieving an accuracy of up to 95%. The excellence of BERT in managing large and complex datasets provides a more precise sentiment analysis, which can significantly improve service quality and customer loyalty, while increasing company revenue by up to 15%. This research advances to the field of sentiment analysis by validating the effectiveness of BERT over traditional models through extensive comparative analysis and practical case studies.

The increasing prevalence of diabetes necessitates the development of effective early detection methods to mitigate its health impacts. This paper investigates the impact of feature transformation and machine learning (ML) models on the early detection of diabetes using a binary tabular classification dataset. We explore three feature transformation techniques, no transformation, normalization, and min-max scaling, to assess their influence on the performance of various ML models. To comprehensively evaluate the effectiveness of these preprocessing techniques, we experimented with twelve different ML models, including both traditional algorithms and ensemble methods. A publicly available dataset has been used for this research, containing 768 samples and 8 features. To ensure their effectiveness, the models are assessed using several evaluation metrics, including accuracy, precision, recall, F1-score, and Area Under the Receiver Operating Characteristic Curve (AUC-ROC). Among the ML models, Light Gradient Boosting Machine (LGBM) achieved the highest accuracy of 82.91% when min-max scaling was applied to the data. Our results demonstrate the varying effectiveness of different combinations of feature transformation techniques and ML models in enhancing diabetes detection performance. Furthermore, it has been observed that the ensemble models generally achieved better performance than traditional ML models. These findings provide valuable insights for optimizing preprocessing and model selection strategies in the development of robust early diabetes detection systems.

Progressively more advanced and complex models are proposed to address problems related to computer vision, forecasting, Internet of Things, Big Data and so on. However, these disciplines require preprocessing steps to obtain meaningful results. One of the most common problems addressed in this stage is the presence of missing values. Understanding the reason why missingness occurs helps to select data imputation methods that are more adequate to complete these missing values. Missing at Random synthetic generation presents challenges such as achieving extreme missingness rates and preserving the consistency of the mechanism. To address these shortcomings, three new methods that generate synthetic missingness under the Missing at Random mechanism are proposed in this work and compared to a baseline model. This comparison considers a benchmark covering 33 data sets and five missingness rates $(10\%, 20\%, 40\%, 60\%, 80\%)$ . Seven data imputation methods are compared to evaluate the proposals, ranging from traditional methods to deep learning methods. The results demonstrate that the proposals are aligned with the baseline method in terms of the performance and ranking of data imputation methods. Thus, three new feasible and consistent alternatives for synthetic missingness generation under Missing at Random are presented.

The number of deaths from lung cancer reached 1.8 million in 2020, ranking first among all cancers. Early diagnosis has been found to improve the survival rate of lung cancer patients after treatment in clinical care. Computed tomography (CT) is a technique commonly used in the early detection of lung cancer to determine the benignity or malignancy of lung nodules. Manual analysis of CT results is less efficient and its accuracy is affected by physicians’ experience levels. Segmenting lung nodules in a computer-aided diagnosis (CAD) system can effectively improve the efficiency and accuracy of the diagnosis. In this paper, we evaluate several deep learning segmentation models (including UNet, SegNet, GCN, FCN, DeepLabV3+, PspNet TransUNet, SwinUNet) and examine the effects of different preprocessing methods on the models to explore the best preprocessing and training strategies for lung nodule segmentation. Specifically, we investigate the effects of two different data preprocessing methods (adding a lung mask and croping the region of interest) on the segmentation results, where better segmentation results are achieved by including the nodal data of the region of interest without the lung mask. Through a comprehensive comparison, TransUNet achieves the best segmentation accuracy, with DICE indices of 0.887, 0.871, 0.75, and 0.744 tested on four datasets, respectively.

Advancements in Internet of Things devices allow for the collection and analysis of big data. Moreover, cloud computing has emerged as an ideal platform for big data analysis because it quickly provides computing resources for analysis at scale. However, concerns exist regarding the privacy and security of this information. Secure computation is a technology that enables computation while maintaining data encryption. In this study, we present a new secure computation based on a client-server model, in which a set of servers perform computations using the inputs of multiple clients. We adopt the $(k, n)$ threshold secret sharing approach, where an input $s$ is divided into $n$ shares and can be recovered from shares with a threshold $k$ . However, conventional secure computation using $(k,n)$ threshold secret sharing generally requires the condition $n \geq 2k-1$ and communication among multiple servers for each multiplication. To the best of our knowledge, no previous study has solved this problem completely. We demonstrate that processes that require communication can be concentrated in the preprocessing phase and propose a secure computation using $(k,n)$ threshold secret sharing that does not require communication during the secure computation phase, even when multiplication is performed. Moreover, we show that the number of communications does not depend on the number of multiplications, unlike in conventional methods. As communication often requires more processing time than the actual secure computation, our method makes it possible to realize faster overall processing than conventional methods. We also present an extensive security analysis and experimental simulation of the proposed method. Our proposed method achieves information-theoretic security against semi-honest adversaries under certain conditions with $n < 2k-1$ .

Coronary Artery Disease (CAD) is an increasingly prevalent disorder that significantly affects both longevity and quality of life, particularly among people aged 30 to 60. Lifestyle, genetics, nutrition, and stress are contributing factors to the increasing mortality rates. Therefore, there is a need for low-cost automated technology to diagnose CAD early and help medical practitioners treat chronic illnesses effectively. However, machine learning methods are typically designed to perform well with large datasets and may not be well-suited for smaller clinical datasets that contain categorical features. To address this issue, alternative approaches such as Data Preprocessing (DP), feature selection techniques, and Hyperparameter tuning (HP) are necessary to achieve optimal performance and accuracy on such datasets. Data preprocessing is also crucial to obtain accurate results by eliminating noise, handling missing values, and dealing with outliers. To address the challenges associated with feature selection, manual selection of hyperparameter tuning, and optimization, we have developed a novel model called BSOXGB (BorutaShap feature selection based Optuna hyperparameter tuning of eXtremely Gradient Boost). The proposed model achieves a remarkable accuracy of 97.70%, outperforming other classifiers like Random Forest (RF), AdaBoost (AB), CatBoost (CB), and ExtraTrees (ET) on the publicly available Z-Alizadeh Sani dataset. BSOXGB, with only 9 relevant features out of 56, has the highest classification accuracy, demonstrating its potential as a practical solution for automatically detecting and diagnosing CAD in the real world.

Intrusion detection and classification have gained significant attention recently due to the increased utilization of networks. For this purpose, there are different types of Network Intrusion Detection System (NIDS) approaches developed in the conventional works, which mainly focus on identifying the intrusions from the datasets with the help of classification techniques. Still, it is limited by the significant problems of inefficiency in handling large dimensional datasets, high computational complexity, false detection, and more time consumption for training the models. To solve these problems, this research intends to develop an innovative clustering-based classification methodology to precisely detect intrusions from the different types of IDS datasets. Here, the most recent and extensively used IDS datasets such as NSL-KDD, CICIDS, and Bot-IoT have been employed for detecting intrusions. Data preprocessing has been performed to normalize the dataset to eliminate irrelevant attributes and organize the features. Then, the data separation is applied by forming the clusters by using an intelligent Anticipated Distance-based Clustering (ADC) incorporated with the Density-Based Spatial clustering of applications with noise (DBScan) algorithm. It helps to find the distance and density measures for grouping the attributes into the clusters, which increases the efficiency of classification. Here, the most suitable optimal parameters are selected using the Perpetual Pigeon Galvanized Optimization (PPGO) technique. The extracted features are used for training and testing the dataset samples. Consequently, the Likelihood Naïve Bayes (LNB) classification approach is implemented to accurately predict the classified label as to whether normal or attack. During the evaluation, the performance of the proposed IDS framework is validated and compared using various evaluation metrics. The results show that the proposed ADC-DBScan-LNB model outperforms the other techniques with improved performance outcomes.

Data preprocessing is crucial in the Machine Learning pipeline because the models’ learning ability directly affects the quality of data and the underlying information acquired from this stage. Nevertheless, surprisingly, there are many alternatives for each transformation task, which makes an inexperienced user overwhelmed. A simple Python-based Auto-preprocessing architecture for Automated Machine Learning is developed to offer automated, interactive, and data-driven support to help the users perform data preprocessing tasks efficiently. The suggested method provides valuable insights into a dataset and can handle standard data preprocessing tasks adeptly. Initially, it detects the data problem and presents it to the end-user using compelling visualizations. Then, it recommends the most effective data cleaning and preparation method to the user after evaluating the state-of-the-art candidate techniques. For evaluation, the proposed architecture is employed on ten different and diverse datasets for automatic data preprocessing before passing it to an ML algorithm. The results are then compared with the results generated by the same ML algorithm but implemented on manually preprocessed data. The results have shown that not only did this approach make the whole process uncomplicated and facile, but it was also able to improve the performance of the model significantly.

In this paper, a brief survey of data preprocessing methods is presented. Specifically, the data preprocessing methods used in the smart grid (SG) domain are surveyed. Also, with the advent of SG, data collection on a large scale became possible. The data is essential for electricity demand, generation and price forecasting, which plays an important role in making energy efficient decisions, and long and short term predictions regarding energy generation, consumption and storage. However, the forecasting accuracy decreases when data is used in raw form. Hence, data preprocessing is considered essential. This paper provides an overview of the data preprocessing methods and a detailed discussion of the methods used in the existing literature. A comparison of the methods is also given. A survey of closely related survey papers is also presented and the papers are compared based on their contributions. Moreover, based on the discussion of the data preprocessing methods, a narrative is built with a critical analysis. Finally, future research directions are discussed to guide the readers.

Missing data are a universal data quality problem in many domains, leading to misleading analysis and inaccurate decisions. Much research has been done to investigate the different mechanisms of missing data and the proper techniques in handling various data types. In the last decade, machine learning has been utilized to replace conventional methods to address the problem of missing values more efficiently. By studying and analyzing recently proposed methods using machine learning approaches, vital adoptions in accuracy, performance, and time consumed can be highlighted. This study aimed to help data analysts and researchers address the limitations of machine learning imputation methods by conducting a systematic literature review to provide a comprehensive overview of using such methods to impute missing values. Novel proposed machine learning approaches used for data imputation are analyzed and summarized to assist researchers in selecting a proper machine learning method based on several factors and settings. The review was performed on research studies published between 2016 and 2021 on adopting machine learning to impute missing values, focusing on their strengths and limitations. A total of 684 research articles from various scientific databases were analyzed using search engines, and 94 of them were selected as primary studies. Finally, several recommendations were given to guide future researchers in applying machine learning to impute missing values.

Spot welding is a crucial process step in various industries. However, classification of spot welding quality is still a tedious process due to the complexity and sensitivity of the test material, which drain conventional approaches to its limits. In this article, we propose an approach for quality inspection of spot weldings using images from laser thermography data. We propose data preparation approaches based on the underlying physics of spot-welded joints, heated with pulsed laser thermography by analyzing the intensity over time and derive dedicated data filters to generate training datasets. Subsequently, we utilize convolutional neural networks to classify weld quality and compare the performance of different models against each other. We achieve competitive results in terms of classifying the different welding quality classes compared to traditional approaches, reaching an accuracy of more than 95 percent. Finally, we explore the effect of different augmentation methods.

Medical experts employ electroencephalography (EEG) for analyzing the electrical activity in the brain to infer disorders. However, the time costs of human experts are very high, and the examination of EEGs by such experts, therefore, accounts for a plethora of medical resources. In this study, an improved one-dimensional CNN-only system of 25 layers has been proposed to identify abnormal and normal adult EEG signals using a single EEG montage without using any explicit feature extraction technique. Most of the previous systems based on deep learning, that have been proposed to solve this problem, use extremely deep architectures containing very large numbers of layers. This study also presents an independent preprocessing module that has been exhaustively evaluated for optimal parameters with the target of adult EEG signal classification. The achieved accuracy of the proposed classifier as a part of the decision support system is 82.24%, which is a substantial improvement of ~3% over the previous best reported classifier of comparable depth. The system also exhibits significantly higher F1-score and sensitivity as well as lower loss. The proposed system is intended to be a part of an expert system for overall brain health evaluation.

Optical motion capture (MOCAP) is a commonly used technology to record the motion of non-rigid objects with high accuracy in 3D space. However, the MOCAP data has to be processed further before it can be used. The scattered reconstructed motion data must constitute a human configuration by labelling process according to the predefined template, and the missing markers have to be reconstructed to produce a stable motion trajectory. In this work, we propose a novel labelling method for motion sequences. First, a novel graph matching method is employed to determine the connection relationship of the scattered motion data for a single frame. Then, Kalman filtering is used for tracking in the motion sequence. As for the challenge coming from missing markers, we propose a new motion data preprocessing method considering the bone length constraint, which represents the information of variation in the relative position of adjacent markers. The processed motion data is input into a Long-Short Term Memory (LSTM) model to recover the missing markers and de-noise the motion data. The experiment conducted on our own dataset proves that our labelling method achieves a similar effect to Cortex, which is a commonly used commercial motion data analysis software. The experiment on CMU dataset demonstrates that our missing marker reconstruction method can achieve an art-of-state result. The labelling code will be pulished on https://github.com/Lijianfang6930/Graph-Matching-for-Marker-Labelling.

Methods to quantify the complexity of trajectory datasets are still a missing piece in benchmarking human trajectory prediction models. In order to gain a better understanding of the complexity of trajectory prediction tasks and following the intuition, that more complex datasets contain more information, an approach for quantifying the amount of information contained in a dataset from a prototype-based dataset representation is proposed. The dataset representation is obtained by first employing a non-trivial spatial sequence alignment, which enables a subsequent learning vector quantization (LVQ) stage. A large-scale complexity analysis is conducted on several human trajectory prediction benchmarking datasets, followed by a brief discussion on indications for human trajectory prediction and benchmarking.

The datasets that appear in publications are curated and have been split into training, testing and validation sub-datasets by domain experts. Consequently, machine learning models typically perform well on such split-by-hand prepared datasets. Whereas preparing real-world datasets into curated split training, testing and validation sub-dataset requires extensive effort. Usually, repetitive random splits are carried out and trained and evaluated on until reaching out a good score on the evaluation metrics. In this paper, an algorithmic method is proposed for preparing the sub-datasets splits for machine learning models. The objective of the proposed method is to achieve an evenly representative splits out of the dataset with standard and algorithmic way that reduce the perplexity of random splitting.

Due to the rapid developments in technology and the sudden expansion of social media use, Dialect Arabic has become an important source of data that needs to be addressed when building Arabic corpora. In this paper, thirty-three Arabic corpora are surveyed to show that despite all of the developments in the literature, Saudi dialect (SD) corpora still need further expansion. This paper contributes to the literature on SD corpora by creating the largest Saudi corpus – the King Saud University Saudi Corpus (KSUSC) – with +1B total words, including +119M SD words. The KSUSC not only is the newest and largest SD corpus but is also diverse, covering 26 domains in text collected from five different sources. This paper also contributes to the literature by developing a new incremental preprocessing system that is used to create relevant lexicons that are then used to clean and normalize the collected data. This incremental system is scalable and can be adapted for different resources and dialects. Moreover, the collection process for building the KSUSC is discussed in detail, and the challenges in collecting SD text with respect to each platform are highlighted. By the end of this paper, different design criteria are proposed and used with the KSUSC to conclude that the resulting corpus can be of great benefit to researchers who are interested in integrating the corpus with their own work or using its resulting lexicons with Saudi-based NLP tasks.

Many machine learning algorithms and almost all deep learning architectures are incapable of processing plain texts in their raw form. This means that their input to the algorithms must be numerical in order to solve classification or regression problems. Hence, it is necessary to encode these categorical variables into numerical values using encoding techniques. Categorical features are common and often of high cardinality. One-hot encoding in such circumstances leads to very high dimensional vector representations, raising memory and computability concerns for machine learning models. This paper proposes a deep-learned embedding technique for categorical features encoding on categorical datasets. Our technique is a distributed representation for categorical features where each category is mapped to a distinct vector, and the properties of the vector are learned while training a neural network. First, we create a data vocabulary that includes only categorical data, and then we use word tokenization to make each categorical data a single word. After that, feature learning is introduced to map all of the categorical data from the vocabulary to word vectors. Three different datasets provided by the University of California Irvine (UCI) are used for training. The experimental results show that the proposed deep-learned embedding technique for categorical data provides a higher F1 score of 89% than 71% of one-hot encoding, in the case of the Long short-term memory (LSTM) model. Moreover, the deep-learned embedding technique uses less memory and generates fewer features than one-hot encoding.

This work presents two methods that facilitate a 3D reconstruction of microscopic blood vessels in the volume slightly larger than 1 mm3. The source of the data are histological serial sections, i.e., microscopic images of probes, stained with immunohistochemistry. Odd and even sections have different stainings in our primary data set. Thus, firstly, an approach to register an alternately-stained series is presented. With image filtering and a feature-detection-based registration we obtain a registered stack of 148 serial sections. The series has missing sections, locally damaged sections, artifacts from acquisition. All these hinder correct connectivity of blood vessels. With our second approach we interpolate the missing information while maintaining the connectivity. We achieve this with deformations based on dense optical flow. The presented methodology is applicable to further histological series. A combination of both approaches allows us to reconstruct more than 76% larger volumes. An important detail was the composition mode of images. Summarizing, we use methods from image processing and computer vision to create large-scale 3D models from immunostained histological serial sections.

Imbalanced class distribution and missing data are two common problems and occurrences in water quality anomaly detection domain. Learning algorithms in an imbalanced dataset can yield an overrated classification accuracy driven by a bias towards the majority class at the expense of the minority class. On the other hand, missing values in data can induce complexity in the learning classifiers during data analysis. These two problems pose substantial challenges to the performance of learning algorithms in real-life water quality anomaly detection problems. Hence, the need for them to be carefully considered and addressed to achieve better performance. In this paper, the performance of a range of several combinations of techniques to deal with imbalanced classes in the context of binary-imbalanced water quality anomaly detection problem and the presence of missing values is extensively compare. The methods considered include seven missing data and eight resampling methods, on ten different learning state-of-the-art classifiers taking into account diversity in their learning philosophies. The different classifiers are evaluated using stratified 5-fold cross-validation, based on three performance evaluation metrics namely accuracy, ROC-AUC and F1-measure. Further experiments are carried out on nineteen variants of homogeneous and heterogeneous ensemble techniques embedded with resampling and missing value strategies during their training phase as well as an optimized deep neural network model. The experimental results show an improvement in the performance of the learning classifiers, especially when dealing with the class imbalance problem (on the one hand) and the incomplete data problem (on the other hand). Furthermore, the neural network model exhibit superior performance when dealing with both problems.

The instance reduction is one of the data preprocessing methods and aims to remove noises and (or) redundant instances from the training set. In the instance reduction, one of the most representative techniques is the edition method which can remove harmful instances from the training set to improve the prediction accuracy of k nearest neighbor (KNN). Nevertheless, most of existing edition methods still have some drawbacks, such as the parameter dependency, high computational time and relatively low accuracy. To solve these problems, we present a new fast parameter-free edition method based on local sets with natural neighbors (ELS). In ELS, we define a new concept of local sets by introducing natural neighbors. ELS can use the local sets to keep more reasonable class boundaries and effectively filtering out noisy instances (including global outliers). The main advantages of ELS are that (a) it is parameter-free; (b) it can remove global outliers and noisy instances; (c) it is relatively fast. Experiments clearly verify that (a) ELS outperforms existing representative edition methods in improving the prediction accuracy of KNN; (b) ELS can improve the performance of the condensation method and hybrid method in terms of both accuracy and reduction; (c) ELS consumes relatively low running time.

Multistep wind speed forecasting is of great significance for wind energy utilization. To further improve the performance of multistep wind speed forecasting, a new multistep forecasting strategy is proposed based on the basic wind speed multistep forecasting strategy and the corresponding forecasting system is established. The system consists of three parts, including a wavelet decomposition preprocessing module, nonlinear autoregressive artificial neural network and nonlinear autoregressive exogenous artificial neural network composite prediction module, and support vector machine classifier postprocessing module. The system analyzes historical wind speed data from two different angles to obtain dual time series features. Then, the double time series feature is used to improve the accuracy and stability of the multipart forecast. The predictive performance of the system is verified through two experiments. Experiment I tests the prediction accuracy and computing resource consumption of the system. Experiment II tests the stability of the system prediction under different wind conditions. The results show that, compared with the traditional multistep prediction strategy system, the new system has a better prediction accuracy and more stable prediction performance.

The Terracotta Warriors are terracotta sculptures created for China's first emperor more than 2,000 years ago. They are among the most precious unearthed cultural relics of China. However, these relics have been predominantly found in fragments. Fragment classification is currently performed manually on enormous quantities of fragments, which is a time-consuming, inaccurate, and subjective task for archaeologists and conservators. In this study, an automatic method based on a deep learning network combined with template guidance is proposed to classify 3D fragments of the Terracotta Warriors. The fragments are initially classified using PointNet. Then, misclassified fragments are secondly categorized based on their best match to a complete Terracotta Warrior model. Extensive experiments were performed to verify the effectiveness of the proposed method. The promising results demonstrate that the method is the most accurate technique for classifying 3D Terracotta Warrior fragments to date. Moreover, the proposed method can significantly increase the efficiency of future fragment reassembly for the Terracotta Warriors.

GPS trajectories are always embedded with errors, due to the weather or environmental variables. Existing trajectory repairing methods have employed Kalman filters or sequential data cleaning methods. Kalman filter or its variants change all observed measurements, while generally most measurements are originally accurate. Sequential data cleaning methods are mainly applied on one-dimensional data sequences, and when encountering multi-dimensional trajectories, their performance will be compromised due to that the features of multi-dimensional trajectories are not fully utilized. To address these issues, we propose to repair GPS trajectory with movement tendencies, speed change tendency, travel distance tendency and repair distance tendency. We formalize the tendency based trajectory repairing, and propose an exact solution to find the repair which minimize movement tendency score. Then we propose high quality candidate selection and dynamic error range estimation, to improve the efficiency and effectiveness of exact solution. Experiments on three data sets demonstrate the superiority of our proposal.

Smoking is associated with cancer, cardiovascular disease and premature death and can cause severe fire hazards. To assist with smoking cessation, this paper presents a wireless body area network-based system consisting of two off-the-shelf devices, one smartphone and one smartwatch, to detect smoking events by mining the inertial sensor data from both devices. In the system, an end-to-end trainable unified model is implemented by combining a variational autoencoder with a random forest to classify the collected data into smoking and nonsmoking after data preprocessing. The variational autoencoder is adopted to learn the feature representation and deal with the class imbalance problem, and the stochastic decision forest is adopted to guide the global optimization of the parameter learning process. Extensive validation of our scheme is performed on a large dataset we collected, and our scheme yields quite promising results in terms of accuracy and efficiency.

Rotating machinery plays a critical role in many significant fields. However, the unpredictable machinery faults may lead to the severe damage and losses. Hence, it is of great value to explore the precise approaches for fault diagnosis. With the development of the intelligent fault diagnosis methods based on deep learning, convolutional neural network (CNN) has aroused the attention of researchers in machinery fault diagnosis. In the light of the reduction of difficulty in feature learning and the improvement of final diagnosis accuracy, data preprocessing is necessary and crucial in CNN-based fault diagnosis methods. This review focuses on CNN-based fault diagnosis approaches in rotating machinery. Firstly, data preprocessing methods are overviewed. Then, we emphatically analyze and discuss several main techniques applied in CNN-based intelligent diagnosis, principally including the fast Fourier transform, wavelet transform, data augmentation, S-transform, and cyclic spectral analysis. Finally, the potential challenges and research objects are prospected on data preprocessing in intelligent fault diagnosis of rotary machinery.

Microfossils, tiny fossils whose study requires the use of a microscope, have been widely applied in many fields of earth, life, and environmental sciences. The abundance and high diversity of microfossils, as well as the need for rapid identification, call for automated methods to classify microfossils. In this study, we constructed an open dataset of three-dimensional (3D) microfossils and proposed a deep learning-based approach for microfossil classification. The dataset, named `Archives of Digital Morphology' (ADMorph), currently contains more than ten thousand 3D models from five classes of 410 million-year-old fishes. The deep learning-based method includes data preprocessing, feature extraction, and 3D microfossil model classification. To assess the method performance and dataset representability, we performed extensive experiments. Compared with multiview convolutional neural networks (MVCNN) (91.54%), PointNet (64.13%), and VoxNet (78.15%), the method proposed herein had higher accuracy (97.60%) on the experimental dataset. We also verified data preprocessing (92.36%) and feature extraction (97.10%). We combined them to obtain the macroaveraging accuracy of 97.60%, the highest accuracy of 100%, and the lowest accuracy of 88.78%. We suggest that the proposed method can be applied to other 3D fossils and biomorphological research fields. The fast-accumulating 3D fossil models might become a source of information-rich datasets for deep learning.

Machine learning for author name disambiguation is usually conducted on the training and test subsets of labeled data created for a specific task. As a result, disambiguation models learned on heterogeneous labeled data are often inapplicable for other purposes that either do not use the same labeled data or do not make use of any labeled data at all. This article explores the idea of transfer learning in a new context, author name disambiguation. We focus on cases where a disambiguation task lacking labeled training data uses models trained on labeled data generated for other tasks. For this purpose, two labeled source datasets are used for training of disambiguation models to be applied to three test target datasets that are deficient of labeled training data. Our results show that transfer learning can produce disambiguation performances similar to those achievable by traditional machine learning in which training and test datasets come from the same labeled data source. The good performance through transfer learning are possible when training source datasets have similar feature distributions as test target datasets. This study suggests that through transfer learning, rich disambiguation models in previous studies can be retained and reused across ambiguous bibliographic data from different fields and data sources, motivating further research on how to correct feature distribution differences between source and target datasets to expand the application of transfer learning in author name disambiguation beyond the model sharing explored in this research.

A novel hybrid ensemble framework is developed to forecast the short-term wind speed, which consists of a data preprocessing technique, data-driven based forecasting algorithms, and an improved Jaya algorithm. In the data preprocessing process, the pauta criterion is employed to find out the outliers, and the variational mode decomposition algorithm decompose the original series to extract the trend and time-frequency information of the historical inputs. The data-driven forecasting algorithms, such as BP, LSSVM, ANFIS, and Elman, are exploited as the original predictor of the framework, while the weights of the predictors are computed by an improved optimization algorithm-CLSJaya. Based on the experimental results of two time-scale datasets from three sites, the proposed framework successfully overcomes the limitations of the individual forecasting models and achieves promising forecasting accuracy.

Abnormal testing data can severely reduce model performance if not processed properly. In this paper, we propose a preprocessing system to handle different types of commonly seen abnormal testing data. The system consists of an aberrant data detector and an aberrant data corrector. The aberrant data detector is responsible for classifying the type of incoming data. Based on the data type, the aberrant data corrector will take different actions to amend testing data. Users can then apply their preferred prediction methods on the corrected testing data. Specifically, corrupted and adversarial images are used as examples of abnormal data. We show that corrupted data can be reconstructed through a Gaussian locally linear mappings method, and the prediction performance of adversarial samples can be improved by using the nearest neighbors as a surrogate. We compare the proposed aberrant data detector and corrector with existing and well-recognized alternatives. These approaches are published individually and do not put two components together as a pre-processing system. The numerical outcomes show that our proposed components, standing alone, are competitive. The proposed system is a generic method that can be applied to different downstream predictive models. We use three existing prediction methods to illustrate the general usage of the proposed system and its capability of improving prediction efficacy.

This study investigated large-scale semi-supervised training (SST) to improve acoustic models for automatic speech recognition. The conventional self-training, the recently proposed committee-based SST using heterogeneous neural networks and the lattice-based SST were examined and compared. The large-scale SST was studied in deep neural network acoustic modeling with respect to the automatic transcription quality, the importance data filtering, the training data quantity and other data attributes of a large quantity of multi-genre unsupervised live data. We found that the SST behavior on large-scale ASR tasks was very different from the behavior obtained on small-scale SST: 1) big data can tolerate a certain degree of mislabeling in the automatic transcription for SST. It is possible to achieve further performance gains with more unsupervised fresh data, and even the automatic transcriptions have a certain degree of errors; 2) the audio attributes, transcription quality and importance of the fresh data are more important than the increased data quantity for large-scale SST; and 3) there are large differences in performance gains on different recognition tasks, such that the benefits highly depend on the selected data attributes of unsupervised data and the data scale of the baseline ASR system. Furthermore, we proposed a novel utterance filtering approach based on active learning to improve the data selection in large-scale SST. The experimental results showed that the SST with the proposed data filtering yields a 2-11% relative word error rate reduction on five multi-genre recognition tasks, even with the baseline acoustic model that was already well trained on a 10000-hr supervised dataset.

Quantile summarization is a useful tool in data streams management and mining that can efficiently capture the distribution of the data. A quantile of a sequence of points is the point with a given rank in the sequence. Given a sequence of uncertain points S on the real line, each represented by one-dimensional probability density function (pdf), we study the problem of incrementally maintaining quantile summaries on S such that for any query with a given rank, the summaries can provide a point as the quantile within a given error. We define quantile on uncertain data with discrete or continuous pdf in terms of two error metrics under possible worlds semantics. For an answer to a quantile query on uncertain data, we give the methods for calculating the value of the error and thereby discussing the high-level features of the summaries that can answer approximate quantile query under the two error metrics. We propose an online, space efficient algorithm to compute such summary data on uncertain data streams. The experimental results show that our algorithm substantially outperforms other techniques, such as Monte Carlo and averaging methods, in terms of query error and space for storing the summary data.

In the development of the wind power industry, short-term wind speed forecasting is necessary, and many researchers have made substantial efforts to establish wind speed prediction models. However, realizing the accurate prediction of wind speeds remains a challenging task. The current prediction models do not consider the preprocessing of the data, and each model has various shortcomings. Considering the disadvantages of the available models, in this paper, an advanced combined forecasting system is applied that utilizes a data preprocessing strategy and parameter optimization strategy to obtain accurate prediction values. The proposed prediction system employs linear and nonlinear models that can take into account the characteristics of wind speed sequences, successfully combine the advantages of various single models, and yield accurate and stable prediction values. Finally, according to the experimental analysis and discussion, the proposed combined prediction system outperforms the compared models in prediction. In conclusion, the powerful combined prediction model provides a feasible scheme for wind power prediction.

The performance evaluation of imputation algorithms often involves the generation of missing values. Missing values can be inserted in only one feature (univariate configuration) or in several features (multivariate configuration) at different percentages (missing rates) and according to distinct missing mechanisms, namely, missing completely at random, missing at random, and missing not at random. Since the missing data generation process defines the basis for the imputation experiments (configuration, missing rate, and missing mechanism), it is essential that it is appropriately applied; otherwise, conclusions derived from ill-defined setups may be invalid. The goal of this paper is to review the different approaches to synthetic missing data generation found in the literature and discuss their practical details, elaborating on their strengths and weaknesses. Our analysis revealed that creating missing at random and missing not at random scenarios in datasets comprising qualitative features is the most challenging issue in the related work and, therefore, should be the focus of future work in the field.

This survey gives a comprehensive overview of tensor techniques and applications in machine learning. Tensor represents higher order statistics. Nowadays, many applications based on machine learning algorithms require a large amount of structured high-dimensional input data. As the set of data increases, the complexity of these algorithms increases exponentially with the increase of vector size. Some scientists found that using tensors instead of the original input vectors can effectively solve these high-dimensional problems. This survey introduces the basic knowledge of tensor, including tensor operations, tensor decomposition, some tensor-based algorithms, and some applications of tensor in machine learning and deep learning for those who are interested in learning tensors. The tensor decomposition is highlighted because it can effectively extract structural features of data and many algorithms and applications are based on tensor decomposition. The organizational framework of this paper is as follows. In part one, we introduce some tensor basic operations, including tensor decomposition. In part two, applications of tensor in machine learning and deep learning, including regression, supervised classification, data preprocessing, and unsupervised classification based on low rank tensor approximation algorithms are introduced detailly. Finally, we briefly discuss urgent challenges, opportunities and prospects for tensor.

The intelligent transportation system (ITS) concept was introduced to increase road safety, manage traffic efficiently, and preserve our green environment. Nowadays, ITS applications are becoming more data-intensive and their data are described using the “5Vs of Big Data”. Thus, to fully utilize such data, big data analytics need to be applied. The Internet of vehicles (IoV) connects the ITS devices to cloud computing centres, where data processing is performed. However, transferring huge amount of data from geographically distributed devices creates network overhead and bottlenecks, and it consumes the network resources. In addition, following the centralized approach to process the ITS big data results in high latency which cannot be tolerated by the delay-sensitive ITS applications. Fog computing is considered a promising technology for real-time big data analytics. Basically, the fog technology complements the role of cloud computing and distributes the data processing at the edge of the network, which provides faster responses to ITS application queries and saves the network resources. However, implementing fog computing and the lambda architecture for real-time big data processing is challenging in the IoV dynamic environment. In this regard, a novel architecture for real-time ITS big data analytics in the IoV environment is proposed in this paper. The proposed architecture merges three dimensions including intelligent computing (i.e. cloud and fog computing) dimension, real-time big data analytics dimension, and IoV dimension. Moreover, this paper gives a comprehensive description of the IoV environment, the ITS big data characteristics, the lambda architecture for real-time big data analytics, several intelligent computing technologies. More importantly, this paper discusses the opportunities and challenges that face the implementation of fog computing and real-time big data analytics in the IoV environment. Finally, the critical issues and future research directions section discusses some issues that should be considered in order to efficiently implement the proposed architecture.

Software fault prediction is the very important research topic for software quality assurance. Data-driven approaches provide robust mechanisms to deal with software fault prediction. However, the prediction performance of the model highly depends on the quality of the data set. Many software data sets suffer from the problem of class imbalance. In this regard, undersampling is a popular data pre-processing method in dealing with the class imbalance problem; easy ensemble presents a robust approach to achieve a high classification rate and address the biases toward majority class samples. However, imbalance class is not the only issue that harms the performance of classifiers. Some noisy data and irrelevant and redundant features may also reduce the performance of predictive accuracy of the classifier. In this paper, we propose two-stage data pre-processing, which incorporates feature selection and rough set-based K nearest neighbour rule (KNN) noise filter afore executing easy ensemble rough-KNN noise-filtered easy ensemble (RKEE). In the first stage, we eliminate the irrelevant and redundant features by the feature ranking algorithm, and in the second stage, we handle the imbalance class problem by using rough-KNN noise filter to eliminate noisy samples from both the minority and the majority class and also handle the uncertainty and the overlapping problem from both the minority and the majority class. Experimental evaluation on real-world software projects, such as NASA and Eclipse data set, is performed in order to demonstrate the effectiveness of our proposed approach. Furthermore, this paper comprehensively investigates the influencing factor in our approach, such as the impact of the rough set theory on noise-filter, the relationship between model performance and imbalance ratio, and so on. Comprehensive experiments indicate that the proposed approach shows outstanding performance with significance in terms of area-under-the-curve.

In a smart grid environment, advanced metering infrastructure (AMI) and intelligent sensors have been deployed extensively. As a result, large-scale and fine-grained smart grid data are more convenient to be collected, in which outliers exist pervasively, caused by system failures, environmental effects, and human interventions. Outlier deletion is always implemented in data preprocessing for improving data quality. However, due to the fact that real records that reflect rare and unusual patterns are also recognized as outliers, outlier mining is necessary to be carried out with the aim of discovering knowledge on abnormal patterns in power generation, transmission, distribution, transformation, and consumption. To the best of our knowledge, a comprehensive and systematic review of outlier data treatment methods is still lacked in the smart grid environment. We, in this paper, aim at presenting the review of outlier data treatment methods toward smart grid applications and categorize them into outlier rejection and outlier mining groups. Since we do this survey from the perspective of data-driven analytics and data mining methods, information security technologies are barely discussed in this paper. Based on a general overview of outlier data treatment methods, we make the contribution of providing the application scenarios of outlier rejection and outlier mining in the smart grid environment. With the construction of smart grid throughout the world, dealing with outlier data has become more crucial for the security and reliability of power system operation. Therefore, we also discuss some future challenges of outlier data treatment toward smart energy management.

Missing or incorrect data caused by improper operations can seriously compromise security investigation. Missing data can not only damage the integrity of the information but also lead to the deviation of the data mining and analysis. Therefore, it is necessary to implement the imputation of missing value in the phase of data preprocessing to reduce the possibility of data missing as a result of human error and operations. The performances of existing imputation approaches of missing value cannot satisfy the analysis requirements due to its low accuracy and poor stability, especially the rapid decreasing imputation accuracy with the increasing rate of missing data. In this paper, we propose a novel missing value imputation algorithm based on the evidence chain (MIAEC), which first mines all relevant evidence of missing values in each data tuple and then combines this relevant evidence to build the evidence chain for further estimation of missing values. To extend MIAEC for large-scale data processing, we apply the map-reduce programming model to realize the distribution and parallelization of MIAEC. Experimental results show that the proposed approach can provide higher imputation accuracy compared with the missing data imputation algorithm based on naive Bayes, the mode imputation algorithm, and the proposed missing data imputation algorithm based on K-nearest neighbor. MIAEC has higher imputation accuracy and its imputation accuracy is also assured with the increasing rate of missing value or the position change of missing value. MIAEC is also proved to be suitable for the distributed computing platform and can achieve an ideal speedup ratio.

Educational data preprocessing from log files represents a time-consuming phase of the knowledge discovery process. It consists of data cleaning, user identification, session identification, and path completion phase. This paper attempts to identify phases, which are necessary in the case of preprocessing of educational data for further application of learning analytics methods. Since the sequential patterns analysis is considered suitable for estimating of discovered knowledge, this paper tries answering the question, which of these preprocessing phases has a significant impact on discovered knowledge in general, as well as in the meaning of quality and quantity of found sequence patterns. Therefore, several data preprocessing techniques for session identification and path completion were applied to prepare log files with different levels of data preprocessing. The results showed that the session identification technique using the reference length, calculated from the sitemap, had a significant impact on the quality of extracted sequence rules. The path completion technique had a significant impact only on the quantity of extracted sequence rules. The found results together with the results of the previous systematic research in educational data preprocessing can improve the automation of the educational data preprocessing phase as well as it can contribute to the development of learning analytics tools suitable for different groups of stakeholders engaged in the educational data mining research activities.