Search

Art has become an important new investment vehicle. Thus, interest is growing in art price prediction as a tool for assessing the returns and risks of art investments. Both traditional statistical methods and machine learning methods have been used to predict art prices. However, both methods incur substantial human costs for data preprocessing for the construction of prediction models, necessitating a reduction in the workload. In this study, we propose the zero-shot classification method to perform automatic annotation in data processing for art price prediction by leveraging large language models (LLMs). The proposed method can perform annotation without new training data. Thus, it minimizes human costs. Our experiments demonstrated that the 4-bit quantized Llama-3 70B model, which can run on a local server, achieved the most accurate (over 0.9) automatic annotation of different art forms using LLMs, performing slightly better than the GPT-4o model from OpenAI. These results are practical for data preprocessing and comparable with the results of previous machine learning methods.

In contemporary society, the rapid progression of urbanization and technological advancements has led to a substantial increase in the number of vehicles, consequently elevating the rate of traffic accidents. To minimize the human and economic losses resulting from these accidents, extensive research has been conducted over the past decades. Recent studies utilizing Grid Maps and time-series methods have shown promising results in identifying factors related to traffic accidents and predicting accident occurrence rates. However, most existing research employs data without thoroughly analyzing its definitive correlation with traffic accidents, focusing solely on an overarching integration process or exclusively considering road conditions, while neglecting environmental factors surrounding the roads. Therefore, this paper focuses on a detailed analysis of factors contributing to traffic accidents in Seoul, the capital city of South Korea, limiting the study period from January 2020 to December 2021. The data encompasses various aspects such as traffic accidents, weather conditions, standard node links, traffic speed, Points of Interest (POI), solar altitude and azimuth, speed bumps, and traffic surveillance cameras. Most of this data is available from national public institutions, with some being computed through specific formulas. Since weather and POI data possess a wide range of features, a Pearson correlation analysis is conducted to extract features relevant to traffic accidents. The extracted features are then compiled and normalized to facilitate the deep learning model’s training, thereby constructing a comprehensive dataset.

Electromyography (EMG) is used to recognize user finger gestures for applications in real-time interfaces. Finger movements are classified by preprocessing to extract the features from the collected EMG data, which are then used for machine learning. The data were extracted using the overlapped segmentation method to ensure sufficient training data. The preprocessing of EMG data uses standard formulae, such as integrated EMG (IEMG) and mean absolute value (MAV). Furthermore, preprocessing involves using original data, simple moving average (SMA), and Fast Fourier transform (FFT) for feature extraction. Subsequently, these preprocessed data sets are used to train machine learning models, facilitating a comparative analysis. Four machine learning models were used: eXtreme Gradient Boost, Random Forest, k-Nearest Neighbors, and Logistic Regression. The experimental results revealed the best accuracy from preprocessing using a simple moving average followed by a Fourier transform, but classification was not possible using all nine finger movements. On the other hand, it showed more than 90% accuracy because the model learned by reducing it to a specific finger gesture. Rest movements, index finger taps, and force-taps movements achieved the highest accuracy, approximately 95%.

The prediction of loan defaults is crucial for banks and financial institutions due to its impact on earnings, and it also plays a significant role in shaping credit scores. This task is a challenging one, and as the demand for loans increases, so does the number of applications. Traditional methods of checking eligibility are time-consuming and laborious, and they may not always accurately identify suitable loan recipients. As a result, some applicants may default on their loans, causing financial losses for banks. Artificial Intelligence, using Machine Learning and Deep Learning techniques, can provide a more efficient solution. These techniques can use various classification algorithms to predict which applicants will likely be eligible for loans. This study uses five Machine Learning classification algorithms (Gaussian Naive Bayes, AdaBoost, Gradient Boosting, K Neighbors Classifier, Decision Trees, Random Forest, and Logistic Regression) and eight Deep Learning algorithms (MLP, CNN, LSTM, Transformer, GRU, Autoencoder, ResNet, and DenseNet). The use of Ensemble Methods and SMOTE with SMOTE-TOMEK Techniques also has a positive impact on the results. Four metrics are used to evaluate the effectiveness of these algorithms: accuracy, precision, recall, and F1-measure. The study found that DenseNet and ResNet were the most accurate predictive models. These findings highlight the potential of predictive modeling in identifying credit disapproval among vulnerable consumers in a sea of loan applications.

The Internet of Things (IoT) is transforming how we live and work, and its applications are widespread, spanning smart homes, industrial monitoring, smart cities, healthcare, agriculture, and retail. Considering its wide range of applications, addressing the security challenges arising from IoT devices’ massive collection and transmission of user data is vital. Intrusion detection systems (IDS) based on deep learning techniques offer new means and research directions for resolving IoT security issues. Deep learning models can process large volumes of data and extract complex patterns, making them generally more effective than traditional rule based IDSs. While deep learning techniques are gradually gaining popularity in IDS applications, current research needs a comprehensive summary of deep learning-based IDS in IoT. This paper introduces intrusion detection technologies, followed by a detailed comparison, analysis, and discussion of deep learning models, datasets, feature extraction and classifiers, data preprocessing techniques, and experimental design of the models. It also highlights the challenges and issues associated with deep learning models and relevant techniques for IDS. Finally, it concludes by providing recommendations to assist researchers in this domain.

Random splitting strategy is a common approach for training, testing, and validating object detection algorithms based on deep learning. Is common for datasets to have images extracted from video sources, in which there are frames with high spatial correlation, i.e., frames with rotated positions or different view angles of the same object. These highly correlated frames may lead to information leakage in training, if these frames are not well-distributed. In this work, it is shown that datasets created with highly spatial correlation frames from the same video have information leakage if using the random splitting strategy to distribute the image into the sub-datasets. It proposed a clustering dataset split algorithm in which images are distributed randomly in the sub-datasets in a pack or clusters instead of a single image at the time. The clusters are created by extracting the image features from a video of the dataset using an image-text pre-trained model, CLIP, and reducing the feature vector dimensionality with t-Distributed Stochastic Neighbor embedding (t-SNE). In this reduced dimensional representation, images are separated into clusters using a clustering algorithms like DBSCAN, OPTICS, and Agglomerative Clustering. These clusters are distributed into the train, test, and validation datasets randomly to avoiding information leakage by highly spatial correlation frames. YOLOv8 is used as the object detector algorithm to test the dataset splitting.

Realistic anomalous data noise is usually locally distributed in nature. To address the challenge posed by clustered anomalous noise in time series data, which not only reduces the accuracy of forecasting models but also presents a unique challenge due to its clustered nature and existing methods’ limitations in handling such noise, this study introduces a novel denoising algorithm leveraging an optimized Long Short-Term Memory (LSTM) network, named ‘Denoising Time Window Optimized LSTM’ (DTW-LSTM), which innovatively identifies and rectifies clustered anomalous noise in time series data, significantly enhancing prediction accuracy by adeptly balancing the denoising process and data integrity preservation, as evidenced by comprehensive experimental validations. The algorithm employs a sliding time window mechanism to analyze the original series and its Fourier approximation, identifying clustered anomalies through a clustering model. It then uses a symmetrical data mixing and crossover technique to enhance the data quality in noisy segments. This process preserves the integrity of time series data by maintaining a balance between the original and fitted series, leading to a significant improvement in prediction accuracy. Experimental results indicate that the algorithm can adeptly discern the presence of clustered anomalous noise within the current time window and effectively rectify sections of data containing such noise. By surpassing the performance benchmarks set by contemporary forecasting models, this algorithm has established itself as the new SOTA method in the field of time series forecasting. Each sub-model contributes to the superior accuracy and reliability of the algorithm.

Optimizing machine learning (ML) model performance relies heavily on appropriate data preprocessing techniques. Despite the widespread use of standardization and normalization, empirical comparisons across different models, dataset sizes, and domains remain sparse. This study bridges this gap by evaluating five machine learning algorithms- Support Vector Machine (SVM), Logistic Regression (LR), Random Forest (RF), Extreme Gradient Boosting (XGBoost), and Adaptive Boosting (AdaBoost)- on datasets of varying sizes from the business, health, and agriculture domains. This study assessed the models without scaling, with standardized data, and with normalized data. The comparative analysis reveals that while standardization consistently improves the performance of linear models like SVM and LR for large and medium datasets, normalization enhances the performance of linear models for small datasets. Moreover, this study employs SHapley Additive exPlanations (SHAP) summary plots to understand how each feature contributes to the model’s performance interpretability with unscaled and scaled datasets. This study provides practical guidelines for selecting appropriate scaling techniques based on the characteristics of datasets and compatibility with various algorithms. Finally, this investigation laid the foundation for data preprocessing and feature engineering across diverse models and domains which offers actionable insights for practitioners.

In the era of digital transformation, customer feedback has become crucial for improving service quality. This study aims to enhance customer satisfaction through sentiment analysis utilizing machine learning techniques, with additional case studies conducted to ensure comprehensive method validation. Traditional sentiment analysis methods frequently fail to manage the complexity and volume of feedback data, yielding to less accurate insights. To address this challenge, we analyzed six machine learning models: Naïve Bayes, Support Vector Machine (SVM), Long Short-Term Memory (LSTM), Random Forest, AdaBoost, and BERT, with a particular focus on BERT. Our results demonstrate that BERT outperformed the other models in terms of both accuracy and processing speed, achieving an accuracy of up to 95%. The excellence of BERT in managing large and complex datasets provides a more precise sentiment analysis, which can significantly improve service quality and customer loyalty, while increasing company revenue by up to 15%. This research advances to the field of sentiment analysis by validating the effectiveness of BERT over traditional models through extensive comparative analysis and practical case studies.

The increasing prevalence of diabetes necessitates the development of effective early detection methods to mitigate its health impacts. This paper investigates the impact of feature transformation and machine learning (ML) models on the early detection of diabetes using a binary tabular classification dataset. We explore three feature transformation techniques, no transformation, normalization, and min-max scaling, to assess their influence on the performance of various ML models. To comprehensively evaluate the effectiveness of these preprocessing techniques, we experimented with twelve different ML models, including both traditional algorithms and ensemble methods. A publicly available dataset has been used for this research, containing 768 samples and 8 features. To ensure their effectiveness, the models are assessed using several evaluation metrics, including accuracy, precision, recall, F1-score, and Area Under the Receiver Operating Characteristic Curve (AUC-ROC). Among the ML models, Light Gradient Boosting Machine (LGBM) achieved the highest accuracy of 82.91% when min-max scaling was applied to the data. Our results demonstrate the varying effectiveness of different combinations of feature transformation techniques and ML models in enhancing diabetes detection performance. Furthermore, it has been observed that the ensemble models generally achieved better performance than traditional ML models. These findings provide valuable insights for optimizing preprocessing and model selection strategies in the development of robust early diabetes detection systems.

Progressively more advanced and complex models are proposed to address problems related to computer vision, forecasting, Internet of Things, Big Data and so on. However, these disciplines require preprocessing steps to obtain meaningful results. One of the most common problems addressed in this stage is the presence of missing values. Understanding the reason why missingness occurs helps to select data imputation methods that are more adequate to complete these missing values. Missing at Random synthetic generation presents challenges such as achieving extreme missingness rates and preserving the consistency of the mechanism. To address these shortcomings, three new methods that generate synthetic missingness under the Missing at Random mechanism are proposed in this work and compared to a baseline model. This comparison considers a benchmark covering 33 data sets and five missingness rates $(10\%, 20\%, 40\%, 60\%, 80\%)$ . Seven data imputation methods are compared to evaluate the proposals, ranging from traditional methods to deep learning methods. The results demonstrate that the proposals are aligned with the baseline method in terms of the performance and ranking of data imputation methods. Thus, three new feasible and consistent alternatives for synthetic missingness generation under Missing at Random are presented.

The number of deaths from lung cancer reached 1.8 million in 2020, ranking first among all cancers. Early diagnosis has been found to improve the survival rate of lung cancer patients after treatment in clinical care. Computed tomography (CT) is a technique commonly used in the early detection of lung cancer to determine the benignity or malignancy of lung nodules. Manual analysis of CT results is less efficient and its accuracy is affected by physicians’ experience levels. Segmenting lung nodules in a computer-aided diagnosis (CAD) system can effectively improve the efficiency and accuracy of the diagnosis. In this paper, we evaluate several deep learning segmentation models (including UNet, SegNet, GCN, FCN, DeepLabV3+, PspNet TransUNet, SwinUNet) and examine the effects of different preprocessing methods on the models to explore the best preprocessing and training strategies for lung nodule segmentation. Specifically, we investigate the effects of two different data preprocessing methods (adding a lung mask and croping the region of interest) on the segmentation results, where better segmentation results are achieved by including the nodal data of the region of interest without the lung mask. Through a comprehensive comparison, TransUNet achieves the best segmentation accuracy, with DICE indices of 0.887, 0.871, 0.75, and 0.744 tested on four datasets, respectively.

Advancements in Internet of Things devices allow for the collection and analysis of big data. Moreover, cloud computing has emerged as an ideal platform for big data analysis because it quickly provides computing resources for analysis at scale. However, concerns exist regarding the privacy and security of this information. Secure computation is a technology that enables computation while maintaining data encryption. In this study, we present a new secure computation based on a client-server model, in which a set of servers perform computations using the inputs of multiple clients. We adopt the $(k, n)$ threshold secret sharing approach, where an input $s$ is divided into $n$ shares and can be recovered from shares with a threshold $k$ . However, conventional secure computation using $(k,n)$ threshold secret sharing generally requires the condition $n \geq 2k-1$ and communication among multiple servers for each multiplication. To the best of our knowledge, no previous study has solved this problem completely. We demonstrate that processes that require communication can be concentrated in the preprocessing phase and propose a secure computation using $(k,n)$ threshold secret sharing that does not require communication during the secure computation phase, even when multiplication is performed. Moreover, we show that the number of communications does not depend on the number of multiplications, unlike in conventional methods. As communication often requires more processing time than the actual secure computation, our method makes it possible to realize faster overall processing than conventional methods. We also present an extensive security analysis and experimental simulation of the proposed method. Our proposed method achieves information-theoretic security against semi-honest adversaries under certain conditions with $n < 2k-1$ .

Coronary Artery Disease (CAD) is an increasingly prevalent disorder that significantly affects both longevity and quality of life, particularly among people aged 30 to 60. Lifestyle, genetics, nutrition, and stress are contributing factors to the increasing mortality rates. Therefore, there is a need for low-cost automated technology to diagnose CAD early and help medical practitioners treat chronic illnesses effectively. However, machine learning methods are typically designed to perform well with large datasets and may not be well-suited for smaller clinical datasets that contain categorical features. To address this issue, alternative approaches such as Data Preprocessing (DP), feature selection techniques, and Hyperparameter tuning (HP) are necessary to achieve optimal performance and accuracy on such datasets. Data preprocessing is also crucial to obtain accurate results by eliminating noise, handling missing values, and dealing with outliers. To address the challenges associated with feature selection, manual selection of hyperparameter tuning, and optimization, we have developed a novel model called BSOXGB (BorutaShap feature selection based Optuna hyperparameter tuning of eXtremely Gradient Boost). The proposed model achieves a remarkable accuracy of 97.70%, outperforming other classifiers like Random Forest (RF), AdaBoost (AB), CatBoost (CB), and ExtraTrees (ET) on the publicly available Z-Alizadeh Sani dataset. BSOXGB, with only 9 relevant features out of 56, has the highest classification accuracy, demonstrating its potential as a practical solution for automatically detecting and diagnosing CAD in the real world.

We present a spatial-spectral decoupling framework (SDF) to improve the performance of hyperspectral image classification, it mainly contains three modules, including data preprocessing, feature representation, and collaborative decision-making. Specifically, the data preprocessing module based on band selection (BS) network can effectively emphasize useful spectral bands while suppressing redundant ones. Besides, the feature representation module is based on spatial-spectral decoupling (SD) network to avoid information confusion between the spatial and the spectral domains. In addition, the collaborative decision-making mechanism based on joint optimization can maintain the discriminative properties of different branches and enhance mutual facilitation among them. Finally, the experimental results validate the effectiveness and superiority of our SDF. [ABSTRACT FROM AUTHOR]

Intrusion detection and classification have gained significant attention recently due to the increased utilization of networks. For this purpose, there are different types of Network Intrusion Detection System (NIDS) approaches developed in the conventional works, which mainly focus on identifying the intrusions from the datasets with the help of classification techniques. Still, it is limited by the significant problems of inefficiency in handling large dimensional datasets, high computational complexity, false detection, and more time consumption for training the models. To solve these problems, this research intends to develop an innovative clustering-based classification methodology to precisely detect intrusions from the different types of IDS datasets. Here, the most recent and extensively used IDS datasets such as NSL-KDD, CICIDS, and Bot-IoT have been employed for detecting intrusions. Data preprocessing has been performed to normalize the dataset to eliminate irrelevant attributes and organize the features. Then, the data separation is applied by forming the clusters by using an intelligent Anticipated Distance-based Clustering (ADC) incorporated with the Density-Based Spatial clustering of applications with noise (DBScan) algorithm. It helps to find the distance and density measures for grouping the attributes into the clusters, which increases the efficiency of classification. Here, the most suitable optimal parameters are selected using the Perpetual Pigeon Galvanized Optimization (PPGO) technique. The extracted features are used for training and testing the dataset samples. Consequently, the Likelihood Naïve Bayes (LNB) classification approach is implemented to accurately predict the classified label as to whether normal or attack. During the evaluation, the performance of the proposed IDS framework is validated and compared using various evaluation metrics. The results show that the proposed ADC-DBScan-LNB model outperforms the other techniques with improved performance outcomes.

Data preprocessing is crucial in the Machine Learning pipeline because the models’ learning ability directly affects the quality of data and the underlying information acquired from this stage. Nevertheless, surprisingly, there are many alternatives for each transformation task, which makes an inexperienced user overwhelmed. A simple Python-based Auto-preprocessing architecture for Automated Machine Learning is developed to offer automated, interactive, and data-driven support to help the users perform data preprocessing tasks efficiently. The suggested method provides valuable insights into a dataset and can handle standard data preprocessing tasks adeptly. Initially, it detects the data problem and presents it to the end-user using compelling visualizations. Then, it recommends the most effective data cleaning and preparation method to the user after evaluating the state-of-the-art candidate techniques. For evaluation, the proposed architecture is employed on ten different and diverse datasets for automatic data preprocessing before passing it to an ML algorithm. The results are then compared with the results generated by the same ML algorithm but implemented on manually preprocessed data. The results have shown that not only did this approach make the whole process uncomplicated and facile, but it was also able to improve the performance of the model significantly.

In this paper, a brief survey of data preprocessing methods is presented. Specifically, the data preprocessing methods used in the smart grid (SG) domain are surveyed. Also, with the advent of SG, data collection on a large scale became possible. The data is essential for electricity demand, generation and price forecasting, which plays an important role in making energy efficient decisions, and long and short term predictions regarding energy generation, consumption and storage. However, the forecasting accuracy decreases when data is used in raw form. Hence, data preprocessing is considered essential. This paper provides an overview of the data preprocessing methods and a detailed discussion of the methods used in the existing literature. A comparison of the methods is also given. A survey of closely related survey papers is also presented and the papers are compared based on their contributions. Moreover, based on the discussion of the data preprocessing methods, a narrative is built with a critical analysis. Finally, future research directions are discussed to guide the readers.

Missing data are a universal data quality problem in many domains, leading to misleading analysis and inaccurate decisions. Much research has been done to investigate the different mechanisms of missing data and the proper techniques in handling various data types. In the last decade, machine learning has been utilized to replace conventional methods to address the problem of missing values more efficiently. By studying and analyzing recently proposed methods using machine learning approaches, vital adoptions in accuracy, performance, and time consumed can be highlighted. This study aimed to help data analysts and researchers address the limitations of machine learning imputation methods by conducting a systematic literature review to provide a comprehensive overview of using such methods to impute missing values. Novel proposed machine learning approaches used for data imputation are analyzed and summarized to assist researchers in selecting a proper machine learning method based on several factors and settings. The review was performed on research studies published between 2016 and 2021 on adopting machine learning to impute missing values, focusing on their strengths and limitations. A total of 684 research articles from various scientific databases were analyzed using search engines, and 94 of them were selected as primary studies. Finally, several recommendations were given to guide future researchers in applying machine learning to impute missing values.

Massive residential power consumption information provides data support for the mining and analysis of load patterns. This article proposes a complete framework for load pattern identification, which mainly includes the clustering module and the classification module. Considering that the high-dimensional load profiling dataset will bring a heavy computational burden, multiple dimensional scaling is introduced in the process of data preprocessing. Then, an innovative mixture model based on regular vine copula mixture model (RVMM) is adopted for clustering typical load patterns. Finally, a random forest (RF) classifier constructed with certain load characteristic indexes and RVMM clustering results is employed as a supervised classification model to predict the categories of subsequent new customers, and the accuracy is calculated by the 10-fold cross-validation. It is demonstrated in the case study that the proposed RVMM algorithm exhibits better performance in the clustering validity evaluation. Besides, higher accuracy is achieved by the RF classifier. [ABSTRACT FROM AUTHOR]

Causal classification concerns the estimation of the net effect of a treatment on an outcome of interest at the instance level, i.e., of the individual treatment effect (ITE). For binary treatment and outcome variables, causal classification models produce ITE estimates that essentially allow one to rank instances from a large positive effect to a large negative effect. Often, as in uplift modeling (UM), one is merely interested in this ranking, rather than in the ITE estimates themselves. In this regard, we investigate the potential of learning to rank (L2R) techniques to learn a ranking of the instances directly. We propose a unified formalization of different binary causal classification performance measures from the UM literature and explore how these can be integrated into the L2R framework. Additionally, we introduce a new metric for UM with L2R called the promoted cumulative gain (PCG). We employ the L2R technique LambdaMART to optimize the ranking according to PCG and show improved results over the use of standard L2R metrics and equal to improved results when compared with state-of-the-art UM. Finally, we show how L2R techniques can be used to specifically optimize for the top- $k$ k fraction of the ranking in a UM context, however, these results do not generalize to the test set. [ABSTRACT FROM AUTHOR]

Source number estimation is an important problem in signal processing. The performances of traditional methods in this field are severely degraded under some extreme conditions, such as relatively short observed signal length, low signal-to-noise ratio (SNR), or arbitrary noise covariance structure. Recently, machine learning has been used for source number estimation with different data formats as input data due to the advantages of not requiring subjective parameter settings and being data-driven. However, it also has shortcomings such as fluctuating performance and requiring more training samples. In view of this problem, based on the statistical characteristics of the eigenvalues of the covariance matrix, this letter proposes two preprocessing schemes to further manifest the input features of the network model. The input features transformed by the proposed preprocessing methods have specific distributions, which can improve the discriminability between features, thereby enhancing the prediction ability of the network. Simulation results verify that the proposed preprocessing schemes are capable of improving the estimation performance while reducing the fluctuation of the performance. [ABSTRACT FROM AUTHOR]

This article proposes a fully integrated output-capacitor-free low-dropout regulator (LDO) for mobile applications. To overcome the limited output voltage range of typical analog LDOs, our design uses a rail-to-rail voltage-difference-to-time-converter (VDTC) and a charge pump (CP) to achieve a wide output range. Using a self-calibrating clock generator (SCCG) removes the need for an external clock source and adaptively tunes the clock frequency, enabling fast transient responses while minimizing quiescent current. A tunable undershoot compensator (TUC) mitigates voltage droop by detecting the drop in the output voltage due to a sharp increase in load current and compensating the output voltage immediately. The proposed LDO is fabricated in a 65-nm low power (LP) CMOS process and demonstrates a maximum load current capacity of 270 mA. The input and output voltage ranges of the LDO are 0.5–1.2 and 0.15–1.15 V, respectively, with 12.7- $\mu \text{A}$ quiescent current and 99.99% peak current efficiency. The measured undershoot and settling time are 150 mV and 100 ns at a slew rate of 200 mA/3 ns, respectively, achieving a figure of merit (FoM) of 0.183 fs. [ABSTRACT FROM AUTHOR]

Job shops are an important production environment for low-volume high-variety manufacturing. Its scheduling has recently been formulated as an integer linear programming (ILP) problem to take advantages of popular mixed-integer linear programming (MILP) methods, e.g., branch-and-cut. When considering a large number of parts, MILP methods may experience difficulties. To address this, a critical but much overlooked issue is formulation tightening. The idea is that if problem constraints can be transformed to directly delineate the problem convex hull in the data preprocessing stage, then a solution can be obtained by using linear programming (LP) methods without combinatorial difficulties. The tightening process, however, is fundamentally challenging because of the existence of integer variables. In this article, an innovative and systematic approach is established for the first time to tighten the formulations of individual parts, each with multiple operations, in the data preprocessing stage. It is a major advancement of our previous work on problems with binary and continuous variables to integer variables. The idea is to first link integer variables to binary variables by innovatively combining constraints so that the integer variables are uniquely determined by the binary variables. With binary and continuous variables only, it is proved that the vertices of the convex hull can be obtained based on vertices of the LP problem after relaxing binary requirements. These vertices are then converted to tightened constraints for general use. This approach significantly improves our previous results on tightening individual operations. Numerical results demonstrate significant benefits on solution quality and computational efficiency. This approach also applies to other complex ILP and MILP problems with similar characteristics and fundamentally changes the way how such problems are formulated and solved. Note to Practitioners—Scheduling is an important but difficult problem in planning and operation of job shops. The problem has been recently formulated in an integer linear programming (ILP) form to take advantage of popular mixed-integer linear programming methods. Given an ILP problem, there must exist a linear programming (LP) formulation so that all of its vertices are also the vertices to the ILP problem. If such an LP problem can be found in the data preprocessing stage, then the corresponding ILP problem is tight and can be solved by using an LP method without difficulties. In this article, an innovative and systematic approach is established to tighten the formulations of individual parts, each with one or multiple operations. It is a major advancement of our previous work on problems with binary and continuous variables by novel exploitation of the relationship between integer and binary variables in job-shop scheduling. The resulting tightened constraints are characterized by part parameters and the length of the scheduling horizon and can be easily adjusted for other data sets. Results demonstrate significant benefits on solution quality and computational efficiency. This approach also applies to other complex ILP and MILP problems with similar characteristics and fundamentally changes the way how such problems are formulated and solved. [ABSTRACT FROM AUTHOR]

Spot welding is a crucial process step in various industries. However, classification of spot welding quality is still a tedious process due to the complexity and sensitivity of the test material, which drain conventional approaches to its limits. In this article, we propose an approach for quality inspection of spot weldings using images from laser thermography data. We propose data preparation approaches based on the underlying physics of spot-welded joints, heated with pulsed laser thermography by analyzing the intensity over time and derive dedicated data filters to generate training datasets. Subsequently, we utilize convolutional neural networks to classify weld quality and compare the performance of different models against each other. We achieve competitive results in terms of classifying the different welding quality classes compared to traditional approaches, reaching an accuracy of more than 95 percent. Finally, we explore the effect of different augmentation methods.

Medical experts employ electroencephalography (EEG) for analyzing the electrical activity in the brain to infer disorders. However, the time costs of human experts are very high, and the examination of EEGs by such experts, therefore, accounts for a plethora of medical resources. In this study, an improved one-dimensional CNN-only system of 25 layers has been proposed to identify abnormal and normal adult EEG signals using a single EEG montage without using any explicit feature extraction technique. Most of the previous systems based on deep learning, that have been proposed to solve this problem, use extremely deep architectures containing very large numbers of layers. This study also presents an independent preprocessing module that has been exhaustively evaluated for optimal parameters with the target of adult EEG signal classification. The achieved accuracy of the proposed classifier as a part of the decision support system is 82.24%, which is a substantial improvement of ~3% over the previous best reported classifier of comparable depth. The system also exhibits significantly higher F1-score and sensitivity as well as lower loss. The proposed system is intended to be a part of an expert system for overall brain health evaluation.

Optical motion capture (MOCAP) is a commonly used technology to record the motion of non-rigid objects with high accuracy in 3D space. However, the MOCAP data has to be processed further before it can be used. The scattered reconstructed motion data must constitute a human configuration by labelling process according to the predefined template, and the missing markers have to be reconstructed to produce a stable motion trajectory. In this work, we propose a novel labelling method for motion sequences. First, a novel graph matching method is employed to determine the connection relationship of the scattered motion data for a single frame. Then, Kalman filtering is used for tracking in the motion sequence. As for the challenge coming from missing markers, we propose a new motion data preprocessing method considering the bone length constraint, which represents the information of variation in the relative position of adjacent markers. The processed motion data is input into a Long-Short Term Memory (LSTM) model to recover the missing markers and de-noise the motion data. The experiment conducted on our own dataset proves that our labelling method achieves a similar effect to Cortex, which is a commonly used commercial motion data analysis software. The experiment on CMU dataset demonstrates that our missing marker reconstruction method can achieve an art-of-state result. The labelling code will be pulished on https://github.com/Lijianfang6930/Graph-Matching-for-Marker-Labelling.

Methods to quantify the complexity of trajectory datasets are still a missing piece in benchmarking human trajectory prediction models. In order to gain a better understanding of the complexity of trajectory prediction tasks and following the intuition, that more complex datasets contain more information, an approach for quantifying the amount of information contained in a dataset from a prototype-based dataset representation is proposed. The dataset representation is obtained by first employing a non-trivial spatial sequence alignment, which enables a subsequent learning vector quantization (LVQ) stage. A large-scale complexity analysis is conducted on several human trajectory prediction benchmarking datasets, followed by a brief discussion on indications for human trajectory prediction and benchmarking.

The datasets that appear in publications are curated and have been split into training, testing and validation sub-datasets by domain experts. Consequently, machine learning models typically perform well on such split-by-hand prepared datasets. Whereas preparing real-world datasets into curated split training, testing and validation sub-dataset requires extensive effort. Usually, repetitive random splits are carried out and trained and evaluated on until reaching out a good score on the evaluation metrics. In this paper, an algorithmic method is proposed for preparing the sub-datasets splits for machine learning models. The objective of the proposed method is to achieve an evenly representative splits out of the dataset with standard and algorithmic way that reduce the perplexity of random splitting.

Due to the rapid developments in technology and the sudden expansion of social media use, Dialect Arabic has become an important source of data that needs to be addressed when building Arabic corpora. In this paper, thirty-three Arabic corpora are surveyed to show that despite all of the developments in the literature, Saudi dialect (SD) corpora still need further expansion. This paper contributes to the literature on SD corpora by creating the largest Saudi corpus – the King Saud University Saudi Corpus (KSUSC) – with +1B total words, including +119M SD words. The KSUSC not only is the newest and largest SD corpus but is also diverse, covering 26 domains in text collected from five different sources. This paper also contributes to the literature by developing a new incremental preprocessing system that is used to create relevant lexicons that are then used to clean and normalize the collected data. This incremental system is scalable and can be adapted for different resources and dialects. Moreover, the collection process for building the KSUSC is discussed in detail, and the challenges in collecting SD text with respect to each platform are highlighted. By the end of this paper, different design criteria are proposed and used with the KSUSC to conclude that the resulting corpus can be of great benefit to researchers who are interested in integrating the corpus with their own work or using its resulting lexicons with Saudi-based NLP tasks.

Many machine learning algorithms and almost all deep learning architectures are incapable of processing plain texts in their raw form. This means that their input to the algorithms must be numerical in order to solve classification or regression problems. Hence, it is necessary to encode these categorical variables into numerical values using encoding techniques. Categorical features are common and often of high cardinality. One-hot encoding in such circumstances leads to very high dimensional vector representations, raising memory and computability concerns for machine learning models. This paper proposes a deep-learned embedding technique for categorical features encoding on categorical datasets. Our technique is a distributed representation for categorical features where each category is mapped to a distinct vector, and the properties of the vector are learned while training a neural network. First, we create a data vocabulary that includes only categorical data, and then we use word tokenization to make each categorical data a single word. After that, feature learning is introduced to map all of the categorical data from the vocabulary to word vectors. Three different datasets provided by the University of California Irvine (UCI) are used for training. The experimental results show that the proposed deep-learned embedding technique for categorical data provides a higher F1 score of 89% than 71% of one-hot encoding, in the case of the Long short-term memory (LSTM) model. Moreover, the deep-learned embedding technique uses less memory and generates fewer features than one-hot encoding.

In batch processes, the ability to learn from previous process data results in high-value and batch-improved products. For batch processes with constraints and state-dependent uncertainty, this article presents a conic iterative learning control (ILC) approach, which uses cone theory to incorporate historical process data into optimization-based ILC design. The proposed conic ILC approach uses rank conditioning to select distinct data samples and conic mapping to map the data to the to-be-optimized control input variables, since adding all historical process data would be computationally intensive. Our method yields a tradeoff between learning ability from historical experience and computational efficiency from solving the optimization problem. Provable constraint satisfaction and robust stability are considered separately. To demonstrate the proven properties and effectiveness of the approach, we present a case study of the injection molding process. [ABSTRACT FROM AUTHOR]

Weight-bearing is a common behavior, whereas a load impact on human gait symmetry is seldom considered. This study aims to propose and verify a gait symmetry detection model, by using the gait symmetry under different weight-bearing states to establish a weight-bearing symmetry classification rule, and to measure the consistency of movement through multiple gait symmetry experiments. In details, a classifier combined with statistical learning is presented to quantify the signal difference between left and right legs. The quantization result is incorporated as an index to judge the magnitude of symmetry. Meanwhile, a dynamic time warping (DTW) algorithm is introduced to calculate the signal similarity of two legs in time domain as well as another index. The symmetry trend on leg signals is also considered and described as a third index. By casting these indexes in the proposed detection model, we employ several existing nonlinear classification methods to effectively disclose the difference between legs. Moreover, the information between legs can also be extracted through time warping, yielding the detection model competitive. The validity of the model is verified by hypothetical test. Compared with some state-of-the-art methods, our model achieves an improvement over 5% in the consistency measure of the calf. The symmetry measure of calf and thigh is also test. The proposed method outperforms other methods with the improvement of 12.5%. [ABSTRACT FROM AUTHOR]

An under-sampling digital predistortion based on a multi-tone mixing feedback technique (MTM-DPD) is presented in this article. In contrast to the under-sampling digital predistortion (DPD) based on conventional receivers, the proposed method could simultaneously reduce the sampling rate and acquisition bandwidth using a multi-tone mixing (MTM)-based feedback channel. The MTM-DPD is implemented with an evolved direct learning architecture, where the coefficients of a predistorter are estimated based on the aliased feedback signal, preprocessed input signal, and preprocessed kernels of the DPD model. To preprocess the input signal and kernels, the behavioral model of the MTM-based feedback channel and its corresponding identification algorithm are analyzed in detailed. Experimental results show that for a gallium nitride (GaN)-based power amplifier (PA) with 40-MHz input, under the excitation of a 17 tone LO signal with frequency interval 7.68 MHz, the MTM-DPD with sampling rate 15.36 MSPS and acquisition bandwidth 15.36 MHz could improve the adjacent channel power ratio (ACPR) from −30.1 to −46.5 dBc and the error vector magnitude (EVM) from 9.13% to 1.76%. Compared with the conventional full sampling DPD (FS-DPD), the MTM-DPD could reduce the cost of analog-to-digital converters (ADCs) in feedback channel significantly while maintaining satisfactory linearization performance. [ABSTRACT FROM AUTHOR]

Deep learning models are widely used in the field of Device-Free Localization (DFL), and their performances rely heavily on the labeled dataset. In this letter, a novel data preprocessing method is proposed for exploiting the full potential of a collected dataset in DFL. Firstly, we propose to employ a pix-level multimodal representation of radio images for a fusion of both amplitude and phase features from Channel State Information (CSI). Then, a novel Conditional Generative Adversarial Network with Auxiliary Classifier (AC-GAN) model is used to generate artificial samples for further expanding the collected dataset. We further employ a regression formulation to train the Convolutional Neural Networks (CNNs) for positioning. The experimental results show that using the proposed method gains approximately 18% Root-Mean-Square-Error (RMSE) improvement over the CSI amplitude-only method and 22% over the CSI phase-only method. [ABSTRACT FROM AUTHOR]

Myocardial infarctions and heart failure are the cause of more than 17 million deaths annually worldwide. ST-segment elevation myocardial infarctions (STEMI) require timely treatment, because delays of minutes have serious clinical impacts. Machine learning can provide alternative ways to predict heart failure and identify genes involved in heart failure. For these scopes, we applied a Random Forests classifier enhanced with feature elimination to microarray gene expression of 111 patients diagnosed with STEMI, and measured the classification performance through standard metrics such as the Matthews correlation coefficient (MCC) and area under the receiver operating characteristic curve (ROC AUC). Afterwards, we used the same approach to rank all genes by importance, and to detect the genes more strongly associated with heart failure. We validated this ranking by literature review and gene set enrichment analysis. Our classifier employed to predict heart failure achieved MCC = +0.87 and ROC AUC = 0.918, and our analysis identified KLHL22, WDR11, OR4Q3, GPATCH3, and FAH as top five protein-coding genes related to heart failure. Our results confirm the effectiveness of machine learning feature elimination in predicting heart failure from gene expression, and the top genes found by our approach will be able to help biologists and cardiologists further our understanding of heart failure. [ABSTRACT FROM AUTHOR]

This work presents two methods that facilitate a 3D reconstruction of microscopic blood vessels in the volume slightly larger than 1 mm3. The source of the data are histological serial sections, i.e., microscopic images of probes, stained with immunohistochemistry. Odd and even sections have different stainings in our primary data set. Thus, firstly, an approach to register an alternately-stained series is presented. With image filtering and a feature-detection-based registration we obtain a registered stack of 148 serial sections. The series has missing sections, locally damaged sections, artifacts from acquisition. All these hinder correct connectivity of blood vessels. With our second approach we interpolate the missing information while maintaining the connectivity. We achieve this with deformations based on dense optical flow. The presented methodology is applicable to further histological series. A combination of both approaches allows us to reconstruct more than 76% larger volumes. An important detail was the composition mode of images. Summarizing, we use methods from image processing and computer vision to create large-scale 3D models from immunostained histological serial sections.

Imbalanced class distribution and missing data are two common problems and occurrences in water quality anomaly detection domain. Learning algorithms in an imbalanced dataset can yield an overrated classification accuracy driven by a bias towards the majority class at the expense of the minority class. On the other hand, missing values in data can induce complexity in the learning classifiers during data analysis. These two problems pose substantial challenges to the performance of learning algorithms in real-life water quality anomaly detection problems. Hence, the need for them to be carefully considered and addressed to achieve better performance. In this paper, the performance of a range of several combinations of techniques to deal with imbalanced classes in the context of binary-imbalanced water quality anomaly detection problem and the presence of missing values is extensively compare. The methods considered include seven missing data and eight resampling methods, on ten different learning state-of-the-art classifiers taking into account diversity in their learning philosophies. The different classifiers are evaluated using stratified 5-fold cross-validation, based on three performance evaluation metrics namely accuracy, ROC-AUC and F1-measure. Further experiments are carried out on nineteen variants of homogeneous and heterogeneous ensemble techniques embedded with resampling and missing value strategies during their training phase as well as an optimized deep neural network model. The experimental results show an improvement in the performance of the learning classifiers, especially when dealing with the class imbalance problem (on the one hand) and the incomplete data problem (on the other hand). Furthermore, the neural network model exhibit superior performance when dealing with both problems.

The instance reduction is one of the data preprocessing methods and aims to remove noises and (or) redundant instances from the training set. In the instance reduction, one of the most representative techniques is the edition method which can remove harmful instances from the training set to improve the prediction accuracy of k nearest neighbor (KNN). Nevertheless, most of existing edition methods still have some drawbacks, such as the parameter dependency, high computational time and relatively low accuracy. To solve these problems, we present a new fast parameter-free edition method based on local sets with natural neighbors (ELS). In ELS, we define a new concept of local sets by introducing natural neighbors. ELS can use the local sets to keep more reasonable class boundaries and effectively filtering out noisy instances (including global outliers). The main advantages of ELS are that (a) it is parameter-free; (b) it can remove global outliers and noisy instances; (c) it is relatively fast. Experiments clearly verify that (a) ELS outperforms existing representative edition methods in improving the prediction accuracy of KNN; (b) ELS can improve the performance of the condensation method and hybrid method in terms of both accuracy and reduction; (c) ELS consumes relatively low running time.

Multistep wind speed forecasting is of great significance for wind energy utilization. To further improve the performance of multistep wind speed forecasting, a new multistep forecasting strategy is proposed based on the basic wind speed multistep forecasting strategy and the corresponding forecasting system is established. The system consists of three parts, including a wavelet decomposition preprocessing module, nonlinear autoregressive artificial neural network and nonlinear autoregressive exogenous artificial neural network composite prediction module, and support vector machine classifier postprocessing module. The system analyzes historical wind speed data from two different angles to obtain dual time series features. Then, the double time series feature is used to improve the accuracy and stability of the multipart forecast. The predictive performance of the system is verified through two experiments. Experiment I tests the prediction accuracy and computing resource consumption of the system. Experiment II tests the stability of the system prediction under different wind conditions. The results show that, compared with the traditional multistep prediction strategy system, the new system has a better prediction accuracy and more stable prediction performance.

The Terracotta Warriors are terracotta sculptures created for China's first emperor more than 2,000 years ago. They are among the most precious unearthed cultural relics of China. However, these relics have been predominantly found in fragments. Fragment classification is currently performed manually on enormous quantities of fragments, which is a time-consuming, inaccurate, and subjective task for archaeologists and conservators. In this study, an automatic method based on a deep learning network combined with template guidance is proposed to classify 3D fragments of the Terracotta Warriors. The fragments are initially classified using PointNet. Then, misclassified fragments are secondly categorized based on their best match to a complete Terracotta Warrior model. Extensive experiments were performed to verify the effectiveness of the proposed method. The promising results demonstrate that the method is the most accurate technique for classifying 3D Terracotta Warrior fragments to date. Moreover, the proposed method can significantly increase the efficiency of future fragment reassembly for the Terracotta Warriors.

GPS trajectories are always embedded with errors, due to the weather or environmental variables. Existing trajectory repairing methods have employed Kalman filters or sequential data cleaning methods. Kalman filter or its variants change all observed measurements, while generally most measurements are originally accurate. Sequential data cleaning methods are mainly applied on one-dimensional data sequences, and when encountering multi-dimensional trajectories, their performance will be compromised due to that the features of multi-dimensional trajectories are not fully utilized. To address these issues, we propose to repair GPS trajectory with movement tendencies, speed change tendency, travel distance tendency and repair distance tendency. We formalize the tendency based trajectory repairing, and propose an exact solution to find the repair which minimize movement tendency score. Then we propose high quality candidate selection and dynamic error range estimation, to improve the efficiency and effectiveness of exact solution. Experiments on three data sets demonstrate the superiority of our proposal.

Smoking is associated with cancer, cardiovascular disease and premature death and can cause severe fire hazards. To assist with smoking cessation, this paper presents a wireless body area network-based system consisting of two off-the-shelf devices, one smartphone and one smartwatch, to detect smoking events by mining the inertial sensor data from both devices. In the system, an end-to-end trainable unified model is implemented by combining a variational autoencoder with a random forest to classify the collected data into smoking and nonsmoking after data preprocessing. The variational autoencoder is adopted to learn the feature representation and deal with the class imbalance problem, and the stochastic decision forest is adopted to guide the global optimization of the parameter learning process. Extensive validation of our scheme is performed on a large dataset we collected, and our scheme yields quite promising results in terms of accuracy and efficiency.

Rotating machinery plays a critical role in many significant fields. However, the unpredictable machinery faults may lead to the severe damage and losses. Hence, it is of great value to explore the precise approaches for fault diagnosis. With the development of the intelligent fault diagnosis methods based on deep learning, convolutional neural network (CNN) has aroused the attention of researchers in machinery fault diagnosis. In the light of the reduction of difficulty in feature learning and the improvement of final diagnosis accuracy, data preprocessing is necessary and crucial in CNN-based fault diagnosis methods. This review focuses on CNN-based fault diagnosis approaches in rotating machinery. Firstly, data preprocessing methods are overviewed. Then, we emphatically analyze and discuss several main techniques applied in CNN-based intelligent diagnosis, principally including the fast Fourier transform, wavelet transform, data augmentation, S-transform, and cyclic spectral analysis. Finally, the potential challenges and research objects are prospected on data preprocessing in intelligent fault diagnosis of rotary machinery.

Microfossils, tiny fossils whose study requires the use of a microscope, have been widely applied in many fields of earth, life, and environmental sciences. The abundance and high diversity of microfossils, as well as the need for rapid identification, call for automated methods to classify microfossils. In this study, we constructed an open dataset of three-dimensional (3D) microfossils and proposed a deep learning-based approach for microfossil classification. The dataset, named `Archives of Digital Morphology' (ADMorph), currently contains more than ten thousand 3D models from five classes of 410 million-year-old fishes. The deep learning-based method includes data preprocessing, feature extraction, and 3D microfossil model classification. To assess the method performance and dataset representability, we performed extensive experiments. Compared with multiview convolutional neural networks (MVCNN) (91.54%), PointNet (64.13%), and VoxNet (78.15%), the method proposed herein had higher accuracy (97.60%) on the experimental dataset. We also verified data preprocessing (92.36%) and feature extraction (97.10%). We combined them to obtain the macroaveraging accuracy of 97.60%, the highest accuracy of 100%, and the lowest accuracy of 88.78%. We suggest that the proposed method can be applied to other 3D fossils and biomorphological research fields. The fast-accumulating 3D fossil models might become a source of information-rich datasets for deep learning.

Machine learning for author name disambiguation is usually conducted on the training and test subsets of labeled data created for a specific task. As a result, disambiguation models learned on heterogeneous labeled data are often inapplicable for other purposes that either do not use the same labeled data or do not make use of any labeled data at all. This article explores the idea of transfer learning in a new context, author name disambiguation. We focus on cases where a disambiguation task lacking labeled training data uses models trained on labeled data generated for other tasks. For this purpose, two labeled source datasets are used for training of disambiguation models to be applied to three test target datasets that are deficient of labeled training data. Our results show that transfer learning can produce disambiguation performances similar to those achievable by traditional machine learning in which training and test datasets come from the same labeled data source. The good performance through transfer learning are possible when training source datasets have similar feature distributions as test target datasets. This study suggests that through transfer learning, rich disambiguation models in previous studies can be retained and reused across ambiguous bibliographic data from different fields and data sources, motivating further research on how to correct feature distribution differences between source and target datasets to expand the application of transfer learning in author name disambiguation beyond the model sharing explored in this research.

A novel hybrid ensemble framework is developed to forecast the short-term wind speed, which consists of a data preprocessing technique, data-driven based forecasting algorithms, and an improved Jaya algorithm. In the data preprocessing process, the pauta criterion is employed to find out the outliers, and the variational mode decomposition algorithm decompose the original series to extract the trend and time-frequency information of the historical inputs. The data-driven forecasting algorithms, such as BP, LSSVM, ANFIS, and Elman, are exploited as the original predictor of the framework, while the weights of the predictors are computed by an improved optimization algorithm-CLSJaya. Based on the experimental results of two time-scale datasets from three sites, the proposed framework successfully overcomes the limitations of the individual forecasting models and achieves promising forecasting accuracy.

Telecom companies are increasing their efforts in customer retention because acquiring new customers often costs much more than retaining existing ones. Therefore, it is important for operators to predict customer churns rapidly and accurately. Machine learning (ML) has been widely used for predictive churn modeling. However, classical ML methods require manual feature selection and time-consuming data preprocessing steps. To overcome these limitations, there is a paradigm shift toward deep learning (DL) for predicting churners. Although DL appears to be promising, the existing literature lacks comparative analysis of ML and DL techniques using benchmark churn datasets. Additionally, various DL architectures must be empirically evaluated to determine which type works best on churn data. We present a system for comparative analysis of learning architectures. Two benchmarked datasets, Cell2Cell and KDD Cup, serve as a use case of our system to provide insights on the extent of improvement DL can bring over classical ML. Four popular evaluation measures are used to compare the performance of popular DL architectures. Our experiments found that convolutional neural networks gave the best results in both use cases. [ABSTRACT FROM AUTHOR]