Search

Highlights • We propose an active semi-supervised learning algorithm with multiple criteria. • The criteria are representativeness, diversity, and variance reduction of a model. • We use clustering information to develop representativeness and diversity criteria. • The proposed algorithm is useful to avoid selection of undesirable samples. Abstract In many practical machine learning problems, the acquisition of labeled data is often expensive and time consuming. To reduce this labeling cost, active learning has been introduced in many scientific fields. This study considers the problem of active learning of a regression model in the context of an optimal experimental design. Classical optimal experimental design approaches are based on the least square errors of labeled samples. Recently, a couple of active learning approaches that take advantage of both labeled and unlabeled data have been developed based on Laplacian regularized regression models with a single criterion. However, these approaches are susceptible to selecting undesirable samples when the number of initially labeled samples is small. To address this susceptibility, this study proposes an active learning method that considers multiple complementary criteria. These criteria include sample representativeness, diversity information, and variance reduction of the Laplacian regularization model. Specifically, we developed novel density and diversity criteria based on a clustering algorithm to identify the samples that are representative of their distributions, while minimizing their redundancy. Experiments were conducted on synthetic and benchmark data to compare the performance of the proposed method with that of existing methods. Experimental results demonstrate that the proposed active learning algorithm outperforms its existing counterparts. [ABSTRACT FROM AUTHOR]

Classification is an important task in various areas. In many real-world applications, class imbalance and overlapping problems have been reported as major issues in the application of traditional classification algorithms. An imbalance problem occurs when training data contain considerably more representatives of one class than of other classes. Class overlap occurs when a region in the data space contains a similar number of data for each class. When a class overlap occurs in imbalanced data sets, classification becomes even more complicated. Although various approaches have been proposed to deal separately with class imbalance and overlapping problems, only a few studies have attempted to address both problems simultaneously. In this paper, we propose an overlap-sensitive margin (OSM) classifier based on a modified fuzzy support vector machine and k -nearest neighbor algorithm to address imbalanced and overlapping data sets. The main idea of the proposed OSM classifier is to separate the data space into soft- and hard-overlap regions using the modified fuzzy support vector machine algorithm. The separated spaces are then classified using the decision boundaries of the support vector machine and 1-nearest neighbor algorithms. Furthermore, by separating a data set into soft- and hard-overlap regions, one can determine which part of the data is to be examined more closely for classification in real-world situations. Experiments using synthetic and real-world data sets demonstrated that the proposed OSM classifier outperformed existing methods for imbalanced and overlapping situations. [ABSTRACT FROM AUTHOR]

Content-based filtering (CBF), one of the most successful recommendation techniques, is based on correlations between contents. CBF uses item information, represented as attributes, to calculate the similarities between items. In this study, we propose a novel CBF method that uses a multiattribute network to effectively reflect several attributes when calculating correlations to recommend items to users. In the network analysis, we measure the similarities between directly and indirectly linked items. Moreover, our proposed method employs centrality and clustering techniques to consider the mutual relationships among items, as well as determine the structural patterns of these interactions. This mechanism ensures that a variety of items are recommended to the user, which improves the performance. We compared the proposed approach with existing approaches using MovieLens data, and found that our approach outperformed existing methods in terms of accuracy and robustness. Our proposed method can address the sparsity problem and over-specialization problem that frequently affect recommender systems. Furthermore, the proposed method depends only on ratings data obtained from a user's own past information, and so it is not affected by the cold start problem. [ABSTRACT FROM AUTHOR]

Forecasting time series data is one of the most important issues involved in numerous applications in real life. Time series data have been analyzed in either the time or frequency domains. The objective of this study is to propose a forecasting method based on wavelet filtering. The proposed method decomposes the original time series into the trend and variation parts and constructs a separate model for each part. Simulation and real case studies were conducted to examine the properties of the proposed method under various scenarios and compare its performance with time series forecasting models without wavelet filtering. The results from both simulated and real data showed that the proposed method based on wavelet filtering yielded more accurate results than the models without wavelet filtering in terms of mean absolute percentage error criterion. [ABSTRACT FROM AUTHOR]

Feature selection based on an ensemble classifier has been recognized as a crucial technique for modeling high-dimensional data. Feature selection based on the random forests model, which is constructed by aggregating multiple decision tree classifiers, has been widely used. However, a lack of stability and balance in decision trees decreases the robustness of random forests. This limitation motivated us to propose a feature selection method based on newly designed nearest-neighbor ensemble classifiers. The proposed method finds significant features by using an iterative procedure. We performed experiments with 20 datasets of microarray gene expressions to examine the property of the proposed method and compared it with random forests. The results demonstrated the effectiveness and robustness of the proposed method, especially when the number of features exceeds the number of observations. [ABSTRACT FROM AUTHOR]

Abstract: Multivariate control charts have been widely used in many industries to monitor and diagnose processes characterized by a large number of quality characteristics. Usually, these characteristics are highly correlated with each other. The direct use of conventional multivariate control charts for situations with highly correlated characteristics may lead to increased rates of false alarms. Principal component analysis (PCA) control charts have been widely used to address problems posed by such high correlations by transforming the set of correlated variables to an uncorrelated set of variables and then identifying the PCs with highest contribution which then allows one to reduce dimensionality. However, an assumption that the data are normally distributed underlies the construction of the control limits of traditional PCA control charts. This assumption has limited the use of PCA control charts in nonnormal situations found in many modern systems. This study presents the development of nonparametric PCA control charts that do not require any distributional assumptions for their construction. We propose to use nonparametric techniques, kernel density estimation, and bootstrapping to establish the control limits of these charts. A simulation study was conducted to evaluate the performance of the proposed charts and compare them with traditional PCA control charts. The comparative performance in terms of average run length showed that the proposed nonparametric PCA control charts performed better than the parametric PCA control charts in nonnormal situations. [Copyright &y& Elsevier]

Abstract: Process monitoring and diagnosis have been widely recognized as important and critical tools in system monitoring for detection of abnormal behavior and quality improvement. Although traditional statistical process control (SPC) tools are effective in simple manufacturing processes that generate a small volume of independent data, these tools are not capable of handling the large streams of multivariate and autocorrelated data found in modern systems. As the limitations of SPC methodology become increasingly obvious in the face of ever more complex processes, data mining algorithms, because of their proven capabilities to effectively analyze and manage large amounts of data, have the potential to resolve the challenging problems that are stretching SPC to its limits. In the present study we attempted to integrate state-of-the-art data mining algorithms with SPC techniques to achieve efficient monitoring in multivariate and autocorrelated processes. The data mining algorithms include artificial neural networks, support vector regression, and multivariate adaptive regression splines. The residuals of data mining models were utilized to construct multivariate cumulative sum control charts to monitor the process mean. Simulation results from various scenarios indicated that data mining model-based control charts performs better than traditional time-series model-based control charts. [Copyright &y& Elsevier]

Abstract: Feature selection has received considerable attention in various areas as a way to select informative features and to simplify the statistical model through dimensional reduction. One of the most widely used methods for dimensional reduction includes principal component analysis (PCA). Despite its popularity, PCA suffers from a lack of interpretability of the original feature because the reduced dimensions are linear combinations of a large number of original features. Traditionally, two or three dimensional loading plots provide information to identify important original features in the first few principal component dimensions. However, the interpretation of what constitutes a loading plot is frequently subjective, particularly when large numbers of features are involved. In this study, we propose an unsupervised feature selection method that combines weighted principal components (PCs) with a thresholding algorithm. The weighted PC is obtained by the weighted sum of the first k PCs of interest. Each of the k loading values in the weighted PC reflects the contribution of each individual feature. We also propose a thresholding algorithm that identifies the significant features. Our experimental results with both the simulated and real datasets demonstrated the effectiveness of the proposed unsupervised feature selection method. [ABSTRACT FROM AUTHOR]

Abstract: Functional data analysis has gained significant attention from a variety of disciplines. In the present study we propose an effective clustering procedure to categorize a number of profiles that are formed with nonlinear functions. The proposed clustering procedure first smoothes the data and then transforms the smoothed data to obtain their functional form. The coefficients of the function obtained from the preceding transformation step are used for clustering. Simulation studies under various scenarios indicated that our proposed clustering procedure correctly identified the true clusters and yielded better clustering results than a latent class cluster analysis, one of the existing clustering methods. Furthermore, the effectiveness of the proposed clustering procedure was demonstrated using real pain data in which the main objective is to characterize the responses of 144 spinal cord dorsal horn neurons with graded thermal stimuli that range from 37°C to 51°C in 2°C increments. The results showed that the proposed clustering strategy can successfully elicit natural grouping of the neurons with similar response patterns to graded thermal stimuli. [Copyright &y& Elsevier]

Abstract: The main purpose of this study is to develop an effective classification procedure that discriminates between normal spectra and cancerous spectra in near infrared (NIR) spectroscopic data in which the classes are highly imbalanced and overlapped. Our proposed procedure consists of several steps. First, to ensure the comparability between spectra, normalization was done by dividing each spectral point by the area of the total intensity of the spectrum. Second, clustering analysis was performed with these normalized spectra to separate the spectra that represent the normal pattern from a mixed group that contains both normal and tumor spectra. Third, we conducted two-stage classification, the first being an effort to construct a classification model with the labels obtained from the preceding clustering analysis and the second being a classification to focus on the mixed group classified from the first classification model. To increase the accuracy, the second classification model was constructed based on the selected features that capture important characteristics of the spectral data. Our proposed procedure was evaluated by its classification ability in testing samples using a leave-one-out cross validation technique, yielding acceptable classification accuracy. [Copyright &y& Elsevier]

Abstract: Feature selection in metabolomics can identify important metabolite features that play a significant role in discriminating between various conditions among samples. In this paper, we propose an efficient feature selection method for high-resolution nuclear magnetic resonance (NMR) spectra obtained from time-course experiments. Our proposed approach combines linear-mixed effects (LME) models with a multiple testing procedure based on a false discovery rate. The proposed LME approach is illustrated using NMR spectra with 574 metabolite features obtained for an experiment to examine metabolic changes in response to sulfur amino acid intake. The experimental results showed that classification models constructed with the features selected by the proposed approach resulted in lower rates of misclassification than those models with full features. Furthermore, we compared the LME approach with the two-sample t-test approach that oversimplifies the time-course factor. [Copyright &y& Elsevier]

High-resolution nuclear magnetic resonance (NMR) spectroscopy has provided a new means for detection and recognition of metabolic changes in biological systems in response to pathophysiological stimuli and to the intake of toxins or nutrition. To identify meaningful patterns from NMR spectra, various statistical pattern recognition methods have been applied to reduce their complexity and uncover implicit metabolic patterns. In this paper, we present a genetic algorithm (GA)-based feature selection method to determine major metabolite features to play a significant role in discrimination of samples among different conditions in high-resolution NMR spectra. In addition, an orthogonal signal filter was employed as a preprocessor of NMR spectra in order to remove any unwanted variation of the data that is unrelated to the discrimination of different conditions. The results of k-nearest neighbors and the partial least squares discriminant analysis of the experimental NMR spectra from human plasma showed the potential advantage of the features obtained from GA-based feature selection combined with an orthogonal signal filter. [Copyright &y& Elsevier]

Dynamic programming (DP) is a mathematical programming approach for optimizing a system that changes over time and is a common approach for developing intelligent systems. Expert systems that are intelligent must be able to adapt dynamically over time. An optimal DP policy identifies the optimal decision dependent on the current state of the system. Hence, the decisions controlling the system can intelligently adapt to changing system states. Although DP has existed since Bellman introduced it in 1957, exact DP policies are only possible for problems with low dimension or under very limiting restrictions. Fortunately, advances in computational power have given rise to approximate DP (ADP). However, most ADP algorithms are still computationally-intractable for high-dimensional problems. This paper specifically considers continuous-state DP problems in which the state variables are multicollinear. The issue of multicollinearity is currently ignored in the ADP literature, but in the statistics community it is well known that high multicollinearity leads to unstable (high variance) parameter estimates in statistical modeling. While not all real world DP applications involve high multicollinearity, it is not uncommon for real cases to involve observed state variables that are correlated, such as the air quality ozone pollution application studied in this research. Correlation is a common occurrence in observed data, including sources in meteorology, energy, finance, manufacturing, health care, etc. ADP algorithms for continuous-state DP achieve an approximate solution through discretization of the state space and model approximations. Typical state space discretizations involve full-dimensional grids or random sampling. The former option requires exponential growth in the number of state points as the state space dimension grows, while the latter option is typically inefficient and requires an intractable number of state points. The exception is computationally-tractable ADP methods based on a design and analysis of computer experiments (DACE) approach. However, the DACE approach utilizes ideal experimental designs that are (nearly) orthogonal, and a multicollinear state space will not be appropriately represented by such ideal experimental designs. While one could directly build approximations over the multicollinear state space, the issue of unstable model approximations remains unaddressed. Our approach for handling multicollinearity employs data mining methods for two purposes: (1) to reduce the dimensionality of a DP problem and (2) to orthogonalize a multicollinear DP state space and enable the use of a computationally-efficient DACE-based ADP approach. Our results demonstrate the risk of ignoring high multicollinearity, quantified by high variance inflation factors representing model instability. Our comparisons using an air quality ozone pollution case study provide guidance on combining feature selection and feature extraction to guarantee orthogonality while achieving over 95% dimension reduction and good model accuracy. [ABSTRACT FROM AUTHOR]

Abstract: The main objective of the present paper is to characterize smoking behavior among older adults by assessing the psychological distress, physical health status, alcohol use, and demographic variables in relations to the current smoking. We targeted 466 senior American smokers who are 65 years of age or older from the 2006 National Survey on Drug Use and Health (NSDUH, 2006). We employed a decision tree algorithm to conduct classification analysis to find the relationship between the average numbers of cigarette use per day. The results showed that the most important explanatory variable for prediction of the average number of cigarette use per day is the age when first started smoking cigarettes every day, followed by education level, and psychological distress. These results suggest that social workers need to provide more customized and individualized intervention to older adults. [Copyright &y& Elsevier]

Abstract: This paper presents a statistical analysis on the environmental impact of airport deicing activities at Dallas-Fort Worth (D/FW) International Airport. The focus of this paper is on aircraft deicing, which typically uses a spray of aircraft deicing and anti-icing fluids (ADAF). ADAF has a high concentration of ethylene/propylene/diethylene glycol, which shears off airfoil surfaces during takeoff and drips to the runways during taxiing. A significant portion of the glycol runs off and mixes with the airport’s receiving waters during heavy deicing periods, resulting in bacterial growth that causes an increase in chemical oxygen demand (COD) and a subsequent reduction in dissolved oxygen (DO) in the receiving waters. In this study, statistical methods for data mining were employed to evaluate the impact of airport deicing activities on COD and DO in the receiving waters immediately surrounding D/FW Airport. In particular, decision tree models were developed to determine important explanatory variables for predicting levels of COD and DO in the airport’s waterways. The decision tree modeling and analysis of COD determined north–south wind, glycol usage at a specific deicing pad, and monitoring site to be significant explanatory variables. The impact of glycol usage on DO was apparent as every decision tree at least one group with a median DO below 4.0mg/l, and these low-DO groups were associated with high glycol usage. These results are crucial to D/FW Airport in their goal to minimize the potential adverse impact of deicing activities on the water quality in waterways proximate to the airport. The advantages of data-driven modeling and analysis are its cost-effectiveness due to its potential to be implemented without making major changes in physical systems, ease of application, and usefulness in making future management decisions. [Copyright &y& Elsevier]

• We propose a predictive traffic control system for the autonomous vehicle routing problem. • We develop an applicable control system to complement the existing routing algorithm. • Proposed method can predict and prevent traffic congestion. • A real times simulation presents the utility and superiority of the proposed method. This study aimed to resolve a real-world traffic problem in a large-scale plant. Autonomous vehicle systems (AVSs), which are designed to use multiple vehicles to transfer materials, are widely used to transfer wafers in semiconductor manufacturing. Traffic control is a significant challenge with AVSs because all vehicles must be monitored and controlled in real time, to cope with uncertainties such as congestion. However, existing traffic control systems, which are primarily designed and controlled by human experts, are insufficient to prevent heavy congestion that impedes production. In this study, we developed a traffic control system based on machine learning predictions, and a routing method that dynamically determines AVS routes with reduced congestion rates. We predicted congestion for critical bottleneck areas, and utilized the predictions for adaptive routing control of all vehicles to avoid congestion. We conducted an experimental evaluation to compare the predictive performance of four popular algorithms. We performed a simulation study based on data from semiconductor fabrication to demonstrate the utility and superiority of the proposed method. The experimental results showed that AVSs with the proposed approach outperformed the existing approach in terms of delivery time, transfer time, and queuing time. We found that adopting machine learning-based traffic control can enhance the performance of existing AVSs and reduce the burden on the human experts who monitor and control AVSs. [ABSTRACT FROM AUTHOR]