Search

Wafer bin maps (WBMs) data, presented as images, play a critical role in identifying defects in the semiconductor industry. Thus, accurately classifying WBM defect patterns is essential to maintain high quality and enhance the overall yield. However, the task of labeling and classifying WBM data, which are generated daily in the tens of thousands or more, presents a challenge for experts. Recently, with advancements in artificial intelligence research, there has been a surge in efforts to automatically classify WBM defect patterns. Nevertheless, existing studies have primarily focus on classifying known defect patterns using labels. However, in the real-world semiconductor industry, new defect patterns are constantly emerging in addition to the known patterns. In this study, we propose the contrastive deep clustering (CODEC) for wafer bin maps that identifies new defective patterns in WBMs while simultaneously clustering these patterns into multiple defects without using labels. We use a contrastive loss function to address the challenges associated with a limited number of novel defect patterns. We demonstrate the effectiveness of our proposed methodology in accurately classifying new defect patterns using open data WM-811 k. [ABSTRACT FROM AUTHOR]

Root cause analysis (RCA) methods for effectively identifying the critical causes of abnormal processes have attracted attention because manufacturing processes have grown in scale and complexity. However, the existing methods for building automatic RCA models suffer from the disadvantage of typically requiring expert knowledge. In addition, without a dataset representing the causal relationship of multivariate processes, it is difficult to provide useful information for RCA. Although data-driven RCA methods have been proposed, most are based on classification models. Given that product quality is defined as a continuous variable in many manufacturing industries, classification models are limited in deriving root causes affecting the product quality level. In this article, we propose a regression model-based RCA method, which we call quality-discriminative localization, consisting of a convolutional neural network (CNN)-based activation mapping of multisensor signal data. In our proposed method, the CNN predicts the product quality of a continuous variable. Activation mapping then extracts causal maps that highlight significant sensor signals for each product. To identify the root causes, we generate a root cause map from the weighted sum of quality and causal maps. We consider root causes as locations of abnormal processes and processing times from localized activation scores on the root cause map. We experimentally demonstrate the usefulness of the proposed method with simulated data and real process data from a steel manufacturing process. Our results show that the proposed method successfully identifies root causes with distinct sensor signal patterns. [ABSTRACT FROM AUTHOR]

Multichannel signal data analysis has been crucial in various industrial applications, such as human activity recognition, vehicle failure predictions, and manufacturing equipment monitoring. Recently, deep neural networks have come into use for multichannel signal data because of their ability to automatically extract useful features from complex multichannel signals. However, deep neural networks are black-box models whose internal working mechanisms cannot be put in a form readily understood by humans. To address this issue, we have proposed an uncertainty-aware hierarchical segment-channel attention model that consists of a time segment and channel level attentions. The hierarchical attention mechanism enables a neural network to identify important time segments and channels critical for prediction, making the model explainable. In addition, the model uses variational inferences to provide uncertainty information that yields a confidence interval that can be easily explained. We conducted experiments on simulated and real-world datasets to demonstrate the usefulness and applicability of our method. The results confirm that our method can attend to important time segments and sensors while achieving better classification performance. [ABSTRACT FROM AUTHOR]

In real-time strategy games, players collect resources, control various units, and create strategies to win. The creation of winning strategies requires accurately analyzing previous games; therefore, it is important to be able to identify the key situations that determined the outcomes of those games. However, previous studies have mainly focused on predicting game results. In this study, we propose a methodology to predict outcomes and to identify information about the turning points that determine outcomes in StarCraft Ⅱ, one of the most popular real-time strategy games. We used replay data from StarCraft Ⅱ that is similar to video data providing continuous multiple images. First, we trained a result prediction model using 3D-residual networks (3D-ResNet) and replay data to improve prediction performance by utilizing in-game spatiotemporal information. Second, we used gradient-weighted class activation mapping to extract information defining the key situations that significantly influenced the outcomes of the game. We then proved that the proposed method outperforms by comparing 2D-residual networks (2D-ResNet) using only one time-point information and 3D-ResNet with multiple time-point information. We verified the usefulness of our methodology on a 3D-ResNet with a gradient class activation map linked to a StarCraft Ⅱ replay dataset. [ABSTRACT FROM AUTHOR]

Automatic identification of defect patterns in wafer bin maps (WBMs) stands as a challenging problem for the semiconductor manufacturing industry. Deep convolutional neural networks have recently shown decent progress in learning spatial patterns in WBMs, but only at the expense of explicit manual supervision. Unfortunately, a clean set of labeled WBM samples is often limited in both size and quality, especially during rapid process development or early production stages. In this study, we propose a self-supervised learning framework that makes the most out of unlabeled data to learn beforehand rich visual representations for data-efficient WBM defect pattern classification. After self-supervised pre-training based on noise-contrastive estimation, the network is fine-tuned on the available labeled data to classify WBM defect patterns. We argue that self-supervised pre-training with a vast amount of unlabeled data substantially improves classification performance when labels are scarce. We demonstrate the effectiveness of our work on a real-world public WBM dataset, WM-811K. The code is available at https://github.com/hgkahng/WaPIRL. [ABSTRACT FROM AUTHOR]

Highlights • We propose an active semi-supervised learning algorithm with multiple criteria. • The criteria are representativeness, diversity, and variance reduction of a model. • We use clustering information to develop representativeness and diversity criteria. • The proposed algorithm is useful to avoid selection of undesirable samples. Abstract In many practical machine learning problems, the acquisition of labeled data is often expensive and time consuming. To reduce this labeling cost, active learning has been introduced in many scientific fields. This study considers the problem of active learning of a regression model in the context of an optimal experimental design. Classical optimal experimental design approaches are based on the least square errors of labeled samples. Recently, a couple of active learning approaches that take advantage of both labeled and unlabeled data have been developed based on Laplacian regularized regression models with a single criterion. However, these approaches are susceptible to selecting undesirable samples when the number of initially labeled samples is small. To address this susceptibility, this study proposes an active learning method that considers multiple complementary criteria. These criteria include sample representativeness, diversity information, and variance reduction of the Laplacian regularization model. Specifically, we developed novel density and diversity criteria based on a clustering algorithm to identify the samples that are representative of their distributions, while minimizing their redundancy. Experiments were conducted on synthetic and benchmark data to compare the performance of the proposed method with that of existing methods. Experimental results demonstrate that the proposed active learning algorithm outperforms its existing counterparts. [ABSTRACT FROM AUTHOR]

We propose a method to select the subset of features in multiple linear regression models that considers the collinearity between features. The proposed method first detects collinear groups of features and then uses collinear groupwise feature selection constraints to estimate the coefficients of the regression model. The constraints simultaneously control the number of features selected and predefined collinear feature groups. We manage the multicollinearity in the regression model by controlling the parameters of the fusion group constraint. To address the NP-hard problem of the proposed method, we propose a modified discrete first-order algorithm. We use simulation and real-world data to demonstrate the usefulness of the proposed method by comparing it to existing regularization and discrete optimization-based methods in terms of predictive accuracy, bias, and variance. The comparison confirms that the proposed method outperforms the alternatives. [ABSTRACT FROM AUTHOR]

Classification is an important task in various areas. In many real-world applications, class imbalance and overlapping problems have been reported as major issues in the application of traditional classification algorithms. An imbalance problem occurs when training data contain considerably more representatives of one class than of other classes. Class overlap occurs when a region in the data space contains a similar number of data for each class. When a class overlap occurs in imbalanced data sets, classification becomes even more complicated. Although various approaches have been proposed to deal separately with class imbalance and overlapping problems, only a few studies have attempted to address both problems simultaneously. In this paper, we propose an overlap-sensitive margin (OSM) classifier based on a modified fuzzy support vector machine and k -nearest neighbor algorithm to address imbalanced and overlapping data sets. The main idea of the proposed OSM classifier is to separate the data space into soft- and hard-overlap regions using the modified fuzzy support vector machine algorithm. The separated spaces are then classified using the decision boundaries of the support vector machine and 1-nearest neighbor algorithms. Furthermore, by separating a data set into soft- and hard-overlap regions, one can determine which part of the data is to be examined more closely for classification in real-world situations. Experiments using synthetic and real-world data sets demonstrated that the proposed OSM classifier outperformed existing methods for imbalanced and overlapping situations. [ABSTRACT FROM AUTHOR]

One-class classification plays a key role in the detection of outliers and abnormalities. Recently, several attempts have been made to extend the application of one-class classification techniques to statistical process control problems, where many of these one-class classification-based approaches have used a support vector data description algorithm. The monitoring statistics for a support vector data description-based control chart are sufficiently defined. However, the control limits are not obvious because the procedure used to derive the control limit does not include a method for controlling the false alarm rate (i.e., Type I error rate), which clearly limits its use in process monitoring. In this study, we propose a new multivariate control chart based on a technique for optimal false alarm-controlled support vector data description, which minimizes the radius of a spherically shaped boundary so that it includes the normal data that are equal to an assigned constant value. By modifying this constant value, users can precisely control the proportion of abnormal data determined by the spherically shaped boundary, which equals the expected Type I error rate. We demonstrated the usefulness of the proposed charts in experiments with simulated data and real process data based on a thin film transistor–liquid crystal display. [ABSTRACT FROM AUTHOR]

Statistical process control techniques are widely used for quality control to monitor the stability of a process over time. In modern manufacturing systems with complex and variable processes, appropriate control chart techniques that can efficiently address nonnormal processes are required. Furthermore, in real manufacturing environments, process changes occur frequently because of various factors such as product and setpoint changes, catalyst degradation, seasonal variations, and sensor drift. However, conventional control chart schemes cannot necessarily accommodate all possible future conditions of a process because they are formulated based on information recorded in the early stages of the process. Several attempts have been made to accommodate process changes over time. In the present paper, we propose a time-adaptive support vector data description-based control chart that can address not only nonnormal in-control observations, but also time-varying processes. The effectiveness and applicability of the proposed chart was demonstrated through experiments with simulated data and real data from the metal frame process in mobile device manufacturing. [ABSTRACT FROM AUTHOR]

Content-based filtering (CBF), one of the most successful recommendation techniques, is based on correlations between contents. CBF uses item information, represented as attributes, to calculate the similarities between items. In this study, we propose a novel CBF method that uses a multiattribute network to effectively reflect several attributes when calculating correlations to recommend items to users. In the network analysis, we measure the similarities between directly and indirectly linked items. Moreover, our proposed method employs centrality and clustering techniques to consider the mutual relationships among items, as well as determine the structural patterns of these interactions. This mechanism ensures that a variety of items are recommended to the user, which improves the performance. We compared the proposed approach with existing approaches using MovieLens data, and found that our approach outperformed existing methods in terms of accuracy and robustness. Our proposed method can address the sparsity problem and over-specialization problem that frequently affect recommender systems. Furthermore, the proposed method depends only on ratings data obtained from a user's own past information, and so it is not affected by the cold start problem. [ABSTRACT FROM AUTHOR]

Defect patterns exhibited in wafer bin maps (WBMs) can provide essential clues about critical process failures to field engineers. In modern manufacturing processes, the automatic WBM defect pattern classification is critical for yield improvement. Although it is difficult to collect sufficient labels while a lot of unlabeled data is given, most existing studies have mainly used only labeled WBM data. Moreover, the unlabeled out-of-distribution (OOD) WBMs are inevitably collected. It degrades the performance of semi-supervised models. To this end, we propose a method for safe wafer bin map classification with self-supervised contrastive learning (SWaCo) to effectively exploit the unlabeled data with OOD. We propose a loss function to utilize label information in the pre-training step to learn more suitable representations for the downstream WBM classification task. The negative labeled examples of the same class as the anchor are used for additional positive examples. Moreover, we investigate proper data augmentation for WBM self-supervised contrastive learning. To evaluate the performance and the applicability of the proposed method, experiments are conducted on a public benchmark WBM dataset, WM-811K. The results demonstrate that the proposed method achieves better classification accuracy than existing methods, especially when only a small amount of class information is given. [ABSTRACT FROM AUTHOR]

This paper proposes a fused lasso model to identify significant features in the spectroscopic signals obtained from a semiconductor manufacturing process, and to construct a reliable virtual metrology (VM) model. Analysis of spectroscopic signals involves combinations of multiple samples collected over time, each with a vast number of highly correlated features. This leads to enormous amounts of data, which is a challenge even for modern-day computers to handle. To simplify such complex spectroscopic signals, dimension reduction is critical. The fused lasso is a regularized regression method that performs automatic variable selection for the predictive modeling of highly correlated datasets such as those of spectroscopic signals. Furthermore, the fused lasso is especially useful for analyzing high-dimensional data in which the features exhibit a natural order, as is the case in spectroscopic signals. In this paper, we conducted an experimental study to demonstrate the usefulness of a fused lasso-based VM model and compared it with other VM models based on the lasso and elastic-net models. The results showed that the VM model constructed with features selected by the fused lasso algorithm yields more accurate and robust predictions than the lasso- and elastic net-based VM models. To the best of our knowledge, ours is the first attempt to apply a fused lasso to VM modeling. [ABSTRACT FROM AUTHOR]

Clustering analysis can facilitate the extraction of implicit patterns in a dataset and elicit its natural groupings without requiring prior classification information. Numerous researchers have focused recently on graph-based clustering algorithms because their graph structure is useful in modeling the local relationships among observations. These algorithms perform reasonably well in their intended applications. However, no consensus exists about which of them best satisfies all the conditions encountered in a variety of real situations. In this study, we propose a graph-based clustering algorithm based on a novel density-of-graph structure. In the proposed algorithm, a density coefficient defined for each node is used to classify dense and sparse nodes. The main structures of clusters are identified through dense nodes and sparse nodes that are assigned to specific clusters. Experiments on various simulation datasets and benchmark datasets were conducted to examine the properties of the proposed algorithm and to compare its performance with that of existing spectral clustering and modularity-based algorithms. The experimental results demonstrated that the proposed clustering algorithm performed better than its competitors; this was especially true when the cluster structures in the data were inherently noisy and nonlinearly distributed. [ABSTRACT FROM AUTHOR]

Control charts have been widely used to improve manufacturing processes by reducing variations and defects. In particular, multivariate control charts have been effectively applied with monitoring processes that contain many correlated variables. Most existing multivariate control charts are vulnerable to misclassification errors that originate because of the hypothesis tests. In particular, these often cause the generation of a large number of false alarms. In this paper, we propose a procedure to reduce false alarms by combining a multivariate control chart and data mining algorithms. Simulation and real case studies demonstrate that the proposed method effectively reduces the false alarm rate. [ABSTRACT FROM AUTHOR]

An introduction is presented in which the editor discusses various reports published within the issue which include issues about supervised learning, the relevance of vector machine, and data mining methodology.

Forecasting time series data is one of the most important issues involved in numerous applications in real life. Time series data have been analyzed in either the time or frequency domains. The objective of this study is to propose a forecasting method based on wavelet filtering. The proposed method decomposes the original time series into the trend and variation parts and constructs a separate model for each part. Simulation and real case studies were conducted to examine the properties of the proposed method under various scenarios and compare its performance with time series forecasting models without wavelet filtering. The results from both simulated and real data showed that the proposed method based on wavelet filtering yielded more accurate results than the models without wavelet filtering in terms of mean absolute percentage error criterion. [ABSTRACT FROM AUTHOR]

Feature selection based on an ensemble classifier has been recognized as a crucial technique for modeling high-dimensional data. Feature selection based on the random forests model, which is constructed by aggregating multiple decision tree classifiers, has been widely used. However, a lack of stability and balance in decision trees decreases the robustness of random forests. This limitation motivated us to propose a feature selection method based on newly designed nearest-neighbor ensemble classifiers. The proposed method finds significant features by using an iterative procedure. We performed experiments with 20 datasets of microarray gene expressions to examine the property of the proposed method and compared it with random forests. The results demonstrated the effectiveness and robustness of the proposed method, especially when the number of features exceeds the number of observations. [ABSTRACT FROM AUTHOR]

In novelty detection, support vector data description (SVDD) is a one-class classification technique that constructs a boundary to differentiate novel from normal patterns. However, boundaries constructed by SVDD do not consider the density of the data. Data points located in low density regions are more likely to be novel patterns because they are remote from their neighbors. This study presents a density-focused SVDD (DFSVDD), for which its boundary considers both shape and the dense region of the data. Two distance measures, the kernel distance and the density distance, are combined to construct the DFSVDD boundary. The kernel distance can be obtained by solving a quadratic optimization, while support vectors are used to obtain the density distance. A simulation study was conducted to evaluate the performance of the proposed DFSVDD and was then compared with the traditional SVDD. The proposed method performed better than SVDD in terms of the area under the receiver operating characteristic curve. Copyright © 2014 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

Relevant statistical modeling and analysis of dental data can improve diagnostic and treatment procedures. The purpose of this study is to demonstrate the use of various data mining algorithms to characterize patients with dentofacial deformities. A total of 72 patients with skeletal malocclusions who had completed orthodontic and orthognathic surgical treatments were examined. Each patient was characterized by 22 measurements related to dentofacial deformities. Clustering analysis and visualization grouped the patients into three different patterns of dentofacial deformities. A feature selection approach based on a false discovery rate was used to identify a subset of 22 measurements important in categorizing these three clusters. Finally, classification was performed to evaluate the quality of the measurements selected by the feature selection approach. The results showed that feature selection improved classification accuracy while simultaneously determining which measurements were relevant. [ABSTRACT FROM AUTHOR]

Abstract: Multivariate control charts have been widely used in many industries to monitor and diagnose processes characterized by a large number of quality characteristics. Usually, these characteristics are highly correlated with each other. The direct use of conventional multivariate control charts for situations with highly correlated characteristics may lead to increased rates of false alarms. Principal component analysis (PCA) control charts have been widely used to address problems posed by such high correlations by transforming the set of correlated variables to an uncorrelated set of variables and then identifying the PCs with highest contribution which then allows one to reduce dimensionality. However, an assumption that the data are normally distributed underlies the construction of the control limits of traditional PCA control charts. This assumption has limited the use of PCA control charts in nonnormal situations found in many modern systems. This study presents the development of nonparametric PCA control charts that do not require any distributional assumptions for their construction. We propose to use nonparametric techniques, kernel density estimation, and bootstrapping to establish the control limits of these charts. A simulation study was conducted to evaluate the performance of the proposed charts and compare them with traditional PCA control charts. The comparative performance in terms of average run length showed that the proposed nonparametric PCA control charts performed better than the parametric PCA control charts in nonnormal situations. [Copyright &y& Elsevier]

Abstract: Process monitoring and diagnosis have been widely recognized as important and critical tools in system monitoring for detection of abnormal behavior and quality improvement. Although traditional statistical process control (SPC) tools are effective in simple manufacturing processes that generate a small volume of independent data, these tools are not capable of handling the large streams of multivariate and autocorrelated data found in modern systems. As the limitations of SPC methodology become increasingly obvious in the face of ever more complex processes, data mining algorithms, because of their proven capabilities to effectively analyze and manage large amounts of data, have the potential to resolve the challenging problems that are stretching SPC to its limits. In the present study we attempted to integrate state-of-the-art data mining algorithms with SPC techniques to achieve efficient monitoring in multivariate and autocorrelated processes. The data mining algorithms include artificial neural networks, support vector regression, and multivariate adaptive regression splines. The residuals of data mining models were utilized to construct multivariate cumulative sum control charts to monitor the process mean. Simulation results from various scenarios indicated that data mining model-based control charts performs better than traditional time-series model-based control charts. [Copyright &y& Elsevier]

We propose new multivariate control charts that can effectively deal with massive amounts of complex data through their integration with classification algorithms. We call the proposed control chart the ‘Probability of Class (PoC) chart’ because the values of PoC, obtained from classification algorithms, are used as monitoring statistics. The control limits of PoC charts are established and adjusted by the bootstrap method. Experimental results with simulated and real data showed that PoC charts outperform Hotelling's T 2 control charts. Further, a simulation study revealed that a small proportion of out-of-control observations are sufficient for PoC charts to achieve the desired performance. [ABSTRACT FROM AUTHOR]

Statistical process control tools have been used routinely to improve process capabilities through reliable on-line monitoring and diagnostic processes. In the present paper, we propose a novel multivariate control chart that integrates a support vector machine (SVM) algorithm, a bootstrap method, and a control chart technique to improve multivariate process monitoring. The proposed chart uses as the monitoring statistic the predicted probability of class (PoC) values from an SVM algorithm. The control limits of SVM-PoC charts are obtained by a bootstrap approach. A simulation study was conducted to evaluate the performance of the proposed SVM-PoC chart and to compare it with other data mining-based control charts and Hotelling's T2 control charts under various scenarios. The results showed that the proposed SVM-PoC charts outperformed other multivariate control charts in nonnormal situations. Further, we developed an exponential weighed moving average version of the SVM-PoC charts for increasing sensitivity to small shifts. [ABSTRACT FROM AUTHOR]

Abstract: Feature selection has received considerable attention in various areas as a way to select informative features and to simplify the statistical model through dimensional reduction. One of the most widely used methods for dimensional reduction includes principal component analysis (PCA). Despite its popularity, PCA suffers from a lack of interpretability of the original feature because the reduced dimensions are linear combinations of a large number of original features. Traditionally, two or three dimensional loading plots provide information to identify important original features in the first few principal component dimensions. However, the interpretation of what constitutes a loading plot is frequently subjective, particularly when large numbers of features are involved. In this study, we propose an unsupervised feature selection method that combines weighted principal components (PCs) with a thresholding algorithm. The weighted PC is obtained by the weighted sum of the first k PCs of interest. Each of the k loading values in the weighted PC reflects the contribution of each individual feature. We also propose a thresholding algorithm that identifies the significant features. Our experimental results with both the simulated and real datasets demonstrated the effectiveness of the proposed unsupervised feature selection method. [ABSTRACT FROM AUTHOR]

Abstract: Functional data analysis has gained significant attention from a variety of disciplines. In the present study we propose an effective clustering procedure to categorize a number of profiles that are formed with nonlinear functions. The proposed clustering procedure first smoothes the data and then transforms the smoothed data to obtain their functional form. The coefficients of the function obtained from the preceding transformation step are used for clustering. Simulation studies under various scenarios indicated that our proposed clustering procedure correctly identified the true clusters and yielded better clustering results than a latent class cluster analysis, one of the existing clustering methods. Furthermore, the effectiveness of the proposed clustering procedure was demonstrated using real pain data in which the main objective is to characterize the responses of 144 spinal cord dorsal horn neurons with graded thermal stimuli that range from 37°C to 51°C in 2°C increments. The results showed that the proposed clustering strategy can successfully elicit natural grouping of the neurons with similar response patterns to graded thermal stimuli. [Copyright &y& Elsevier]

Abstract: The main purpose of this study is to develop an effective classification procedure that discriminates between normal spectra and cancerous spectra in near infrared (NIR) spectroscopic data in which the classes are highly imbalanced and overlapped. Our proposed procedure consists of several steps. First, to ensure the comparability between spectra, normalization was done by dividing each spectral point by the area of the total intensity of the spectrum. Second, clustering analysis was performed with these normalized spectra to separate the spectra that represent the normal pattern from a mixed group that contains both normal and tumor spectra. Third, we conducted two-stage classification, the first being an effort to construct a classification model with the labels obtained from the preceding clustering analysis and the second being a classification to focus on the mixed group classified from the first classification model. To increase the accuracy, the second classification model was constructed based on the selected features that capture important characteristics of the spectral data. Our proposed procedure was evaluated by its classification ability in testing samples using a leave-one-out cross validation technique, yielding acceptable classification accuracy. [Copyright &y& Elsevier]

Abstract: Feature selection in metabolomics can identify important metabolite features that play a significant role in discriminating between various conditions among samples. In this paper, we propose an efficient feature selection method for high-resolution nuclear magnetic resonance (NMR) spectra obtained from time-course experiments. Our proposed approach combines linear-mixed effects (LME) models with a multiple testing procedure based on a false discovery rate. The proposed LME approach is illustrated using NMR spectra with 574 metabolite features obtained for an experiment to examine metabolic changes in response to sulfur amino acid intake. The experimental results showed that classification models constructed with the features selected by the proposed approach resulted in lower rates of misclassification than those models with full features. Furthermore, we compared the LME approach with the two-sample t-test approach that oversimplifies the time-course factor. [Copyright &y& Elsevier]

High-resolution nuclear magnetic resonance (NMR) spectroscopy has provided a new means for detection and recognition of metabolic changes in biological systems in response to pathophysiological stimuli and to the intake of toxins or nutrition. To identify meaningful patterns from NMR spectra, various statistical pattern recognition methods have been applied to reduce their complexity and uncover implicit metabolic patterns. In this paper, we present a genetic algorithm (GA)-based feature selection method to determine major metabolite features to play a significant role in discrimination of samples among different conditions in high-resolution NMR spectra. In addition, an orthogonal signal filter was employed as a preprocessor of NMR spectra in order to remove any unwanted variation of the data that is unrelated to the discrimination of different conditions. The results of k-nearest neighbors and the partial least squares discriminant analysis of the experimental NMR spectra from human plasma showed the potential advantage of the features obtained from GA-based feature selection combined with an orthogonal signal filter. [Copyright &y& Elsevier]

Abstract: Successful identification of the important metabolite features in high-resolution nuclear magnetic resonance (NMR) spectra is a crucial task for the discovery of biomarkers that have the potential for early diagnosis of disease and subsequent monitoring of its progression. Although a number of traditional features extraction/selection methods are available, most of them have been conducted in the original frequency domain and disregarded the fact that an NMR spectrum comprises a number of local bumps and peaks with different scales. In the present study a complex wavelet transform that can handle multiscale information efficiently and has an energy shift-insensitive property is proposed as a method to improve feature extraction and classification in NMR spectra. Furthermore, a multiple testing procedure based on a false discovery rate (FDR) was used to identify important metabolite features in the complex wavelet domain. Experimental results with real NMR spectra showed that classification models constructed with the complex wavelet coefficients selected by the FDR-based procedure yield lower rates of misclassification than models constructed with original features and conventional wavelet coefficients. [Copyright &y& Elsevier]

Class imbalance problems have been reported as a major issue in various applications. Classification becomes further complicated when an imbalance occurs in time series data sets. To address time series data, it is necessary to consider their characteristics (i.e., high dimensionality, high correlations, and multimodality). Oversampling is a well-known approach for addressing this problem; however, such an approach does not appropriately consider the characteristics of time series data. This paper addresses these limitations by presenting a model-based oversampling approach, a boundary-focused generative adversarial network (BFGAN). The proposed BFGAN employs a specifically designed additional label for reflecting the importance of a sample's position in data space. Furthermore, the BFGAN generates artificial samples after taking into consideration a sample's multimodality and importance by using a suitable modified GAN structure. We present empirical results that reveal a significant improvement in the quality of the generated data when the proposed BFGAN is used as an oversampling algorithm for an imbalanced multimodal time series data set. [ABSTRACT FROM AUTHOR]

Biosignals provide information about the onset of certain diseases or health conditions, aiding in diagnosing diseases and monitoring health conditions quickly and accurately. However, the inter-subject variability within biosignals hampers the model performance. In this study, we propose an inter-subject similar loss to learn representations robust to inter-subject variability. This loss promotes subject invariance, improves the generalizability of the representation, and allows better representations to be learned even with fewer training subjects. The proposed framework consists of two complementary loss functions: (1) supervised contrastive loss and (2) inter-subject similar loss. We evaluated the classification performance of the proposed method on three public biosignal datasets. The experimental results demonstrate that the proposed method outperforms the comparison methods and that the subject-invariant representation performs well on unseen subjects. Code is available at: https://github.com/KimHyeon-Ji/CSGR-Bio.git [ABSTRACT FROM AUTHOR]

In the semiconductor manufacturing processes, a wafer bin map (WBM) represents electrical test results. In WBMs, defective dies often form specific local patterns; such patterns are usually caused by failure from specific processes or equipment. Thus, identifying the local patterns is crucial for finding the processes or equipment responsible for the fault. Various statistical and machine learning methods have been developed for WBM classification; however, most of the existing studies considered single WBMs. This study proposes an explainable neural network for multiple WBMs classification, named a hierarchical spatial-test attention network. Our method has a hierarchical structure that reflects the characteristics of multiple WBMs. The method has two levels of attention mechanisms to the spatial and test levels, allowing the model to attend to more and less important parts when classifying WBMs. Furthermore, we propose a spatial attention probability conveyance mechanism and test-level attention entropy penalty to improve the classification performance and interpretability of the proposed method. We applied our method on a real-world multiple WBMs dataset to demonstrate the usefulness and applicability of our method. The results confirmed that the proposed method could accurately classify defect patterns while correctly identifying defect patterns’ test and location. [ABSTRACT FROM AUTHOR]

Over the past few years, the growing interest in adapting wearable and mobile devices for smart healthcare and living systems has led to an expansion of studies on human activity recognition (HAR). Recently, many HAR studies have focused on deep supervised learning, which automatically extracts meaningful features through extensive labeled data, is widely employed and demonstrated impressive progress. However, several issues, such as privacy sensitivity, cost intensity, and restricted laboratory conditions in HAR, impede the scalability of human activity labels, and the lack of labeled data degrades the performance of supervised learning. To address these challenges, we aim to extract useful representations without labels through self-supervised learning. Especially, we propose signal-wise self-supervised learning (SWISS) to learn signal interactions, taking into account the characteristics of inertial measurement unit used in sensor-based HAR data. First, we design data-level and feature-level reconstruction tasks for masked signals. These tasks compel the model to learn signal interactions by reconstructing masked signals in both the original data space and the low-dimensional feature space. Additionally, we design a new model architecture called the signal token transformer (STT), which is suitable for encoding both temporal features and signal interactions. We demonstrate the effectiveness of our approach on four publicly available human activity datasets. Furthermore, we show that STT can identify important signals and facilitate the learning of signal interactions through a self-attention mechanism. The code is available at https://github.com/DMQA-SR/HAR_SWISS. [ABSTRACT FROM AUTHOR]

Bayesian networks (BNs) have been widely used in causal analysis because they can express the statistical relationship between significant variables. To gain superior causal analysis results, numerous studies have emphasized the importance of combining a knowledge‐based approach and a data‐based approach. However, combining these two approaches is a difficult task because it can reduce the effectiveness of the BN structure learning. Further, the learning schemes of BNs for computational efficiency can cause an inadequate causal analysis. To address these problems, we propose a knowledge‐driven BN structure calibration algorithm for rich causal semantics. We first present an algorithm that can efficiently identify the subnetworks that can be altered to satisfy the learning condition of the BNs. We then reflect experts' knowledge to reduce erroneous causalities from the learned network. Experiments on various simulation and benchmark data sets were conducted to examine the properties of the proposed method and to compare its performance with an existing method. Further, an experimental study with real data from semiconductor fabrication plants demonstrated that the proposed method provided superior performance in improving structural accuracy. [ABSTRACT FROM AUTHOR]

Feature ranking is a widely used feature selection method. It uses importance scores to evaluate features and selects those with high scores. Conventional unsupervised feature ranking methods do not consider the information on cluster structures; therefore, these methods may be unable to select the relevant features for clustering analysis. To address this limitation, we propose a feature ranking algorithm based on silhouette decomposition. The proposed algorithm calculates the ensemble importance scores by decomposing the average silhouette widths of random subspaces. By doing so, the contribution of a feature in generating cluster structures can be represented more clearly. Experiments on different benchmark data sets examined the properties of the proposed algorithm and compared it with the existing ensemble-based feature ranking methods. The experiments demonstrated that the proposed algorithm outperformed its existing counterparts. [ABSTRACT FROM AUTHOR]

We propose a method for learning multi-agent policies to compete against multiple opponents. The method consists of recurrent neural network-based actor-critic networks and deterministic policy gradients that promote cooperation between agents by communication. The learning process does not require access to opponents’ parameters or observations because the agents are trained separately from the opponents. The actor networks enable the agents to communicate using forward and backward paths while the critic network helps to train the actors by delivering them gradient signals based on their contribution to the global reward. Moreover, to address nonstationarity due to the evolving of other agents, we propose approximate model learning using auxiliary prediction networks for modeling the state transitions, reward function, and opponent behavior. In the test phase, we use competitive multi-agent environments to demonstrate by comparison the usefulness and superiority of the proposed method in terms of learning efficiency and goal achievements. The comparison results show that the proposed method outperforms the alternatives. [ABSTRACT FROM AUTHOR]

Deep reinforcement learning (DRL) is a powerful approach that combines reinforcement learning (RL) and deep learning to address complex decision-making problems in high-dimensional environments. Although DRL has been remarkably successful, its low sample efficiency necessitates extensive training times and large amounts of data to learn optimal policies. These limitations are more pronounced in the context of multi-agent reinforcement learning (MARL). To address these limitations, various studies have been conducted to improve DRL. In this study, we propose an approach that combines a masked reconstruction task with QMIX (M-QMIX). By introducing a masked reconstruction task as an auxiliary task, we aim to achieve enhanced sample efficiency—a fundamental limitation of RL in multi-agent systems. Experiments were conducted using the StarCraft II micromanagement benchmark to validate the effectiveness of the proposed method. We used 11 scenarios comprising five easy, three hard, and three very hard scenarios. We particularly focused on using a limited number of time steps for each scenario to demonstrate the improved sample efficiency. Compared to QMIX, the proposed method is superior in eight of the 11 scenarios. These results provide strong evidence that the proposed method is more sample-efficient than QMIX, demonstrating that it effectively addresses the limitations of DRL in multi-agent systems. [ABSTRACT FROM AUTHOR]

With the development of deep learning technology, deep reinforcement learning (DRL) has successfully built intelligent agents in sequential decision-making problems through interaction with image-based environments. However, learning from unlimited interaction is impractical and sample inefficient because training an agent requires many trial and error and numerous samples. One response to this problem is sample-efficient DRL, a research area that encourages learning effective state representations in limited interactions with image-based environments. Previous methods could effectively surpass human performance by training an RL agent using self-supervised learning and data augmentation to learn good state representations from a given interaction. However, most of the existing methods only consider similarity of image observations so that they are hard to capture semantic representations. To address these challenges, we propose spatio-temporal and action-based contrastive representation (STACoRe) learning for sample-efficient DRL. STACoRe performs two contrastive learning to learn proper state representations. One uses the agent's actions as pseudo labels, and the other uses spatio-temporal information. In particular, when performing the action-based contrastive learning, we propose a method that automatically selects data augmentation techniques suitable for each environment for stable model training. We train the model by simultaneously optimizing an action-based contrastive loss function and spatio-temporal contrastive loss functions in an end-to-end manner. This leads to improving sample efficiency for DRL. We use 26 benchmark games in Atari 2600 whose environment interaction is limited to only 100k steps. The experimental results confirm that our method is more sample efficient than existing methods. The code is available at https://github.com/dudwojae/STACoRe. • Contrastive learning using the agent's actions as pseudo labels improves sample efficiency. • Automatically selecting the optimal data augmentation improves training stability. • Contrastive learning using spatio-temporal information further improves sample efficiency. [ABSTRACT FROM AUTHOR]

Traditional control charts, such as Hotelling’s T 2 , are effective in detecting abnormal patterns. However, most control charts do not take into account a time-varying property in a process. In the present study, we propose a parameter-less self-organizing map-based control chart that can handle a situation in which changes occur in the distribution or parameter of the target observations. The control limits of the proposed chart are determined by estimating the empirical level of significance on the percentile using the bootstrap method. Experimental results obtained by using simulated data and actual process data from the manufacturing process for a thin-film transistor-liquid crystal display demonstrate the effectiveness and usefulness of the proposed algorithm. [ABSTRACT FROM AUTHOR]

Multivariate statistical process control techniques have been widely used to improve processes by reducing variation and preventing defects. In modern manufacturing, because of the complexity and variability of processes, traditional multivariate control charts such as Hotelling's T 2 cannot efficiently handle situations in which the patterns of process observations are nonlinear, multimodal, and time varying. In the present study, we propose a nonparametric control chart, which is capable of adaptively monitoring time-varying and multimodal processes. Experiments with simulated and real process data from a thin film transistor-liquid crystal display (TFT-LCD) demonstrate the effectiveness and accuracy of the proposed method. [ABSTRACT FROM AUTHOR]

Successful identification of the significant features in complex mass spectral fingerprints is a crucial task in discriminating states or differences in natural systems (e.g., diseased vs. healthy, treated vs. untreated, and male vs. female) that are visualized using mass spectrometry technology. In this study, we present an ensemble regularization method that combines three regularization regression models to generate more robust results. Specifically, the coefficients from each of three regularization models were bootstrapped and the means and standard deviations of these coefficients were calculated. After obtaining these estimated statistics of the coefficients, we performed a hypothesis test for each feature. Finally, we determined the significant features that were simultaneously selected by the three hypothesis tests. Mass spectral data from six different extracts of mosquito cuticles were used to evaluate the performance of the proposed method. The purpose of this spectral analysis was to determine the major features needed to differentiate married-female mosquitoes having the potential to cause malaria infection from others. In addition, we compared the proposed ensemble feature selection method with random forest, a widely used feature selection algorithm. We found that the proposed method outperformed random forest in terms of feature selection efficiency. [ABSTRACT FROM AUTHOR]

Background. To date, investigating respiratory disease patients visiting the emergency departments related with fined dust is limited. This study aimed to analyze the effects of two variable-weather and air pollution on respiratory disease patients who visited emergency departments. Methods. This study utilized the National Emergency Department Information System (NEDIS) database. The meteorological data were obtained from the National Climate Data Service. Each weather factor reflected the accumulated data of 4 days: a patient's visit day and 3 days before the visit day. We utilized the RandomForestRegressor of scikit-learn for data analysis. Result. The study included 525,579 participants. This study found that multiple variables of weather and air pollution influenced the respiratory diseases of patients who visited emergency departments. Most of the respiratory disease patients had acute upper respiratory infections [J00–J06], influenza [J09–J11], and pneumonia [J12–J18], on which PM10 following temperature and steam pressure was the most influential. As the top three leading causes of admission to the emergency department, pneumonia [J12–J18], acute upper respiratory infections [J00–J06], and chronic lower respiratory diseases [J40–J47] were highly influenced by PM10. Conclusion. Most of the respiratory patients visiting EDs were diagnosed with acute upper respiratory infections, influenza, and pneumonia. Following temperature, steam pressure and PM10 had influential relations with these diseases. It is expected that the number of respiratory disease patients visiting the emergency departments will increase by day 3 when the steam pressure and temperature values are low, and the variables of air pollution are high. The number of respiratory disease patients visiting the emergency departments will increase by day 3 when the steam pressure and temperature values are low, and the variables of air pollution are high. [ABSTRACT FROM AUTHOR]