Search

In recent years, generative adversarial networks (GANs) have been proposed to generate simulated images, and some works of literature have applied GAN to the analysis of numerical data in many fields, such as the prediction of building energy consumption and the prediction and identification of liver cancer stages. However, these studies are based on sufficient data volume. In the current era of globalization, the demand for rapid decision‐making is increasing, but the data available in a short period of time is scarce. As a result, machine learning may not provide precise results. Obtaining more information from a small number of samples has become an important issue. Therefore, this study aimed to modify the generative adversarial network structure for learning with small numerical datasets, starting with the Wasserstein GAN (WGAN) as the GAN architecture, and using mega‐trend‐diffusion (MTD) to limit the bound of virtual samples that the GAN generates. The model verification of our proposed structure was conducted with two datasets in the UC Irvine Machine Learning Repository, and the performance was evaluated using three criteria: accuracy, standard deviation, and p‐value. The experiment result shows that, using this improved GAN architecture (WGAN_MTD), small sample data can also be used to generate virtual samples that are similar to real samples through GAN. Published version

Learning with small data is challenging for most algorithms in regard to building statistically robust models. In previous studies, virtual sample generation (VSG) approaches have been verified as effective in terms of meeting this challenge. However, most VSG methods were developed for numerical inputs. Therefore, to address situations where data has nominal inputs and continuous outputs, a systemic VSG procedure is proposed to generate samples based on fuzzy techniques to further enhance modelling capability. Based on the concept of the data preprocess in the M5′ model tree, we reveal a useful procedure by which to extract the fuzzy relations between nominal inputs and continuous outputs. Further, with the idea of nonparametric operations, we employ trend similarity to present the fuzzy relations between inputs and outputs. Then, these relations are represented by possibility distributions, and sample candidates are created based on these distributions. Finally, the candidates filtered using $$\alpha$$ -cut are regarded as qualified virtual samples. In the experiments, we demonstrate the effectiveness of our approach through a comparison with two other VSG approaches using five public datasets and two prediction models. Moreover, three parameters used in our approaches are discussed. However, determining how to find the most fit parameters requires further study in the future.

In many small-data-learning problems, owing to the incomplete data structure, explicit information for decision makers is limited. Although machine learning algorithms are extensively applied to extract knowledge, most of them are developed without considering whether the training sets can fully represent the population properties. Focusing on small data which contains nominal inputs and continuous outputs, this paper develops an effective sample generating procedure based on fuzzy theories to tackle the learning issue by data preprocessing. According to the derived fuzzy relations between categories and continuous outputs, the possibilities of the combinations of categories (virtual samples) can be aggregated when continuous outputs are given. Proper virtual samples are further selected by using fuzzy alpha-cut on the possibility distributions, and these are added to the training sets to form new ones. In the experiment, sixteen datasets taken from the UC Irvine Machine Learning Repository are examined with back-propagation neural networks and support vector regressions. The results reveal that the forecasting accuracies of the two models are significantly improved when they are built with the proposed new training sets. Moreover, the results also indicate the proposed method outperforms bootstrap aggregating and the synthetic minority over-sampling technique-Nominal-Continuous with the greatest amount of statistical support.

A small dataset often makes it difficult to build a reliable learning model, and thus some researchers have proposed virtual sample generation (VSG) methods to add artificial samples into small datasets to extend the data size. However, for some datasets the assumption of the distribution of data in the VSG methods may be vague, and when data only has a few attributes, such approaches may not work effectively. Other researchers thus proposed attribute extension methods to generate attributes to convert data into a higher dimensional space. Unfortunately, the resulting dataset may become a sparse dataset with many null or zero values in extended attributes, and then a large quantity of such attributes will reduce the representativeness of instances for the learning model. Therefore, based on fuzzy theories, this paper proposes a novel sample attribute extending (SEA) method to extend a suitable quantity of attributes to improve small dataset learning. In order to verify the validity of the SEA method, using SVR and BPNN, this paper adopts two real cases and two public datasets to conduct the learning of the predictive model, and uses the paired t-test to statistically examine the significance of improvement. The experimental results show that the proposed SEA method can effectively improve the learning accuracy of small datasets.

Over the past few decades, a few learning algorithms have been proposed to extract knowledge from data. The majority of these algorithms have been developed with the assumption that training sets can denote populations. When the training sets contain only a few properties of their populations, the algorithms may extract minimal and/or biased knowledge for decision makers. This study develops a systematic procedure based on fuzzy theories to create new training sets by rebuilding the possible sample distributions, where the procedure contains new functions that estimate domains and a sample generating method. In this study, two real cases of a leading company in the thin film transistor liquid crystal display (TFT-LCD) industry are examined. Two learning algorithms—a back-propagation neural network and support vector regression—are employed for modeling, and two sample generation approaches—bootstrap aggregating (bagging) and the synthetic minority over-sampling technique (SMOTE)—are employed to compare the accuracy of the models. The results indicate that the proposed method outperforms bagging and the SMOTE with the greatest amount of statistical support.

Compressing new product development schedules has become an important strategy for firms to earn more market share in recent years, because of the globalizing competition. Nevertheless, engineers often have to face the issue of small data learning in this context, since only few pilot runs are allowed to avoid lowering the yield rates of mass production. Bagging has shown to be an effective approach to deal with small data. However, when adopting multiple models in bagging to learn numeric forecasting tasks without the prior weights, bagging often takes the averages of the predictions of the models as the compromised ones, which are very probably affected by extreme values and thus become less precise. Accordingly, this study develops a systematic procedure to generate weights for prediction aggregation by employing box-and-whisker plots to build the possible distributions of the predictive errors of the algorithms with membership functions. In the experiment, a real case taken from a thin film transistor liquid crystal display maker is studied with four distinct algorithms and bagging; and the results show the errors of the predictions aggregated by the proposed weights are significantly lower than those of the four algorithms and the averaging method in bagging in general.

The small data learning issue has existed for over one hundred years (since 1908) when the Student's t-distribution was first developed. Few statistical tools can evaluate a population appropriately if the sample size is too small; small samples can be remedied through virtual sample generation (VSG) methods, which are widely used in industry and machine learning. However, most VSG methods were developed for data having only numerical attributes, very few studies have dealt with nominal attributes and cause domain estimation limitations. Therefore, this paper proposes a method that generates virtual samples based on the discrete degrees of nominal attributes, and then estimates the general population domains by fuzzy membership functions. A backpropagation neural network model and a support vector regression model are used to test the efficiency of the proposed method, while the Wilcoxon-sign test is used to test the difference with raw data sets. The result shows that the proposed method can reduce the mean absolute error and enhance classification accuracy by generating virtual samples that have nominal attributes.

Small data-set learning problems are attracting more attention because of the short product lifecycles caused by the increasing pressure of global competition. Although statistical approaches and machine learning algorithms are widely applied to extract information from such data, these are basically developed on the assumption that training samples can represent the properties of the whole population. However, as the properties that the training samples contain are limited, the knowledge that the learning algorithms extract may also be deficient. Virtual sample generation approaches, used as a kind of data pretreatment, have proved their effectiveness when handling small data-set problems. By considering the relationships among attributes in the value generation procedure, this research proposes a non-parametric process to learn the trend similarities among attributes, and then uses these to estimate the corresponding ranges that attribute values may be located in when other attribute values are given. T...

In most highly competitive manufacturing industries, the sample sizes are usually very small in pilot runs, in order to quickly launch new products. However, it is always difficult for engineers to improve the quality in mass production runs based on the limited data obtained in this way. Past research has demonstrated that adding artificial samples can be an effective approach when learning with small data-sets. However, a prior analysis of the data is needed to deduce the appropriate sample distributions within which the artificial samples are generated. Johnson transformation is one of the well-known models that can be applied to bring data close to a normal distribution with the satisfaction of certain statistical assumptions. The sample size required for such data transformation methods is usually large, and this thus motivates the efforts of the current study to develop a new method which is suitable for small data-sets. Accordingly, this research proposes the small Johnson Data Transformation metho...

Because the lifecycles of consumable electronic products are now very short, it has become very difficult for manufacturers to precisely determine customer demands with limited historical data. Over the past two decades, the grey model (GM) and its extensions have been shown to be effective tools to deal with short-term time series data. To further enforce the effectiveness of data uncertainty treatment for dynamic integrated-circuit assembly industries, a GM envelopment learning procedure is developed. In our procedure, short term series data is fuzzified to form a fuzzy time series for the purpose of building GM models, in which the final predictions are further aggregated with the proposed weights. The experimental results of a real case and a public dataset indicate that the proposed procedure can further improve the accuracy of predictions given by GM models and thus has practical value in tackling real cases.

Efficiently controlling the early stages of a manufacturing system is an important issue for enterprises. However, the number of samples collected at this point is usually limited due to time and cost issues, making it difficult to understand the real situation in the production process. One of the ways to solve this problem is to use a small data set forecasting tool, such as the various gray approaches. The gray model is a popular forecasting technique for use with small data sets, and while it has been successfully adopted in various fields, it can still be further improved. This paper thus uses a box plot to analyze data features and proposes a new formula for the background values in the gray model to improve forecasting accuracy. The new forecasting model is called BGM(1,1). In the experimental study, one public dataset and one real case are used to confirm the effectiveness of the proposed model, and the experimental results show that it is an appropriate tool for small data set forecasting. Small-data-set forecasting problem is difficult for most manufacturing environments.A forecasting tool using limited data for engineers and managers is more effective and efficient.The proposed method base on the box plot can analyze data features to improve forecasting accuracy with small data sets.The proposed method is considered an appropriate procedure in general to forecast manufacturing outputs based on small samples.

Under the increasing pressure of global competition, product life cycles are becoming shorter and shorter. This means that better methods are needed to analyse the limited information obtained at the trial stage in order to derive useful knowledge that can aid in mass production. Machine learning algorithms, such as data mining techniques, are widely applied to solve this problem. However, a certain amount of training samples is usually required to determine the validity of the information that is obtained. This study uses only a few data points to estimate the range of data attribute domains using a data diffusion method, in order to derive more useful information. Then, based on practical engineering experience, we generate virtual samples with a noise disturbance method to improve the robustness of the predictions derived from a multiple linear regression. One real data set obtained from a large TFT-LCD company is examined in the experiment, and the results show the proposed approach to be effective.

In the past few decades, there were quite a few learning algorithms developed to extract knowledge from data. However, none of the single algorithms can be applicable to learn all the datasets with favor results because data patterns may represent linear and non-linear. Accordingly, the idea of aggregating the predictions of multiple learning models to improve the forecasting accuracy of a single method was proposed. Nevertheless, how to improve the accuracy of the aggregated predictions when learning small datasets is the objective of this study. Based on the distributions of the predictive errors of learning models, the proposed method learns the weights of the models and then tries to aggregate more precise predictions with the weights. The experiment results show the forecasting errors of the predictions aggregated by the proposed method are significantly lower than the predictions of single models and the averaged predictions.

While back-propagation neural networks (BPNN) are effective learning tools for building non-linear models, they are often unstable when using small-data-sets. Therefore, in order to solve this problem, we construct artificial samples, called virtual samples, to improve the learning robustness. This research develops a novel method of virtual sample generation (VSG), named genetic algorithm-based virtual sample generation (GABVSG), which considers the integrated effects and constraints of data attributes. We first determine the acceptable range by using mega-trend diffusion (MTD) functions, and construct the feasibility-based programming (FBP) model with BPNN. A genetic algorithm (GA) is then applied to accelerate the generation of the most-feasible virtual samples. Finally, we use two real cases to verify the performance of the proposed method by comparing the results with two well-known forecasting models, BPNN, support vector machine for regression (SVR) and one newly published approach MTD method [1]. The experimental results indicate that the performance of the GABVSG method is superior to that of using original training data without artificial samples. Consequently, the proposed method can improve learning performance significantly when working with small samples.

Virtual sample generation approaches have been used with small data sets to enhance classification performance in a number of reports. The appropriate estimation of data distribution plays an important role in this process, with performance usually better for data sets that have a simple distribution rather than a complex one. Mixed-type data sets often have a multi-modal distribution instead of a simple, uni-modal one. This study thus proposes a new approach to detect multi-modality in data sets, to avoid the problem of inappropriately using a uni-modal distribution. We utilize the common k-means clustering method to detect possible clusters, and, based on the clustered sample sets, a Weibull variate is developed for each of these to produce multi-modal virtual data. In this approach, the degree of error variation in the Weibull skewness between the original and virtual data is measured and used as the criterion for determining the sizes of virtual samples. Six data sets with different training data sizes are employed to check the performance of the proposed method, and comparisons are made based on the classification accuracies. The results using non-parametric testing show that the proposed method has better classification performance to that of the recently presented Mega-Trend-Diffusion method. We propose a new method to assess the modality of small data.We construct a unique scheme to systematically generate multi-modal virtual samples.In this scheme, we suggest a criterion to control the size of virtual samples.We utilize six data sets to show the superiority of the proposed method.

Electronics product life cycles are becoming shorter and shorter because of the severe global competition. In such highly competitive industry, it has become an important strategy to accelerate new products launching to the market to earn more shares. However, the lead times of pilot runs are usually long in new product development (NPD) processes, and reducing pilot runs has thus become one of the key tasks of manufacturing systems. Specifically, since the shorter a test period is the smaller sample size one can obtain, making that to find a small data learning method for a manufacturing system being a new challenge. Facing the problem, this work, based on the box plots and the fuzzy techniques, develops an approach to systematically generate synthetic samples to help stabilize the learning process for the used back-propagation neural network (BPN). A real learning task taken from the Array process of a TFT-LCD manufacturer (a new high-resolution product of 4K2K in 2013) is employed as an example to illustrate the details of the proposed method. The task contains nine inputs and 72 output manufacturing attributes, but only with 20 samples. It is quite difficult for most existing modeling algorithms to deal with such a high dimensional situation when the sample size is small. The experiment results show that the proposed approach can effectively improve the robustness and preciseness of a BPN forecasting model. In addition to the reduction of pilot runs, more process knowledge is obtained in the input-output analysis.

In the current highly competitive manufacturing environment, it is important to have effective and efficient control of manufacturing systems to obtain and maintain competitive advantages. However, developing appropriate forecasting models for such systems can be challenging in their early stages, as the sample sizes are usually very small, and thus there is limited data available for analysis. The technique of virtual sample generation is one way to address this issue, but this method is usually not directly applied to time series data. This research thus develops a Latent Information function to analyze data features and extract hidden information, in order to learn from small data sets considering timing factors. The experimental results obtained using the Synthetic Control Chart Time Series and aluminum price datasets show that the proposed method can significantly improve forecasting accuracy, and thus is considered an appropriate procedure to forecast manufacturing outputs based on small samples.

Small-data problems are commonly encountered in the early stages of a new manufacturing procedure, presenting challenges to both academics and practitioners, as good performance is difficult to achieve with learning models when there is a lack of sufficient data. Virtual sample generation (VSG) has been shown to be an effective method to overcome this issue in a wide range of studies in various fields. Such works usually assume that the relations among attributes are independent of each other, and produce synthetic data by using sample distributions of these. However, the VSG technique may be ineffective if the real data has interrelated attributes. Therefore, this research provides a novel procedure to generate related virtual samples with non-linear attribute dependency. To construct a relational model between the independent and dependent attributes, we employ gene expression programming (GEP) to find the most suitable mathematical model. One practical dataset and three real UCI datasets are presented in this paper to verify the effectiveness of the proposed method, and the results show that the proposed approach has better learning accuracy with regard to a back-propagation neural (BPN) network than that of the well-known mega-trend-diffusion (MTD) and the multi regression analysis (MRA) approaches.

In the early stages of manufacturing systems, it is often difficult to obtain sufficient data to make accurate forecasts. Grey system theory is one of the approaches to deal with this issue, as it uses fairly small sets to construct forecasting models. Among published grey models, the current non-equigap grey models can deal with data having unequal gaps, and have been applied in various fields. However, these models usually use fixed modeling procedures that do not consider data growth trend differences. This paper utilizes the trend and potency tracking method to determine the parameter @a of the background value to build an adaptive non-equigap grey model to improve forecasting performance. The experimental results indicate that the proposed method considers that data occurrence properties can obtain better forecasting results.

Machine learning algorithms are widely applied to extract useful information, but the sample size is often an important factor in determining their reliability. The key issue that makes small dataset learning tasks difficult is that the information that such datasets contain cannot fully represent the characteristics of the entire population. The principal approach of this study to overcome this problem is systematically adding artificial samples to fill the data gaps; this research employs the mega-trend-diffusion technique to generate virtual samples to extend the data size. In this paper, a real, small dataset learning task in the array process of a thin-film transistor liquid-crystal display (TFT-LCD) panel manufacturer is proposed, where there are only 20 samples used for learning the relationship between 15 inputs and 36 output attributes. The experiment results show that the approach is effective in building robust back-propagation neural network (BPN) and support vector regression (SVR) models. In...

Constructing an accurate prediction model from a small training data set is an important but difficult task in the field of forecasting. This is because when the data size is small, the incomplete data may mean that the model produced cannot sufficiently represent the true data structure or cause the model training to be overfitted. To address this issue, this paper presents an approach that combines multiple prediction models to extract data information in multiple facets. In the multi-model approach, a compromise weight method is proposed to determine the relative reliability of each of the prediction model. The methods used include multiple regression, artificial neural network, and support vector machines for regression. A thin-film transistor liquid crystal display manufacturing case study is used to illustrate the details of this research. The empirical results not only show that the proposed multi-model can reduce the manufacturing variation and increase the production yield, but also can propose a...

In most highly competitive manufacturing industries, the sample sizes of new products are usually very small in pilot runs because the production schedules are very tight. To obtain the expected quality in mass production runs using limited data is, therefore, always a challenging issue for engineers. Although machine learning algorithms are widely applied to this task, the training sample size is a key weakness when determining the manufacturing parameters. In order to extract more robust information for engineers from the small datasets, this research, based on regression analysis and fuzzy techniques, develops an effective procedure for new production pattern constructions. In addition, a case study of TFT-LCD manufacturing in 2009 is taken as an example to illustrate the presented approach. The experimental results show that it is possible to develop a robust forecasting model which can provide more precise manufacturing predictions with the limited data acquired from pilot runs.

It is difficult for learning models to achieve high classification performance with imbalanced data sets. To conquer the problem, this study presents a strategy involving the reduction of size of majority data set and the generation of synthetic samples of minority data set. Parkinson's disease data set is used to examine and to compare the performance of classification methods. The paired t-tests are also used to show the effectiveness of the proposed method comparing with that of the other methods.

Since the product life cycles are getting shorter and shorter, the issue of small data set learning has drawn more and more attentions in both academics and enterprises. Many methods have been proposed to improve the learning performance of small data set. In these methods, the virtual sample generation approach is the most popular technique for improving small data learning. In the process of virtual sample generation, the attribute independence in small data is the key part to determine the learning performance, because it is the necessary assumption before generating virtual samples. However, in the real world, attributes in the data set usually are not mutual independent. Therefore, this paper proposes a new process to generate independent virtual samples based on the box-and-whisker plot domain estimation. In order to validate the effectiveness of the proposed method, one data set is used to calculate the classification accuracy average and standard deviation based on the support vector machine. The results of the experiment show that the presented method has a superior classification performance than other methods.

This paper proposes a method that focuses on creating new data attributes by using fuzzy operations for solving small dataset learning problems. Using the idea of fuzzy rules, the membership value of antecedents in each rule can be extracted from the data point. Therefore, in this research, those membership values will be deemed as new data features and the data dimensionality will be extended. To test the effectiveness of the proposed method, the data set with new data features and the one with no special treatment will be utilized respectively to build predictive models. Paired t-test is carried out to see how effective the proposed method can improve the learning on the basis of small sample sets.

In the current rapidly changing manufacturing conditions, controlling manufacturing systems effectively and efficiently is a critical issue for enterprises, especially in their early stages. However, it is often difficult to make correct decisions, with the insufficient information available at such times. We thus develop a two-stage modeling procedure to build a predictive model using few samples. We first use three conventional approaches to establish forecasting models, and then implement pre-testing with the proposed grey-based fitness measuring index to determine the weights to create a hybrid model. Two datasets, including color filter manufacturing data and the Asia-Pacific Economic Cooperation energy database, are evaluated in the experiment, and the results show that the proposed method not only has good forecasting performance, but also reduces the influence forecasting errors. Accordingly, the proposed procedure is thus considered a feasible approach for small-data-set forecasting.

Data mining methods have been successfully used for analyzing production data in manufacturing, and generally the production samples are considered derived from a population from the statistical view. However, the assumption is often challenged, because a production system is normally highly interrelated with a manufacturing environment that is time-dependent. Therefore, instead of treating data as elements from a population following a certain distribution, this research considers that, in the whole course of data collection, a total data set might reasonably be divided into three phases as a more accurate production profile: early phase, mature phase, and oversized phase. In the early phase, the data set is usually small and the information extracted is fragile; in the mature phase, the data collected can provide sufficient and stable knowledge to the management; and in the oversized phase, since the system has substantially changed, many old data are no longer current so that their representability has declined. Based on this concept, the two critical points that segment the total data set into three parts are systematically determined here using neural network technologies; and the manufacturing model can be reformed for advancing its predictive capability.

Data quantity is the main issue in the small data set problem, because usually insufficient data will not lead to a robust classification performance. How to extract more effective information from a small data set is thus of considerable interest. This paper proposes a new attribute construction approach which converts the original data attributes into a higher dimensional feature space to extract more attribute information by a similarity-based algorithm using the classification-oriented fuzzy membership function. Seven data sets with different attribute sizes are employed to examine the performance of the proposed method. The results show that the proposed method has a superior classification performance when compared to principal component analysis (PCA), kernel principal component analysis (KPCA), and kernel independent component analysis (KICA) with a Gaussian kernel in the support vector machine (SVM) classifier.

Small data set problems have been widely considered in many fields, where increasing the prediction ability is the most important goal. This study considers the data structure to identify new data points in a more precise manner, and is thus able to achieve improved prediction capability. The proposed method, named structure-based data transformation, consists of two steps. The first step is using the density-based spatial clustering of applications with noise (DBSCAN) algorithm to separate data sets into clusters, which generates the number of clusters dynamically. The second step is to build up the data transformation function, in which the new attributes are computed using fuzzy membership functions obtained by the corresponding membership grades in each cluster. Three real cases are selected to compare the proposed forecasting model with the linear regression (LR), backpropagation neural network (BPNN), and support vector machine for regression (SVR) methods. The result show that the structure-based data transformation method has better performance than when using the raw data with regard to the error improving rate, mean square error (MSE), and standard deviation (STD).

Product life cycles are becoming shorter, especially in the electronics industry. The issue of time to market has thus become a core competency for firms to increase market share. In order to shorten the cycle time from product design to mass production, engineers must often make decisions under uncertain conditions with limited information. Although machine learning algorithms can help derive useful information, the smallest training sample size required to establish robust models is important to know, as with insufficient data size the models produced may be unreliable. This research develops a two-phase procedure for small-data-set learning problems at pilot run stage and takes the multi-layer ceramic capacitor case as an example to figure out a precise model which concretely represents the learned process knowledge to help shorten the lead-time before mass production. The results reveal that it is possible to rapidly develop a model of production with limited data from pilot runs.

To achieve effective and efficient decision making in a highly competitive business environment, an enterprise must have an appropriate forecasting technique that can meet the requirements of both timeliness and accuracy. Accordingly, in the early stages, building a forecasting model with incomplete information and limited samples is very important to a business. Grey system theory is one of the prediction methods that can be built with a small sample and yet has a strong ability to make short-term predictions. The purpose of this study is to come up with an improved forecasting model based on the concept of this theory to enlarge the applicability of the grey forecasting model in various situations. By extending the data transforming approach, this method generalizes a building procedure for the grey model to grasp the data outline and information trend. Specifically, a novel inverse accumulating generation operator is developed to enable omnidirectional forecasting. The research utilizes observations of the titanium alloy fatigue limit along with temperature changes as raw data to verify the performance of the proposed method. The experimental results show that not only can this method expand the application scope of the grey forecasting model, but also improve its forecasting accuracy.

Manufacturing forecast problems have been widely discussed in recent years, where more accurate predictions could reduce the overall manufacturing costs. This study uses the case of ensuring the heights of thin film transistor---liquid crystal display photo-spacers. It is a small sample size prediction problem, because the data available for analysis is limited on the manufacturing lines. A new approach is developed to deal with this problem, which involves three steps. The first step is using K-means clustering to separate data into K clusters, while the second step is to compute the possibility through the fuzzy membership function in each cluster for attribute extension. The last step is to put the data with new generate attributes into a backpropagation neural network (BPNN) machine learning algorithm. Two performance evaluation methods, cross-validation and data specification testing, are selected to compare the proposed method with three popular prediction models: linear regression, support vector machine for regression (SVR), and BPNN. The results show that the proposed method outperforms the others with regard to the total errors, mean square error, and standard deviation.

Objective: Medical data sets are usually small and have very high dimensionality. Too many attributes will make the analysis less efficient and will not necessarily increase accuracy, while too few data will decrease the modeling stability. Consequently, the main objective of this study is to extract the optimal subset of features to increase analytical performance when the data set is small. Methods: This paper proposes a fuzzy-based non-linear transformation method to extend classification related information from the original data attribute values for a small data set. Based on the new transformed data set, this study applies principal component analysis (PCA) to extract the optimal subset of features. Finally, we use the transformed data with these optimal features as the input data for a learning tool, a support vector machine (SVM). Six medical data sets: Pima Indians' diabetes, Wisconsin diagnostic breast cancer, Parkinson disease, echocardiogram, BUPA liver disorders dataset, and bladder cancer cases in Taiwan, are employed to illustrate the approach presented in this paper. Results: This research uses the t-test to evaluate the classification accuracy for a single data set; and uses the Friedman test to show the proposed method is better than other methods over the multiple data sets. The experiment results indicate that the proposed method has better classification performance than either PCA or kernel principal component analysis (KPCA) when the data set is small, and suggest creating new purpose-related information to improve the analysis performance. Conclusion: This paper has shown that feature extraction is important as a function of feature selection for efficient data analysis. When the data set is small, using the fuzzy-based transformation method presented in this work to increase the information available produces better results than the PCA and KPCA approaches.

Cross-validation is a widely used model evaluation method in data mining applications. However, it usually takes a lot of effort to determine the appropriate parameter values, such as training data size and the number of experiment runs, to implement a validated evaluation. This study develops an efficient cross-validation method called Complexity-based Efficient (CBE) cross-validation for binary classification problems. CBE cross-validation establishes a complexity index, called the CBE index, by exploring the geometric structure and noise of data. The CBE index is used to calculate the optimal training data size and the number of experiment runs to reduce model evaluation time when dealing with computationally expensive classification data sets. A simulated and three real data sets are employed to validate the performance of the proposed method in the study, while the validation methods compared are repeated random sub-sampling validation and K-fold cross-validation. The results show that CBE cross-validation, repeated random sub-sampling validation and K-fold cross-validation have similar validation performance, except that the training time required for CBE cross-validation is indeed lower than that for the other two methods.

Enterprises usually install a computerized information system to improve production efficiency. However, operational problems still occur from time to time, with different products usually requiring different solutions. This study discusses operational problems and proposes a diagnostic method for integrated shop-floor systems that manufacture multiple products. This study uses the multivariable statistics method to conduct a relevance analysis to determine the important attributes that influence production operations. Then, a neural network is used as the diagnostic system to detect operational problems. Support vector learning machines (SVM) are used to confirm the correct product classification. Finally, the diagnostic results are stored in a case-based reasoning system database for future use.

Early in the process of developing a new product, on hand data is usually scarce and it is hard to assess product reliability. Many previous studies thus assume prior product failure distribution in the area of reliability estimation. However, when real data distribution is different from the assumed one, the reliability estimation accuracy is biased. This paper proposed a modeling method with four steps to build a reliability growth model for new products without assuming prior distribution. A sequential test procedure and the Bayesian theorem are mainly employed in the method presented. The results indicate that the proposed method can successfully build a reliability growth model for new products.

Global competition has shortened product life cycles and makes the trend of industrial demand not easily forecasted. Therefore, one of the key points that will enable enterprises to survive and succeed is the ability to adapt to this dynamic environment. However, the available data, such as demand and sales, are often limited in the early periods of product life cycles, making traditional forecasting techniques unreliable for decision making. Although various forecasting methods currently exist, their utility is often limited by insufficient data and indefinite data distribution. The grey prediction model is one of the potential approaches for small sample forecast, although it's often hard to amend according to the sample characteristics in practice, owing to its fixed modeling method. This research tries to use the trend and potency tracking method (TPTM) to analyze sample behavior, extract the concealed information from data, and utilize the trend and potency value to construct an adaptive grey prediction model, AGM (1,1), based on grey theory. The experimental results show that the proposed model can improve the prediction accuracy for small samples.

Building a decision support system (DSS) using small data sets usually results in uncertain knowledge, likely leading to incorrect decisions and causing a large losses. However, gathering sufficient samples for building a DSS often has significant costs in many cases. To solve this problem, a case study of a particular business decision-making procedure in which only small data sets are available is discussed. The learning accuracy for the modeling phase in the DSS was improved using the mega-trend-diffusion technique, which includes two learning tools: Back-propagation network and Bayesian network. The case study, a business diversification decision for an oil company, shows that the proposed technique contributes to increasing the prediction precision using very limited experience.

In manufacturing systems, only a small training dataset can be obtained in the early stages. A small training dataset usually leads to low learning accuracy with regard to classification of machine learning, and the knowledge derived is often fragile, and this is called the small sample problem. This research mainly aims at overcoming this problem using a special nonlinear classification technique to generate virtual samples to enlarge the training dataset for learning improvement. This research proposes a new sample generation method, named non-linear virtual sample generation (NVSG), which combines a unique group discovery technique and a virtual sample generation method using parametric equations of hypersphere. By applying a back-propagation neural network (BPN) as the learning tool, the computational experiments obtained from the simulated dataset and the real dataset quoted from the Iris Plant Database show that the learning accuracy can be significantly improved using NVSG method for very small training datasets.

If the production process, production equipment, or material changes, it becomes necessary to execute pilot runs before mass production in manufacturing systems. Using the limited data obtained from pilot runs to shorten the lead time to predict future production is this worthy of study. Although, artificial neural networks are widely utilized to extract management knowledge from acquired data, sufficient training data is the fundamental assumption. Unfortunately, this is often not achievable for pilot runs because there are few data obtained during trial stages and theoretically this means that the knowledge obtained is fragile. The purpose of this research is to utilize bootstrap to generate virtual samples to fill the information gaps of sparse data. The results of this research indicate that the prediction error rate can be significantly decreased by applying the proposed method to a very small data set.

Support vector machines (SVM) are widely applied to various classification problems. However, most SVM need lengthy computation time when faced with a large and complicated dataset. This research develops a clustering algorithm for efficient learning. The method mainly categorizes data into clusters, and finds critical data in clusters as a substitute for the original data to reduce the computational complexity. The computational experiments presented in this paper show that the clustering algorithm significantly advances SVM learning efficiency.

Having high learning accuracies, neural networks are widely applied for solving function approximation problems. Nevertheless, it is difficult to train a neural network to recognize a non-linear function using a small training sample set. Because the errors between real values and estimated values are significant and almost impossible to figure out using insufficient samples. This study develops an algorithm combining segmentation technique and artificial samples to overcome this situation. The strategy is to shorten the model range to minimize the total estimation error. Another supportive strategy known as artificial samples generation is also employed to fill information gaps. At the end of this research, the results of the computational examples indicate that learning accuracy can be significantly improved using the proposed method involving a very small data set.

Acquiring knowledge in manufacturing systems in the early stages always has a challenging task due to the lack of sufficient data. This makes it hard for the derived management model to reach a reliable and stable level. Li and Lin (2006) developed a useful method that can deal with the problem of knowledge acquisition based on a small data set. However, this method assumes all data are collected at the same time, since they treat the data set as a source (from one population) of a priori knowledge for learning. In fact, instead of being a random data set, these collected data can be time dependent, that is, they tend to be a sequence of observations, occurring at different times. The consideration of this dependence property in the data will benefit the knowledge acquisition in the early stages by expanding the learning model from an independent model to a dependent model. This research expanded the intervalized kernel density estimator (IKDE) presented in Li and Lin (2006) to a more general form to improve the learning model in the early stages. The general model, called GIKDE, joints the concepts of time series and stochastic processes in order to deal with both independent and dependent data sets. The Virtual Sample Generation process based on GIKDE was also developed to produce extra information for expediting the learning. Results obtained from the application of the model to a control charts data, using a back-propagation neural network as the learning tool, show that this unique approach is an effective method of knowledge acquisition for a manufacturing system in the early stages.

For many years, scientists have engaged in profiling altered genes to help diagnose related cancers. However, the size of the sample to develop a new profile of cancer genes in the beginning stage is usually small because of costly procedure. Researchers are often disturbed by the analytical method because there has been no effective technique to deal with such small sample size situations in cancer genes diagnosis. The purpose of the study was to employ a new method, mega-trend-diffusion technique, to improve the accuracy of gene diagnosis for bladder cancer on a very limited number of samples. The modeling results showed that when the number of training data increased, the learning accuracy of the bladder cancer diagnosis was enhanced stably, from 82% to 100%. Compared with traditional methods, this study provides a new approach of a reliable model for small dataset analysis. Although the study treats bladder cancer as an example, it is believed that the findings can be generalized to other diseases with limited sample size.

From computational learning theory, sample size in machine learning problems indeed affects the learning performance. Since only few samples can be obtained in the early stages of a system and fewer exemplars usually lead to a low learning accuracy, this research compares different machine learning methods through their classification accuracies to improve small-data-set learning. Techniques used in this paper include the mega-trend diffusion technique, a backpropagation neural network, a support vector machine, and decision trees to explore the machine learning issue with two real medical data sets concerning cancer. The result of the experiment shows that the mega-trend diffusion technique and backpropagation approaches are effective methods of small-data-set learning.

Neural networks are widely utilized to extract management knowledge from acquired data, but having enough real data is not always possible. In the early stages of dynamic flexible manufacturing system (FMS) environments, only a litter data is obtained, and this means that the scheduling knowledge is often unreliable. The purpose of this research is to utilize data expansion techniques for an obtained small data set to improve the accuracy of machine learning for FMS scheduling. This research proposes a mega-trend-diffusion technique to estimate the domain range of a small data set and produce artificial samples for training the modified backpropagation neural network (BPNN). The tool used is the Pythia software. The results of the FMS simulation model indicate that learning accuracy can be significantly improved when the proposed method is applied to a very small data set.

Owing to the factors of cost and time limit, the number of samples is usually small in the early stages of manufacturing systems. When the number of available data is small, traditional statistic techniques have difficulty to obtain robust analyses. Therefore, based on a uni-modality distribution assumption, many researchers have proposed virtual sample generation methods to expand the training sample size to enhance the performance of small data set learning. In practice, small data may be following a multi-modality distribution. Therefore, in order to solve multi-modal small data sets, this study proposes a new approach to create multi-modality Weibull virtual samples, where we use the maximal p-value to estimate parameters of the Weibull distribution. In addition, the soft DBSCAN method is used to identify a suitable number of modalities. One data set is employed to check the performance of the proposed method, and comparisons are made by the prediction on root mean square error. The results using a paired t-test show that the proposed method has a superior prediction performance than that of the mega-trend-diffusion method using a uni-modality triangular membership function.

In many small dataset learning tasks, however, owing to the incomplete data structure, the explicit information for decision making is limited. This research aims to learn more information hidden inside the incomplete data by adding more samples to strengthen data structures. Based on the prior knowledge provided by the M5' model tree, the proposed research mechanism generates artificial samples to enhance the crisp data structures. Besides, the ability to handle nominal attributes is also provided in this research, while it is usually lacking in most sample generation approaches. In the experiments, the results show that the knowledge acquisition capability and the predictive accuracies of M5' are improved.

Science learned models based on limited data are usually fragile, researchers suggest the adoption of virtual samples to improve the prediction model. In this study, nonparametric statistical tool, Kolmogorov-Smirnov test, is introduced to examine the distribution of virtual samples without any assumption about the underlying population. The examination procedure would help control the quality of the generated virtual samples, such that the prediction model can be more robust with the basis of high quality virtual samples. Experimental results show that the prediction model with statistical test procedure performs better than the original one, with more stable and improved accuracies, and the examination procedure can effectively lower the prediction error.