Search

Incompleteness is one of the problematic data quality challenges in real-world machine learning tasks. A large number of studies have been conducted for addressing this challenge. However, most of the existing studies focus on the classification task and only a limited number of studies for symbolic regression with missing values exist. In this work, a new imputation method for symbolic regression with incomplete data is proposed. The method aims to improve both the effectiveness and efficiency of imputing missing values for symbolic regression. This method is based on genetic programming (GP) and weighted K-nearest neighbors (KNN). It constructs GP-based models using other available features to predict the missing values of incomplete features. The instances used for constructing such models are selected using weighted KNN. The experimental results on real-world data sets show that the proposed method outperforms a number of state-of-the-art methods with respect to the imputation accuracy, the symbolic regression performance, and the imputation time. [ABSTRACT FROM AUTHOR]

Genetic programming (GP) has been successfully applied to classification. However, GP may evolve biased classifiers when encountering the problem of class imbalance. These biased classifiers are often not reliable to be applied to some real-world applications. High dimensionality makes it more difficult for classifiers to effectively separate the majority class and the minority class. The use of GP to handle the joint effect of high dimensionality and class imbalance has not been heavily investigated. In this paper, we propose a GP approach to high-dimensional imbalanced classification, with the goals of increasing the classification performance as well as saving training time. To achieve this goal, a new fitness function is developed to solve the problem of class imbalance, and moreover, a strategy is proposed to reuse previous good GP individuals for improving efficiency. The proposed method is examined on ten high-dimensional imbalanced datasets. Experimental results show that, for high-dimensional imbalanced classification, the proposed method generally outperforms other GP methods and traditional classification algorithms using sampling methods to solve the problem of class imbalance. [ABSTRACT FROM AUTHOR]

Genetic programming (GP) represents a well-known and widely used evolutionary computation technique that has shown promising results in optimisation, classification, and symbolic regression problems. However, similar to many other techniques, the performance of GP deteriorates for solving highly complex tasks. Transfer learning can improve the learning ability of GP, which can be seen from previous research on including, but not limited to, symbolic regression and Boolean problems. However, utilising transfer learning to tackle image-related, specifically, image classification, problems in GP is limited. This paper aims at proposing a new method for employing transfer learning in GP to extract and transfer knowledge in order to tackle complex texture image classification problems. To assess the improvement gained from using the extracted knowledge, the proposed method is examined and compared against the baseline GP method and a state-of-the-art method on three publicly available and commonly used texture image classification datasets. The obtained results indicate that the reuse of the extracted knowledge from an image dataset has significant impact on improving the performance in learning different rotated versions of the same dataset, as well as other related image datasets. Further, it is found that the proposed approach in the very first generation of the evolutionary process produces better classification accuracy than the final classification accuracy obtained by the baseline method after 50 generations. [ABSTRACT FROM AUTHOR]

In edge detection, designing new techniques to combine local features is expected to improve detection performance. However, how to effectively design combination techniques remains an open issue. In this study, an automatic design approach is proposed to combine local edge features using Bayesian programs (models) evolved by genetic programming (GP). Multivariate density is used to estimate prior probabilities for edge points and non-edge points. Bayesian programs evolved by GP are used to construct composite features after estimating the relevant multivariate density. The results show that GP has the ability to effectively evolve Bayesian programs. These evolved programs have higher detection accuracy than the combination of local features by directly using the multivariate density (of these local features) in a simple Bayesian model. From evolved Bayesian programs, the proposed GP system has potential to effectively select features to construct Bayesian programs for performance improvement. [ABSTRACT FROM AUTHOR]

Web services, which can be described as functionality modules invoked over a network as part of a larger application are often used in software development. Instead of occasionally incorporating some of these services in an application, they can be thought of as fundamental building blocks that are combined in a process known as Web service composition. Manually creating compositions from a large number of candidate services is very time consuming, and developing techniques for achieving this objective in an automated manner becomes an active research field. One promising group of techniques encompasses evolutionary computing, which can effectively tackle the large search spaces characteristic of the composition problem. Therefore, this paper proposes the use of genetic programming for Web service composition, investigating three variations to ensure the creation of functionally correct solutions that are also optimised according to their quality of service. A variety of comparisons are carried out between these variations and two particle swarm optimisation approaches, with results showing that there is likely a trade-off between execution time and the quality of solutions when employing genetic programming and particle swarm optimisation. Even though genetic programming has a higher execution time for most datasets, the results indicate that it scales better than particle swarm optimisation. [ABSTRACT FROM AUTHOR]

In classification problems, a large number of features are typically used to describe the problem's instances. However, not all of these features are useful for classification. Feature selection is usually an important pre-processing step to overcome the problem of 'curse of dimensionality'. Feature selection aims to choose a small number of features to achieve similar or better classification performance than using all features. This paper presents a particle swarm Optimization (PSO)-based multi-objective feature selection approach to evolving a set of non-dominated feature subsets which achieve high classification performance. The proposed algorithm uses local search techniques to improve a Pareto front and is compared with a pure multi-objective PSO algorithm, three well-known evolutionary multi-objective algorithms and a current state-of-the-art PSO-based multi-objective feature selection approach. Their performances are examined on 12 benchmark datasets. The experimental results show that in most cases, the proposed multi-objective algorithm generates better Pareto fronts than all other methods. [ABSTRACT FROM AUTHOR]

Gaussian-based filtering techniques have been popularly applied to edge detection. However, how to effectively tune parameters of Gaussian filters and how to effectively combine different Gaussian filters are still open issues. In this study, a new genetic programming (GP) approach is proposed to automatically tune parameters of Gaussian filters and automatically combine different types of Gaussian filters to extract edge features. In general, it is time-consuming for GP to evolve edge detectors using a large training image dataset. To efficiently evolve edge detectors from a large training image dataset, we propose sampling techniques (randomly selecting training images) to evolve Gaussian-based edge detectors. A sampling technique only using part of a set of images obtains similar performance to the training data using all of these images. The evolved edge detectors from the proposed sampling technique perform better than the Gaussian gradient and rotation invariant surround suppression. Based on the analysis of GP evolving edge detectors, it is suggested that combining Gaussian filters should be based on different types of Gaussian filters, and the Gaussian gradient should be considered as a major filter in these combinations. [ABSTRACT FROM AUTHOR]

Particle swarm optimization (PSO) is a metaheuristic whose quality of results deteriorates significantly in optimization problems subject to noise. The underlying reason to such a deterioration is that the effect of noise hinders the ability of particles to distinguish good from bad solutions, leading them to suffer from deception, blindness and disorientation. A deceived particle is not partially attracted to the true best solution in its neighborhood, a blinded particle misses an opportunity to improve upon its personal best solution, and a disoriented particle mistakenly prefers a worse solution. These conditions need to be addressed via noise mitigation mechanisms to prevent (or at least reduce) such a deterioration. Single-evaluation methods are the name by which we refer to PSO algorithms that address the effect of noise without performing additional function evaluations. The first of these algorithms was PSO with evaporation (PSO-E), which was proposed to reduce blindness in the swarms, and reports have suggested that it succeeds at finding better solutions than the regular PSO in different stochastic and dynamic optimization problems. However, PSO-E depends on an evaporation factor whose value is determined empirically, and the swarm is always at risk of exhibiting divergent behaviour. In this article, we propose a method to determine a priori the evaporation factor for PSO-E, and we also propose a new PSO with probabilistic updates (PSO-PU) to prevent the risk of divergence. Additionally, we take a different approach and develop a new PSO with average neighborhoods (PSO-AN) to blur the effect of noise and thereby reduce deception. Experiments on 20 large-scale benchmark functions subject to different levels of noise show that the regular PSO (lacking a noise mitigation mechanism) generally finds better solutions than PSO-E and PSO-PU because their approaches cause too much disorientation. However, PSO-AN finds better solutions than the regular PSO thanks to the improved quality of its neighborhood best solutions that partially attract the swarm towards better regions of the search space. [ABSTRACT FROM AUTHOR]

In edge detection, constructing features with rich responses on different types of edges is a challenging problem. Genetic programming (GP) has been previously employed to construct features. Normally, the values of the features constructed by GP are calculated from raw observations. Some existing work has considered the distributions of the raw observations, but these features only poorly indicate class label probabilities. To construct features with rich responses on different types of edges, the distributions of the observations from GP programs are investigated in this study. The values of the constructed features are obtained from estimated distributions, rather than directly using the observations. These features themselves indicate probabilities for the target labels. Basic rotation-invariant features from gradients, image quality, and local histograms are used to construct new composite features. The results show that the invariant features constructed by GP combine advantages from the basic features, reduce drawbacks from basic features alone, and improve the detection performance. In terms of the quantitative and qualitative evaluations, features constructed by GP with distribution estimation are better than the combinations from a Bayesian model and a linear support vector machine approach. [ABSTRACT FROM AUTHOR]

Learning classifier systems (LCSs), an established evolutionary computation technique, are over 30 years old with much empirical testing and foundations of theoretical understanding. XCS is a well-tested LCS model that generates optimal (i.e., maximally general and accurate) classifier rules in the final solution. Previous work has hypothesized the evolution mechanisms in XCS by identifying the bounds of learning and population requirements. However, no work has shown exactly how an optimum rule is evolved or especially identifies whether the methods within an LCS are being utilized effectively. In this paper, we introduce a method to trace the evolution of classifier rules generated in an XCS-based classifier system. Specifically, we introduce the concept of a family tree, termed parent-tree, for each individual classifier rule generated in the system during training, which describes the whole generational process for that classifier. Experiments are conducted on two sample Boolean problem domains, i.e., multiplexer and count ones problems using two XCS-based systems, i.e., standard XCS and XCS with code-fragment actions. The analysis of parent-trees reveals, for the first time in XCS, that no matter how specific or general the initial classifiers are, all the optimal classifiers are converged through the mechanism 'be specific then generalize' near the final stages of evolution. Populations where the initial classifiers were slightly more specific than the known 'ideal' specificity in the target solutions evolve faster than either very specific, ideal or more general starting classifier populations. Consequently introducing the 'flip mutation' method and reverting the conventional wisdom back to apply rule discovery in the match set has demonstrated benefits in binary classification problems, which has implications in using XCS for knowledge discovery tasks. It is further concluded that XCS does not directly utilize all relevant information or all breeding strategies to evolve the optimum solution, indicating areas for performance and efficiency improvement in XCS-based systems. [ABSTRACT FROM AUTHOR]

This article presents a performance study on the effect of synchronicity in communications and neighborhood size in Particle Swarm Optimization (PSO) on large-scale optimization problems. The algorithms under study are the Synchronous PSO (S-PSO), the Asynchronous PSO (A-PSO), and the recently proposed Random Asynchronous PSO (RA-PSO), all of which are evaluated upon the set of benchmark functions presented at the IEEE CEC'2010 special session and competition on large-scale global optimization. Results show that RA-PSO has the best general performance in large neighborhoods, while S-PSO has the best one in small neighborhoods. Rigorous statistical analyses support our observations and provide a guideline on the performance of the algorithms under different neighborhood sizes. [ABSTRACT FROM AUTHOR]

The main goal of the research direction is to extract building blocks of knowledge from a problem domain. Once extracted successfully, these building blocks are to be used in learning more complex problems of the domain, in an effort to produce a scalable learning classifier system (LCS). However, whilst current LCS (and other evolutionary computation techniques) discover good rules, they also create sub-optimum rules. Therefore, it is difficult to separate good building blocks of information from others without extensive post-processing. In order to provide richness in the LCS alphabet, code fragments similar to tree expressions in genetic programming are adopted. The accuracy-based XCS concept is used as it aims to produce maximally general and accurate classifiers, albeit the rule base requires condensation (compaction) to remove spurious classifiers. Serendipitously, this work on scalability of LCS produces compact rule sets that can be easily converted to the optimum population. The main contribution of this work is the ability to clearly separate the optimum rules from others without the need for expensive post-processing for the first time in LCS. This paper identifies that consistency of action in rich alphabets guides LCS to optimum rule sets. [ABSTRACT FROM AUTHOR]

Adaptation during evolution has been an important focus of research in training neural networks. Cooperative coevolution has played a significant role in improving standard evolution of neural networks by organizing the training problem into modules and independently solving them. The number of modules required to represent a neural network is critical to the success of evolution. This paper proposes a framework for the adaptation of the number of modules during evolution. The framework is called adaptive modularity cooperative coevolution. It is used for training recurrent neural networks on grammatical inference problems. The results shows that the proposed approach performs better than its counterparts as the dimensionality of the problem increases. [ABSTRACT FROM AUTHOR]

Tournament selection is one of the most commonly used parent selection schemes in genetic programming (GP). While it has a number of advantages over other selection schemes, it still has some issues that need to be thoroughly investigated. Two of the issues are associated with the sampling process from the population into the tournament. The first one is the so-called 'multi-sampled' issue, where some individuals in the population are picked up (sampled) many times to form a tournament. The second one is the 'not-sampled' issue, meaning that some individuals are never picked up when forming tournaments. In order to develop a more effective selection scheme for GP, it is necessary to understand the actual impacts of these issues in standard tournament selection. This paper investigates the behaviour of different sampling replacement strategies through mathematical modelling, simulations and empirical experiments. The results show that different sampling replacement strategies have little impact on selection pressure and cannot effectively tune the selection pressure in dynamic evolution. In order to conduct effective parent selection in GP, research focuses should be on developing automatic and dynamic selection pressure tuning methods instead of alternative sampling replacement strategies. Although GP is used in the empirical experiments, the findings revealed in this paper are expected to be applicable to other evolutionary algorithms. [ABSTRACT FROM AUTHOR]