Search

Abstract In this paper, we propose a weighted multiple classifier framework based on random projections. Similar to the mechanism of other homogeneous ensemble methods, the base classifiers in our approach are obtained by a learning algorithm on different training sets generated by projecting the original up-space training set to lower dimensional down-spaces. We then apply a Least SquarE−based method to weigh the outputs of the base classifiers so that the contribution of each classifier to the final combined prediction is different. We choose Decision Tree as the learning algorithm in the proposed framework and conduct experiments on a number of real and synthetic datasets. The experimental results indicate that our framework is better than many of the benchmark algorithms, including three homogeneous ensemble methods (Bagging, RotBoost, and Random Subspace), several well-known algorithms (Decision Tree, Random Neural Network, Linear Discriminative Analysis, K Nearest Neighbor, L2-loss Linear Support Vector Machine, and Discriminative Restricted Boltzmann Machine), and random projection-based ensembles with fixed combining rules with regard to both classification error rates and F1 scores. [ABSTRACT FROM AUTHOR]

Highlights • Orthogonal projection is proposed to mitigate the domain shift problem. • Semantic feature representation is included to alleviate visual category ambiguity. • Deep model is applied to improve the performance. • Extensive experiments show the superiority of our algorithm. Abstract To mitigate the problems of visual ambiguity and domain shift in conventional zero-shot learning (ZSL), in this paper, we propose a novel method, namely, dual-verification network (DVN), which accepts features and attributes in a pairwise manner as input and verifies the result in both the attribute and feature spaces. First, the DVN projects a feature onto an orthogonal space, where the projected feature has maximum correlation with its corresponding attribute and is orthogonal to all the other attributes. Second, we adopt the concept of semantic feature representation, which computes the relationship between the semantic feature and class labels. Based on this concept, we project the attributes onto the feature space by extending the attributes and labels from the class level to instance level. In addition, we employ a deep architecture and utilize the cross entropy loss to train an end-to-end network for dual verification. Extensive experiments in ZSL and generalized ZSL are performed on four well-known datasets, and the results show that the proposed DVN exhibits a competitive performance relative to the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

As an important preprocessing step for data mining, attribute reduction has become a hot research topic in rough set theory. In practice, many real data may vary dynamically with time, therefore, reduct will change dynamically under the variation of objects and attributes in decision systems. The classical attribute reduction methods need to recompute from scratch, which are ineffective to deal with dynamic decision systems. How to implement updating reducts by utilizing previous results is vital for improving the efficiency of attribute reduction approaches. In the paper, we firstly introduce incremental mechanisms of computing reduct when objects and attributes of the decision system change dynamically. Then, incremental methods are developed to update reduct when attributes and objects increase simultaneously. Finally, a series of experiments are conducted to validate the proposed incremental attribute reduction methods. Experimental results show that they are effective to update reduct with change of attributes and objects in the decision systems. [ABSTRACT FROM AUTHOR]

With the increase in available extensible markup language (XML) documents, numerous approaches to querying have been proposed in the literature. XPath queries and Twig pattern queries are the two basic approaches, directly affecting the efficiency of XML operations. Distributive manipulation of massive XML data is challenging. This paper aims to develop an efficient distributed XML query processing method using MapReduce, which simultaneously processes several queries on large volumes of XML data. First, we split up a large-scale XML data file into file-splits and put them in a distributed storage system. Then, we present an efficient algorithm to compute different fragments of the document tree using the MapReduce framework in parallel. In order to efficiently handle a large amount of XML data, we built a partition index and used a random access mechanism for specific queries. The experiment results show that our proposed approach is efficient with good scalability. [ABSTRACT FROM AUTHOR]

The detection of gravitational waves with ground-based laser-interferometric detectors requires sensitivity to changes in distance much smaller than the diameter of atomic nuclei. Though sophisticated machinery and techniques have been developed over the past few decades to isolate such instruments from non-astrophysical noise, the detectors are still susceptible to instrumental and environmental noise transients known as “glitches,” which hinder searches for transient gravitational waves. The Gravity Spy project is an effort to comprehensively classify the glitches that afflict gravitational wave detectors into morphological families by combining the strengths of machine learning algorithms and citizen scientists. This paper presents the initial Gravity Spy dataset used for citizen scientist and machine learning classification – a static, accessible, documented dataset for testing machine learning supervised classification. Previous versions of this dataset used in [8, 53] did not include all current classes and also for some of the classes, some samples were pruned and added. This set consists of time–frequency images of LIGO glitches and their associated metadata. These glitches are organized by time–frequency morphology into 22 classes for which descriptions and representative images are presented. Results from the application of state-of-the-art supervised classification methods to this dataset are presented in order to provide baselines for future glitch classification work. Standard splitting for training, validation, and testing sets are also presented to facilitate the comparison between different machine learning methods. The baseline methods are selected from both traditional and more recent deep learning approaches. An ensemble framework is developed that demonstrates that combining various classifiers can yield a more accurate model for classification. The ensemble classifier, trained with the standard training set, achieves 98.21% accuracy on the standard test set. [ABSTRACT FROM AUTHOR]

Hidden suspicious accounts are sparsely connected in social graphs; however, certain suspicious messages are usually forwarded in bulk to extend their overall propagation scope. Existing anti-attack methods only detect single messages or accounts. Because most algorithms rely on the connection relations among the accounts in social graphs, they may repeatedly detect the same account. Furthermore, these hidden suspicious accounts cannot be identified and eliminated completely. Therefore, messages forwarded by hidden suspicious accounts should be merged, and the accounts should be eliminated simultaneously rather than individually. This paper introduces the forwarding message tree, which combines accounts based on the relations among their forwarded messages. Our approach clearly exposes the inner relations among hidden suspicious accounts and conveniently deletes those accounts. First, we present the forwarding message tree and identify six effective features: the forwarding layer relation, propagation depth, propagation breadth, repeated forwarding behavior, propagation speed, and average tree weight. Next, to illustrate the effectiveness of these features, we incorporate them into machine learning algorithms. The detection accuracy and false-positive rates for a real dataset collected from an online social network are 95.32% and 0.5%, respectively. Most of the proposed features rate at the top of a gain ranking. We conclude that the forwarding message tree can indeed detect and delete hidden suspicious accounts. [ABSTRACT FROM AUTHOR]

Multiple-Instance Learning (MIL) refers to the problem wherein each object is a bag consisting of multiple instances and only the bags' labels are provided. MIL data can contain irrelevant, redundant, and noisy components, which makes feature-extraction preprocessing essential for performance improvement. In this paper, we propose a Multiple-Instance Feature Extraction (MIFE) framework to design algorithms at both the bag and instance levels based on the Maximum Trace-Difference criterion, which simultaneously maximizes between-class scattering and minimizes within-class scattering. MIFE not only treats the existing Multiple-Instance Discriminant Analysis algorithm as an instance-level realization but also enables us to adopt different bag-level distances to design corresponding bag-level algorithms. In particular, we introduce the Class-to-Bag (C2B) and Bag-to-Bag (B2B) distances into the MIFE framework and obtain the MIFE-C2B and MIFE-B2B algorithms, respectively. The experimental results show that both MIFE-C2B and MIFE-B2B obtain competitive classification performance, and MIFE-B2B obtains the best performance on most tested datasets. The dimensionality reduction results show that both MIFE-C2B and MIFE-B2B obtain their best performance with no more than approximately 30% of the original dimensions on most tested datasets. [ABSTRACT FROM AUTHOR]

Random Vector Functional-link (RVFL) networks, a class of learner models, can be regarded as feed-forward neural networks built with a specific randomized algorithm, i.e., the input weights and biases are randomly assigned and fixed during the training phase, and the output weights are analytically evaluated by the least square method. In this paper, we provide some insights into RVFL networks and highlight some practical issues and common pitfalls associated with RVFL-based modelling techniques. Inspired by the folklore that “all high-dimensional random vectors are almost always nearly orthogonal to each other”, we establish a theoretical result on the infeasibility of RVFL networks for universal approximation, if a RVFL network is built incrementally with random selection of the input weights and biases from a fixed scope, and constructive evaluation of its output weights. This work also addresses the significance of the scope setting of random weights and biases in respect to modelling performance. Two numerical examples are employed to illustrate our findings, which theoretically and empirically reveal some facts and limits of such class of randomized learning algorithms. [ABSTRACT FROM AUTHOR]

Active learning, one of large and important branches in machine learning and data mining, aims to build an accurate learning model with a relatively small number of labeled points which are chosen actively by the constructed learning model. Active learning algorithms can play an important role when the cost of obtaining labels of data points is expensive. However, the active learning algorithms for the relevance vector machine regression, one of the famous kernel methods for the regression due to its sparseness, Bayesian characteristics, and kernel properties, have not been established yet, whilst those for other kernel methods are extensively developed. In this paper, we propose a transductive relevance vector machine to construct the active learning algorithm for relevance vector machine based a sparse Bayesian kernel regression model. The proposed method obtains its basis vectors from the unlabeled data set as well as the labeled one. Then, we also suggest three querying strategies which uses only the relevance vectors automatically selected by the developed model for active selection for data points to be labeled. Applied to several artificial and real data sets, the proposed method show significantly more accurate results than the benchmark, the random selection, while maintaining the sparse characteristic of the traditional relevance vector machines. [ABSTRACT FROM AUTHOR]

Pedestrian detection from videos is a useful technique in intelligent transportation systems. Some key challenges of accurate pedestrian detection are the large variations in pedestrian appearance as the pedestrians assume different poses and the different camera views that are involved. This makes the generic visual descriptors unreliable for real-world pedestrian detection. In this paper, we propose a high-level human-specific descriptor for detecting pedestrians in multiple videos. More specifically, by obtaining the feature matrix from a sliding window, we use multiple mapping vectors to project the original feature matrix into different mask spaces. Inspired by the part-based model [12], it is natural to formulate the pedestrian detection into a multiple-instance learning (MIL) framework. Afterward, we adopt an MI-SVM [9] to solve it. To evaluate the proposed detection algorithm, we implement the pedestrian detection algorithm in FPGA, which can process over 30 fps. Moreover, our method outperforms many existing object detection algorithms in terms of accuracy. [ABSTRACT FROM AUTHOR]

Graphics Processing Units (GPUs) are effective tools for improving the efficiency of many computationally demanding algorithms. GPUs have been particularly effective at speeding up the evaluation stage of evolutionary machine learning systems. The speedups obtained in these tasks, depend on many factors: dataset characteristics, the parallel strategy of the GPU code and the fit of the GPU code within the rest of the learning system. A solid understanding of all these factors is required to choose and adjust the most suitable GPU strategy in different scenarios. In this paper we present a large-scale performance evaluation of two GPU strategies for speeding up the evaluation of evolutionary machine learning systems. We use highly-tuneable synthetic problems to exhaustively explore the space of problem characteristics and determine the type of problems where each strategy performs best. The lessons learnt from this extensive evaluation are further confirmed by running experiments on a broad range of real-world datasets. Through this thorough evaluation we obtain a solid understanding of the capabilities and limitations of the evaluated GPU strategies for boosting the efficiency of evolutionary machine learning systems. [ABSTRACT FROM AUTHOR]

Attribute reduction is an important preprocessing step in pattern recognition, machine learning and data mining. As an effective method for attribute reduction, rough set theory offers a useful and formal methodology. It retains the discernibility power of the original datasets; thus, attribute reduction has been extensively studied in rough set theory. However, the inefficiency of the existing attribute reduction algorithms limits the application of rough sets. In this paper, we first analyse the limitations of existing attribute reduction algorithms. Then, a novel measure of attribute quality, called the relative discernibility degree, is proposed based on the discernibility. Theoretical analysis shows that this measure can find relative dispensable attributes and remain unchanged after removing the relative dispensable attributes and redundant objects in the process of selecting attributes. This property can be used to reduce the search space and accelerate the heuristic process of attribute reduction. Consequently, a new attribute reduction algorithm is proposed from the viewpoint of discernibility. Further- more, the relationships among the reduction definitions of the algebra view, information view and discernibility view are derived. Some non-equivalent relationships among these views of rough set theory in inconsistent decision tables are discovered. A set of numerical experiments was conducted on UCI datasets. Experimental results show that the proposed algorithm is effective and efficient and is applicable to the case of large-scale datasets. [ABSTRACT FROM AUTHOR]

In this paper, a dynamic mentoring scheme along with a self-regulation scheme have been incorporated in the standard Particle Swarm Optimization (PSO) algorithm to empower the searching particles with human-like characteristics. The algorithm is referred to as a Dynamic Mentoring and Self-Regulation based Particle Swarm Optimization (DMeSR-PSO) algorithm. Based on their experiences, the particles are divided into three groups, viz., the mentor group, the mentee group and the independent learner group where the number of particles in each group is dynamically changing in every iteration. In human learning psychology, mentoring is regarded as a powerful and effective learning process and independent learners are the ones who do not need mentoring and are capable of performing self-regulation of their own knowledge. Therefore, the particles in each of the above three groups have different learning strategies for their velocity updates where the mentors are equipped with a strong self-belief based search, the mentees are taking guidance from the mentors and the independent learners employ self-perception strategy. The DMeSR-PSO algorithm has been extensively evaluated using the simple unimodal and multimodal benchmark functions from CEC2005, more complex shifted and rotated benchmark functions fromCEC2013 and also based on eight real- world problems from CEC2011. The results have been compared with six state-of-the-art PSO variants and five meta-heuristic algorithms for the CEC2005 problems. Further, a comparative analysis on CEC2013 benchmark functions with different PSO variants has also been presented. Finally, DMeSR-PSO's performance on the real-world problems is compared with the top two algorithms from the CEC2011 competition. The results indicate that the proposed learning strategies help DMeSR-PSO to achieve faster convergence and provide better solutions in most of the problems with a 95% confidence level, yielding an effective optimization algorithm for real-world applications. [ABSTRACT FROM AUTHOR]

Recovering low-rank and sparse matrices from incomplete or corrupted observations is an important problem in statistics, machine learning, computer vision, as well as signal and image processing. In theory, this problem can be solved by the natural convex joint/mixed relaxations (i.e., l 1 -norm and trace norm) under certain conditions. However, all current provable algorithms suffer from superlinear per-iteration cost, which severely limits their applicability to large-scale problems. In this paper, we propose a scalable, provable and structured robust bilinear factorization (RBF) method to recover low-rank and sparse matrices from missing and grossly corrupted data, i.e., robust matrix completion (RMC), or incomplete and grossly corrupted measurements, i.e., compressive principal component pursuit (CPCP). Specifically, we first present two small-scale matrix trace norm regularized bilinear factorization models for RMC and CPCP problems, in which repetitively calculating SVD of a large-scale matrix is replaced by updating two much smaller factor matrices. Then, we apply the alternating direction method of multipliers (ADMM) to efficiently solve the RMC problems. Finally, we provide the convergence analysis of our algorithm, and extend it to address general CPCP problems. Experimental results verified both the efficiency and effectiveness of our method compared with the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Data clustering aims to segment a database into groups of objects based on the similarity among these objects. Due to its unsupervised nature, the search for a good-quality solution can become a complex process. There is currently a wide range of clustering algorithms, and selecting the best one for a given problem can be a slow and costly process. In 1976, Rice formulated the Algorithm Selection Problem (ASP), which postulates that the algorithm performance can be predicted based on the structural characteristics of the problem. Meta-learning brings the concept of learning about learning; that is, the meta-knowledge obtained from the algorithm learning process allows the improvement of the algorithm performance. Meta-learning has a major intersection with data mining in classification problems, in which it is normally used to recommend algorithms. The present paper proposes new ways to obtain meta-knowledge for clustering tasks. Specifically, two contributions are explored here: (1) a new approach to characterize clustering problems based on the similarity among objects; and (2) new methods to combine internal indices for ranking algorithms based on their performance on the problems. Experiments were conducted to evaluate the recommendation quality. The results show that the new meta-knowledge provides high-quality algorithm selection for clustering tasks. [ABSTRACT FROM AUTHOR]

Multiple Kernel Learning (MKL) can boost classification performance through using multiple kernels rather than a single fixed one. Unlike the traditional MKL with the implicit kernels, Multiple Empirical Kernel Learning (MEKL) explicitly maps input data into multiple feature spaces. This paper focuses on MEKL and proposes an effective Threefold Structural MEKL (TSMEKL). The first fold structure is the space structural information between different mapped feature spaces. The second one is the class discriminant information within each mapped feature space. The third one is the cluster structural information of samples in each mapped feature space. The classical MEKL mainly pays attention to the first two structures, but neglects the last one. The proposed TSMEKL introduces the cluster structural information into MEKL. Doing so can simultaneously utilize the space, the class, and the cluster information in the way from globality to locality. Therefore, TSMEKL utilizes threefold structural information to result in the improvement of classification performance. To the best of our knowledge, it is the first time to introduce the cluster information into the MEKL framework. The main advantage of the developed TSMEKL is considering different folds of data information to improve classification performance. The experimental results validate the feasibility and effectiveness of TSMEKL. Moreover, we discuss the theoretical and experimental generalization risk bound of the proposed algorithm in terms of the Rademacher complexity. [ABSTRACT FROM AUTHOR]

Minimum spanning trees (MSTs) have long been used in data mining, pattern recognition and machine learning. However, it is difficult to apply traditional MST algorithms to a large dataset since the time complexity of the algorithms is quadratic. In this paper, we present a fast MST (FMST) algorithm on the complete graph of N points. The proposed algorithm employs a divide-and-conquer scheme to produce an approximate MST with theoretical time complexity of O ( N 1.5 ) , which is faster than the conventional MST algorithms with O ( N 2 ) . It consists of two stages. In the first stage, called the divide-and-conquer stage, K -means is employed to partition a dataset into N clusters. Then an exact MST algorithm is applied to each cluster and the produced N MSTs are connected in terms of a proposed criterion to form an approximate MST. In the second stage, called the refinement stage, the clusters produced in the first stage form N - 1 neighboring pairs, and the dataset is repartitioned into N - 1 clusters with the purpose of partitioning the neighboring boundaries of a neighboring pair into a cluster. With the N - 1 clusters, another approximate MST is constructed. Finally, the two approximate MSTs are combined into a graph and a more accurate MST is generated from it. The proposed algorithm can be regarded as a framework, since any exact MST algorithm can be incorporated into the framework to reduce its running time. Experimental results show that the proposed approximate MST algorithm is computationally efficient, and the approximation is close to the exact MST so that in practical applications the performance does not suffer. [ABSTRACT FROM AUTHOR]

Abstract: It is well known that data structures are essential in obtaining efficient algorithms, having a major importance in the software development process. A proper instantiation of a data structure can greatly impact the performance and the efficiency of the software system. In this paper we are focusing on the problem of customizing data structures instances during the execution of software systems using a supervised learning approach. In order to customize a data structure instance according to the software system’s current execution context, a neural network will be used. We experimentally evaluated our technique on a case study, emphasizing the advantages of the proposed approach. The obtained experimental results highlight the potential of our proposal in using a supervised learning based approach for dynamically configuring data structures instances. [Copyright &y& Elsevier]

Abstract: Non-stationary classification problems concern the changes on data distribution over a classifier lifetime. To face this problem, learning algorithms must conciliate essential, but difficult to gather, attributes like good classification performance, stability and low associated costs, like processing time and memory. This paper presents an extension of the K-associated optimal graph learning algorithm to cope with classification over non-stationary domains. The algorithm relies on a graph structure consisting of many disconnected components (subgraphs). Such graph enhances data representation by fitting locally groups of data according to a purity measure, which, in turn, quantifies the overlapping between vertices of different classes. As a result, the graph can be used to accurately estimate the probability of unlabeled data to belong to a given class. The proposed algorithm is benefited from the dynamical evolution of the graph by updating its set of components when new data is presented along time, by removing old components as new components arise. Experimental results on artificial and real domains and further statistical analysis show that the proposed algorithm is an effective solution to non-stationary classification problems. [Copyright &y& Elsevier]

Abstract: The possibility of missing or incomplete data is often ignored when describing statistical or machine learning methods, but as it is a common problem in practice, it is relevant to consider. A popular strategy is to fill in the missing values by imputation as a pre-processing step, but for many methods this is not necessary, and can yield sub-optimal results. Instead, appropriately estimating pairwise distances in a data set directly enables the use of any machine learning methods using nearest neighbours or otherwise based on distances between samples. In this paper, it is shown how directly estimating distances tends to result in more accurate results than calculating distances from an imputed data set, and an algorithm to calculate the estimated distances is presented. The theoretical framework operates under the assumption of a multivariate normal distribution, but the algorithm is shown to be robust to violations of this assumption. The focus is on numerical data with a considerable proportion of missing values, and simulated experiments are provided to show accurate performance on several data sets. [Copyright &y& Elsevier]

Highlights: [•] This paper presents a new multiobjective swarm hybrid learning algorithm, MPSON. [•] We also propose MGAN to comparison with MPSON based on performance measures. [•] The aim of our algorithms is to obtain simple and accurate RBF networks simultaneously. [•] We validate the algorithms on datasets with different features, samples and classes. [•] Results show MPSON is the best algorithm with the accuracy-complexity balance for RBF network design. [Copyright &y& Elsevier]

Abstract: This paper formulates a twin-hypersphere support vector machine (THSVM) classifier for binary recognition. Similar to the twin support vector machine (TWSVM) classifier, this THSVM determines two hyperspheres by solving two related support vector machine (SVM)-type problems, each one is smaller than the classical SVM, which makes the THSVM be more efficient than the classical SVM. In addition, the THSVM avoids the matrix inversions in its two dual quadratic programming problems (QPPs) compared with the TWSVM. By considering the characteristics of the dual QPPs of THSVM, an efficient Gilbert’s algorithm for the THSVM based on the reduced convex hull (RCH) instead of directly optimizing its pair of QPPs is further presented. Computational results on several synthetic as well as benchmark datasets indicate the significant advantages of the THSVM classifier in the computational time and test accuracy. [Copyright &y& Elsevier]

Abstract: Selection of cutting parameters in machining operations is an essential task to reduce cost of the products and increase quality. This paper presents an optimization approach based on artificial bee colony algorithm for optimal selection of cutting parameters in multi-pass turning operations. The objective is to find the optimized cutting parameters in the turning operations. A comparison of evolutionary-based optimization techniques to solve multi-pass turning optimization problems is presented. The results of the proposed approach for the case studies are compared with previously published results by using other optimization techniques in the literature. [Copyright &y& Elsevier]

Abstract: Classification is one of the key issues in the fields of decision sciences and knowledge discovery. In this paper, we present a new classification method based on gravitational potential energy between particles. The basic principle of gravitation based classification (GBC) algorithm is to find the equilibrium condition of the classifier, which is modeled as a classifier line between two groups of fixed particles. In the proposed approach, the input data is a collection of masses, which interact with each other based on Newton’s universal law of gravitation and the laws of motion. We present a convex formulation for this problem that always converges to a global optimum solution. The proposed method has been compared with some well-known classification approaches, and the results confirm the high performance of the proposed method. [Copyright &y& Elsevier]

Abstract: This paper proposes the Simplified Structure Evolving Method (SSEM) for fuzzy system identification, which improves our earlier work on the Structure Evolving Learning Method for fuzzy systems (SELM). The improvement is that SSEM applies a scheme that starts with the simplest fuzzy rule set with only one fuzzy rule (instead of 2 n fuzzy rules as in SELM, where n is the number of input variables), whilst retaining all the advantages of SELM. SELM is able to solve the problem of the exponential increase of fuzzy rules, however, it requires a basic fuzzy rule set which is exponential to the number of input variables (2 n fuzzy rules) as a starting point. The improvement offered by SSEM enables automatic feature selection and system structure identification, and avoids inefficient rules and inefficient variable involvement for system identification. This improvement enables fuzzy systems to be applicable to problems of any input dimension. Three benchmark examples with high dimension inputs are given to illustrate the advantages of the proposed algorithm. [Copyright &y& Elsevier]

Abstract: In this paper, we analyse the single processor total weighted completion time scheduling problem with the learning effect, where the processing time of each job is a non-increasing function dependent on the sum of the normal processing times of preceding jobs. We prove that the considered problem is at least NP-hard. Moreover, a pseudopolynomial time dynamic programming algorithm that optimally solves the problem with a step learning function (curve) is constructed. Furthermore, fast approximation algorithms for the general version of the problem, where job processing times are described by arbitrary functions dependent on the sum of the normal job processing times, are provided. Their efficiency is verified numerically and for Weighted Shortest Processing Times algorithm a worst case analysis is also performed. [Copyright &y& Elsevier]

Abstract: The machine learning community has traditionally used correct classification rates or accuracy (C) values to measure classifier performance and has generally avoided presenting classification levels of each class in the results, especially for problems with more than two classes. C values alone are insufficient because they cannot capture the myriad of contributing factors that differentiate the performance of two different classifiers. Receiver Operating Characteristic (ROC) analysis is an alternative to solve these difficulties, but it can only be used for two-class problems. For this reason, this paper proposes a new approach for analysing classifiers based on two measures: C and sensitivity (S) (i.e., the minimum of accuracies obtained for each class). These measures are optimised through a two-stage evolutionary process. It was conducted by applying two sequential fitness functions in the evolutionary process, including entropy (E) for the first stage and a new fitness function, area (A), for the second stage. By using these fitness functions, the C level was optimised in the first stage, and the S value of the classifier was generally improved without significantly reducing C in the second stage. This two-stage approach improved S values in the generalisation set (whereas an evolutionary algorithm (EA) based only on the S measure obtains worse S levels) and obtained both high C values and good classification levels for each class. The methodology was applied to solve 16 benchmark classification problems and two complex real-world problems in analytical chemistry and predictive microbiology. It obtained promising results when compared to other competitive multi-class classification algorithms and a multi-objective alternative based on E and S. [Copyright &y& Elsevier]

Abstract: This paper presents a new feature discovery approach called FEADIS that strengthens learning algorithms with discovered features. The discovered features are formed by various mathematical functions including ceil, mod, sin, and similar. These features are constructed in an iterative manner to improve gradually its learning performance. We demonstrate FEADIS capabilities by testing different types of datasets including periodical datasets. From the results, we conclude that FEADIS increases the performance of learning algorithms in a wide range of datasets for nominal or numeric target feature. Furthermore, most of the well known classifiers without FEADIS strengthening have severe difficulty in handling datasets that have periodical functional relations between input features and target feature – a difficulty circumvented by their potential use of FEADIS. [Copyright &y& Elsevier]

Abstract: Model trees are a particular case of decision trees employed to solve regression problems. They have the advantage of presenting an interpretable output, helping the end-user to get more confidence in the prediction and providing the basis for the end-user to have new insight about the data, confirming or rejecting hypotheses previously formed. Moreover, model trees present an acceptable level of predictive performance in comparison to most techniques used for solving regression problems. Since generating the optimal model tree is an NP-Complete problem, traditional model tree induction algorithms make use of a greedy top-down divide-and-conquer strategy, which may not converge to the global optimal solution. In this paper, we propose a novel algorithm based on the use of the evolutionary algorithms paradigm as an alternate heuristic to generate model trees in order to improve the convergence to globally near-optimal solutions. We call our new approach evolutionary model tree induction (E-Motion). We test its predictive performance using public UCI data sets, and we compare the results to traditional greedy regression/model trees induction algorithms, as well as to other evolutionary approaches. Results show that our method presents a good trade-off between predictive performance and model comprehensibility, which may be crucial in many machine learning applications. [Copyright &y& Elsevier]

Abstract: Clustering is a popular non-directed learning data mining technique for partitioning a dataset into a set of clusters (i.e. a segmentation). Although there are many clustering algorithms, none is superior on all datasets, and so it is never clear which algorithm and which parameter settings are the most appropriate for a given dataset. This suggests that an appropriate approach to clustering should involve the application of multiple clustering algorithms with different parameter settings and a non-taxing approach for comparing the various segmentations that would be generated by these algorithms. In this paper we are concerned with the situation where a domain expert has to evaluate several segmentations in order to determine the most appropriate segmentation (set of clusters) based on his/her specified objective(s). We illustrate how a data mining process model could be applied to address this problem. [Copyright &y& Elsevier]