Search

Ensemble pruning is substantial for the successful application of an ensemble system. A novel method for Ensemble Pruning via BackTracking algorithm (EnPBT) was proposed by us in our previous work. Backtracking algorithm can systematically search for the solutions of a problem in a depth-first and jumping manner, suitable for solving all those large-scale combinatorial problems. The validity of EnPBT algorithm has been verified in our previous work. However, the relatively slow pruning speed might be a drawback of EnPBT. Aiming at this problem, an efficient and novel ensemble pruning algorithm is proposed in this paper, i.e. One-Path and Two-Trips (OPTT) ensemble pruning algorithm. It is very fast in pruning speed, while its classification performance has no significant difference with EnPBT, as demonstrated in the experimental results of this work. In short, OPTT achieves a proper trade-off between pruning effectiveness and efficiency.

Ensemble pruning is crucial for the consideration of both efficiency and predictive accuracy of an ensemble system. This paper proposes a new Competitive technique for Ensemble Pruning based on Cross-Validation (CEPCV). The data to be learnt by neural computing models are mostly drifting with time and environment, therefore a dynamic ensemble pruning method is indispensable for practical applications, while the proposed CEPCV method is just the kind of dynamic ensemble pruning method, which can realize on-line ensemble pruning and take full advantage of potentially valuable information. The algorithm naturally inherits the predominance of cross-validation technique, which implies that those networks regarded as winners in selective competitions and chosen into the pruned ensemble have the ''strongest'' generalization capability. It is essentially based on the strategy of ''divide and rule, collect the wisdom'', and might alleviate the local minima problem of many conventional ensemble pruning approaches only at the cost of a little greater computational cost, which is acceptable to most applications of ensemble learning. The comparative experiments among the four ensemble pruning algorithms, including: CEPCV and the state-of-the-art Directed Hill Climbing Ensemble Pruning (DHCEP) algorithm and two baseline methods, i.e. BSM, which chooses the Best Single Model in the initial ensemble based on their performances on the pruning set, and ALL, which reserves all network members of the initial ensemble, on ten benchmark classification tasks, demonstrate the effectiveness and validity of CEPCV.

Decision tree is a divide and conquer classification method used in machine learning. Most pruning methods for decision trees minimize a classification error rate. In uncertain domains, some sub-trees that do not decrease the error rate can be relevant in pointing out some populations of specific interest or to give a representation of a large data file. A new pruning method (called DI pruning) is presented here. It takes into account the complexity of sub-trees and is able to keep sub-trees with leaves yielding to determine relevant decision rules, although they do not increase the classification efficiency. DI pruning allows to assess the quality of the data used for the knowledge discovery task. In practice, this method is implemented in the UnDeT software.