Search

Publisher Summary This chapter discusses learning approaches to prune the successor set(s). It studies the exclusion of forbidden states or move sequences and localizing the search using the notion of relevance. The chapter distinguishes between on-the-fly and offline learning. One of the most effective approaches to tackle large problem spaces is to prune (i.e., cutoff branches from) the search tree. There are multiple reasons for pruning. Some branches might not lead to a goal state, others lead to inferior solutions; some result in positions already reached on different paths, and others are redundant; though these might lead to a solution, there are still alternatives that also lead to a solution. All state space pruning techniques reduce the node-branching factor of the search tree such that fewer successor nodes have to be analyzed. Since a smaller part of the state space is generated, pruning saves both search time and space. However, there might be a trade-off between the two. Some techniques require rather complex data structures, such that the maintenance of pruning information may be involved. Static pruning techniques detect pruning knowledge prior to the main search routine. Other pruning rules may not be known to the search algorithm at its start and have to be inferred during the execution of the program. This leads to layered search algorithms. In the top-level search, the search algorithms search for problem solutions, and in frequently invoked lower-level searches, pruning knowledge is refined.

Pruning a decision tree is considered by some researchers to be the most important part of tree building in noisy domains. While, there are many approaches to pruning, an alternative approach of averaging over decision trees has not received as much attention. We perform an empirical comparison of pruning with the approach of averaging over decision trees. For this comparison we use a computationally efficient method of averaging, namely averaging over the extended, fanned set of a tree. Since there are a wide range of approaches to pruning, we compare tree averaging with a traditional pruning approach, along with an optimal pruning approach.

We evaluate the performance of weakest-link decision tree pruning using cross-validation. This technique maps tree pruning into a problem of tree selection: Find the best tree, i.e the right-sized tree, from a set of trees ranging in size from the unpruned tree to a null tree. For small samples (no more than 200 cases), extensive empirical evidence supports the following conclusions relative to tree selection: (a) 10-fold cross-validation is nearly unbiased; (b) not pruning a covering tree is highly biased; (c) with at least 100 samples, 10-fold cross-validation usually outperforms not pruning; (d) with at least 50 samples, a strategy based on 2-fold cross-validation is generally more effective than both 10-fold cross-validation and not pruning; (e) with fewer than 50 samples, estimator accuracy is highly variable, with a substantial risk of overoptimism for not pruning on noisy data.