Constructing Hierarchical Rule Systems

Rule systems have failed to attract much interest in large data analysis problems because they tend to be too simplistic to be use- ful or consist of too many rules for human interpretation. We present a method that constructs a hierarchical rule system, with only a small number of rules at each stage of the hierarchy. Lower levels in this hi- erarchy focus on outliers or areas of the feature space where only weak evidence for a rule was found in the data. Rules further up, at higher lev- els of the hierarchy, describe increasingly general and strongly supported aspects of the data. We demonstrate the proposed method's usefulness on several classification benchmark data sets using a fuzzy rule induction process as the underlying learning algorithm. The results demonstrate how the rule hierarchy allows to build much smaller rule systems and how the model—especially at higher levels of the hierarchy—remains in- terpretable. The presented method can be applied to a variety of local learning systems in a similar fashion.