Feature-Based and Adaptive Rule Adaptation in Dynamic Environments

Abstract Rule-based systems have been used increasingly to augment learning algorithms for annotating data. Rules alleviate many of the shortcomings inherent in pure algorithmic approaches, in cases algorithms are not working well or lack from enough training data. However, in dynamic curation environments where data are constantly changing, there is a need to craft and adapt rules to keep them applicable and precise. Rule adaptation has been proven to be painstakingly difficult and error-prone, as an analyst is needed for examining the precision of rules and applying different modifications to adapt the imprecise ones. In this paper, we present an autonomic and conceptual approach to adapt data annotation rules. Our approach offloads analysts from adapting rules; it boosts rules to annotate a larger number of items using a set of high-level conceptual features, e.g. topic. We utilize a Bayesian multi-armed-bandit algorithm, an online learning algorithm that adapts rules based on the feedback collects from the curation environment over time. We propose a summarization technique, which offers a set of high-level conceptual features for annotating items by identifying the semantical relationships among them. We conduct experiments on different curation domains and compare the performance of our approach with systems relying on analysts for adapting rules. The experimental results show that our approach has a comparative performance to analysts in adapting rules.