Learning a Nonlinear Combination of Generalized Heterogeneous Classifiers

Finding an effective way to combine the base learners is an essential part of constructing a heterogeneous ensemble of classifiers. In this paper, we propose a framework for heterogeneous ensembles, which investigates using an artificial neural network to learn a nonlinear combination of the base classifiers. In the proposed framework, a set of heterogeneous classifiers are stacked to produce the first-level outputs. Then these outputs are augmented using several combination functions to construct the inputs of the second-level classifier. We conduct a set of extensive experiments on 121 datasets and compare the proposed method with other established and state-of-the-art heterogeneous methods. The results demonstrate that the proposed scheme outperforms many heterogeneous ensembles, and is superior compared to singly tuned classifiers. The proposed method is also compared to several homogeneous ensembles and performs notably better. Our findings suggest that the improvements are even more significant on larger datasets.