Search

• Multi-criteria approaches improve the performance of semi-supervised decision trees. • Scale inconsistencies can be effectively handled via proximity measures. • Using target relations enhances predictive performance of semi-supervised decision trees. Scale inconsistency is a widely encountered issue in multi-output learning problems. Specifically, target sets with multiple real valued or a mixture of categorical and real valued variables require addressing the scale differences to obtain predictive models with sufficiently good performance. Data transformation techniques are often employed to solve that problem. However, these operations are susceptible to different shortcomings such as changing the statistical properties of the data and increase the computational burden. Scale differences also pose problem in semi-supervised learning (SSL) models as they require processing of unsupervised information where distance measures are commonly employed. Classical distance metrics can be criticized as they lose efficiency when variables exhibit type or scale differences, too. Besides, in higher dimensions distance metrics cause problems due to loss of discriminative power. This paper introduces alternative semi-supervised tree-based strategies that are robust to scale differences both in terms of feature and target variables. We propose use of a scale-invariant proximity measure by means of tree-based ensembles to preserve the original characteristics of the data. We update classical tree derivation procedure to a multi-criteria form to resolve scale inconsistencies. We define proximity based clustering indicators and extend the supervised model with unsupervised criteria. Our experiments show that proposed method significantly outperforms its benchmark learning model that is predictive clustering trees. [ABSTRACT FROM AUTHOR]

• Multi-objective approaches change the learning trajectory for the better. • Using targets as additional inputs improves the generalization power of the classifiers. • The order of adding targets is important in chaining. • Exploiting target relations augments the learning process. Multitask (multi-target or multi-output) learning (MTL) deals with simultaneous prediction of several outputs. MTL approaches rely on the optimization of a joint score function over the targets. However, defining a joint score in global models is problematic when the target scales are different. To address such problems, single target (i.e. local) learning strategies are commonly employed. Here we propose alternative tree-based learning strategies to handle the issue with target scaling in global models, and to identify the learning order for chaining operations in local models. In the first proposal, the problems with target scaling are resolved using alternative splitting strategies which consider the learning tasks in a multi-objective optimization framework. The second proposal deals with the problem of ordering in the chaining strategies. We introduce an alternative estimation strategy, minimum error chain policy, that gradually expands the input space using the estimations that approximate to true characteristics of outputs, namely out-of-bag estimations in tree-based ensemble framework. Our experiments on benchmark datasets illustrate the success of the proposed multitask extension of trees compared to the decision trees with de facto design especially for datasets with large number of targets. In line with that, minimum error chain policy improves the performance of the state-of-the-art chaining policies. [ABSTRACT FROM AUTHOR]