Search

In high-dimensional prediction settings, it remains challenging to reliably estimate the test performance. To address this challenge, a novel performance estimation framework is presented. This framework, called Learn2Evaluate, is based on learning curves by fitting a smooth monotone curve depicting test performance as a function of the sample size. Learn2Evaluate has several advantages compared to commonly applied performance estimation methodologies. Firstly, a learning curve offers a graphical overview of a learner. This overview assists in assessing the potential benefit of adding training samples and it provides a more complete comparison between learners than performance estimates at a fixed subsample size. Secondly, a learning curve facilitates in estimating the performance at the total sample size rather than a subsample size. Thirdly, Learn2Evaluate allows the computation of a theoretically justified and useful lower confidence bound. Furthermore, this bound may be tightened by performing a bias correction. The benefits of Learn2Evaluate are illustrated by a simulation study and applications to omics data. [ABSTRACT FROM AUTHOR]

Abstract: Sovereign default models that differ in their treatment of unobservable country, regional and time heterogeneities are systematically compared. The analysis is based on annual data over the 1983–2002 period for 96 developing economies. Inference-based criteria and parameter plausibility overwhelmingly favour more complex models that allow the link between the probability response and the fundamentals to vary over time and across countries. However, out-of-sample forecast evaluation using several loss functions and equal-predictive-ability tests suggests that simplicity beats complexity. Parsimonious pooled logit models produce the most accurate sovereign default forecasts and outperform the naive benchmarks. [Copyright &y& Elsevier]