Search

The mathematician and bio‐scientist Sam Karlin is quoted stating that "The purpose of models is not to fit the data but to sharpen the question". In this paper, we describe a journey between questions, models and data analysis to reach specific goals. This journey is typical in industrial, engineering, biology and social science applications. It contrasts regulated clinical research where a statistical analysis plan is declared before data collection. We consider random forests, ridge regression, lasso and elastic nets. To make our point, we use a case study of 63 sensors collected in the testing of an electronic system. The paper lists a sequence of questions and how they were tackled by statistical analysis to meet the analysis goal. Eventually, we were able to provide a robust parsimonious and effective model for predicting the system condition using a subset of the 63 sensors. In handling this problem, we develop and apply several innovative methods and insights that can prove useful in other contexts. [ABSTRACT FROM AUTHOR]

Ensemble methods, such as Bagging, Boosting, or Random Forests, often enhance the prediction performance of single learners on both classification and regression tasks. In the context of regression, we propose a gradient boosting‐based algorithm incorporating a diversity term with the aim of constructing different learners that enrich the ensemble while achieving a trade‐off of some individual optimality for global enhancement. Verifying the hypotheses of Biau and Cadre's theorem (2021, Advances in contemporary statistics and econometrics—Festschrift in honour of Christine Thomas‐Agnan, Springer), we present a convergence result ensuring that the associated optimization strategy reaches the global optimum. In the experiments, we consider a variety of different base learners with increasing complexity: stumps, regression trees, Purely Random Forests, and Breiman's Random Forests. Finally, we consider simulated and benchmark datasets and a real‐world electricity demand dataset to show, by means of numerical experiments, the suitability of our procedure by examining the behavior not only of the final or the aggregated predictor but also of the whole generated sequence. [ABSTRACT FROM AUTHOR]

The understanding of many physical and engineering problems involves running complex computational models. Such models take as input a high number of numerical and physical explanatory variables. The information on these underlying input parameters is often limited or uncertain. It is therefore important, based on the relationships between the input variables and the output, to identify and prioritize the most influential inputs. One may use global sensitivity analysis (GSA) methods which aim at ranking input random variables according to their importance in the output uncertainty, or even quantify the global influence of a particular input on the output. Using sensitivity metrics to ignore less important parameters is a form of dimension reduction in the model's input parameter space. This suggests the use of meta-modeling as a quantitative approach for nonparametric GSA, where the original input/output relation is first approximated using various statistical regression techniques. Subsequently, the main goal of our work is to provide a comprehensive review paper in the domain of sensitivity analysis focusing on some interesting connections between random forests and GSA. The idea is to use a random forests methodology as an efficient non-parametric approach for building meta-models that allow an efficient sensitivity analysis. Apart its easy applicability to regression problems, the random forests approach presents further strong advantages by its ability to implicitly deal with correlation and high dimensional data, to handle interactions between variables and to identify informative inputs using a permutation based RF variable importance index which is easy and fast to compute. We further review an adequate set of tools for quantifying variable importance which are then exploited to reduce the model's dimension enabling otherwise infeasible sensibility analysis studies. Numerical results from several simulations and a data exploration on a real dataset are presented to illustrate the effectiveness of such an approach. • Global Sensitivity Analysis ranks inputs according to their importance on output. • Random Forests is an efficient non-parametric approach for building meta-models. • Random forests variable importance measure is used to define sensitivity measures. • Provide a comprehensive review paper in sensitivity analysis using random forests. • Focus on connections between random forests and Global Sensitivity Analysis. [ABSTRACT FROM AUTHOR]