Bootstrap Assessment of the Stability of Multivariable Models

Assessing the instability of a multivariable model is important but is rarely done in practice. Model instability occurs when selected predictors—and for multivariable fractional polynomial modeling, selected functions of continuous predictors—are sensitive to small changes in the data. Bootstrap analysis is a useful technique for investigating variations among selected models in samples drawn at random with replacement. Such samples mimic datasets that are structurally similar to that under study and that could plausibly have arisen instead. The bootstrap inclusion fraction of a candidate variable usefully indicates the importance of the variable. We describe Stata tools for stability analysis in the context of the mfpcommand for multivariable model building. We offer practical guidance and illustrate the application of the tools to a study in prostate cancer.