P1-189 Net benefit fraction as a modified population attributable fraction to evaluate the clinical performance of a risk function

Introduction Net benefit (NB) has been suggested to verify the clinical usefulness of a model beyond its discrimination and calibration but it has a complex meaning in prediction models. Here we define net benefit fraction (NBF) using Population Attributable Fraction (PAF) index. Methods According to Vickers and Elkin (2006), in models that predict diseases to treat high risk individuals, NB is defined as true positive (TP) rate minus weighted false positive (wFP) rate; the weight is the odds at the threshold probability for treatment (Pt/[1-Pt]). Dividing NB by its maximum, incidence of disease, gives the portion of incidence prevented by the treatment. A shortcoming of NB is that it assumes the treatment reduces the incidence in TPs to zero. On the other hand, PAF is defined as reduction of the risk to below the threshold, that is, to that of low risk population. However, PAF does not take into account the false positives. Therefore, we suggest NBF calculated as PAF minus wFP rate, the latter divided by incidence. We applied the method to calculate the clinical performance of Framingham risk function at routine threshold probability of 20% to predict cardiovascular diseases in a population based cohort of 6224 Iranians aged 30–74 years with 10-year follow-up. Results dividing NB by incidence resulted 17% and PAF shows 43% decrease in incidence, but NBF shows just 8% advantage for treatment according to the model. Conclusion NBF seems to be a challengeable issue in policy making using risk functions.