Abstract A127: Comparison of discriminatory power and accuracy of three lung cancer risk models

Background: Since lung cancer only occurs in a small fraction of long-term smokers, there is a need to develop risk prediction models to identify high-risk subgroups. Three lung cancer models, constructed using clinical and epidemiological variables, predicted absolute risk of lung cancer: one based on a cohort of patients recruited for the CARET study, and two constructed from case-control studies conducted in Houston, Texas and Liverpool, England. Given their potential application to primary chemo-prevention strategies and screening trials, it is important to compare the accuracy of these three models in an independent population. Methods: We used data for 3197 lung cancer patients and 1703 cancer-free controls recruited to an ongoing case-control study of lung cancer at Harvard School of Public Health and Massachusetts General Hospital (Boston, MA). We estimated 5-year lung cancer risk for each risk model and compared the discriminatory power, as measured by the area under the the receiver-operator characteristic curve, accuracy, as measured by the positive predictive value and negative predictive value, and clinical utility of these models, as measured with scaled rectangles. Results: Overall, the discriminatory power for the Liverpool Lung Project (LLP) (AUC = 0.69, 95% CI = 0.67–0.71) and Spitz models (AUC = 0.69, 95%CI = 0.66–0.71) were comparable, while the Bach model had significantly lower power (AUC =0.66, 95% CI = 0.64–0.69; P=0.02). Positive predictive values were highest with the Spitz model (0.882) compared to 0.809 for the Bach model and 0.759 for the LLP model. In contrast, the negative predictive values were highest for the LLP model (0.560) compared to 0.450 for the Spitz model and 0.447 for the Bach. The Spitz and Bach models had lower sensitivity but higher specificity compared to the LLP model. For instance, 26.6% of all lung cancer cases have a five-year absolute risk of lung cancer ≥ 2.5% for the Spitz model compared to 66.7% of all cases for the LLP model. However, only 5.6% of all healthy controls have a five-year absolute risk of lung cancer ≥ 2.5% with the Spitz model compared to 33.4% of all controls for the LLP model. Conclusion: We observed modest differences in discriminatory among the three lung cancer risk models. The level of the discriminatory powers of these three lung cancer risk models was moderate at best, which highlights the difficulty in developing effective risk models. There is considerable room for improvement in model performance by incorporating additional risk factors, such as genetic risk factors, to increase discriminatory power and accuracy, while still maintaining clinical utility. Citation Information: Cancer Prev Res 2010;3(1 Suppl):A127.