Challenges in comparing risk-adjusted bypass surgery mortality results: results from the Cooperative Cardiovascular Project

We sought to evaluate the predictive accuracy of four bypass surgery mortality clinical risk models and to examine the extent to which hospitals' risk-adjusted surgical outcomes vary depending on which risk-adjustment method is applied.Cardiovascular "report cards" often compare risk-adjusted surgical outcomes; however, it is unclear to what extent the risk-adjustment process itself may affect these metrics.As part of the Cooperative Cardiovascular Project's Pilot Revascularization Study, we compared the predictive accuracy of four bypass clinical risk models among 3,654 Medicare patients undergoing surgery at 28 hospitals in Alabama and Iowa. We also compared the agreement in hospital-level risk-adjusted bypass outcome performance ratings depending on which of the four risk models was applied.Although the four risk models had similar discriminatory abilities (C-index, 0.71 to 0.74), certain models tended to overpredict mortality in higher-risk patients. There was high correlation between a hospital's risk-adjusted mortality rates regardless of which of the four models was used (correlation between risk-adjusted rating, 0.93 to 0.97). In contrast, there was limited agreement in which hospitals were identified as "performance outliers" depending on which risk-adjustment model was used and how outlier status was defined.A hospital's risk-adjusted bypass surgery mortality rating, relative to its peers, was consistent regardless of the risk-adjustment model applied, supporting their use as a means of provider performance feedback. Designation of performance outliers, however, can vary significantly depending on the benchmark and methods used for this determination.