Search

Postsurgical hospital acquired infections (HAIs) are viewed as a quality benchmark in healthcare due to their association with morbidity, mortality and high cost. The prediction of HAIs allows for implementing prevention strategies at early stages to reduce postsurgical complications. In the United States, the National Surgical Quality Improvement Program (NSQIP) maintains a registry considered a “gold-standard” for HAIs outcome reporting, but it relies heavily on costly manual chart review and therefore only includes a small percent of surgery cases from each participating site. Most HAI prediction models rely on a wide range of weak risk factors, which are combined into models with many parameters and require larger sample sizes than available from NSQIP at a single health system. In this study, we propose an alternative approach to develop a robust prediction model, using the few NSQIP cases efficiently. Rather than training the HAIs prediction models directly on the small number of NSQIP patients, we leverage a simple detection model which detects HAIs after the fact on postoperative data and use this detection model to label a large non-NSQIP perioperative dataset on which prediction models are constructed. Detection models rely on strong signals requiring fewer samples to learn. We evaluate this approach in a single academic health system with 115,202 surgeries (10,354 in NSQIP). The prediction models were evaluated on the NSQIP “gold-standard” labels. While organ-space surgical site infection showed comparable performance, the proposed model demonstrated better performance for prediction of superficial surgical site infection, sepsis or septic shock, pneumonia, and urinary tract infection.

The adoption of predictive modeling for clinical decision support is accelerating in healthcare, however, the need for large sample sizes puts smaller health systems at a disadvantage. Small health systems have insufficient positive cases to build models are left with three choices. First, they can obtain already trained models, which are often too generic. Second, they can participate in research networks, building a model through a network-wide data set. Since small hospitals can only contribute small amounts of data influencing the resulting shared model minimally, this approach yields only minimal specialization. The third option is transfer learning, where a model previously trained on a large population is refined to the specific population, which carries the danger of over-specializing to the idiosyncrasies of the small data set. In this paper, we present a novel paradigm, consensus modeling, that allows a small health system to collaborate with a larger system to build a model specific to the smaller system without sharing any data instances. The method is similar to transfer learning in that it refines models from the larger system to be specific to the small system, but through iterative refinement, the larger system alleviates the risk of over-specializing to the small system. We evaluated the approach on predicting postoperative complications at two health systems with 9,044 and 38,545 patients. The model obtained from the proposed consensus modeling paradigm achieved a predictive performance on the small system that is as good as the transfer learning approach (AUC 0.71 vs 0.71) but significantly outperformed the transfer learning approach on the large dataset (AUC 0.80 vs 0.65) suggesting significantly reduced over-specializing.