Search

ObjectiveAfter deploying a clinical prediction model, subsequently collected data can be used to fine-tune its predictions and adapt to temporal shifts. Because model updating carries risks of over-updating/fitting, we study online methods with performance guarantees.Materials and methodsWe introduce 2 procedures for continual recalibration or revision of an underlying prediction model: Bayesian logistic regression (BLR) and a Markov variant that explicitly models distribution shifts (MarBLR). We perform empirical evaluation via simulations and a real-world study predicting Chronic Obstructive Pulmonary Disease (COPD) risk. We derive "Type I and II" regret bounds, which guarantee the procedures are noninferior to a static model and competitive with an oracle logistic reviser in terms of the average loss.ResultsBoth procedures consistently outperformed the static model and other online logistic revision methods. In simulations, the average estimated calibration index (aECI) of the original model was 0.828 (95%CI, 0.818-0.938). Online recalibration using BLR and MarBLR improved the aECI towards the ideal value of zero, attaining 0.265 (95%CI, 0.230-0.300) and 0.241 (95%CI, 0.216-0.266), respectively. When performing more extensive logistic model revisions, BLR and MarBLR increased the average area under the receiver-operating characteristic curve (aAUC) from 0.767 (95%CI, 0.765-0.769) to 0.800 (95%CI, 0.798-0.802) and 0.799 (95%CI, 0.797-0.801), respectively, in stationary settings and protected against substantial model decay. In the COPD study, BLR and MarBLR dynamically combined the original model with a continually refitted gradient boosted tree to achieve aAUCs of 0.924 (95%CI, 0.913-0.935) and 0.925 (95%CI, 0.914-0.935), compared to the static model's aAUC of 0.904 (95%CI, 0.892-0.916).DiscussionDespite its simplicity, BLR is highly competitive with MarBLR. MarBLR outperforms BLR when its prior better reflects the data.ConclusionsBLR and MarBLR can improve the transportability of clinical prediction models and maintain their performance over time.

BackgroundAlthough community-acquired pneumonia (CAP) is one of the most common infections in children, no tools exist to risk stratify children with suspected CAP. We developed and validated a prediction model to risk stratify and inform hospitalization decisions in children with suspected CAP.MethodsWe performed a prospective cohort study of children aged 3 months to 18 years with suspected CAP in a pediatric emergency department. Primary outcome was disease severity, defined as mild (discharge home or hospitalization for 24 hours), or severe (intensive care unit stay for >24 hours, septic shock, vasoactive agents, positive-pressure ventilation, chest drainage, extracorporeal membrane oxygenation, or death). Ordinal logistic regression and bootstrapped backwards selection were used to derive and internally validate our model.ResultsOf 1128 children, 371 (32.9%) developed moderate disease and 48 (4.3%) severe disease. Severity models demonstrated excellent discrimination (optimism-corrected c-indices of 0.81) and outstanding calibration. Severity predictors in the final model included respiratory rate, systolic blood pressure, oxygenation, retractions, capillary refill, atelectasis or pneumonia on chest radiograph, and pleural effusion.ConclusionsWe derived and internally validated a score that accurately predicts disease severity in children with suspected CAP. Once externally validated, this score has potential to facilitate management decisions by providing individualized risk estimates that can be used in conjunction with clinical judgment to improve the care of children with suspected CAP.

Rationale & objectiveThe Kidney Failure Risk Equation (KFRE) is a simple widely validated prediction model using age, sex, estimated glomerular filtration rate, and urinary albumin-creatinine ratio to predict the risk for end-stage kidney disease. Data are limited for its applicability to kidney transplant recipients.Study designValidation study of the KFRE as a post hoc analysis of the Folic Acid for Vascular Outcomes Reduction in Transplantation (FAVORIT) Trial.Setting & participantsAdult kidney transplant recipients with functioning kidney allografts at least 6 months posttransplantation from 30 centers in the United States, Canada, and Brazil. Participants with estimated glomerular filtration rates

Rationale & objectiveThe Kidney Failure Risk Equation (KFRE) is a simple widely validated prediction model using age, sex, estimated glomerular filtration rate, and urinary albumin-creatinine ratio to predict the risk for end-stage kidney disease. Data are limited for its applicability to kidney transplant recipients.Study designValidation study of the KFRE as a post hoc analysis of the Folic Acid for Vascular Outcomes Reduction in Transplantation (FAVORIT) Trial.Setting & participantsAdult kidney transplant recipients with functioning kidney allografts at least 6 months posttransplantation from 30 centers in the United States, Canada, and Brazil. Participants with estimated glomerular filtration rates