Improving risk prediction model quality in the critically ill: data linkage study

Background: A previous National Institute for Health and Care Research study [Harrison DA, Ferrando-Vivas P, Shahin J, Rowan KM. Ensuring comparisons of health-care providers are fair: development and validation of risk prediction models for critically ill patients. Health Serv Deliv Res 2015;3(41)] identified the need for more research to understand risk factors and consequences of critical care and subsequent outcomes. Objectives: First, to improve risk models for adult general critical care by developing models for mortality at fixed time points and time-to-event outcomes, end-stage renal disease, type 2 diabetes, health-care utilisation and costs. Second, to improve risk models for cardiothoracic critical care by enhancing risk factor data and developing models for longer-term mortality. Third, to improve risk models for in-hospital cardiac arrest by enhancing risk factor data and developing models for longer-term mortality and critical care utilisation. Design: Risk modelling study linking existing data. Setting: NHS adult critical care units and acute hospitals in England. Participants: Patients admitted to an adult critical care unit or experiencing an in-hospital cardiac arrest. Interventions: None. Main outcome measures: Mortality at hospital discharge, 30 days, 90 days and 1 year following critical care unit admission; mortality at 1 year following discharge from acute hospital; new diagnosis of end-stage renal disease or type 2 diabetes; hospital resource use and costs; return of spontaneous circulation sustained for > 20 minutes; survival to hospital discharge and 1 year; and length of stay in critical care following in-hospital cardiac arrest. Data sources: Case Mix Programme, National Cardiac Arrest Audit, UK Renal Registry, National Diabetes Audit, National Adult Cardiac Surgery Audit, Hospital Episode Statistics and Office for National Statistics. Results: Data were linked for 965,576 critical care admissions between 1 April 2009 and 31 March 2016, and 83,939 in-hospital cardiac arrests between 1 April 2011 and 31 March 2016. For admissions to adult critical care units, models for 30-day mortality had similar predictors and performance to those for hospital mortality and did not reduce heterogeneity. Models for longer-term outcomes reflected increasing importance of chronic over acute predictors. New models for end-stage renal disease and diabetes will allow benchmarking of critical care units against these important outcomes and identification of patients requiring enhanced follow-up. The strongest predictors of health-care costs were prior hospitalisation, prior dependency and chronic conditions. Adding pre- and intra-operative risk factors to models for cardiothoracic critical care gave little improvement in performance. Adding comorbidities to models for in-hospital cardiac arrest provided modest improvements but were of greater importance for longer-term outcomes. Limitations: Delays in obtaining linked data resulted in the data used being 5 years old at the point of publication: models will already require recalibration. Conclusions: Data linkage provided enhancements to the risk models underpinning national clinical audits in the form of additional predictors and novel outcomes measures. The new models developed in this report may assist in providing objective estimates of potential outcomes to patients and their families. Future work: (1) Develop and test care pathways for recovery following critical illness targeted at those with the greatest need; (2) explore other relevant data sources for longer-term outcomes; (3) widen data linkage for resource use and costs to primary care, outpatient and emergency department data. Study registration: This study is registered as NCT02454257. Funding details: This project was funded by the National Institute for Health and Care Research (NIHR) Health and Social Care Delivery Research programme and will be published in full in Health and Social Care Delivery Research; Vol. 10, No. 39. See the NIHR Journals Library website for further project information.