Search

Introduction: We developed a multimarker blood test result interpretation tool for the clinical dementia practice, including phosphorylated (P-)tau181, amyloid-beta (Abeta)42/40, glial fibrillary acidic protein (GFAP), and neurofilament light (NfL)., Methods: We measured the plasma biomarkers with Simoa (n = 1199), applied LASSO regression for biomarker selection and receiver operating characteristics (ROC) analyses to determine diagnostic accuracy. We validated our findings in two independent cohorts and constructed a visualization approach., Results: P-tau181, GFAP, and NfL were selected. This combination had area under the curve (AUC) = 83% to identify amyloid positivity in pre-dementia stages, AUC = 87%-89% to differentiate Alzheimer's or controls from frontotemporal dementia, AUC = 74%-76% to differentiate Alzheimer's or controls from dementia with Lewy bodies. Highly reproducible AUCs were obtained in independent cohorts. The resulting visualization tool includes UpSet plots to visualize the stand-alone biomarker results and density plots to visualize the biomarker results combined., Discussion: Our multimarker blood test interpretation tool is ready for testing in real-world clinical dementia settings., Highlights: We developed a multimarker blood test interpretation tool for clinical dementia practice. Our interpretation tool includes plasma biomarkers P-tau, GFAP, and NfL. Our tool is particularly useful for Alzheimer's and frontotemporal dementia diagnosis., (© 2024 The Author(s). Alzheimer's & Dementia published by Wiley Periodicals LLC on behalf of Alzheimer's Association.)

Background: Prediction models can identify fall-prone individuals. Prediction models can be based on either data from research cohorts (cohort-based) or routinely collected data (RCD-based). We review and compare cohort-based and RCD-based studies describing the development and/or validation of fall prediction models for community-dwelling older adults., Methods: Medline and Embase were searched via Ovid until January 2023. We included studies describing the development or validation of multivariable prediction models of falls in older adults (60+). Both risk of bias and reporting quality were assessed using the PROBAST and TRIPOD, respectively., Results: We included and reviewed 28 relevant studies, describing 30 prediction models (23 cohort-based and 7 RCD-based), and external validation of two existing models (one cohort-based and one RCD-based). The median sample sizes for cohort-based and RCD-based studies were 1365 [interquartile range (IQR) 426-2766] versus 90 441 (IQR 56 442-128 157), and the ranges of fall rates were 5.4% to 60.4% versus 1.6% to 13.1%, respectively. Discrimination performance was comparable between cohort-based and RCD-based models, with the respective area under the receiver operating characteristic curves ranging from 0.65 to 0.88 versus 0.71 to 0.81. The median number of predictors in cohort-based final models was 6 (IQR 5-11); for RCD-based models, it was 16 (IQR 11-26). All but one cohort-based model had high bias risks, primarily due to deficiencies in statistical analysis and outcome determination., Conclusions: Cohort-based models to predict falls in older adults in the community are plentiful. RCD-based models are yet in their infancy but provide comparable predictive performance with no additional data collection efforts. Future studies should focus on methodological and reporting quality., (© The Author(s) 2024. Published by Oxford University Press on behalf of the British Geriatrics Society.)

Background: Falls involve dynamic risk factors that change over time, but most studies on fall-risk factors are cross-sectional and do not capture this temporal aspect. The longitudinal clinical notes within electronic health records (EHR) provide an opportunity to analyse fall risk factor trajectories through Natural Language Processing techniques, specifically dynamic topic modelling (DTM). This study aims to uncover fall-related topics for new fallers and track their evolving trends leading up to falls., Methods: This case-cohort study utilised primary care EHR data covering information on older adults between 2016 and 2019. Cases were individuals who fell in 2019 but had no falls in the preceding three years (2016-18). The control group was randomly sampled individuals, with similar size to the cases group, who did not endure falls during the whole study follow-up period. We applied DTM on the clinical notes collected between 2016 and 2018. We compared the trend lines of the case and control groups using the slopes, which indicate direction and steepness of the change over time., Results: A total of 2,384 fallers (cases) and an equal number of controls were included. We identified 25 topics that showed significant differences in trends between the case and control groups. Topics such as medications, renal care, family caregivers, hospital admission/discharge and referral/streamlining diagnostic pathways exhibited a consistent increase in steepness over time within the cases group before the occurrence of falls., Conclusions: Early recognition of health conditions demanding care is crucial for applying proactive and comprehensive multifactorial assessments that address underlying causes, ultimately reducing falls and fall-related injuries., (© The Author(s) 2024. Published by Oxford University Press on behalf of the British Geriatrics Society.)

Background: Objective prognostic information is essential for good clinical decision making. In case of unknown diseases, scarcity of evidence and limited tacit knowledge prevent obtaining this information. Prediction models can be useful, but need to be not only evaluated on how well they predict, but also how stable these models are under fast changing circumstances with respect to development of the disease and the corresponding clinical response. This study aims to provide interpretable and actionable insights, particularly for clinicians. We developed and evaluated two regression tree predictive models for in-hospital mortality of COVID-19 patient at admission and 24 hours (24 h) after admission, using a national registry. We performed a retrospective analysis of observational routinely collected data., Methods: Two regression tree models were developed for admission and 24 h after admission. The complexity of the trees was managed via cross validation to prevent overfitting. The predictive ability of the model was assessed via bootstrapping using the Area under the Receiver-Operating-Characteristic curve, Brier score and calibration curves. The tree models were assessed on the stability of their probabilities and predictive ability, on the selected variables, and compared to a full-fledged logistic regression model that uses variable selection and variable transformations using splines. Participants included COVID-19 patients from all ICUs participating in the Dutch National Intensive Care Evaluation (NICE) registry, who were admitted at the ICU between February 27, 2020, and November 23, 2021. From the NICE registry, we included concerned demographic data, minimum and maximum values of physiological data in the first 24 h of ICU admission and diagnoses (reason for admission as well as comorbidities) for model development. The main outcome measure was in-hospital mortality. We additionally analysed the Length-of-Stay (LoS) per patient subgroup per survival status., Results: A total of 13,369 confirmed COVID-19 patients from 70 ICUs were included (with mortality rate of 28%). The optimism-corrected AUROC of the admission tree (with seven paths) was 0.72 (95% CI: 0.71-0.74) and of the 24 h tree (with 11 paths) was 0.74 (0.74-0.77). Both regression trees yielded good calibration and variable selection for both trees was stable. Patient subgroups comprising the tree paths had comparable survival probabilities as the full-fledged logistic regression model, survival probabilities were stable over six COVID-19 surges, and subgroups were shown to have added predictive value over the individual patient variables., Conclusions: We developed and evaluated regression trees, which operate at par with a carefully crafted logistic regression model. The trees consist of homogenous subgroups of patients that are described by simple interpretable constraints on patient characteristics thereby facilitating shared decision-making., (© 2023. The Author(s).)

Background: Heart failure (HF) is a common cardiac syndrome with a high disease burden and poor prognosis in our aging populations. Understanding the characteristics of patients with newly diagnosed HF is essential for improving care and outcomes. The AMSTERDAM-HF study is aimed to examine the population characteristics of patients with incident HF., Methods: We performed a retrospective dynamic cohort study in the Amsterdam general practice network consisting of 904,557 individuals. Incidence HF rates, geographical demographics, patient characteristics, risk factors, symptoms prior to HF diagnosis, and prognosis were reported., Results: The study identified 10,067 new cases of HF over 6,816,099 person-years. The median age of patients was 77 years (25th-75th percentile: 66-85), and 48% were male. The incidence rate of HF was 213.44 per 100,000 patient-years, and was higher in male versus female patients (incidence rate ratio: 1.08, 95%-CI:1.04-1.13). Hypertension (men 46.3% and women 55.8%), coronary artery disease (men 36% and women 25%) and diabetes mellitus (men 30.5% and women 26.8%) were the most common risk factors. Dyspnoea and oedema were key reported symptoms prior to HF diagnosis. Survival rates at 10-year follow-up were poor, particularly in men (36.4%) compared to women (39.7%). Incidence rates, comorbidity burden and prognosis were worse in city districts with high ethnic diversity and low socio-economic position., Conclusion: Our study provides insights into incident HF in a contemporary Western European, multi-ethnic, urban population. It highlights notable sex, age, and geographical differences in incidence rates, risk factors, symptoms and prognosis., Competing Interests: Declaration of Competing Interest The authors report no relationships that could be construed as a conflict of interest., (Copyright © 2023 The Author(s). Published by Elsevier B.V. All rights reserved.)

Objective: To review and critically appraise published and preprint reports of prognostic models of in-hospital mortality of patients in the intensive-care unit (ICU) based on neural representations (embeddings) of clinical notes., Methods: PubMed and arXiv were searched up to August 1, 2022. At least two reviewers independently selected the studies that developed a prognostic model of in-hospital mortality of intensive-care patients using free-text represented as embeddings and extracted data using the CHARMS checklist. Risk of bias was assessed using PROBAST. Reporting on the model was assessed with the TRIPOD guideline. To assess the machine learning components that were used in the models, we present a new descriptive framework based on different techniques to represent text and provide predictions from text. The study protocol was registered in the PROSPERO database (CRD42022354602)., Results: Eighteen studies out of 2,825 were included. All studies used the publicly-available MIMIC dataset. Context-independent word embeddings are widely used. Model discrimination was provided by all studies (AUROC 0.75-0.96), but measures of calibration were scarce. Seven studies used both structural clinical variables and notes. Model discrimination improved when adding clinical notes to variables. None of the models was externally validated and often a simple train/test split was used for internal validation. Our critical appraisal demonstrated a high risk of bias in all studies and concerns regarding their applicability in clinical practice., Conclusion: All studies used a neural architecture for prediction and were based on one publicly available dataset. Clinical notes were reported to improve predictive performance when used in addition to only clinical variables. Most studies had methodological, reporting, and applicability issues. We recommend reporting both model discrimination and calibration, using additional data sources, and using more robust evaluation strategies, including prospective and external validation. Finally, sharing data and code is encouraged to improve study reproducibility., Competing Interests: Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2023 The Authors. Published by Elsevier Inc. All rights reserved.)

Objectives: Heart failure (HF) is a prevalent syndrome with considerable disease burden, healthcare utilization and costs. Timely diagnosis is essential to improve outcomes. This study aimed to compare the diagnostic performance of B-type natriuretic peptide (BNP) and N-terminal proBNP (NT-proBNP) in detecting HF in primary care. Our second aim was to explore if personalized thresholds (using age, sex, or other readily available parameters) would further improve diagnostic accuracy over universal thresholds., Methods: A retrospective study was performed among patients without prior HF who underwent natriuretic peptide (NP) testing in the Amsterdam General Practice Network between January 2011 and December 2021. HF incidence was based on registration out to 90 days after NP testing. Diagnostic accuracy was evaluated with AUROC, sensitivity and specificity based on guideline-recommended thresholds (125 ng/L for NT-proBNP and 35 ng/L for BNP). We used inverse probability of treatment weighting to adjust for confounding., Results: A total of 15,234 patients underwent NP testing, 6,870 with BNP (4.5 % had HF), and 8,364 with NT-proBNP (5.7 % had HF). NT-proBNP was more accurate than BNP, with an AUROC of 89.9 % (95 % CI: 88.4-91.2) vs. 85.9 % (95 % CI 83.5-88.2), with higher sensitivity (95.3 vs. 89.7 %) and specificity (59.1 vs. 58.0 %). Differentiating NP cut-off by clinical variables modestly improved diagnostic accuracy for BNP and NT-proBNP compared with a universal threshold., Conclusions: NT-proBNP outperforms BNP for detecting HF in primary care. Personalized instead of universal diagnostic thresholds led to modest improvement., (© 2023 the author(s), published by De Gruyter, Berlin/Boston.)

Background: - Subgroup discovery (SGD) is the automated splitting of the data into complex subgroups. Various SGD methods have been applied to the medical domain, but none have been extensively evaluated. We assess the numerical and clinical quality of SGD methods., Method: - We applied the improved Subgroup Set Discovery (SSD++), Patient Rule Induction Method (PRIM) and APRIORI - Subgroup Discovery (APRIORI-SD) algorithms to obtain patient subgroups on observational data of 14,548 COVID-19 patients admitted to 73 Dutch intensive care units. Hospital mortality was the clinical outcome. Numerical significance of the subgroups was assessed with information-theoretic measures. Clinical significance of the subgroups was assessed by comparing variable importance on population and subgroup levels and by expert evaluation., Results: - The tested algorithms varied widely in the total number of discovered subgroups (5-62), the number of selected variables, and the predictive value of the subgroups. Qualitative assessment showed that the found subgroups make clinical sense. SSD++ found most subgroups (n = 62), which added predictive value and generally showed high potential for clinical use. APRIORI-SD and PRIM found fewer subgroups (n = 5 and 6), which did not add predictive value and were clinically less relevant., Conclusion: - Automated SGD methods find clinical subgroups that are relevant when assessed quantitatively (yield added predictive value) and qualitatively (intensivists consider the subgroups significant). Different methods yield different subgroups with varying degrees of predictive performance and clinical quality. External validation is needed to generalize the results to other populations and future research should explore which algorithm performs best in other settings., Competing Interests: Declaration of competing interest The authors declare that they have no conflict of interests., (Copyright © 2023 The Author(s). Published by Elsevier Ltd.. All rights reserved.)

Background: Nephrotoxic drugs frequently cause acute kidney injury (AKI) in adult intensive care unit (ICU) patients. However, there is a lack of large pharmaco-epidemiological studies investigating the associations between drugs and AKI. Importantly, AKI risk factors may also be indications or contraindications for drugs and thereby confound the associations. Here, we aimed to estimate the associations between commonly administered (potentially) nephrotoxic drug groups and AKI in adult ICU patients whilst adjusting for confounding., Methods: In this multicenter retrospective observational study, we included adult ICU admissions to 13 Dutch ICUs. We measured exposure to 44 predefined (potentially) nephrotoxic drug groups. The outcome was AKI during ICU admission. The association between each drug group and AKI was estimated using etiological cause-specific Cox proportional hazard models and adjusted for confounding. To facilitate an (independent) informed assessment of residual confounding, we manually identified drug group-specific confounders using a large drug knowledge database and existing literature., Results: We included 92 616 ICU admissions, of which 13 492 developed AKI (15%). We found 14 drug groups to be associated with a higher hazard of AKI after adjustment for confounding. These groups included established (e.g. aminoglycosides), less well established (e.g. opioids) and controversial (e.g. sympathomimetics with α- and β-effect) drugs., Conclusions: The results confirm existing insights and provide new ones regarding drug associated AKI in adult ICU patients. These insights warrant caution and extra monitoring when prescribing nephrotoxic drugs in the ICU and indicate which drug groups require further investigation., Competing Interests: The authors declare that they have no conflict of interest., (© The Author(s) 2023. Published by Oxford University Press on behalf of the ERA.)

We aimed to assess the added predictive performance that free-text Dutch consultation notes provide in detecting colorectal cancer in primary care, in comparison to currently used models. We developed, evaluated and compared three prediction models for colorectal cancer (CRC) in a large primary care database with 60,641 patients. The prediction model with both known predictive features and free-text data (with TabTxt AUROC: 0.823) performs statistically significantly better (p < 0.05) than the other two models with only tabular (as used nowadays) and text data, respectively (AUROC Tab: 0.767; Txt: 0.797). The specificity of the two models that use demographics and known CRC features (with specificity Tab: 0.321; TabTxt: 0.335) are higher than that of the model with only free-text (specificity Txt: 0.234). The Txt and, to a lesser degree, TabTxt model are well calibrated, while the Tab model shows slight underprediction at both tails. As expected with an outcome prevalence below 0.01, all models show much uncalibrated predictions in the extreme upper tail (top 1%). Free-text consultation notes show promising results to improve the predictive performance over established prediction models that only use structured features. Clinical future implications for our CRC use case include that such improvement may help lowering the number of referrals for suspected CRC to medical specialists., (© 2023. The Author(s).)

Objective: We conducted a systematic review to characterize and critically appraise developed prediction models based on structured electronic health record (EHR) data for adverse drug event (ADE) diagnosis and prognosis in adult hospitalized patients., Materials and Methods: We searched the Embase and Medline databases (from January 1, 1999, to July 4, 2022) for articles utilizing structured EHR data to develop ADE prediction models for adult inpatients. For our systematic evidence synthesis and critical appraisal, we applied the Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS)., Results: Twenty-five articles were included. Studies often did not report crucial information such as patient characteristics or the method for handling missing data. In addition, studies frequently applied inappropriate methods, such as univariable screening for predictor selection. Furthermore, the majority of the studies utilized ADE labels that only described an adverse symptom while not assessing causality or utilizing a causal model. None of the models were externally validated., Conclusions: Several challenges should be addressed before the models can be widely implemented, including the adherence to reporting standards and the adoption of best practice methods for model development and validation. In addition, we propose a reorientation of the ADE prediction modeling domain to include causality as a fundamental challenge that needs to be addressed in future studies, either through acquiring ADE labels via formal causality assessments or the usage of adverse event labels in combination with causal prediction modeling., (© The Author(s) 2023. Published by Oxford University Press on behalf of the American Medical Informatics Association.)

Background: Falls in older people are common and morbid. Prediction models can help identifying individuals at higher fall risk. Electronic health records (EHR) offer an opportunity to develop automated prediction tools that may help to identify fall-prone individuals and lower clinical workload. However, existing models primarily utilise structured EHR data and neglect information in unstructured data. Using machine learning and natural language processing (NLP), we aimed to examine the predictive performance provided by unstructured clinical notes, and their incremental performance over structured data to predict falls., Methods: We used primary care EHR data of people aged 65 or over. We developed three logistic regression models using the least absolute shrinkage and selection operator: one using structured clinical variables (Baseline), one with topics extracted from unstructured clinical notes (Topic-based) and one by adding clinical variables to the extracted topics (Combi). Model performance was assessed in terms of discrimination using the area under the receiver operating characteristic curve (AUC), and calibration by calibration plots. We used 10-fold cross-validation to validate the approach., Results: Data of 35,357 individuals were analysed, of which 4,734 experienced falls. Our NLP topic modelling technique discovered 151 topics from the unstructured clinical notes. AUCs and 95% confidence intervals of the Baseline, Topic-based and Combi models were 0.709 (0.700-0.719), 0.685 (0.676-0.694) and 0.718 (0.708-0.727), respectively. All the models showed good calibration., Conclusions: Unstructured clinical notes are an additional viable data source to develop and improve prediction models for falls compared to traditional prediction models, but the clinical relevance remains limited., (© The Author(s) 2023. Published by Oxford University Press on behalf of the British Geriatrics Society. All rights reserved. For permissions, please email: journals.permissions@oup.com.)

Background: Timely diagnosis of heart failure (HF) is essential to optimize treatment opportunities that improve symptoms, quality of life, and survival. While most patients consult their general practitioner (GP) prior to HF, the early stages of HF may be difficult to identify. An integrated clinical support tool may aid in identifying patients at high risk of HF. We therefore constructed a prediction model using routine health care data., Methods: Our study involved a dynamic cohort of patients (≥35 years) who consulted their GP with either dyspnoea and/or peripheral oedema within the Amsterdam metropolitan area from 2011 to 2020. The outcome of interest was incident HF, verified by an expert panel. We developed a regularized, cause-specific multivariable proportional hazards model (TARGET-HF). The model was evaluated with bootstrapping on an isolated validation set and compared to an existing model developed with hospital insurance data as well as patient age as a sole predictor., Results: Data from 31,905 patients were included (40% male, median age 60 years) of whom 1,301 (4.1%) were diagnosed with HF over 124,676 person-years of follow-up. Data were allocated to a development (n = 25,524) and validation (n = 6,381) set. TARGET-HF attained a C-statistic of 0.853 (95% CI, 0.834 to 0.872) on the validation set, which proved to provide a better discrimination than C = 0.822 for age alone (95% CI, 0.801 to 0.842, P < 0.001) and C = 0.824 for the hospital-based model (95% CI, 0.802 to 0.843, P < 0.001)., Conclusion: The TARGET-HF model illustrates that routine consultation codes can be used to build a performant model to identify patients at risk for HF at the time of GP consultation., (© The Author(s) 2022. Published by Oxford University Press.)

Importance: Proton-pump inhibitors (PPIs) have been associated with the risk of colonization with drug-resistant bacteria; however, possible confounding by lifestyle-associated factors and disease severity casts doubt on this association, and whether the risk is dose dependent is not known., Objectives: To assess the association between PPI use and the risk of acquiring drug-resistant Enterobacterales and to examine interactions with possible microbiome-altering agents., Design, Setting, and Participants: This nested case-control study involved 2239 hospitalized adult (aged ≥18 years) patients identified from the microbiology laboratory database of Amsterdam University Medical Centers between December 31, 2018, and January 6, 2021. Patients in the case group had newly detected extended-spectrum β-lactamase (ESBL)- or carbapenemase-producing Enterobacterales (identified by clinical specimens). Risk-set sampling was used to assign patients with negative results for ESBL- and carbapenemase-producing Enterobacterales to the control group, who were then matched on a 5:1 ratio with patients in the case group by age and culture date. A second validation case-control study included matched pairs (1:1 ratio; 94 in each group) of patients who were prospectively enrolled., Exposures: Proton pump inhibitor use and clinical data at 30 days (primary exposure) and 90 days (secondary exposure) before the date of culture., Main Outcomes and Measures: Adjusted incidence rate ratios (aIRRs) of ESBL- or carbapenemase-producing Enterobacterales acquisition by PPI dose and time risk windows (30 days for the primary outcome and 90 days for the secondary outcome) were estimated using conditional logistic regression models., Results: Among 2239 hospitalized patients (51.1% male; mean [SD] age, 60.9 [16.7] years), 374 were in the case group (51.6% male; mean [SD] age, 61.1 [16.5] years) and 1865 were in the matched control group (51.0% male; mean [SD] age, 60.9 [16.7] years). The aIRR for PPI use overall was 1.48 (95% CI, 1.15-1.91) at 30 days. Sensitivity analyses and the analysis of the pair-matched study with prospectively enrolled patients (aIRR, 2.96, 95% CI, 1.14-7.74) yielded similar results; findings were consistent in subgroups and corroborated by a negative-control exposure analysis. No association with microbiome-disturbing agents was found; laxatives and antibiotics were independently associated with a more than 2-fold increase in the risk of acquisition (antibiotics: aIRR, 2.78 [95% CI, 2.14-3.59]; laxatives: aIRR, 2.26 [95% CI. 1.73-2.94])., Conclusions and Relevance: In this study, after careful control for confounding and sensitivity analyses, PPI use was associated with increases in the risk of acquiring ESBL- or carbapenemase-producing Enterobacterales among adult hospitalized patients. These findings emphasize the need for judicious use of PPIs.

To reduce adverse drug events (ADEs), hospitals need a system to support them in monitoring ADE occurrence routinely, rapidly, and at scale. Natural language processing (NLP), a computerized approach to analyze text data, has shown promising results for the purpose of ADE detection in the context of pharmacovigilance. However, a detailed qualitative assessment and critical appraisal of NLP methods for ADE detection in the context of ADE monitoring in hospitals is lacking. Therefore, we have conducted a scoping review to close this knowledge gap, and to provide directions for future research and practice. We included articles where NLP was applied to detect ADEs in clinical narratives within electronic health records of inpatients. Quantitative and qualitative data items relating to NLP methods were extracted and critically appraised. Out of 1,065 articles screened for eligibility, 29 articles met the inclusion criteria. Most frequent tasks included named entity recognition (n = 17; 58.6%) and relation extraction/classification (n = 15; 51.7%). Clinical involvement was reported in nine studies (31%). Multiple NLP modelling approaches seem suitable, with Long Short Term Memory and Conditional Random Field methods most commonly used. Although reported overall performance of the systems was high, it provides an inflated impression given a steep drop in performance when predicting the ADE entity or ADE relation class. When annotating corpora, treating an ADE as a relation between a drug and non-drug entity seems the best practice. Future research should focus on semi-automated methods to reduce the manual annotation effort, and examine implementation of the NLP methods in practice., Competing Interests: The authors declare that they have no conflict of interest., (Copyright: © 2023 Murphy et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.)

Objectives: Capturing frailty using a quick tool has proven to be challenging. We hypothesise that this is due to the complex interactions between frailty domains. We aimed to identify these interactions and assess whether adding interactions between domains improves mortality predictability., Methods: In this retrospective cohort study, we selected all patients aged 70 or older who were admitted to one Dutch hospital between April 2015 and April 2016. Patient characteristics, frailty screening (using VMS (Safety Management System), a screening tool used in Dutch hospital care), length of stay, and mortality within three months were retrospectively collected from electronic medical records. To identify predictive interactions between the frailty domains, we constructed a classification tree with mortality as the outcome using five variables: the four VMS-domains (delirium risk, fall risk, malnutrition, physical impairment) and their sum. To determine if any domain interactions were predictive for three-month mortality, we performed a multivariable logistic regression analysis., Results: We included 4,478 patients. (median age: 79 years; maximum age: 101 years; 44.8% male) The highest risk for three-month mortality included patients that were physically impaired and malnourished (23% (95%-CI 19.0-27.4%)). Subgroups had comparable three-month mortality risks based on different domains: malnutrition without physical impairment (15.2% (96%-CI 12.4-18.6%)) and physical impairment and delirium risk without malnutrition (16.3% (95%-CI 13.7-19.2%))., Discussion: We showed that taking interactions between domains into account improves the predictability of three-month mortality risk. Therefore, when screening for frailty, simply adding up domains with a cut-off score results in loss of valuable information., Competing Interests: Declaration of Competing Interest The authors have no personal or financial conflicts of interest., (Copyright © 2022. Published by Elsevier B.V.)

Purpose: To assess, validate and compare the predictive performance of models for in-hospital mortality of COVID-19 patients admitted to the intensive care unit (ICU) over two different waves of infections. Our models were built with high-granular Electronic Health Records (EHR) data versus less-granular registry data., Methods: Observational study of all COVID-19 patients admitted to 19 Dutch ICUs participating in both the national quality registry National Intensive Care Evaluation (NICE) and the EHR-based Dutch Data Warehouse (hereafter EHR). Multiple models were developed on data from the first 24 h of ICU admissions from February to June 2020 (first COVID-19 wave) and validated on prospective patients admitted to the same ICUs between July and December 2020 (second COVID-19 wave). We assessed model discrimination, calibration, and the degree of relatedness between development and validation population. Coefficients were used to identify relevant risk factors., Results: A total of 1533 patients from the EHR and 1563 from the registry were included. With high granular EHR data, the average AUROC was 0.69 (standard deviation of 0.05) for the internal validation, and the AUROC was 0.75 for the temporal validation. The registry model achieved an average AUROC of 0.76 (standard deviation of 0.05) in the internal validation and 0.77 in the temporal validation. In the EHR data, age, and respiratory-system related variables were the most important risk factors identified. In the NICE registry data, age and chronic respiratory insufficiency were the most important risk factors., Conclusion: In our study, prognostic models built on less-granular but readily-available registry data had similar performance to models built on high-granular EHR data and showed similar transportability to a prospective COVID-19 population. Future research is needed to verify whether this finding can be confirmed for upcoming waves., Competing Interests: Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2022 The Author(s). Published by Elsevier B.V. All rights reserved.)

Objective: Early identification of older people at risk of falling is the cornerstone of fall prevention. Many fall prediction tools exist but their external validity is lacking. External validation is a prerequisite before application in clinical practice. Models developed with electronic health record (EHR) data are especially challenging because of the uncontrolled nature of routinely collected data. We aimed to externally validate our previously developed and published prediction model for falls, using a large cohort of community-dwelling older people derived from primary care EHR data., Design: Retrospective analysis of a prospective cohort drawn from EHR data., Setting and Participants: Pseudonymized EHR data were collected from individuals aged ≥65 years, who were enlisted in any of the participating 59 general practices between 2015 and 2020 in the Netherlands., Methods: Ten predictors were defined and obtained using the same methods as in the development study. The outcome was 1-year fall and was obtained from free text. Both reproducibility and transportability were evaluated. Model performance was assessed in terms of discrimination using the area under the receiver operating characteristic curve (ROC-AUC), and in terms of calibration, using calibration-in-the-large, calibration slope and calibration plots., Results: Among 39,342 older people, 5124 (13.4%) fell in the 1-year follow-up. The characteristics of the validation and the development cohorts were similar. ROC-AUCs of the validation and development cohort were 0.690 and 0.705, respectively. Calibration-in-the-large and calibration slope were 0.012 and 0.878, respectively. Calibration plots revealed overprediction for high-risk groups in a small number of individuals., Conclusions and Implications: Our previously developed prediction model for falls demonstrated good external validity by reproducing its predictive performance in the validation cohort. The implementation of this model in the primary care setting could be considered after impact assessment., (Copyright © 2022 The Authors. Published by Elsevier Inc. All rights reserved.)

Background: The number of studies applying machine learning (ML) to predict acute kidney injury (AKI) has grown steadily over the past decade. We assess and critically appraise the state of the art in ML models for AKI prediction, considering performance, methodological soundness, and applicability., Methods: We searched PubMed and ArXiv, extracted data, and critically appraised studies based on the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD), Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS), and Prediction Model Risk of Bias Assessment Tool (PROBAST) guidelines., Results: Forty-six studies from 3166 titles were included. Thirty-eight studies developed a model, five developed and externally validated one, and three studies externally validated one. Flexible ML methods were used more often than deep learning, although the latter was common with temporal variables and text as predictors. Predictive performance showed an area under receiver operating curves ranging from 0.49 to 0.99. Our critical appraisal identified a high risk of bias in 39 studies. Some studies lacked internal validation, whereas external validation and interpretability of results were rarely considered. Fifteen studies focused on AKI prediction in the intensive care setting, and the US-derived Medical Information Mart for Intensive Care (MIMIC) data set was commonly used. Reproducibility was limited as data and code were usually unavailable., Conclusions: Flexible ML methods are popular for the prediction of AKI, although more complex models based on deep learning are emerging. Our critical appraisal identified a high risk of bias in most models: Studies should use calibration measures and external validation more often, improve model interpretability, and share data and code to improve reproducibility., (© The Author(s) 2022. Published by Oxford University Press on behalf of the ERA.)

Background: Currently used prediction tools have limited ability to identify community-dwelling older people at high risk for falls. Prediction models utilizing electronic health records (EHRs) provide opportunities but up to now showed limited clinical value as risk stratification tool, because of among others the underestimation of falls prevalence. The aim of this study was to develop a fall prediction model for community-dwelling older people using a combination of structured data and free text of primary care EHRs and to internally validate its predictive performance., Methods: We used EHR data of individuals aged 65 or older. Age, sex, history of falls, medications, and medical conditions were included as potential predictors. Falls were ascertained from the free text. We employed the Bootstrap-enhanced penalized logistic regression with the least absolute shrinkage and selection operator to develop the prediction model. We used 10-fold cross-validation to internally validate the prediction strategy. Model performance was assessed in terms of discrimination and calibration., Results: Data of 36 470 eligible participants were extracted from the data set. The number of participants who fell at least once was 4 778 (13.1%). The final prediction model included age, sex, history of falls, 2 medications, and 5 medical conditions. The model had a median area under the receiver operating curve of 0.705 (interquartile range 0.700-0.714)., Conclusions: Our prediction model to identify older people at high risk for falls achieved fair discrimination and had reasonable calibration. It can be applied in clinical practice as it relies on routinely collected variables and does not require mobility assessment tests., (© The Author(s) 2021. Published by Oxford University Press on behalf of The Gerontological Society of America.)

External validation of models for the prediction of acute kidney injury (AKI) is rare. We externally validate AKI prediction models in intensive care units. The models were developed on the Medical Information Mart for Intensive Care dataset and validated on the eICU dataset. Traditional machine learning models show limited transportability to the new population (AUROC < 0.8). Models based on recurrent neural networks, which can capture complex relationships between the data, transport well to the new population (AUROC 0.8-0.9). Such models can help clinicians to recognize AKI and improve the outcome.

Identifying prognostic factors (PFs) is often costly and labor-intensive. Routinely collected hospital data provide opportunities to identify clinically relevant PFs and construct accurate prognostic models without additional data-collection costs. This multicenter (66 hospitals) study reports on associations various patient-level variables have with outcomes and costs. Outcomes were in-hospital mortality, intensive care unit (ICU) admission, length of stay, 30-day readmission, 30-day reintervention and in-hospital costs. Candidate PFs were age, sex, Elixhauser Comorbidity Score, prior hospitalizations, prior days spent in hospital, and socio-economic status. Included patients dealt with either colorectal carcinoma (CRC, n = 10,254), urinary bladder carcinoma (UBC, n = 17,385), acute percutaneous coronary intervention (aPCI, n = 25,818), or total knee arthroplasty (TKA, n = 39,214). Prior hospitalization significantly increased readmission risk in all treatments (OR between 2.15 and 25.50), whereas prior days spent in hospital decreased this risk (OR between 0.55 and 0.95). In CRC patients, women had lower risk of in-hospital mortality (OR 0.64), ICU admittance (OR 0.68) and 30-day reintervention (OR 0.70). Prior hospitalization was the strongest PF for higher costs across all treatments (31-64% costs increase/hospitalization). Prognostic model performance (c-statistic) ranged 0.67-0.92, with Brier scores below 0.08. R-squared ranged from 0.06-0.19 for LoS and 0.19-0.38 for costs. Identified PFs should be considered as building blocks for treatment-specific prognostic models and information for monitoring patients after surgery. Researchers and clinicians might benefit from gaining a better insight into the drivers behind (costs) prognosis., (© 2022. The Author(s).)

Background: Building Machine Learning (ML) models in healthcare may suffer from time-consuming and potentially biased pre-selection of predictors by hand that can result in limited or trivial selection of suitable models. We aimed to assess the predictive performance of automating the process of building ML models (AutoML) in-hospital mortality prediction modelling of triage COVID-19 patients at ICU admission versus expert-based predictor pre-selection followed by logistic regression., Methods: We conducted an observational study of all COVID-19 patients admitted to Dutch ICUs between February and July 2020. We included 2,690 COVID-19 patients from 70 ICUs participating in the Dutch National Intensive Care Evaluation (NICE) registry. The main outcome measure was in-hospital mortality. We asessed model performance (at admission and after 24h, respectively) of AutoML compared to the more traditional approach of predictor pre-selection and logistic regression., Findings: Predictive performance of the autoML models with variables available at admission shows fair discrimination (average AUROC = 0·75-0·76 (sdev = 0·03), PPV = 0·70-0·76 (sdev = 0·1) at cut-off = 0·3 (the observed mortality rate), and good calibration. This performance is on par with a logistic regression model with selection of patient variables by three experts (average AUROC = 0·78 (sdev = 0·03) and PPV = 0·79 (sdev = 0·2)). Extending the models with variables that are available at 24h after admission resulted in models with higher predictive performance (average AUROC = 0·77-0·79 (sdev = 0·03) and PPV = 0·79-0·80 (sdev = 0·10-0·17))., Conclusions: AutoML delivers prediction models with fair discriminatory performance, and good calibration and accuracy, which is as good as regression models with expert-based predictor pre-selection. In the context of the restricted availability of data in an ICU quality registry, extending the models with variables that are available at 24h after admission showed small (but significantly) performance increase., (Copyright © 2022 The Author(s). Published by Elsevier B.V. All rights reserved.)

Acute kidney injury (AKI) is an abrupt decrease of kidney function which is common in the intensive care. Many AKI prediction models have been proposed, but an analysis of what is the added value of clinical notes and medical terminologies has not yet been conducted. We developed and internally validated a model to predict AKI that includes not only clinical variables, but also clinical notes and medical terminologies. Our results were overall good (AUROC > 0.80). The best model used only clinical variables (AUROC 0.899).

Background: Recent research demonstrated substantial heterogeneity in the Kidney Disease: Improving Global Outcomes (KDIGO) acute kidney injury (AKI) diagnosis and staging criteria implementations in clinical research. Here we report an additional issue in the implementation of the criteria: the incorrect description and application of a stage 3 serum creatinine (SCr) criterion. Instead of an increase in SCr to or beyond 4.0 mg/dL, studies apparently interpreted this criterion as an increase in SCr by 4.0 mg/dL., Methods: Using a sample of 8124 consecutive intensive care unit (ICU) admissions, we illustrate the implications of such incorrect application. The AKI stage distributions associated with the correct and incorrect stage 3 SCr criterion implementations were compared, both with and without the stage 3 renal replacement therapy (RRT) criterion. In addition, we compared chronic kidney disease presence, ICU mortality rates and hospital mortality rates associated with each of the AKI stages and the misclassified cases., Results: Where incorrect implementation of the SCr stage 3 criterion showed a stage 3 AKI rate of 29%, correct implementation revealed a rate of 34%, mainly due to shifts from stage 1 to stage 3. Without the stage 3 RRT criterion, the stage 3 AKI rates were 9% and 19% after incorrect and correct implementation, respectively. The ICU and hospital mortality rates in cases misclassified as stage 1 or 2 were similar to those in cases correctly classified as stage 1 instead of stage 3., Conclusions: While incorrect implementation of the SCr stage 3 criterion has significant consequences for AKI severity epidemiology, consequences for clinical decision making may be less severe. We urge researchers and clinicians to verify their implementation of the AKI staging criteria., (© The Author(s) 2021. Published by Oxford University Press on behalf of the ERA.)

Introduction: We aimed to develop a prediction model for foot ulcer recurrence in people with diabetes using easy-to-obtain clinical variables and to validate its predictive performance in order to help risk assessment in this high-risk group., Research Design and Methods: We used data from a prospective analysis of 304 people with foot ulcer history who had 18-month follow-up for ulcer outcome. Demographic, disease-related and organization-of-care variables were included as potential predictors. Two logistic regression prediction models were created: model 1 for all recurrent foot ulcers (n=126 events) and model 2 for recurrent plantar foot ulcers (n=70 events). We used 10-fold cross-validation, each including five multiple imputation sets for internal validation. Performance was assessed in terms of discrimination using area under the receiver operating characteristic curve (AUC) (0-1, 1=perfect discrimination), and calibration with the Brier Score (0-1, 0=complete concordance predicted vs observed values) and calibration graphs., Results: Predictors in model 1 were: a younger age, more severe peripheral sensory neuropathy, fewer months since healing of previous ulcer, presence of a minor lesion, use of a walking aid and not monitoring foot temperatures at home. Mean AUC for model 1 was 0.69 (2SD 0.040) and mean Brier Score was 0.22 (2SD 0.011). Predictors in model 2 were: a younger age, plantar location of previous ulcer, fewer months since healing of previous ulcer, presence of a minor lesion, consumption of alcohol, use of a walking aid, and foot care received in a university medical center. Mean AUC for model 2 was 0.66 (2SD 0.023) and mean Brier Score was 0.16 (2SD 0.0048)., Conclusions: These internally validated prediction models predict with reasonable to good calibration and fair discrimination who is at highest risk of ulcer recurrence. The people at highest risk should be monitored more carefully and treated more intensively than others., Trial Registration Number: NTR5403., Competing Interests: Competing interests: None declared., (© Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.)

Objectives: Individuals may respond differently to the same treatment, and there is a need to understand such heterogeneity of causal individual treatment effects. We propose and evaluate a modelling approach to better understand this heterogeneity from observational studies by identifying patient subgroups with a markedly deviating response to treatment. We illustrate this approach in a primary care case-study of antibiotic (AB) prescription on recovery from acute rhino-sinusitis (ARS)., Methods: Our approach consists of four stages and is applied to a large dataset in primary care dataset of 24,392 patients suspected of suffering from ARS. We first identify pre-treatment variables that either confound the relationship between treatment and outcome or are risk factors of the outcome. Second, based on the pre-treatment variables we create Synthetic Random Forest (SRF) models to compute the potential outcomes and subsequently the causal individual treatment effect (ITE) estimates. Third, we perform subgroup discovery using the ITE estimates as outcomes to identify positive and negative responders. Fourth, we evaluate the predictive performance of the identified subgroups for predicting the outcome in two ways: the likelihood ratio test, and whether the subgroups are selected via the Akaike Information Criterion (AIC) using backward stepwise variable selection. We validate the whole modelling strategy by means of 10-fold-cross-validation., Results: Based on 20 pre-treatment variables, four subgroups (three for positive responders and one for negative responders) were identified. The log likelihood ratio tests showed that the subgroups were significant. Variable selection using the AIC kept two of the four subgroups, one for positive responders and one for negative responders. As for the validation of the whole modelling strategy, all reported measures (the number of pre-treatment variables associated with the outcome, number of subgroups, number of subgroups surviving variable selection and coverage) showed little variation., Conclusions: With the proposed approach, we identified subgroups of positive and negative responders to treatment that markedly deviate from the mean response. The subgroups showed additive predictive value of the outcome. The modelling approach strategy was shown to be robust on this dataset. Our approach was thus able to discover understandable subgroups from observational data that have predictive value and which may be considered by the clinical users to get insight into who responds positively or negatively to a proposed treatment., (Copyright © 2021 The Author(s). Published by Elsevier B.V. All rights reserved.)

Acute kidney injury (AKI) is a common and potentially life-threatening condition, which often occurs in the intensive care unit. We propose a machine learning model based on recurrent neural networks to continuously predict AKI. We internally validated its predictive performance, both in terms of discrimination and calibration, and assessed its interpretability. Our model achieved good discrimination (AUC 0.80-0.94). Such a continuous model can support clinicians to promptly recognize and treat AKI patients and may improve their outcomes.