Search

Background: Prognostic models are used widely in the oncology domain to guide medical decision-making. Little is known about the risk of bias of prognostic models developed using machine learning and the barriers to their clinical uptake in the oncology domain., Methods: We conducted a systematic review and searched MEDLINE and EMBASE databases for oncology-related studies developing a prognostic model using machine learning methods published between 01/01/2019 and 05/09/2019. The primary outcome was risk of bias, judged using the Prediction model Risk Of Bias ASsessment Tool (PROBAST). We described risk of bias overall and for each domain, by development and validation analyses separately., Results: We included 62 publications (48 development-only; 14 development with validation). 152 models were developed across all publications and 37 models were validated. 84% (95% CI: 77 to 89) of developed models and 51% (95% CI: 35 to 67) of validated models were at overall high risk of bias. Bias introduced in the analysis was the largest contributor to the overall risk of bias judgement for model development and validation. 123 (81%, 95% CI: 73.8 to 86.4) developed models and 19 (51%, 95% CI: 35.1 to 67.3) validated models were at high risk of bias due to their analysis, mostly due to shortcomings in the analysis including insufficient sample size and split-sample internal validation., Conclusions: The quality of machine learning based prognostic models in the oncology domain is poor and most models have a high risk of bias, contraindicating their use in clinical practice. Adherence to better standards is urgently needed, with a focus on sample size estimation and analysis methods, to improve the quality of these models., (© 2022. The Author(s).)

Background: Most patients who present with depression are treated in primary care by general practitioners (GPs). Relapse of depression is common (at least 50% of patients treated for depression will relapse after a single episode) and leads to considerable morbidity and decreased quality of life for patients. The majority of patients will relapse within 6 months, and those with a history of relapse are more likely to relapse in the future than those with no such history. GPs see a largely undifferentiated case-mix of patients, and once patients with depression reach remission, there is limited guidance to help GPs stratify patients according to risk of relapse. We aim to develop a prognostic model to predict an individual's risk of relapse within 6-8 months of entering remission. The long-term objective is to inform the clinical management of depression after the acute phase., Methods: We will develop a prognostic model using secondary analysis of individual participant data drawn from seven RCTs and one longitudinal cohort study in primary or community care settings. We will use logistic regression to predict the outcome of relapse of depression within 6-8 months. We plan to include the following established relapse predictors in the model: residual depressive symptoms, number of previous depressive episodes, co-morbid anxiety and severity of index episode. We will use a "full model" development approach, including all available predictors. Performance statistics (optimism-adjusted C-statistic, calibration-in-the-large, calibration slope) and calibration plots (with smoothed calibration curves) will be calculated. Generalisability of predictive performance will be assessed through internal-external cross-validation. Clinical utility will be explored through net benefit analysis., Discussion: We will derive a statistical model to predict relapse of depression in remitted depressed patients in primary care. Assuming the model has sufficient predictive performance, we outline the next steps including independent external validation and further assessment of clinical utility and impact., Study Registration: ClinicalTrials.gov ID: NCT04666662.

Background: Shoulder pain is one of the most common presentations of musculoskeletal pain with a 1-month population prevalence of between 7 and 26%. The overall prognosis of shoulder pain is highly variable with 40% of patients reporting persistent pain 1 year after consulting their primary care clinician. Despite evidence for prognostic value of a range of patient and disease characteristics, it is not clear whether these factors also predict (moderate) the effect of specific treatments (such as corticosteroid injection, exercise, or surgery)., Objectives: This study aims to identify predictors of treatment effect (i.e. treatment moderators or effect modifiers) by investigating the association between a number of pre-defined individual-level factors and the effects of commonly used treatments on shoulder pain and disability outcomes., Methods: This will be a meta-analysis using individual participant data (IPD). Eligible trials investigating the effectiveness of advice and analgesics, corticosteroid injection, physiotherapy-led exercise, psychological interventions, and/or surgical treatment in patients with shoulder conditions will be identified from systematic reviews and an updated systematic search for trials, and risk of bias will be assessed. Authors of all eligible trials will be approached for data sharing. Outcomes measured will be shoulder pain and disability, and our previous work has identified candidate predictors. The main analysis will be conducted using hierarchical one-stage IPD meta-analysis models, examining the effect of treatment-predictor interaction on outcome for each of the candidate predictors and describing relevant subgroup effects where significant interaction effects are detected. Random effects will be used to account for clustering and heterogeneity. Sensitivity analyses will be based on (i) exclusion of trials at high risk of bias, (ii) use of restricted cubic splines to model potential non-linear associations for candidate predictors measured on a continuous scale, and (iii) the use of a two-stage IPD meta-analysis framework., Discussion: Our study will collate, appraise, and synthesise IPD from multiple studies to examine potential predictors of treatment effect in order to assess the potential for better and more efficient targeting of specific treatments for individuals with shoulder pain., Systematic Review Registration: PROSPERO CRD42018088298., Competing Interests: Competing interestsGW-J sits on the NICE Guideline Review Panel “Workplace health: Long-term sickness absence and capability for work”. CL receives payment for courses on the assessment and treatment of rotator cuff related shoulder pain.

Over the past few years, evidence synthesis has become essential to investigate and improve the generalizability of medical research findings. This strategy often involves a meta-analysis to formally summarize quantities of interest, such as relative treatment effect estimates. The use of meta-analysis methods is, however, less straightforward in prognosis research because substantial variation exists in research objectives, analysis methods and the level of reported evidence. We present a gentle overview of statistical methods that can be used to summarize data of prognostic factor and prognostic model studies. We discuss how aggregate data, individual participant data, or a combination thereof can be combined through meta-analysis methods. Recent examples are provided throughout to illustrate the various methods., Competing Interests: Competing interestsTPD is an Associate Editor for BMC Diagnostic and Prognostic Research. KGM is an Editor-in-Chief for BMC Diagnostic and Prognostic Research. RDR is a member of the Editorial Board of BMC Diagnostic and Prognostic Research.

Background: Monochorionic twin pregnancies are at high risk of adverse outcomes, but it is not possible to predict which pregnancies will develop complications. The aim of the study was to evaluate, in monochorionic twin pregnancies, whether first-trimester ultrasound (nuchal translucency [NT], crown-rump length [CRL]), and maternal serum biomarkers (alpha-fetoprotein [AFP], soluble fms-like tyrosine kinase-1 [sFlt-1] and placental growth factor [PlGF]), are prognostic factors for fetal adverse outcome composite, twin-twin transfusion syndrome (TTTS), growth restriction, and intrauterine fetal death (IUFD)., Methods: A cohort study of 177 monochorionic diamniotic twin pregnancies. Independent prognostic ability of each factor was assessed by multivariable logistic regression, adjusting for standard prognostic factors. Factors were analysed as continuous data; thus, the reported ORs relate to either 1% change in NT or CRL inter-twin percentage discordance or one unit of measure in each serum biomarker., Results: The odds of the fetal adverse outcome composite were significantly associated with increased NT inter-twin percentage discordance (adjusted OR 1.03 [95% CI 1.01, 1.06]) and CRL inter-twin percentage discordance (adjusted OR 1.17 [95% CI 1.07, 1.29]). TTTS was significantly associated with increased NT discordance (adjusted OR 1.06 [95% CI 1.03, 1.10]) and decreased PlGF (adjusted OR 0.42 [95% CI 0.19, 0.93]). Antenatal growth restriction was significantly associated with increased CRL discordance (adjusted OR 1.20 [95% CI 1.08, 1.34]). Single and double IUFD were associated with decreased PlGF (adjusted OR 0.34 [95% CI 0.12, 0.98]) and (adjusted OR 0.18 [95%CI 0.05, 0.58]) respectively., Conclusions: This study has identified potential individual prognostic factors in the first trimester (fetal biometric and maternal serum biomarkers) that show promise but require further robust evaluation in a larger, prospective series of MC twin pregnancies, so that their usefulness both individually and in combination can be defined., Trial Registration: ISRCTN 13114861 (retrospectively registered)., Competing Interests: This study received ethical approval from East Midlands Research Ethics Committee (15/EM/0240) in June 2015 and the Royal Prince Alfred Hospital Research and Ethics Governance Office (HREC/11/RPAH/472) in January 2016.Not applicableThe authors declare that they have no competing interests.Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Background: Pre-eclampsia, a condition with raised blood pressure and proteinuria is associated with an increased risk of maternal and offspring mortality and morbidity. Early identification of mothers at risk is needed to target management., Methods/design: We aim to systematically review the existing literature to identify prediction models for pre-eclampsia. We have established the International Prediction of Pregnancy Complication Network (IPPIC), made up of 72 researchers from 21 countries who have carried out relevant primary studies or have access to existing registry databases, and collectively possess data from more than two million patients. We will use the individual participant data (IPD) from these studies to externally validate these existing prediction models and summarise model performance across studies using random-effects meta-analysis for any, late (after 34 weeks) and early (before 34 weeks) onset pre-eclampsia. If none of the models perform well, we will recalibrate (update), or develop and validate new prediction models using the IPD. We will assess the differential accuracy of the models in various settings and subgroups according to the risk status. We will also validate or develop prediction models based on clinical characteristics only; clinical and biochemical markers; clinical and ultrasound parameters; and clinical, biochemical and ultrasound tests., Discussion: Numerous systematic reviews with aggregate data meta-analysis have evaluated various risk factors separately or in combination for predicting pre-eclampsia, but these are affected by many limitations. Our large-scale collaborative IPD approach encourages consensus towards well developed, and validated prognostic models, rather than a number of competing non-validated ones. The large sample size from our IPD will also allow development and validation of multivariable prediction model for the relatively rare outcome of early onset pre-eclampsia., Trial Registration: The project was registered on Prospero on the 27 November 2015 with ID: CRD42015029349., Competing Interests: Not applicableNot applicableKM is Editor-in-Chief and KS is an associate editor for BMC Diagnostic and Prognostic Research. The other authors declare that they have no competing interests.Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Background: Early-onset pre-eclampsia with raised blood pressure and protein in the urine before 34 weeks' gestation is one of the leading causes of maternal deaths in the UK. The benefits to the child from prolonging the pregnancy need to be balanced against the risk of maternal deterioration. Accurate prediction models of risks are needed to plan management., Methods: We aim to undertake a multicentre prospective cohort study (Prediction of Risks in Early onset Pre-eclampsia (PREP)) to develop clinical prediction models in women with early-onset pre-eclampsia, for risk of adverse maternal outcomes by 48 h and by discharge. We will externally validate the models in two independent cohorts with 634 and 216 women. In the secondary analyses, we will assess risk of adverse fetal and neonatal outcomes at birth and by discharge., Discussion: The PREP study will quantify the risk of maternal complications at various time points and provide individualised estimates of overall risk in women with early-onset pre-eclampsia to plan the management., Trial Registration: ISRCTN registry, ISRCTN40384046., Competing Interests: The authors declare that they have no competing interests.

Background: Prognosis research studies (e.g. those deriving prognostic models or examining potential predictors of outcome) often collect information on time-varying predictors after their intended moment of use, sometimes using a measurement method different to that which would be used. We aimed to illustrate how estimates of predictor-outcome associations and prognostic model performance obtained from such studies may differ to those at the earlier, intended moment of use., Methods: We analysed data from two primary care cohorts of patients consulting for non-inflammatory musculoskeletal conditions: the Prognostic Research Study (PROG-RES: n  = 296, aged >50 years) and the Primary care Osteoarthritis Screening Trial (POST: n  = 756, >45 years). Both cohorts had collected comparable information on a potentially important time-varying predictor (current pain intensity: 0-10 numerical rating scale), other predictors (age, gender, practice) and outcome (patient-perceived non-recovery at 6 months). Using logistic regression models, we compared the direction and magnitude of predictor-outcome associations and model performance measures under two scenarios: (i) current pain intensity ascertained by the treating general practitioner in the consultation (the intended moment of use) and (ii) current pain intensity ascertained by a questionnaire mailed several days after the consultation., Results: In both cohorts, the predictor-outcome association was substantially weaker for pain measured at the consultation (OR (95% CI): PROG-RES 1.06 (0.95, 1.18); POST 1.04 (0.96, 1.12)) than for pain measured in the questionnaire (PROG-RES 1.34 (1.20, 1.48); POST 1.26 (1.18, 1.34)). The c -statistic of the multivariable model was lower when pain was measured at the consultation ( c -statistic (95% CI): PROG-RES 0.57 (0.51, 0.64); POST 0.66 (0.62, 0.70)) than when pain was measured in the questionnaire (PROG-RES 0.69 (0.63, 0.75); POST 0.72 (0.68, 0.76)), reflecting the lower OR for pain at the consultation., Conclusions: Prognostic research studies ideally should measure time-varying predictors at their intended moment of use and using the intended measurement method. Otherwise, they may produce substantially different estimates of predictor-outcome associations and model performance. Researchers should report when, how and where predictors were measured and identify any significant departures from their intended use that may limit the applicability of findings in practice., Trial Registration: The protocol for the PROG-RES cohort data collection and primary analysis has been published in an open-access journal (Mallen et al., BMC Musculoskelet Disord 7:84, 2006). The POST trial was registered (ISRCTN40721988; date of registration: 21 June 2011; date of enrolment of the first participant: 3 October 2011) and had a pre-specified protocol covering primary analysis. There was no published protocol for the current secondary analyses presented in this manuscript., Competing Interests: The authors declare that they have no competing interests.