Search

When designing a study to develop a new prediction model with binary or time-to-event outcomes, researchers should ensure their sample size is adequate in terms of the number of participants (n) and outcome events (E) relative to the number of predictor parameters (p) considered for inclusion. We propose that the minimum values of n and E (and subsequently the minimum number of events per predictor parameter, EPP) should be calculated to meet the following three criteria: (i) small optimism in predictor effect estimates as defined by a global shrinkage factor of ≥0.9, (ii) small absolute difference of ≤ 0.05 in the model's apparent and adjusted Nagelkerke's R 2 , and (iii) precise estimation of the overall risk in the population. Criteria (i) and (ii) aim to reduce overfitting conditional on a chosen p, and require prespecification of the model's anticipated Cox-Snell R 2 , which we show can be obtained from previous studies. The values of n and E that meet all three criteria provides the minimum sample size required for model development. Upon application of our approach, a new diagnostic model for Chagas disease requires an EPP of at least 4.8 and a new prognostic model for recurrent venous thromboembolism requires an EPP of at least 23. This reinforces why rules of thumb (eg, 10 EPP) should be avoided. Researchers might additionally ensure the sample size gives precise estimates of key predictor effects; this is especially important when key categorical predictors have few events in some categories, as this may substantially increase the numbers required., (© 2018 The Authors. Statistics in Medicine Published by John Wiley & Sons Ltd.)

In the medical literature, hundreds of prediction models are being developed to predict health outcomes in individuals. For continuous outcomes, typically a linear regression model is developed to predict an individual's outcome value conditional on values of multiple predictors (covariates). To improve model development and reduce the potential for overfitting, a suitable sample size is required in terms of the number of subjects (n) relative to the number of predictor parameters (p) for potential inclusion. We propose that the minimum value of n should meet the following four key criteria: (i) small optimism in predictor effect estimates as defined by a global shrinkage factor of ≥0.9; (ii) small absolute difference of ≤ 0.05 in the apparent and adjusted R 2 ; (iii) precise estimation (a margin of error ≤ 10% of the true value) of the model's residual standard deviation; and similarly, (iv) precise estimation of the mean predicted outcome value (model intercept). The criteria require prespecification of the user's chosen p and the model's anticipated R 2 as informed by previous studies. The value of n that meets all four criteria provides the minimum sample size required for model development. In an applied example, a new model to predict lung function in African-American women using 25 predictor parameters requires at least 918 subjects to meet all criteria, corresponding to at least 36.7 subjects per predictor parameter. Even larger sample sizes may be needed to additionally ensure precise estimates of key predictor effects, especially when important categorical predictors have low prevalence in certain categories., (© 2018 John Wiley & Sons, Ltd.)

Competing Interests: Competing interests: None.

Objectives: The ability to efficiently and accurately predict future risk of primary total hip and knee replacement (THR/TKR) in earlier stages of osteoarthritis (OA) has potentially important applications. We aimed to develop and validate two models to estimate an individual's risk of primary THR and TKR in patients newly presenting to primary care., Methods: We identified two cohorts of patients aged ≥40 years newly consulting hip pain/OA and knee pain/OA in the Clinical Practice Research Datalink. Candidate predictors were identified by systematic review, novel hypothesis-free 'Record-Wide Association Study' with replication, and panel consensus. Cox proportional hazards models accounting for competing risk of death were applied to derive risk algorithms for THR and TKR. Internal-external cross-validation (IECV) was then applied over geographical regions to validate two models., Results: 45 predictors for THR and 53 for TKR were identified, reviewed and selected by the panel. 301 052 and 416 030 patients newly consulting between 1992 and 2015 were identified in the hip and knee cohorts, respectively (median follow-up 6 years). The resultant model C-statistics is 0.73 (0.72, 0.73) and 0.79 (0.78, 0.79) for THR (with 20 predictors) and TKR model (with 24 predictors), respectively. The IECV C-statistics ranged between 0.70-0.74 (THR model) and 0.76-0.82 (TKR model); the IECV calibration slope ranged between 0.93-1.07 (THR model) and 0.92-1.12 (TKR model)., Conclusions: Two prediction models with good discrimination and calibration that estimate individuals' risk of THR and TKR have been developed and validated in large-scale, nationally representative data, and are readily automated in electronic patient records., Competing Interests: Competing interests: None declared, (© Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY. Published by BMJ.)

Prediction models in health care use predictors to estimate for an individual the probability that a condition or disease is already present (diagnostic model) or will occur in the future (prognostic model). Publications on prediction models have become more common in recent years, and competing prediction models frequently exist for the same outcome or target population. Health care providers, guideline developers, and policymakers are often unsure which model to use or recommend, and in which persons or settings. Hence, systematic reviews of these studies are increasingly demanded, required, and performed. A key part of a systematic review of prediction models is examination of risk of bias and applicability to the intended population and setting. To help reviewers with this process, the authors developed PROBAST (Prediction model Risk Of Bias ASsessment Tool) for studies developing, validating, or updating (for example, extending) prediction models, both diagnostic and prognostic. PROBAST was developed through a consensus process involving a group of experts in the field. It includes 20 signaling questions across 4 domains (participants, predictors, outcome, and analysis). This explanation and elaboration document describes the rationale for including each domain and signaling question and guides researchers, reviewers, readers, and guideline developers in how to use them to assess risk of bias and applicability concerns. All concepts are illustrated with published examples across different topics. The latest version of the PROBAST checklist, accompanying documents, and filled-in examples can be downloaded from www.probast.org.

Clinical prediction models combine multiple predictors to estimate risk for the presence of a particular condition (diagnostic models) or the occurrence of a certain event in the future (prognostic models). PROBAST (Prediction model Risk Of Bias ASsessment Tool), a tool for assessing the risk of bias (ROB) and applicability of diagnostic and prognostic prediction model studies, was developed by a steering group that considered existing ROB tools and reporting guidelines. The tool was informed by a Delphi procedure involving 38 experts and was refined through piloting. PROBAST is organized into the following 4 domains: participants, predictors, outcome, and analysis. These domains contain a total of 20 signaling questions to facilitate structured judgment of ROB, which was defined to occur when shortcomings in study design, conduct, or analysis lead to systematically distorted estimates of model predictive performance. PROBAST enables a focused and transparent approach to assessing the ROB and applicability of studies that develop, validate, or update prediction models for individualized predictions. Although PROBAST was designed for systematic reviews, it can be used more generally in critical appraisal of prediction model studies. Potential users include organizations supporting decision making, researchers and clinicians who are interested in evidence-based medicine or involved in guideline development, journal editors, and manuscript reviewers.

One-stage individual participant data meta-analysis models should account for within-trial clustering, but it is currently debated how to do this. For continuous outcomes modeled using a linear regression framework, two competing approaches are a stratified intercept or a random intercept. The stratified approach involves estimating a separate intercept term for each trial, whereas the random intercept approach assumes that trial intercepts are drawn from a normal distribution. Here, through an extensive simulation study for continuous outcomes, we evaluate the impact of using the stratified and random intercept approaches on statistical properties of the summary treatment effect estimate. Further aims are to compare (i) competing estimation options for the one-stage models, including maximum likelihood and restricted maximum likelihood, and (ii) competing options for deriving confidence intervals (CI) for the summary treatment effect, including the standard normal-based 95% CI, and more conservative approaches of Kenward-Roger and Satterthwaite, which inflate CIs to account for uncertainty in variance estimates. The findings reveal that, for an individual participant data meta-analysis of randomized trials with a 1:1 treatment:control allocation ratio and heterogeneity in the treatment effect, (i) bias and coverage of the summary treatment effect estimate are very similar when using stratified or random intercept models with restricted maximum likelihood, and thus either approach could be taken in practice, (ii) CIs are generally best derived using either a Kenward-Roger or Satterthwaite correction, although occasionally overly conservative, and (iii) if maximum likelihood is required, a random intercept performs better than a stratified intercept model. An illustrative example is provided., (© 2018 The Authors. Statistics in Medicine Published by John Wiley & Sons, Ltd.)

Background: In patients with rectal cancer who achieve clinical complete response after neoadjuvant chemoradiotherapy, watch and wait is a novel management strategy with potential to avoid major surgery. Study-level meta-analyses have reported wide variation in the proportion of patients with local regrowth. We did an individual participant data meta-analysis to investigate factors affecting occurrence of local regrowth., Methods: We updated search results of a recent systematic review by searching MEDLINE and Embase from Jan 1, 2016, to May 5, 2017, and used expert knowledge to identify published studies reporting on local regrowth in patients with rectal cancer managed by watch and wait after clinical complete response to neoadjuvant chemoradiotherapy. We restricted studies to those that defined clinical complete response using criteria equivalent to São Paulo benchmarks (ie, absence of residual ulceration, stenosis, or mass within the rectum on clinical and endoscopic examination). The primary outcome was 2-year cumulative incidence of local regrowth, estimated with a two-stage random-effects individual participant data meta-analysis. We assessed the effects of clinical and treatment factors using Cox frailty models, expressed as hazard ratios (HRs). From these models, we derived percentage differences in mean θ as an approximation of the effect of measured covariates on between-centre heterogeneity. This study is registered with PROSPERO, number CRD42017070934., Findings: We obtained individual participant data from 11 studies, including 602 patients enrolled between March 11, 1990, and Feb 13, 2017, with a median follow-up of 37·6 months (IQR 25·0-58·7). Ten of the 11 datasets were judged to be at low risk of bias. 2-year cumulative incidence of local regrowth was 21·4% (random-effects 95% CI 15·3-27·6), with high levels of between-study heterogeneity (I 2 =61%). We noted wide between-centre variation in patient, tumour, and treatment characteristics. We found some evidence that increasing cT stage was associated with increased risk of local regrowth (random-effects HR per cT stage 1·40, 95% CI 1·00-1·94; p trend =0·048). In a subgroup of 459 patients managed after 2008 (when pretreatment staging by MRI became standard), 2-year cumulative incidence of local regrowth was 19% (95% CI 13-28) for stage cT1 and cT2 tumours, 31% (26-37) for cT3, and 37% (21-60) for cT4 (random-effects HR per cT stage 1·50, random-effects 95% CI 1·03-2·17; p trend =0·0330). We estimated that measured factors contributed 4·8-45·3% of observed between-centre heterogeneity., Interpretation: In patients with rectal cancer and clinical complete response after chemoradiotherapy managed by watch and wait, we found some evidence that increasing cT stage predicts for local regrowth. These data will inform clinician-patient decision making in this setting. Research is needed to determine other predictors of a sustained clinical complete response., Funding: None., (Copyright © 2018 Elsevier Ltd. All rights reserved.)

If individual participant data are available from multiple studies or clusters, then a prediction model can be externally validated multiple times. This allows the model's discrimination and calibration performance to be examined across different settings. Random-effects meta-analysis can then be used to quantify overall (average) performance and heterogeneity in performance. This typically assumes a normal distribution of 'true' performance across studies. We conducted a simulation study to examine this normality assumption for various performance measures relating to a logistic regression prediction model. We simulated data across multiple studies with varying degrees of variability in baseline risk or predictor effects and then evaluated the shape of the between-study distribution in the C-statistic, calibration slope, calibration-in-the-large, and E/O statistic, and possible transformations thereof. We found that a normal between-study distribution was usually reasonable for the calibration slope and calibration-in-the-large; however, the distributions of the C-statistic and E/O were often skewed across studies, particularly in settings with large variability in the predictor effects. Normality was vastly improved when using the logit transformation for the C-statistic and the log transformation for E/O, and therefore we recommend these scales to be used for meta-analysis. An illustrated example is given using a random-effects meta-analysis of the performance of QRISK2 across 25 general practices.

Univariate meta-analysis concerns a single outcome of interest measured across a number of independent studies. However, many research studies will have also measured secondary outcomes. Multivariate meta-analysis allows us to take these secondary outcomes into account, and can also include studies where the primary outcome is missing. We define the efficiency (E) as the variance of the overall estimate from a multivariate meta-analysis relative to the variance of the overall estimate from a univariate meta-analysis. The extra information gained from a multivariate meta-analysis of n studies is then similar to the extra information gained if a univariate meta-analysis of the primary effect had a further n(1-E)/E studies. The variance contribution of a study's secondary outcomes (its borrowing of strength) can be thought of as a contrast between the variance matrix of the outcomes in that study and the set of variance matrices of all the studies in the meta-analysis. In the bivariate case this is given a simple graphical interpretation as the borrowing of strength plot . We discuss how these findings can also be used in the context of random effects meta-analysis. Our discussion is motivated by a published meta-analysis of ten anti-hypertension clinical trials.

Objective: Measurement error in predictor variables may threaten the validity of clinical prediction models. We sought to evaluate the possible extent of the problem. A secondary objective was to examine whether predictors are measured at the intended moment of model use., Methods: A systematic search of Medline was used to identify a sample of articles reporting the development of a clinical prediction model published in 2015. After screening according to a predefined inclusion criteria, information on predictors, strategies to control for measurement error, and intended moment of model use were extracted. Susceptibility to measurement error for each predictor was classified into low and high risks., Results: Thirty-three studies were reviewed, including 151 different predictors in the final prediction models. Fifty-one (33.7%) predictors were categorized as high risk of error; however, this was not accounted for in the model development. Only 8 (24.2%) studies explicitly stated the intended moment of model use and when the predictors were measured., Conclusion: Reporting of measurement error and intended moment of model use is poor in prediction model studies. There is a need to identify circumstances where ignoring measurement error in prediction models is consequential and whether accounting for the error will improve the predictions., (Copyright © 2018 Elsevier Inc. All rights reserved.)

Many meta-analysis models contain multiple parameters, for example due to multiple outcomes, multiple treatments or multiple regression coefficients. In particular, meta-regression models may contain multiple study-level covariates, and one-stage individual participant data meta-analysis models may contain multiple patient-level covariates and interactions. Here, we propose how to derive percentage study weights for such situations, in order to reveal the (otherwise hidden) contribution of each study toward the parameter estimates of interest. We assume that studies are independent, and utilise a decomposition of Fisher's information matrix to decompose the total variance matrix of parameter estimates into study-specific contributions, from which percentage weights are derived. This approach generalises how percentage weights are calculated in a traditional, single parameter meta-analysis model. Application is made to one- and two-stage individual participant data meta-analyses, meta-regression and network (multivariate) meta-analysis of multiple treatments. These reveal percentage study weights toward clinically important estimates, such as summary treatment effects and treatment-covariate interactions, and are especially useful when some studies are potential outliers or at high risk of bias. We also derive percentage study weights toward methodologically interesting measures, such as the magnitude of ecological bias (difference between within-study and across-study associations) and the amount of inconsistency (difference between direct and indirect evidence in a network meta-analysis).

Background: Prior studies have reported inconsistencies in the baseline risk profile, comorbidity burden and their association with clinical outcomes in women compared to men. More importantly, there is limited data around the sex differences and how these have changed over time in contemporary percutaneous coronary intervention (PCI) practice., Methods and Results: We used the Nationwide Inpatient Sample to identify all PCI procedures based on ICD-9 procedure codes in the United States between 2004-2014 in adult patients. Descriptive statistics were used to describe sex-based differences in baseline characteristics and comorbidity burden of patients. Multivariable logistic regressions were used to investigate the association between these differences and in-hospital mortality, complications, length of stay and total hospital charges. Among 6,601,526 patients, 66% were men and 33% were women. Women were more likely to be admitted with diagnosis of NSTEMI (non-ST elevation acute myocardial infarction), were on average 5 years older (median age 68 compared to 63) and had higher burden of comorbidity defined by Charlson score ≥3. Women also had higher in-hospital crude mortality (2.0% vs 1.4%) and any complications compared to men (11.1% vs 7.0%). These trends persisted in our adjusted analyses where women had a significant increase in the odds of in-hospital mortality men (OR 1.20 (95% CI 1.16,1.23) and major bleeding (OR 1.81 (95% CI 1.77,1.86)., Conclusion: In this national unselected contemporary PCI cohort, there are significant sex-based differences in presentation, baseline characteristics and comorbidity burden. These differences do not fully account for the higher in-hospital mortality and procedural complications observed in women., Competing Interests: The authors have declared that no competing interests exist.

Background: Venous leg ulcers (VLUs) are a common type of complex wound that have a negative impact on people's lives and incur high costs for health services and society. It has been suggested that prolonged high levels of protease activity in the later stages of the healing of chronic wounds may be associated with delayed healing. Protease modulating treatments have been developed which seek to modulate protease activity and thereby promote healing in chronic wounds., Objectives: To determine whether protease activity is an independent prognostic factor for the healing of venous leg ulcers., Search Methods: In February 2018, we searched the following databases: Cochrane Central Register of Controlled Trials (CENTRAL), Ovid MEDLINE, Ovid Embase and CINAHL., Selection Criteria: We included prospective and retrospective longitudinal studies with any follow-up period that recruited people with VLUs and investigated whether protease activity in wound fluid was associated with future healing of VLUs. We included randomised controlled trials (RCTs) analysed as cohort studies, provided interventions were taken into account in the analysis, and case-control studies if there were no available cohort studies. We also included prediction model studies provided they reported separately associations of individual prognostic factors (protease activity) with healing. Studies of any type of protease or combination of proteases were eligible, including proteases from bacteria, and the prognostic factor could be examined as a continuous or categorical variable; any cut-off point was permitted. The primary outcomes were time to healing (survival analysis) and the proportion of people with ulcers completely healed; the secondary outcome was change in ulcer size/rate of wound closure. We extracted unadjusted (simple) and adjusted (multivariable) associations between the prognostic factor and healing., Data Collection and Analysis: Two review authors independently assessed studies for inclusion at each stage, and undertook data extraction, assessment of risk of bias and GRADE assessment. We collected association statistics where available. No study reported adjusted analyses: instead we collected unadjusted results or calculated association measures from raw data. We calculated risk ratios when both outcome and prognostic factor were dichotomous variables. When the prognostic factor was reported as continuous data and healing outcomes were dichotomous, we either performed regression analysis or analysed the impact of healing on protease levels, analysing as the standardised mean difference. When both prognostic factor and outcome were continuous data, we reported correlation coefficients or calculated them from individual participant data.We displayed all results on forest plots to give an overall visual representation. We planned to conduct meta-analyses where this was appropriate, otherwise we summarised narratively., Main Results: We included 19 studies comprising 21 cohorts involving 646 participants. Only 11 studies (13 cohorts, 522 participants) had data available for analysis. Of these, five were prospective cohort studies, four were RCTs and two had a type of case-control design. Follow-up time ranged from four to 36 weeks. Studies covered 10 different matrix metalloproteases (MMPs) and two serine proteases (human neutrophil elastase and urokinase-type plasminogen activators). Two studies recorded complete healing as an outcome; other studies recorded partial healing measures. There was clinical and methodological heterogeneity across studies; for example, in the definition of healing, the type of protease and its measurement, the distribution of active and bound protease species, the types of treatment and the reporting of results. Therefore, meta-analysis was not performed. No study had conducted multivariable analyses and all included evidence was of very low certainty because of the lack of adjustment for confounders, the high risk of bias for all studies except one, imprecision around the measures of association and inconsistency in the direction of association. Collectively the research indicated complete uncertainty as to the association between protease activity and VLU healing., Authors' Conclusions: This review identified very low validity evidence regarding any association between protease activity and VLU healing and there is complete uncertainty regarding the relationship. The review offers information for both future research and systematic review methodology.

This study aims to evaluate the impact of co-morbidity burden on outcomes in patients who undergo percutaneous coronary intervention (PCI). We used the Nationwide Inpatient Sample to identify all PCI procedures undertaken in the United States from 2004 to 2014. We then determined co-morbidity burden for each patient record based on the Charlson Co-morbidity Score. Multivariable logistic regression models were used to examine the association between co-morbidity burden and in-hospital mortality other in-hospital complications. A total of 6,601,526 PCI procedures were included in the analysis. Overall co-morbidity burden increased over time, with severe co-morbidity burden (defined as a CCI score ≥3) increasing from 5.3% in 2004 to 14.2% in 2014 (p <0.0001). After adjustment for confounding factors increasing co-morbidity burden was independently associated with increased odds of in-hospital mortality, complications, length of hospital stay, and total cost of hospitalization post PCI. A CCI score of 1 was independently associated with an increase in the odds of in hospital mortality (odds ratio [OR] 1.19 [95% confidence interval [CI] 1.15 to 1.25]), a score of 2 associated with an almost 1.5-fold increase (OR 1.41 [95% CI 1.34 to 1.48]) and a score of ≥3 a 2-fold increase (OR 1.96 [95% CI 1.86 to 2.07]) compared with no co-morbid burden (CCI score of 0). In conclusion, our results show that co-morbid burden is independently associated with increased risk of in-hospital mortality, in-hospital complications, length of stay, and healthcare costs., (Copyright © 2018 Elsevier Ltd. All rights reserved.)

Percentage study weights in meta-analysis reveal the contribution of each study toward the overall summary results and are especially important when some studies are considered outliers or at high risk of bias. In meta-analyses of test accuracy reviews, such as a bivariate meta-analysis of sensitivity and specificity, the percentage study weights are not currently derived. Rather, the focus is on representing the precision of study estimates on receiver operating characteristic plots by scaling the points relative to the study sample size or to their standard error. In this article, we recommend that researchers should also provide the percentage study weights directly, and we propose a method to derive them based on a decomposition of Fisher information matrix. This method also generalises to a bivariate meta-regression so that percentage study weights can also be derived for estimates of study-level modifiers of test accuracy. Application is made to two meta-analyses examining test accuracy: one of ear temperature for diagnosis of fever in children and the other of positron emission tomography for diagnosis of Alzheimer's disease. These highlight that the percentage study weights provide important information that is otherwise hidden if the presentation only focuses on precision based on sample size or standard errors. Software code is provided for Stata, and we suggest that our proposed percentage weights should be routinely added on forest and receiver operating characteristic plots for sensitivity and specificity, to provide transparency of the contribution of each study toward the results. This has implications for the PRISMA-diagnostic test accuracy guidelines that are currently being produced., (Copyright © 2017 John Wiley & Sons, Ltd.)

Random-effects meta-analyses are very commonly used in medical statistics. Recent methodological developments include multivariate (multiple outcomes) and network (multiple treatments) meta-analysis. Here, we provide a new model and corresponding estimation procedure for multivariate network meta-analysis, so that multiple outcomes and treatments can be included in a single analysis. Our new multivariate model is a direct extension of a univariate model for network meta-analysis that has recently been proposed. We allow two types of unknown variance parameters in our model, which represent between-study heterogeneity and inconsistency. Inconsistency arises when different forms of direct and indirect evidence are not in agreement, even having taken between-study heterogeneity into account. However, the consistency assumption is often assumed in practice and so we also explain how to fit a reduced model which makes this assumption. Our estimation method extends several other commonly used methods for meta-analysis, including the method proposed by DerSimonian and Laird (). We investigate the use of our proposed methods in the context of both a simulation study and a real example., (© 2017, The Authors Biometrics published by Wiley Periodicals, Inc. on behalf of International Biometric Society.)

The use of data from multiple studies or centers for the validation of a clinical test or a multivariable prediction model allows researchers to investigate the test's/model's performance in multiple settings and populations. Recently, meta-analytic techniques have been proposed to summarize discrimination and calibration across study populations. Here, we rather consider performance in terms of net benefit, which is a measure of clinical utility that weighs the benefits of true positive classifications against the harms of false positives. We posit that it is important to examine clinical utility across multiple settings of interest. This requires a suitable meta-analysis method, and we propose a Bayesian trivariate random-effects meta-analysis of sensitivity, specificity, and prevalence. Across a range of chosen harm-to-benefit ratios, this provides a summary measure of net benefit, a prediction interval, and an estimate of the probability that the test/model is clinically useful in a new setting. In addition, the prediction interval and probability of usefulness can be calculated conditional on the known prevalence in a new setting. The proposed methods are illustrated by 2 case studies: one on the meta-analysis of published studies on ear thermometry to diagnose fever in children and one on the validation of a multivariable clinical risk prediction model for the diagnosis of ovarian cancer in a multicenter dataset. Crucially, in both case studies the clinical utility of the test/model was heterogeneous across settings, limiting its usefulness in practice. This emphasizes that heterogeneity in clinical utility should be assessed before a test/model is routinely implemented., (Copyright © 2018 John Wiley & Sons, Ltd.)

Background: Researchers and funders should consider the statistical power of planned Individual Participant Data (IPD) meta-analysis projects, as they are often time-consuming and costly. We propose simulation-based power calculations utilising a two-stage framework, and illustrate the approach for a planned IPD meta-analysis of randomised trials with continuous outcomes where the aim is to identify treatment-covariate interactions., Methods: The simulation approach has four steps: (i) specify an underlying (data generating) statistical model for trials in the IPD meta-analysis; (ii) use readily available information (e.g. from publications) and prior knowledge (e.g. number of studies promising IPD) to specify model parameter values (e.g. control group mean, intervention effect, treatment-covariate interaction); (iii) simulate an IPD meta-analysis dataset of a particular size from the model, and apply a two-stage IPD meta-analysis to obtain the summary estimate of interest (e.g. interaction effect) and its associated p-value; (iv) repeat the previous step (e.g. thousands of times), then estimate the power to detect a genuine effect by the proportion of summary estimates with a significant p-value., Results: In a planned IPD meta-analysis of lifestyle interventions to reduce weight gain in pregnancy, 14 trials (1183 patients) promised their IPD to examine a treatment-BMI interaction (i.e. whether baseline BMI modifies intervention effect on weight gain). Using our simulation-based approach, a two-stage IPD meta-analysis has < 60% power to detect a reduction of 1 kg weight gain for a 10-unit increase in BMI. Additional IPD from ten other published trials (containing 1761 patients) would improve power to over 80%, but only if a fixed-effect meta-analysis was appropriate. Pre-specified adjustment for prognostic factors would increase power further. Incorrect dichotomisation of BMI would reduce power by over 20%, similar to immediately throwing away IPD from ten trials., Conclusions: Simulation-based power calculations could inform the planning and funding of IPD projects, and should be used routinely.

Introduction: The aim of the QUIDS study is to develop a decision support tool for the management of women with symptoms and signs of preterm labour, based on a validated prognostic model using quantitative fetal fibronectin (fFN) concentration, in combination with clinical risk factors., Methods and Analysis: The study will evaluate the Rapid fFN 10Q System (Hologic, Marlborough, Massachusetts, USA) which quantifies fFN in a vaginal swab. In QUIDS part 2, we will perform a prospective cohort study in at least eight UK consultant-led maternity units, in women with symptoms of preterm labour at 22 +0  to 34 +6 weeks gestation to externally validate a prognostic model developed in QUIDS part 1. The effects of quantitative fFN on anxiety will be assessed, and acceptability of the test and prognostic model will be evaluated in a subgroup of women and clinicians (n=30). The sample size is 1600 women (with estimated 96-192 events of preterm delivery within 7 days of testing). Clinicians will be informed of the qualitative fFN result (positive/negative) but be blinded to quantitative fFN result. Research midwives will collect outcome data from the maternal and neonatal clinical records. The final validated prognostic model will be presented as a mobile or web-based application., Ethics and Dissemination: The study is funded by the National Institute of Healthcare Research Health Technology Assessment (HTA 14/32/01). It has been approved by the West of Scotland Research Ethics Committee (16/WS/0068)., Version: Protocol V.2, Date 1 November 2016., Trial Registration Number: ISRCTN 41598423andCPMS: 31277., Competing Interests: Competing interests: SJS and JEN work at the University of Edinburgh, who received £1000 sponsorship from Hologic to support a meeting (The Society of Reproductive Investigation and MRC Centre for Reproductive Health Scientific Symposium on Targeting Inflammation to Improve Reproductive Health across the Lifecourse – August 2017). AS has in the past (over last 5 years; not in the last 3 years) received funding for expenses related to advisory board and internal staff education from Hologic. MC received sponsorship from Hologic to organise an educational teaching focusing on prediction of Preterm Birth at the 2017 annual meeting of the British Maternal and Fetal Medicine Society. Hologic, the makers of fFN have provided analysers and technical support for their use to sites participating in the QUIDS prospective cohort study. They have no access to the data, or other involvement in the conduct, data analysis, interpretation of results or decision to publish the results of the study., (© Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2018. All rights reserved. No commercial use is permitted unless otherwise expressly granted.)

Introduction: The aim of the QUIDS study is to develop a decision support tool for the management of women with symptoms and signs of preterm labour, based on a validated prognostic model using quantitative fetal fibronectin (qfFN) concentration, in combination with clinical risk factors., Methods and Analysis: The study will evaluate the Rapid fFN 10Q System (Hologic, Marlborough, Massachusetts) which quantifies fFN in a vaginal swab. In part 1 of the study, we will develop and internally validate a prognostic model using an individual participant data (IPD) meta-analysis of existing studies containing women with symptoms of preterm labour alongside fFN measurements and pregnancy outcome. An economic analysis will be undertaken to assess potential cost-effectiveness of the qfFN prognostic model. The primary endpoint will be the ability of the prognostic model to rule out spontaneous preterm birth within 7 days. Six eligible studies were identified by systematic review of the literature and five agreed to provide their IPD (n=5 studies, 1783 women and 139 events of preterm delivery within 7 days of testing)., Ethics and Dissemination: The study is funded by the National Institute of Healthcare Research Health Technology Assessment (HTA 14/32/01). It has been approved by the West of Scotland Research Ethics Committee (16/WS/0068)., Prospero Registration Number: CRD42015027590., Version: Protocol version 2, date 1 November 2016., Competing Interests: Competing interests: SJS and JEN work at the University of Edinburgh, who received £1000 sponsorship from Hologic to support a meeting (Scientific Symposium Targeting Inflammation to Improve Reproductive Health across the Lifecourse (August 2017) in collaboration with The Society for Reproductive Investigation and MRC Centre for Reproductive Health). AS has in the past (over last 5 years; not in the last 3 years) received funding for expenses related to advisory board and internal staff education from Hologic. MC received sponsorship from Hologic to organise an educational teaching focusing on prediction of preterm birth at the 2017 annual meeting of the British Maternal and Fetal Medicine Society. Hologic, the makers of fFN, provided analysers and technical support for their use to sites participating in the QUIDS prospective cohort study. They have no access to the data, or other involvement in the conduct, analysis, interpretation or decision to publish the results of the study., (© Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2018. All rights reserved. No commercial use is permitted unless otherwise expressly granted.)

Small-study effects are a common threat in systematic reviews and may indicate publication bias. Their existence is often verified by visual inspection of the funnel plot. Formal tests to assess the presence of funnel plot asymmetry typically estimate the association between the reported effect size and their standard error, the total sample size, or the inverse of the total sample size. In this paper, we demonstrate that the application of these tests may be less appropriate in meta-analysis of survival data, where censoring influences statistical significance of the hazard ratio. We subsequently propose 2 new tests that are based on the total number of observed events and adopt a multiplicative variance component. We compare the performance of the various funnel plot asymmetry tests in an extensive simulation study where we varied the true hazard ratio (0.5 to 1), the number of published trials (N=10 to 100), the degree of censoring within trials (0% to 90%), and the mechanism leading to participant dropout (noninformative versus informative). Results demonstrate that previous well-known tests for detecting funnel plot asymmetry suffer from low power or excessive type-I error rates in meta-analysis of survival data, particularly when trials are affected by participant dropout. Because our novel test (adopting estimates of the asymptotic precision as study weights) yields reasonable power and maintains appropriate type-I error rates, we recommend its use to evaluate funnel plot asymmetry in meta-analysis of survival data. The use of funnel plot asymmetry tests should, however, be avoided when there are few trials available for any meta-analysis., (© 2017 The Authors. Research Synthesis Methods Published by John Wiley & Sons, Ltd.)

Introduction: For tests reporting continuous results, primary studies usually provide test performance at multiple but often different thresholds. This creates missing data when performing a meta-analysis at each threshold. A standard meta-analysis (no imputation [NI]) ignores such missing data. A single imputation (SI) approach was recently proposed to recover missing threshold results. Here, we propose a new method that performs multiple imputation of the missing threshold results using discrete combinations (MIDC)., Methods: The new MIDC method imputes missing threshold results by randomly selecting from the set of all possible discrete combinations which lie between the results for 2 known bounding thresholds. Imputed and observed results are then synthesised at each threshold. This is repeated multiple times, and the multiple pooled results at each threshold are combined using Rubin's rules to give final estimates. We compared the NI, SI, and MIDC approaches via simulation., Results: Both imputation methods outperform the NI method in simulations. There was generally little difference in the SI and MIDC methods, but the latter was noticeably better in terms of estimating the between-study variances and generally gave better coverage, due to slightly larger standard errors of pooled estimates. Given selective reporting of thresholds, the imputation methods also reduced bias in the summary receiver operating characteristic curve. Simulations demonstrate the imputation methods rely on an equal threshold spacing assumption. A real example is presented., Conclusions: The SI and, in particular, MIDC methods can be used to examine the impact of missing threshold results in meta-analysis of test accuracy studies., (© 2017 The Authors. Research Synthesis Methods published by John Wiley & Sons Ltd.)

Multivariate random-effects meta-analysis allows the joint synthesis of correlated results from multiple studies, for example, for multiple outcomes or multiple treatment groups. In a Bayesian univariate meta-analysis of one endpoint, the importance of specifying a sensible prior distribution for the between-study variance is well understood. However, in multivariate meta-analysis, there is little guidance about the choice of prior distributions for the variances or, crucially, the between-study correlation, ρ B ; for the latter, researchers often use a Uniform(-1,1) distribution assuming it is vague. In this paper, an extensive simulation study and a real illustrative example is used to examine the impact of various (realistically) vague prior distributions for ρ B and the between-study variances within a Bayesian bivariate random-effects meta-analysis of two correlated treatment effects. A range of diverse scenarios are considered, including complete and missing data, to examine the impact of the prior distributions on posterior results (for treatment effect and between-study correlation), amount of borrowing of strength, and joint predictive distributions of treatment effectiveness in new studies. Two key recommendations are identified to improve the robustness of multivariate meta-analysis results. First, the routine use of a Uniform(-1,1) prior distribution for ρ B should be avoided, if possible, as it is not necessarily vague. Instead, researchers should identify a sensible prior distribution, for example, by restricting values to be positive or negative as indicated by prior knowledge. Second, it remains critical to use sensible (e.g. empirically based) prior distributions for the between-study variances, as an inappropriate choice can adversely impact the posterior distribution for ρ B , which may then adversely affect inferences such as joint predictive probabilities. These recommendations are especially important with a small number of studies and missing data.

Introduction: Our previous study-level (aggregate data) meta-analysis suggested that vitamin D supplements may be beneficial for bone density specifically in children with vitamin D deficiency. However, the misclassification of vitamin D status inherent in study-level data means that the results are not definitive and cannot provide an accurate assessment of the size of any effect. Therefore, we propose to undertake an individual patient data (IPD) meta-analysis to determine whether the effect of vitamin D supplementation on bone density in children differs according to baseline vitamin D status, and to specifically estimate the effect of vitamin D in children who are vitamin D deficient., Methods and Analysis: This study has been designed to adhere to the Preferred Reporting Items for Systematic Review and Meta-Analyses of IPD statement. We will include randomised placebo-controlled trials of vitamin D supplementation reporting bone density outcomes at least 6 months after the study commenced in children and adolescents (aged <20 years) without coexistent medical conditions or treatments causing osteoporosis. We will update the search of the original review to cover the period 2009-2017, using the same methods as the original review. Fully anonymised data on all randomised patients will be requested. Outcomes will be femoral neck, total hip, lumbar spine and proximal and distal forearm bone mineral density, and total body bone mineral content. A two-stage IPD meta-analysis will be used to examine the effect of baseline serum 25-hydroxyvitamin D (25(OH)D) on treatment effect for each bone density outcome. Restricted maximum likelihood will be used to estimate the random-effects meta-analysis models, with 95% CI for summary effects. Heterogeneity will be assessed by I 2 and potential publication bias (small-study effects) and availability bias by funnel plots, Egger's test and Peter's test., Ethics and Dissemination: Ethics approval will not be required as the data are to be used for the primary purpose for which they were collected and all original individual studies had ethics approval. Results of the IPD meta-analysis will be submitted for publication in a peer-reviewed journal., Prospero Registration Number: CRD42017068772., Competing Interests: Competing interests: None declared., (© Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2018. All rights reserved. No commercial use is permitted unless otherwise expressly granted.)

Introduction: Knee and hip osteoarthritis (OA) is a leading cause of disability worldwide. Therapeutic exercise is a recommended core treatment for people with knee and hip OA, however, the observed effect sizes for reducing pain and improving physical function are small to moderate. This may be due to insufficient targeting of exercise to subgroups of people who are most likely to respond and/or suboptimal content of exercise programmes. This study aims to identify: (1) subgroups of people with knee and hip OA that do/do not respond to therapeutic exercise and to different types of exercise and (2) mediators of the effect of therapeutic exercise for reducing pain and improving physical function. This will enable optimal targeting and refining the content of future exercise interventions., Methods and Analysis: Systematic review and individual participant data meta-analyses. A previous comprehensive systematic review will be updated to identify randomised controlled trials that compare the effects of therapeutic exercise for people with knee and hip OA on pain and physical function to a non-exercise control. Lead authors of eligible trials will be invited to share individual participant data. Trial-level and participant-level characteristics (for baseline variables and outcomes) of included studies will be summarised. Meta-analyses will use a two-stage approach, where effect estimates are obtained for each trial and then synthesised using a random effects model (to account for heterogeneity). All analyses will be on an intention-to-treat principle and all summary meta-analysis estimates will be reported as standardised mean differences with 95% CI., Ethics and Dissemination: Research ethical or governance approval is exempt as no new data are being collected and no identifiable participant information will be shared. Findings will be disseminated via national and international conferences, publication in peer-reviewed journals and summaries posted on websites accessed by the public and clinicians., Prospero Registration Number: CRD42017054049., Competing Interests: Competing interests: None declared., (© Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2017. All rights reserved. No commercial use is permitted unless otherwise expressly granted.)

Multivariate and network meta-analysis have the potential for the estimated mean of one effect to borrow strength from the data on other effects of interest. The extent of this borrowing of strength is usually assessed informally. We present new mathematical definitions of 'borrowing of strength'. Our main proposal is based on a decomposition of the score statistic, which we show can be interpreted as comparing the precision of estimates from the multivariate and univariate models. Our definition of borrowing of strength therefore emulates the usual informal assessment. We also derive a method for calculating study weights, which we embed into the same framework as our borrowing of strength statistics, so that percentage study weights can accompany the results from multivariate and network meta-analyses as they do in conventional univariate meta-analyses. Our proposals are illustrated using three meta-analyses involving correlated effects for multiple outcomes, multiple risk factor associations and multiple treatments (network meta-analysis).

Background: Pre-eclampsia, a condition with raised blood pressure and proteinuria is associated with an increased risk of maternal and offspring mortality and morbidity. Early identification of mothers at risk is needed to target management., Methods/design: We aim to systematically review the existing literature to identify prediction models for pre-eclampsia. We have established the International Prediction of Pregnancy Complication Network (IPPIC), made up of 72 researchers from 21 countries who have carried out relevant primary studies or have access to existing registry databases, and collectively possess data from more than two million patients. We will use the individual participant data (IPD) from these studies to externally validate these existing prediction models and summarise model performance across studies using random-effects meta-analysis for any, late (after 34 weeks) and early (before 34 weeks) onset pre-eclampsia. If none of the models perform well, we will recalibrate (update), or develop and validate new prediction models using the IPD. We will assess the differential accuracy of the models in various settings and subgroups according to the risk status. We will also validate or develop prediction models based on clinical characteristics only; clinical and biochemical markers; clinical and ultrasound parameters; and clinical, biochemical and ultrasound tests., Discussion: Numerous systematic reviews with aggregate data meta-analysis have evaluated various risk factors separately or in combination for predicting pre-eclampsia, but these are affected by many limitations. Our large-scale collaborative IPD approach encourages consensus towards well developed, and validated prognostic models, rather than a number of competing non-validated ones. The large sample size from our IPD will also allow development and validation of multivariable prediction model for the relatively rare outcome of early onset pre-eclampsia., Trial Registration: The project was registered on Prospero on the 27 November 2015 with ID: CRD42015029349., Competing Interests: Not applicableNot applicableKM is Editor-in-Chief and KS is an associate editor for BMC Diagnostic and Prognostic Research. The other authors declare that they have no competing interests.Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

An important question for clinicians appraising a meta-analysis is: are the findings likely to be valid in their own practice-does the reported effect accurately represent the effect that would occur in their own clinical population? To this end we advance the concept of statistical validity-where the parameter being estimated equals the corresponding parameter for a new independent study. Using a simple ('leave-one-out') cross-validation technique, we demonstrate how we may test meta-analysis estimates for statistical validity using a new validation statistic, Vn, and derive its distribution. We compare this with the usual approach of investigating heterogeneity in meta-analyses and demonstrate the link between statistical validity and homogeneity. Using a simulation study, the properties of Vn and the Q statistic are compared for univariate random effects meta-analysis and a tailored meta-regression model, where information from the setting (included as model covariates) is used to calibrate the summary estimate to the setting of application. Their properties are found to be similar when there are 50 studies or more, but for fewer studies Vn has greater power but a higher type 1 error rate than Q. The power and type 1 error rate of Vn are also shown to depend on the within-study variance, between-study variance, study sample size, and the number of studies in the meta-analysis. Finally, we apply Vn to two published meta-analyses and conclude that it usefully augments standard methods when deciding upon the likely validity of summary meta-analysis estimates in clinical practice. © 2017 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd., (© 2017 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd.)

Competing Interests: Competing interests: We have read and understood the BMJ policy on declaration of interests and declare: none.

Hierarchical models such as the bivariate and hierarchical summary receiver operating characteristic (HSROC) models are recommended for meta-analysis of test accuracy studies. These models are challenging to fit when there are few studies and/or sparse data (for example zero cells in contingency tables due to studies reporting 100% sensitivity or specificity); the models may not converge, or give unreliable parameter estimates. Using simulation, we investigated the performance of seven hierarchical models incorporating increasing simplifications in scenarios designed to replicate realistic situations for meta-analysis of test accuracy studies. Performance of the models was assessed in terms of estimability (percentage of meta-analyses that successfully converged and percentage where the between study correlation was estimable), bias, mean square error and coverage of the 95% confidence intervals. Our results indicate that simpler hierarchical models are valid in situations with few studies or sparse data. For synthesis of sensitivity and specificity, univariate random effects logistic regression models are appropriate when a bivariate model cannot be fitted. Alternatively, an HSROC model that assumes a symmetric SROC curve (by excluding the shape parameter) can be used if the HSROC model is the chosen meta-analytic approach. In the absence of heterogeneity, fixed effect equivalent of the models can be applied.

Background: Diet- and physical activity-based interventions in pregnancy have the potential to alter maternal and child outcomes., Objectives: To assess whether or not the effects of diet and lifestyle interventions vary in subgroups of women, based on maternal body mass index (BMI), age, parity, Caucasian ethnicity and underlying medical condition(s), by undertaking an individual patient data (IPD) meta-analysis. We also evaluated the association of gestational weight gain (GWG) with adverse pregnancy outcomes and assessed the cost-effectiveness of the interventions., Data Sources: MEDLINE, EMBASE, Cochrane Central Register of Controlled Trials, Database of Abstracts of Reviews of Effects and Health Technology Assessment database were searched from October 2013 to March 2015 (to update a previous search)., Review Methods: Researchers from the International Weight Management in Pregnancy Collaborative Network shared the primary data. For each intervention type and outcome, we performed a two-step IPD random-effects meta-analysis, for all women (except underweight) combined and for each subgroup of interest, to obtain summary estimates of effects and 95% confidence intervals (CIs), and synthesised the differences in effects between subgroups. In the first stage, we fitted a linear regression adjusted for baseline (for continuous outcomes) or a logistic regression model (for binary outcomes) in each study separately; estimates were combined across studies using random-effects meta-analysis models. We quantified the relationship between weight gain and complications, and undertook a decision-analytic model-based economic evaluation to assess the cost-effectiveness of the interventions., Results: Diet and lifestyle interventions reduced GWG by an average of 0.70 kg (95% CI -0.92 to -0.48 kg; 33 studies, 9320 women). The effects on composite maternal outcome [summary odds ratio (OR) 0.90, 95% CI 0.79 to 1.03; 24 studies, 8852 women] and composite fetal/neonatal outcome (summary OR 0.94, 95% CI 0.83 to 1.08; 18 studies, 7981 women) were not significant. The effect did not vary with baseline BMI, age, ethnicity, parity or underlying medical conditions for GWG, and composite maternal and fetal outcomes. Lifestyle interventions reduce Caesarean sections (OR 0.91, 95% CI 0.83 to 0.99), but not other individual maternal outcomes such as gestational diabetes mellitus (OR 0.89, 95% CI 0.72 to 1.10), pre-eclampsia or pregnancy-induced hypertension (OR 0.95, 95% CI 0.78 to 1.16) and preterm birth (OR 0.94, 95% CI 0.78 to 1.13). There was no significant effect on fetal outcomes. The interventions were not cost-effective. GWG, including adherence to the Institute of Medicine-recommended targets, was not associated with a reduction in complications. Predictors of GWG were maternal age (summary estimate -0.10 kg, 95% CI -0.14 to -0.06 kg) and multiparity (summary estimate -0.73 kg, 95% CI -1.24 to -0.23 kg)., Limitations: The findings were limited by the lack of standardisation in the components of intervention, residual heterogeneity in effects across studies for most analyses and the unavailability of IPD in some studies., Conclusion: Diet and lifestyle interventions in pregnancy are clinically effective in reducing GWG irrespective of risk factors, with no effects on composite maternal and fetal outcomes., Future Work: The differential effects of lifestyle interventions on individual pregnancy outcomes need evaluation., Study Registration: This study is registered as PROSPERO CRD42013003804., Funding: The National Institute for Health Research Health Technology Assessment programme.

Background: In a random effects meta-analysis model, true treatment effects for each study are routinely assumed to follow a normal distribution. However, normality is a restrictive assumption and the misspecification of the random effects distribution may result in a misleading estimate of overall mean for the treatment effect, an inappropriate quantification of heterogeneity across studies and a wrongly symmetric prediction interval., Methods: We focus on problems caused by an inappropriate normality assumption of the random effects distribution, and propose a novel random effects meta-analysis model where a Box-Cox transformation is applied to the observed treatment effect estimates. The proposed model aims to normalise an overall distribution of observed treatment effect estimates, which is sum of the within-study sampling distributions and the random effects distribution. When sampling distributions are approximately normal, non-normality in the overall distribution will be mainly due to the random effects distribution, especially when the between-study variation is large relative to the within-study variation. The Box-Cox transformation addresses this flexibly according to the observed departure from normality. We use a Bayesian approach for estimating parameters in the proposed model, and suggest summarising the meta-analysis results by an overall median, an interquartile range and a prediction interval. The model can be applied for any kind of variables once the treatment effect estimate is defined from the variable., Results: A simulation study suggested that when the overall distribution of treatment effect estimates are skewed, the overall mean and conventional I 2 from the normal random effects model could be inappropriate summaries, and the proposed model helped reduce this issue. We illustrated the proposed model using two examples, which revealed some important differences on summary results, heterogeneity measures and prediction intervals from the normal random effects model., Conclusions: The random effects meta-analysis with the Box-Cox transformation may be an important tool for examining robustness of traditional meta-analysis results against skewness on the observed treatment effect estimates. Further critical evaluation of the method is needed.

Objective: Are first trimester serum pregnancy-associated plasma protein-A (PAPP-A), nuchal translucency (NT) and crown-rump length (CRL) prognostic factors for adverse pregnancy outcomes?, Method: Retrospective cohort, women, singleton pregnancies (UK 2011-2015). Unadjusted and multivariable logistic regression., Outcomes: small for gestational age (SGA), pre-eclampsia (PE), preterm birth (PTB), miscarriage, stillbirth, perinatal mortality and neonatal death (NND)., Results: A total of 12 592 pregnancies: 852 (6.8%) PTB, 352 (2.8%) PE, 1824 (14.5%) SGA, 73 (0.6%) miscarriages, 37(0.3%) stillbirths, 73 perinatal deaths (0.6%) and 38 (0.30%) NND. Multivariable analysis: lower odds of SGA [adjusted odds ratio (aOR) 0.88 (95% CI 0.85,0.91)], PTB [0.92 (95%CI 0.88,0.97)], PE [0.91 (95% CI 0.85,0.97)] and stillbirth [0.71 (95% CI 0.52,0.98)] as PAPP-A increases. Lower odds of SGA [aOR 0.79 (95% CI 0.70,0.89)] but higher odds of miscarriage [aOR 1.75 95% CI (1.12,2.72)] as NT increases, and lower odds of stillbirth as CRL increases [aOR 0.94 95% CI (0.89,0.99)]. Multivariable analysis of three factors together demonstrated strong associations: a) PAPP-A, NT, CRL and SGA, b) PAPP-A and PTB, c) PAPP-A, CRL and PE, d) NT and miscarriage., Conclusions: Pregnancy-associated plasma protein-A, NT and CRL are independent prognostic factors for adverse pregnancy outcomes, particularly PAPP-A and SGA with lower PAPP-A associated with increased risk. © 2017 John Wiley & Sons, Ltd., (© 2017 John Wiley & Sons, Ltd.)

Chronic obstructive pulmonary disease (COPD) is greatly underdiagnosed worldwide and more efficient methods of case-finding are required. We developed and externally validated a risk score to identify undiagnosed COPD using primary care records.We conducted a retrospective cohort analysis of a pragmatic cluster randomised controlled case-finding trial in the West Midlands, UK. Participants aged 40-79 years with no prior diagnosis of COPD received a postal or opportunistic screening questionnaire. Those reporting chronic respiratory symptoms were assessed with spirometry. COPD was defined as presence of relevant symptoms with a post-bronchodilator forced expiratory volume in 1 s/forced vital capacity ratio below the lower limit of normal. A risk score was developed using logistic regression with variables available from electronic health records for 2398 participants who returned a postal questionnaire. This was externally validated among 1097 participants who returned an opportunistic questionnaire to derive the c-statistic, and the sensitivity and specificity of cut-points.A risk score containing age, smoking status, dyspnoea, prescriptions of salbutamol and prescriptions of antibiotics discriminated between patients with and without undiagnosed COPD (c-statistic 0.74, 95% CI 0.68-0.80). A cut-point of ≥7.5% predicted risk had a sensitivity of 68.8% (95% CI 57.3-78.9%) and a specificity of 68.8% (95% CI 65.8.1-71.6%).A novel risk score using routine data from primary care electronic health records can identify patients at high risk for undiagnosed symptomatic COPD. This score could be integrated with clinical information systems to help primary care clinicians target patients for case-finding., Competing Interests: Conflict of interest: Disclosures can be found alongside this article at erj.ersjournals.com, (Copyright ©ERS 2017.)

Objective: To compare the primary care consulting behavior, prior to diagnosis, of people with systemic lupus erythematosus (SLE) with controls, and to develop and validate a risk prediction model to aid earlier SLE diagnosis., Methods: We included a total of 1,739 incident SLE cases practice-matched to 6,956 controls from the UK Clinical Practice Research Datalink. Using logistic regression, odds ratios were calculated for age, sex, consultation rates, selected presenting clinical features, and previous diagnoses in the 5 years preceding diagnosis date. A risk prediction model was developed from pre-selected variables using backward stepwise logistic regression. Model discrimination and calibration were tested in an independent validation cohort of 1,831,747 patients., Results: People with SLE had a significantly higher consultation rate than controls (median 9.2 versus 3.8 per year), which was in part attributable to clinical features that occur in SLE. The final risk prediction model included the variables age, sex, consultation rate, arthralgia or arthritis, rash, alopecia, sicca, Raynaud's phenomenon, serositis, and fatigue. The model discrimination and calibration in the validation sample was good (receiver operating characteristic curve 0.75, 95% confidence interval 0.73, 0.78). However, absolute risk predictions for SLE were typically less than 1% due to the rare nature of SLE., Conclusion: People with SLE consult their general practitioner more frequently and with clinical features attributable to SLE in the 5 years preceding diagnosis, suggesting that there are potential opportunities to reduce diagnostic delay in primary care. A risk prediction model was developed and validated that may be used to identify people at risk of SLE in future clinical practice., (© 2016, American College of Rheumatology.)

Introduction: The aim of this study was to determine the prognostic value of the first urinary albumin/creatinine ratio (ACR) for adverse maternal and neonatal outcomes and how it relates to other prognostic factors., Material and Methods: We performed a retrospective cohort study from December 2009 to February 2012 with analysis of demographic, clinical and biochemical data from two obstetric day assessment units in hospitals in Southeast Scotland. We included 717 pregnant women, with singleton pregnancies after 20 weeks' gestation, referred for evaluation of suspected preeclampsia and having their first ACR performed. The ability of ACR to predict future outcomes was assessed in both univariable and multivariable logistic regression models. The latter assessed its prognostic value independent of (adjusting for) existing prognostic factors. Primary outcome measures were maternal and neonatal composite adverse outcomes, and a secondary outcome was gestation at delivery., Results: In all, 204 women (28.5%) experienced a composite adverse maternal outcome and 146 women (20.4%) experienced a composite adverse neonatal outcome. Multivariate analysis of log-transformed ACR demonstrated that a 1-unit increase in log ACR is associated with an increased odds of adverse maternal [odds ratio 1.60, 95% confidence interval (CI) 1.45-1.80] and adverse neonatal (odds ratio 1.15, 95% CI 1.02-1.29) composite outcomes, and with reduced gestational age at delivery (coefficient: -0.46, 95% CI -0.54 to -0.38)., Conclusions: ACR is an independent prognostic factor for maternal and neonatal adverse outcomes in suspected preeclampsia. ACR may be useful to inform risk predictions within a prognostic model., (© 2017 The Authors. Acta Obstetricia et Gynecologica Scandinavica published by John Wiley & Sons Ltd on behalf of Nordic Federation of Societies of Obstetrics and Gynecology (NFOG).)

Background: The prognosis of early-onset pre-eclampsia (before 34 weeks' gestation) is variable. Accurate prediction of complications is required to plan appropriate management in high-risk women., Objective: To develop and validate prediction models for outcomes in early-onset pre-eclampsia., Design: Prospective cohort for model development, with validation in two external data sets., Setting: Model development: 53 obstetric units in the UK. Model transportability: PIERS (Pre-eclampsia Integrated Estimate of RiSk for mothers) and PETRA (Pre-Eclampsia TRial Amsterdam) studies., Participants: Pregnant women with early-onset pre-eclampsia., Sample Size: Nine hundred and forty-six women in the model development data set and 850 women (634 in PIERS, 216 in PETRA) in the transportability (external validation) data sets., Predictors: The predictors were identified from systematic reviews of tests to predict complications in pre-eclampsia and were prioritised by Delphi survey., Main Outcome Measures: The primary outcome was the composite of adverse maternal outcomes established using Delphi surveys. The secondary outcome was the composite of fetal and neonatal complications., Analysis: We developed two prediction models: a logistic regression model (PREP-L) to assess the overall risk of any maternal outcome until postnatal discharge and a survival analysis model (PREP-S) to obtain individual risk estimates at daily intervals from diagnosis until 34 weeks. Shrinkage was used to adjust for overoptimism of predictor effects. For internal validation (of the full models in the development data) and external validation (of the reduced models in the transportability data), we computed the ability of the models to discriminate between those with and without poor outcomes ( c -statistic), and the agreement between predicted and observed risk (calibration slope)., Results: The PREP-L model included maternal age, gestational age at diagnosis, medical history, systolic blood pressure, urine protein-to-creatinine ratio, platelet count, serum urea concentration, oxygen saturation, baseline treatment with antihypertensive drugs and administration of magnesium sulphate. The PREP-S model additionally included exaggerated tendon reflexes and serum alanine aminotransaminase and creatinine concentration. Both models showed good discrimination for maternal complications, with anoptimism-adjusted c -statistic of 0.82 [95% confidence interval (CI) 0.80 to 0.84] for PREP-L and 0.75 (95% CI 0.73 to 0.78) for the PREP-S model in the internal validation. External validation of the reduced PREP-L model showed good performance with a c -statistic of 0.81 (95% CI 0.77 to 0.85) in PIERS and 0.75 (95% CI 0.64 to 0.86) in PETRA cohorts for maternal complications, and calibrated well with slopes of 0.93 (95% CI 0.72 to 1.10) and 0.90 (95% CI 0.48 to 1.32), respectively. In the PIERS data set, the reduced PREP-S model had a c -statistic of 0.71 (95% CI 0.67 to 0.75) and a calibration slope of 0.67 (95% CI 0.56 to 0.79). Low gestational age at diagnosis, high urine protein-to-creatinine ratio, increased serum urea concentration, treatment with antihypertensive drugs, magnesium sulphate, abnormal uterine artery Doppler scan findings and estimated fetal weight below the 10th centile were associated with fetal complications., Conclusions: The PREP-L model provided individualised risk estimates in early-onset pre-eclampsia to plan management of high- or low-risk individuals. The PREP-S model has the potential to be used as a triage tool for risk assessment. The impacts of the model use on outcomes need further evaluation., Trial Registration: Current Controlled Trials ISRCTN40384046., Funding: The National Institute for Health Research Health Technology Assessment programme.

Background: Unexpected clinical deterioration before 34 weeks gestation is an undesired course in early-onset pre-eclampsia. To safely prolong preterm gestation, accurate and timely prediction of complications is required., Method: Women with confirmed early onset pre-eclampsia were recruited from 53 maternity units in the UK to a large prospective cohort study (PREP-946) for development of prognostic models for the overall risk of experiencing a complication using logistic regression (PREP-L), and for predicting the time to adverse maternal outcome using a survival model (PREP-S). External validation of the models were carried out in a multinational cohort (PIERS-634) and another cohort from the Netherlands (PETRA-216). Main outcome measures were C-statistics to summarise discrimination of the models and calibration plots and calibration slopes., Results: A total of 169 mothers (18%) in the PREP dataset had adverse outcomes by 48 hours, and 633 (67%) by discharge. The C-statistics of the models for predicting complications by 48 hours and by discharge were 0.84 (95% CI, 0.81-0.87; PREP-S) and 0.82 (0.80-0.84; PREP-L), respectively. The PREP-S model included maternal age, gestation, medical history, systolic blood pressure, deep tendon reflexes, urine protein creatinine ratio, platelets, serum alanine amino transaminase, urea, creatinine, oxygen saturation and treatment with antihypertensives or magnesium sulfate. The PREP-L model included the above except deep tendon reflexes, serum alanine amino transaminase and creatinine. On validation in the external PIERS dataset, the reduced PREP-S model showed reasonable calibration (slope 0.80) and discrimination (C-statistic 0.75) for predicting adverse outcome by 48 hours. Reduced PREP-L model showed excellent calibration (slope: 0.93 PIERS, 0.90 PETRA) and discrimination (0.81 PIERS, 0.75 PETRA) for predicting risk by discharge in the two external datasets., Conclusions: PREP models can be used to obtain predictions of adverse maternal outcome risk, including early preterm delivery, by 48 hours (PREP-S) and by discharge (PREP-L), in women with early onset pre-eclampsia in the context of current care. They have a potential role in triaging high-risk mothers who may need transfer to tertiary units for intensive maternal and neonatal care., Trial Registration: ISRCTN40384046 , retrospectively registered.

Stratified medicine utilizes individual-level covariates that are associated with a differential treatment effect, also known as treatment-covariate interactions. When multiple trials are available, meta-analysis is used to help detect true treatment-covariate interactions by combining their data. Meta-regression of trial-level information is prone to low power and ecological bias, and therefore, individual participant data (IPD) meta-analyses are preferable to examine interactions utilizing individual-level information. However, one-stage IPD models are often wrongly specified, such that interactions are based on amalgamating within- and across-trial information. We compare, through simulations and an applied example, fixed-effect and random-effects models for a one-stage IPD meta-analysis of time-to-event data where the goal is to estimate a treatment-covariate interaction. We show that it is crucial to centre patient-level covariates by their mean value in each trial, in order to separate out within-trial and across-trial information. Otherwise, bias and coverage of interaction estimates may be adversely affected, leading to potentially erroneous conclusions driven by ecological bias. We revisit an IPD meta-analysis of five epilepsy trials and examine age as a treatment effect modifier. The interaction is -0.011 (95% CI: -0.019 to -0.003; p = 0.004), and thus highly significant, when amalgamating within-trial and across-trial information. However, when separating within-trial from across-trial information, the interaction is -0.007 (95% CI: -0.019 to 0.005; p = 0.22), and thus its magnitude and statistical significance are greatly reduced. We recommend that meta-analysts should only use within-trial information to examine individual predictors of treatment effect and that one-stage IPD models should separate within-trial from across-trial information to avoid ecological bias. © 2016 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd., (© 2016 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd.)

Meta-analysis using individual participant data (IPD) obtains and synthesises the raw, participant-level data from a set of relevant studies. The IPD approach is becoming an increasingly popular tool as an alternative to traditional aggregate data meta-analysis, especially as it avoids reliance on published results and provides an opportunity to investigate individual-level interactions, such as treatment-effect modifiers. There are two statistical approaches for conducting an IPD meta-analysis: one-stage and two-stage. The one-stage approach analyses the IPD from all studies simultaneously, for example, in a hierarchical regression model with random effects. The two-stage approach derives aggregate data (such as effect estimates) in each study separately and then combines these in a traditional meta-analysis model. There have been numerous comparisons of the one-stage and two-stage approaches via theoretical consideration, simulation and empirical examples, yet there remains confusion regarding when each approach should be adopted, and indeed why they may differ. In this tutorial paper, we outline the key statistical methods for one-stage and two-stage IPD meta-analyses, and provide 10 key reasons why they may produce different summary results. We explain that most differences arise because of different modelling assumptions, rather than the choice of one-stage or two-stage itself. We illustrate the concepts with recently published IPD meta-analyses, summarise key statistical software and provide recommendations for future IPD meta-analyses. © 2016 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd., (© 2016 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd.)

Background: Early-onset pre-eclampsia with raised blood pressure and protein in the urine before 34 weeks' gestation is one of the leading causes of maternal deaths in the UK. The benefits to the child from prolonging the pregnancy need to be balanced against the risk of maternal deterioration. Accurate prediction models of risks are needed to plan management., Methods: We aim to undertake a multicentre prospective cohort study (Prediction of Risks in Early onset Pre-eclampsia (PREP)) to develop clinical prediction models in women with early-onset pre-eclampsia, for risk of adverse maternal outcomes by 48 h and by discharge. We will externally validate the models in two independent cohorts with 634 and 216 women. In the secondary analyses, we will assess risk of adverse fetal and neonatal outcomes at birth and by discharge., Discussion: The PREP study will quantify the risk of maternal complications at various time points and provide individualised estimates of overall risk in women with early-onset pre-eclampsia to plan the management., Trial Registration: ISRCTN registry, ISRCTN40384046., Competing Interests: The authors declare that they have no competing interests.

Background: Prognosis research studies (e.g. those deriving prognostic models or examining potential predictors of outcome) often collect information on time-varying predictors after their intended moment of use, sometimes using a measurement method different to that which would be used. We aimed to illustrate how estimates of predictor-outcome associations and prognostic model performance obtained from such studies may differ to those at the earlier, intended moment of use., Methods: We analysed data from two primary care cohorts of patients consulting for non-inflammatory musculoskeletal conditions: the Prognostic Research Study (PROG-RES: n  = 296, aged >50 years) and the Primary care Osteoarthritis Screening Trial (POST: n  = 756, >45 years). Both cohorts had collected comparable information on a potentially important time-varying predictor (current pain intensity: 0-10 numerical rating scale), other predictors (age, gender, practice) and outcome (patient-perceived non-recovery at 6 months). Using logistic regression models, we compared the direction and magnitude of predictor-outcome associations and model performance measures under two scenarios: (i) current pain intensity ascertained by the treating general practitioner in the consultation (the intended moment of use) and (ii) current pain intensity ascertained by a questionnaire mailed several days after the consultation., Results: In both cohorts, the predictor-outcome association was substantially weaker for pain measured at the consultation (OR (95% CI): PROG-RES 1.06 (0.95, 1.18); POST 1.04 (0.96, 1.12)) than for pain measured in the questionnaire (PROG-RES 1.34 (1.20, 1.48); POST 1.26 (1.18, 1.34)). The c -statistic of the multivariable model was lower when pain was measured at the consultation ( c -statistic (95% CI): PROG-RES 0.57 (0.51, 0.64); POST 0.66 (0.62, 0.70)) than when pain was measured in the questionnaire (PROG-RES 0.69 (0.63, 0.75); POST 0.72 (0.68, 0.76)), reflecting the lower OR for pain at the consultation., Conclusions: Prognostic research studies ideally should measure time-varying predictors at their intended moment of use and using the intended measurement method. Otherwise, they may produce substantially different estimates of predictor-outcome associations and model performance. Researchers should report when, how and where predictors were measured and identify any significant departures from their intended use that may limit the applicability of findings in practice., Trial Registration: The protocol for the PROG-RES cohort data collection and primary analysis has been published in an open-access journal (Mallen et al., BMC Musculoskelet Disord 7:84, 2006). The POST trial was registered (ISRCTN40721988; date of registration: 21 June 2011; date of enrolment of the first participant: 3 October 2011) and had a pre-specified protocol covering primary analysis. There was no published protocol for the current secondary analyses presented in this manuscript., Competing Interests: The authors declare that they have no competing interests.

A random effects meta-analysis combines the results of several independent studies to summarise the evidence about a particular measure of interest, such as a treatment effect. The approach allows for unexplained between-study heterogeneity in the true treatment effect by incorporating random study effects about the overall mean. The variance of the mean effect estimate is conventionally calculated by assuming that the between study variance is known; however, it has been demonstrated that this approach may be inappropriate, especially when there are few studies. Alternative methods that aim to account for this uncertainty, such as Hartung-Knapp, Sidik-Jonkman and Kenward-Roger, have been proposed and shown to improve upon the conventional approach in some situations. In this paper, we use a simulation study to examine the performance of several of these methods in terms of the coverage of the 95% confidence and prediction intervals derived from a random effects meta-analysis estimated using restricted maximum likelihood. We show that, in terms of the confidence intervals, the Hartung-Knapp correction performs well across a wide-range of scenarios and outperforms other methods when heterogeneity was large and/or study sizes were similar. However, the coverage of the Hartung-Knapp method is slightly too low when the heterogeneity is low (I 2  < 30%) and the study sizes are quite varied. In terms of prediction intervals, the conventional approach is only valid when heterogeneity is large (I 2  > 30%) and study sizes are similar. In other situations, especially when heterogeneity is small and the study sizes are quite varied, the coverage is far too low and could not be consistently improved by either increasing the number of studies, altering the degrees of freedom or using variance inflation methods. Therefore, researchers should be cautious in deriving 95% prediction intervals following a frequentist random-effects meta-analysis until a more reliable solution is identified. © 2016 The Authors. Statistics in Medicine Published by John Wiley & Sons Ltd., (© 2016 The Authors. Statistics in Medicine Published by John Wiley & Sons Ltd.)

Objectives: To compare different methods to handle treatment when developing a prognostic model that aims to produce accurate probabilities of the outcome of individuals if left untreated., Study Design and Setting: Simulations were performed based on two normally distributed predictors, a binary outcome, and a binary treatment, mimicking a randomized trial or an observational study. Comparison was made between simply ignoring treatment (SIT), restricting the analytical data set to untreated individuals (AUT), inverse probability weighting (IPW), and explicit modeling of treatment (MT). Methods were compared in terms of predictive performance of the model and the proportion of incorrect treatment decisions., Results: Omitting a genuine predictor of the outcome from the prognostic model decreased model performance, in both an observational study and a randomized trial. In randomized trials, the proportion of incorrect treatment decisions was smaller when applying AUT or MT, compared to SIT and IPW. In observational studies, MT was superior to all other methods regarding the proportion of incorrect treatment decisions., Conclusion: If a prognostic model aims to produce correct probabilities of the outcome in the absence of treatment, ignoring treatments that affect that outcome can lead to suboptimal model performance and incorrect treatment decisions. Explicitly, modeling treatment is recommended., (Copyright © 2016 Elsevier Inc. All rights reserved.)

Background: Many individuals with chronic obstructive pulmonary disease (COPD) remain undiagnosed worldwide. Health-care organisations are implementing case-finding programmes without good evidence of which are the most effective and cost-effective approaches. We assessed the effectiveness and cost-effectiveness of two alternative approaches to targeted case finding for COPD compared with routine practice., Methods: In this cluster-randomised controlled trial, participating general practices in the West Midlands, UK, were randomly assigned (1:1), via a computer-generated block randomisation sequence, to either a targeted case-finding group or a routine care group. Eligible patients were ever-smokers aged 40-79 years without a previously recorded diagnosis of COPD. Patients in the targeted case-finding group were further randomly assigned (1:1) via their household to receive either a screening questionnaire at the general practitioner (GP) consultation (opportunistic) or a screening questionnaire at the GP consultation plus a mailed questionnaire (active). Respondents reporting relevant respiratory symptoms were invited for post-bronchodilator spirometry. Patients, clinicians, and investigators were not masked to allocation, but group allocation was concealed from the researchers who performed the spirometry assessments. Primary outcomes were the percentage of the eligible population diagnosed with COPD within 1 year (defined as post-bronchodilator forced expiratory volume in 1 s [FEV1] to forced vital capacity [FVC] ratio <0·7 in patients with symptoms or a new diagnosis on their GP record) and cost per new COPD diagnosis. Multiple logistic and Poisson regression were used to estimate effect sizes. Costs were obtained from the trial. This trial is registered with ISRCTN, number ISRCTN14930255., Findings: From Aug 10, 2012, to June 22, 2014, 74 818 eligible patients from 54 diverse general practices were randomly assigned and completed the trial. At 1 year, 1278 (4%) cases of COPD were newly detected in 32 789 eligible patients in the targeted case-finding group compared with 337 (1%) cases in 42 029 patients in the routine care group (adjusted odds ratio [OR] 7·45 [95% CI 4·80-11·55], p<0·0001). The percentage of newly detected COPD cases was higher in the active case-finding group (822 [5%] of 15 378) than in the opportunistic case-finding group (370 [2%] of 15 387; adjusted OR 2·34 [2·06-2·66], p<0·0001; adjusted risk difference 2·9 per 100 patients [95% CI 2·3-3·6], p<0·0001). Active case finding was more cost-effective than opportunistic case finding (£333 vs £376 per case detected, respectively)., Interpretation: In this well established primary care system, routine practice identified few new cases of COPD. An active targeted approach to case finding including mailed screening questionnaires before spirometry is a cost-effective way to identify undiagnosed patients and has the potential to improve their health., Funding: National Institute for Health Research., (Copyright © 2016 Elsevier Ltd. All rights reserved.)