Search

Existing guidelines on respiratory syncytial virus (RSV) prophylaxis differ greatly by gestational age (GA) and other underlying risk factors, highlighting the data gaps in RSV disease burden among preterm infants. We will conduct a systematic review and individual participant data (IPD) meta-analysis of RSV global disease burden among preterm-born children. Three databases, Medline, Embase, and Global Health, will be searched for relevant studies on RSV disease burden for 2019 or before in preterm-born children published between 1 January 1995 and 31 December 2021. IPD will be sought by contacting the investigators identified from published literature and from existing collaboration networks. One-stage and 2-stage random-effects meta-analyses will be used to combine information from IPD and non-IPD studies to produce summary RSV burden estimates of incidence rate, hospital admission rate, and in-hospital case fatality ratio. The framework will be extended to examine subgroup(s) with the most substantial RSV disease burden.

In prediction model research, external validation is needed to examine an existing model's performance using data independent to that for model development. Current external validation studies often suffer from small sample sizes and consequently imprecise predictive performance estimates. To address this, we propose how to determine the minimum sample size needed for a new external validation study of a prediction model for a binary outcome. Our calculations aim to precisely estimate calibration (Observed/Expected and calibration slope), discrimination (C-statistic), and clinical utility (net benefit). For each measure, we propose closed-form and iterative solutions for calculating the minimum sample size required. These require specifying: (i) target SEs (confidence interval widths) for each estimate of interest, (ii) the anticipated outcome event proportion in the validation population, (iii) the prediction model's anticipated (mis)calibration and variance of linear predictor values in the validation population, and (iv) potential risk thresholds for clinical decision-making. The calculations can also be used to inform whether the sample size of an existing (already collected) dataset is adequate for external validation. We illustrate our proposal for external validation of a prediction model for mechanical heart valve failure with an expected outcome event proportion of 0.018. Calculations suggest at least 9835 participants (177 events) are required to precisely estimate the calibration and discrimination measures, with this number driven by the calibration slope criterion, which we anticipate will often be the case. Also, 6443 participants (116 events) are required to precisely estimate net benefit at a risk threshold of 8%. Software code is provided.

ObjectiveWhile many studies have consistently found incomplete reporting of regression-based prediction model studies, evidence is lacking for machine learning-based prediction model studies. We aim to systematically review the adherence of Machine Learning (ML)-based prediction model studies to the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) Statement.Study design and settingWe included articles reporting on development or external validation of a multivariable prediction model (either diagnostic or prognostic) developed using supervised ML for individualized predictions across all medical fields (PROSPERO, CRD42019161764). We searched PubMed from 1 January 2018 to 31 December 2019. Data extraction was performed using the 22-item checklist for reporting of prediction model studies (www.TRIPOD-statement.org). We measured the overall adherence per article and per TRIPOD item.ResultsOur search identified 24 814 articles, of which 152 articles were included: 94 (61.8%) prognostic and 58 (38.2%) diagnostic prediction model studies. Overall, articles adhered to a median of 38.7% (IQR 31.0-46.4) of TRIPOD items. No articles fully adhered to complete reporting of the abstract and very few reported the flow of participants (3.9%, 95% CI 1.8 to 8.3), appropriate title (4.6%, 95% CI 2.2 to 9.2), blinding of predictors (4.6%, 95% CI 2.2 to 9.2), model specification (5.2%, 95% CI 2.4 to 10.8), and model’s predictive performance (5.9%, 95% CI 3.1 to 10.9). There was often complete reporting of source of data (98.0%, 95% CI 94.4 to 99.3) and interpretation of the results (94.7%, 95% CI 90.0 to 97.3).ConclusionSimilar to prediction model studies developed using conventional regression-based techniques, the completeness of reporting is poor. Essential information to decide to use the model (i.e. model specification and its performance) is rarely reported. However, some items and sub-items of TRIPOD might be less suitable for ML-based prediction model studies and thus, TRIPOD requires extensions. Overall, there is an urgent need to improve the reporting quality and usability of research to avoid research waste.What is new?Key findings: Similar to prediction model studies developed using regression techniques, machine learning (ML)-based prediction model studies adhered poorly to the TRIPOD statement, the current standard reporting guideline.What this adds to what is known? In addition to efforts to improve the completeness of reporting in ML-based prediction model studies, an extension of TRIPOD for these type of studies is needed.What is the implication, what should change now? While TRIPOD-AI is under development, we urge authors to follow the recommendations of the TRIPOD statement to improve the completeness of reporting and reduce potential research waste of ML-based prediction model studies.

Objective: Stillbirth is a potentially preventable complication of pregnancy. Identifying women at risk can guide decisions on closer surveillance or timing of birth to prevent fetal death.Prognostic models have been developed to predict the risk of stillbirth, but none have yet been externally validated. We externally validated published prediction models for stillbirth using individual participant data (IPD) meta-analysis to assess their predictive performance. Methods: We searched Medline, EMBASE, DH-DATA and AMED databases from inception to December 2020 to identify stillbirth prediction models. We included studies that developed or updated prediction models for stillbirth for use at any time during pregnancy. IPD from cohorts within the International Prediction of Pregnancy Complication (IPPIC) Network were used to externally validate the identified prediction models whose individual variables were available in the IPD. We assessed the risk of bias of the models and IPD using PROBAST, and reported discriminative performance using the C-statistic, and calibration performance using calibration plots, calibration slopeand calibration-in-the-large. We estimated performance measures separately in each study, and then summarised across studies using random-effects meta-analysis. Clinical utility was assessed using net benefit. Results: We identified 17 studies reporting the development of 40 prognostic models for stillbirth. None of the models were previously externally validated, and only a fifth (20%, 8/40) reported the full model equation. We were able to validate three of these models using the IPD from 19 cohort studies (491,201 pregnant women) within the IPPIC Network database. Based on evaluating their development studies, all three models had an overall high risk of bias according to PROBAST. In our IPD meta-analysis, the models had summary C-statistics ranging from 0.53 to 0.65; summary calibration slopes of 0.40to 0.88, and generally with observed risks predictions that were too extreme compared to observed risks; and little to no clinical utility as assessed by net benefit. However, there remained uncertainty in performance for some models due to small available sample sizes. Conclusion: The three validated models generally showed poor and uncertain predictive performancein new data, with limited evidence to support their clinical application. Findings suggest methodological shortcomings in their development including overfitting of models. Further research is needed to further validate these and other models, identify stronger prognostic factors, and to develop more robust prediction models

OBJECTIVE: Stillbirth is a potentially preventable complication of pregnancy. Identifying women at high risk of stillbirth can guide decisions on the need for closer surveillance and timing of delivery in order to prevent fetal death. Prognostic models have been developed to predict the risk of stillbirth, but none has yet been validated externally. In this study, we externally validated published prediction models for stillbirth using individual participant data (IPD) meta-analysis to assess their predictive performance. METHODS: MEDLINE, EMBASE, DH-DATA and AMED databases were searched from inception to December 2020 to identify studies reporting stillbirth prediction models. Studies that developed or updated prediction models for stillbirth for use at any time during pregnancy were included. IPD from cohorts within the International Prediction of Pregnancy Complications (IPPIC) Network were used to validate externally the identified prediction models whose individual variables were available in the IPD. The risk of bias of the models and cohorts was assessed using the Prediction study Risk Of Bias ASsessment Tool (PROBAST). The discriminative performance of the models was evaluated using the C-statistic, and calibration was assessed using calibration plots, calibration slope and calibration-in-the-large. Performance measures were estimated separately in each cohort, as well as summarized across cohorts using random-effects meta-analysis. Clinical utility was assessed using net benefit. RESULTS: Seventeen studies reporting the development of 40 prognostic models for stillbirth were identified. None of the models had been previously validated externally, and the full model equation was reported for only one-fifth (20%, 8/40) of the models. External validation was possible for three of these models, using IPD from 19 cohorts (491 201 pregnant women) within the IPPIC Network database. Based on evaluation of the model development studies, all three models had an overall high risk of bias, according to PROBAST. In the IPD meta-analysis, the models had summary C-statistics ranging from 0.53 to 0.65 and summary calibration slopes ranging from 0.40 to 0.88, with risk predictions that were generally too extreme compared with the observed risks. The models had little to no clinical utility, as assessed by net benefit. However, there remained uncertainty in the performance of some models due to small available sample sizes. CONCLUSIONS: The three validated stillbirth prediction models showed generally poor and uncertain predictive performance in new data, with limited evidence to support their clinical application. The findings suggest methodological shortcomings in their development, including overfitting. Further research is needed to further validate these and other models, identify stronger prognostic factors and develop more robust prediction models. © 2021 The Authors. Ultrasound in Obstetrics & Gynecology published by John Wiley & Sons Ltd on behalf of International Society of Ultrasound in Obstetrics and Gynecology.

OBJECTIVE: Falls are one of the most common side effects associated with antihypertensive medication. In patients at high risk of falls, the additional risk of harm from medication may outweigh the potential benefits in terms of cardiovascular risk reduction. However, it is currently unclear which patients are at high risk of falls. This study aimed to develop and validate a clinical prediction model for risk of hospitalisation or death from falls in adults eligible for antihypertensive treatment. DESIGN AND METHOD: Eligible patients were aged 40 and above, with at least one blood pressure measurement between 130-179 mm Hg. The outcome was a fall resulting in hospitalisation or death within 10 years of entering the study. Model development was conducted in data from CPRD GOLD using a Fine-Gray approach, accounting for the competing risk of death from other causes, with subsequent recalibration at specific time-points using pseudo-values. External validation was conducted using data from CPRD Aurum, with performance assessed through calibration curves, the Observed/Expected (O/E) ratio, C-statistic, and D-statistic, each pooled across GP practices. RESULTS: Analysis included 1,773,224 patients (62,691 events) for model development, and 3,805,322 (206,956 events) for external validation. Conditional on other variables, increasing age, being female, of white ethnicity, being a heavy drinker and living in an area of high social deprivation were associated with an increased risk of falls. Upon external validation, the re-calibrated model showed good discrimination, with pooled C-statistics of 0.843 (95%CI: 0.841 to 0.844) and 0.833 (95%CI: 0.831 to 0.835) at 5 and 10 years respectively. Original model calibration was poor on visual inspection and whilst this improved with re-calibration, under-prediction of risk remained (Observed/Expected ratio 1.839 at 10 years, 95%CI: 1.811 to 1.865) (Figure 1). CONCLUSIONS: STRATIFY-Falls uses commonly recorded clinical characteristics and shows good discrimination on external validation, accurately distinguishing between patients at high and low risk of falls in the next 5-10 years. This model may be used in routine clinical practice to help identify those at high risk of falls, for whom the benefits of antihypertensive medication may be outweighed by the harms.

In 2019 we published a pair of articles inStatistics in Medicinethat describe how to calculate the minimum sample size for developing a multivariable prediction model with a continuous outcome, or with a binary or time‐to‐event outcome. As for any sample size calculation, the approach requires the user to specify anticipated values for key parameters. In particular, for a prediction model with a binary outcome, the outcome proportion and a conservative estimate for the overall fit of the developed model as measured by the Cox‐SnellR2(proportion of variance explained) must be specified. This proposal raises the question of how to identify a plausible value forR2in advance of model development. Our articles suggest researchers should identifyR2from closely related models already published in their field. In this letter, we present details on how to deriveR2using the reportedCstatistic (AUROC) for such existing prediction models with a binary outcome. TheCstatistic is commonly reported, and so our approach allows researchers to obtainR2for subsequent sample size calculations for new models. Stata and R code is provided, and a small simulation study.

The Grading of Recommendations, Assessment, Development and Evaluation (GRADE) approach to rating certainty in the results of research studies was initially developed for therapeutic questions. The approach considers: study design, risk of bias; inconsistency; imprecision; indirectness, publication bias; magnitude of effect; and dose-response. Questions about prognosis bear great relevance for decision-making in health care. Studies of prognosis can inform individuals about their likely outcome: for instance, in patients with a new diagnosis of cancer, are they likely to be alive or dead in five years. Further, prognostic studies can aid decisions in those considering treatment: for instance, is one's risk high enough to use medication to prevent cardiovascular disease. It follows that health care professionals and patients need to know how confident they can be regarding such prognostic estimates. We have previously provided guidance for using the GRADE approach to determine certainty in estimates of future events in broad categories of patients (overall prognosis). Prognostic studies may also provide more or less robust estimates of the association between patient characteristics (such as their age, sex, and coexisting illness) and undesirable or desirable outcomes. GRADE's approach to certainty of the evidence aims to inform clinicians and patients of the trustworthiness of the estimates from systematic reviews of studies addressing such individual prognostic factors.

Objectives\ud To develop and validate a prediction model for fat mass in children aged 4-15 years using routinely available risk factors of height, weight, and demographic information without the need for more complex forms of assessment.\ud \ud Design\ud Individual participant data meta-analysis.\ud \ud Setting\ud Four population based cross sectional studies and a fifth study for external validation, United Kingdom.\ud \ud Participants\ud A pooled derivation dataset (four studies) of 2375 children and an external validation dataset of 176 children with complete data on anthropometric measurements and deuterium dilution assessments of fat mass.\ud \ud Main outcome measure\ud Multivariable linear regression analysis, using backwards selection for inclusion of predictor variables and allowing non-linear relations, was used to develop a prediction model for fat-free mass (and subsequently fat mass by subtracting resulting estimates from weight) based on the four studies. Internal validation and then internal-external cross validation were used to examine overfitting and generalisability of the model’s predictive performance within the four development studies; external validation followed using the fifth dataset.\ud \ud Results\ud Model derivation was based on a multi-ethnic population of 2375 children (47.8% boys, n=1136) aged 4-15 years. The final model containing predictor variables of height, weight, age, sex, and ethnicity had extremely high predictive ability (optimism adjusted R2: 94.8%, 95% confidence interval 94.4% to 95.2%) with excellent calibration of observed and predicted values. The internal validation showed minimal overfitting and good model generalisability, with excellent calibration and predictive performance. External validation in 176 children aged 11-12 years showed promising generalisability of the model (R2: 90.0%, 95% confidence interval 87.2% to 92.8%) with good calibration of observed and predicted fat mass (slope: 1.02, 95% confidence interval 0.97 to 1.07). The mean difference between observed and predicted fat mass was −1.29 kg (95% confidence interval −1.62 to −0.96 kg).\ud \ud Conclusion\ud The developed model accurately predicted levels of fat mass in children aged 4-15 years. The prediction model is based on simple anthropometric measures without the need for more complex forms of assessment and could improve the accuracy of assessments for body fatness in children (compared with those provided by body mass index) for effective surveillance, prevention, and management of clinical and public health obesity.

Multivariate random-effects meta-analysis allows the joint synthesis of correlated results from multiple studies, for example, for multiple outcomes or multiple treatment groups. In a Bayesian univariate meta-analysis of one endpoint, the importance of specifying a sensible prior distribution for the between-study variance is well understood. However, in multivariate meta-analysis, there is little guidance about the choice of prior distributions for the variances or, crucially, the between-study correlation, ρB; for the latter, researchers often use a Uniform(-1,1) distribution assuming it is vague. In this paper, an extensive simulation study and a real illustrative example is used to examine the impact of various (realistically) vague prior distributions for ρB and the between-study variances within a Bayesian bivariate random-effects meta-analysis of two correlated treatment effects. A range of diverse scenarios are considered, including complete and missing data, to examine the impact of the prior distributions on posterior results (for treatment effect and between-study correlation), amount of borrowing of strength, and joint predictive distributions of treatment effectiveness in new studies. Two key recommendations are identified to improve the robustness of multivariate meta-analysis results. First, the routine use of a Uniform(-1,1) prior distribution for ρB should be avoided, if possible, as it is not necessarily vague. Instead, researchers should identify a sensible prior distribution, for example, by restricting values to be positive or negative as indicated by prior knowledge. Second, it remains critical to use sensible (e.g. empirically based) prior distributions for the between-study variances, as an inappropriate choice can adversely impact the posterior distribution for ρB, which may then adversely affect inferences such as joint predictive probabilities. These recommendations are especially important with a small number of studies and missing data.

Univariate meta‐analysis concerns a single outcome of interest measured across a number of independent studies. However, many research studies will have also measured secondary outcomes. Multivariate meta‐analysis allows us to take these secondary outcomes into account and can also include studies where the primary outcome is missing. We define the efficiency E as the variance of the overall estimate from a multivariate meta‐analysis relative to the variance of the overall estimate from a univariate meta‐analysis. The extra information gained from a multivariate meta‐analysis of n studies is then similar to the extra information gained if a univariate meta‐analysis of the primary effect had a further n(1−E)/E studies. The variance contribution of a study's secondary outcomes (its borrowing of strength) can be thought of as a contrast between the variance matrix of the outcomes in that study and the set of variance matrices of all the studies in the meta‐analysis. In the bivariate case this is given a simple graphical interpretation as the borrowing‐of‐strength plot. We discuss how these findings can also be used in the context of random‐effects meta‐analysis. Our discussion is motivated by a published meta‐analysis of 10 antihypertension clinical trials.

Small-study effects are a common threat in systematic reviews and may indicate publication bias. Their existence is often verified by visual inspection of the funnel plot. Formal tests to assess the presence of funnel plot asymmetry typically estimate the association between the reported effect size and their standard error, the total sample size, or the inverse of the total sample size. In this paper, we demonstrate that the application of these tests may be less appropriate in meta-analysis of survival data, where censoring influences statistical significance of the hazard ratio. We subsequently propose 2 new tests that are based on the total number of observed events and adopt a multiplicative variance component. We compare the performance of the various funnel plot asymmetry tests in an extensive simulation study where we varied the true hazard ratio (0.5 to 1), the number of published trials (N=10 to 100), the degree of censoring within trials (0% to 90%), and the mechanism leading to participant dropout (noninformative versus informative). Results demonstrate that previous well-known tests for detecting funnel plot asymmetry suffer from low power or excessive type-I error rates in meta-analysis of survival data, particularly when trials are affected by participant dropout. Because our novel test (adopting estimates of the asymptotic precision as study weights) yields reasonable power and maintains appropriate type-I error rates, we recommend its use to evaluate funnel plot asymmetry in meta-analysis of survival data. The use of funnel plot asymmetry tests should, however, be avoided when there are few trials available for any meta-analysis.

Objectives To explain methods for data synthesis of evidence from diagnostic test accuracy (DTA) studies, and to illustrate different types of analyses that may be performed in a DTA systematic review.\ud \ud Methods We described properties of meta-analytic methods for quantitative synthesis of evidence. We used a DTA review comparing the accuracy of three screening questionnaires for bipolar disorder to illustrate application of the methods for each type of analysis.\ud \ud Results The discriminatory ability of a test is commonly expressed in terms of sensitivity (proportion of those with the condition who test positive) and specificity (proportion of those without the condition who test negative). There is a trade-off between sensitivity and specificity, as an increasing threshold for defining test positivity will decrease sensitivity and increase specificity. Methods recommended for meta-analysis of DTA studies --such as the bivariate or hierarchical summary receiver operating characteristic (HSROC) model --jointly summarise sensitivity and specificity while taking into account this threshold effect, as well as allowing for between study differences in test performance beyond what would be expected by chance. The bivariate model focuses on estimation of a summary sensitivity and specificity at a common threshold while the HSROC model focuses on the estimation of a summary curve from studies that have used different thresholds.\ud \ud Conclusions Meta-analyses of diagnostic accuracy studies can provide answers to important clinical questions. We hope this article will provide clinicians with sufficient understanding of the terminology and methods to aid interpretation of systematic reviews and facilitate better patient care.

Multivariate meta-analysis, which jointly analyzes multiple and possibly correlated outcomes in a single analysis, is becoming increasingly popular in recent years. An attractive feature of the multivariate meta-analysis is its ability to account for the dependence between multiple estimates from the same study. However, standard inference procedures for multivariate meta-analysis require the knowledge of within-study correlations, which are usually unavailable. This limits standard inference approaches in practice. Riley et al. proposed a working model and an overall synthesis correlation parameter to account for the marginal correlation between outcomes, where the only data needed are those required for a separate univariate random-effects meta-analysis. As within-study correlations are not required, the Riley method is applicable to a wide variety of evidence synthesis situations. However, the standard variance estimator of the Riley method is not entirely correct under many important settings. As a consequence, the coverage of a function of pooled estimates may not reach the nominal level even when the number of studies in the multivariate meta-analysis is large. In this paper, we improve the Riley method by proposing a robust variance estimator, which is asymptotically correct even when the model is misspecified (i.e., when the likelihood function is incorrect). Simulation studies of a bivariate meta-analysis, in a variety of settings, show a function of pooled estimates has improved performance when using the proposed robust variance estimator. In terms of individual pooled estimates themselves, the standard variance estimator and robust variance estimator give similar results to the original method, with appropriate coverage. The proposed robust variance estimator performs well when the number of studies is relatively large. Therefore, we recommend the use of the robust method for meta-analyses with relatively large number of studies (e.g., m ≥ 50 ). When the sample size is relatively small, we recommend the use of the robust method under the working independence assumption. We illustrate the proposed method through two meta-analyses.

An important question for clinicians appraising a meta-analysis is: are the findings likely to be valid in their own practice-does the reported effect accurately represent the effect that would occur in their own clinical population? To this end we advance the concept of statistical validity-where the parameter being estimated equals the corresponding parameter for a new independent study. Using a simple ('leave-one-out') cross-validation technique, we demonstrate how we may test meta-analysis estimates for statistical validity using a new validation statistic, Vn, and derive its distribution. We compare this with the usual approach of investigating heterogeneity in meta-analyses and demonstrate the link between statistical validity and homogeneity. Using a simulation study, the properties of Vn and the Q statistic are compared for univariate random effects meta-analysis and a tailored meta-regression model, where information from the setting (included as model covariates) is used to calibrate the summary estimate to the setting of application. Their properties are found to be similar when there are 50 studies or more, but for fewer studies Vn has greater power but a higher type 1 error rate than Q. The power and type 1 error rate of Vn are also shown to depend on the within-study variance, between-study variance, study sample size, and the number of studies in the meta-analysis. Finally, we apply Vn to two published meta-analyses and conclude that it usefully augments standard methods when deciding upon the likely validity of summary meta-analysis estimates in clinical practice. © 2017 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd.

OBJECTIVE: Are first trimester serum pregnancy-associated plasma protein-A (PAPP-A), nuchal translucency (NT) and crown rump length (CRL) prognostic factors for adverse pregnancy outcomes? METHOD: Retrospective cohort women, singleton pregnancies (UK 2011-2015). Unadjusted and multivariable logistic regression, outcomes: small for gestational age (SGA), pre-eclampsia (PE), pre-term birth (PTB), miscarriage, stillbirth, perinatal mortality and neonatal death (NND). RESULTS: 12,592 pregnancies: 852 (6.8%) PTB, 352 (2.8%) PE, 1824 (14.5%) SGA, 73 (0.6%) miscarriages, 37(0.3%) stillbirths, 73 perinatal deaths (0.6%) and 38 (0.30%) NND. Multivariable analysis: lower odds of SGA [adjusted odds ratio (aOR) 0.88 (95% CI 0.85,0.91)], PTB [0.92 (95%CI 0.88,0.97)], PE [0.91 (95% CI 0.85,0.97)] and stillbirth [ 0.71 (95% CI 0.52,0.98)] as PAPP-A increases. Lower odds of SGA [aOR 0.79 (95% CI 0.70,0.89)] but higher odds of miscarriage [aOR 1.75 95% CI (1.12,2.72)] as NT increases, and lower odds of stillbirth as CRL increases [aOR 0.94 95% CI (0.89,0.99)]. Multivariable analysis of three factors together demonstrated strong associations: a) PAPP-A, NT, CRL and SGA, b) PAPP-A and PTB, c) PAPP-A, CRL and PE, d) NT and miscarriage. CONCLUSIONS: PAPP-A, NT and CRL independent prognostic factor for adverse pregnancy outcomes, especially PAPP-A and SGA with lower PAPP-A associated with increased risk.

[This corrects the article DOI: 10.1186/s41512-016-0001-y.].

Stratified medicine utilizes individual-level covariates that are associated with a differential treatment effect, also known as treatment-covariate interactions. When multiple trials are available, meta-analysis is used to help detect true treatment-covariate interactions by combining their data. Meta-regression of trial-level information is prone to low power and ecological bias, and therefore, individual participant data (IPD) meta-analyses are preferable to examine interactions utilizing individual-level information. However, one-stage IPD models are often wrongly specified, such that interactions are based on amalgamating within- and across-trial information. We compare, through simulations and an applied example, fixed-effect and random-effects models for a one-stage IPD meta-analysis of time-to-event data where the goal is to estimate a treatment-covariate interaction. We show that it is crucial to centre patient-level covariates by their mean value in each trial, in order to separate out within-trial and across-trial information. Otherwise, bias and coverage of interaction estimates may be adversely affected, leading to potentially erroneous conclusions driven by ecological bias. We revisit an IPD meta-analysis of five epilepsy trials and examine age as a treatment effect modifier. The interaction is -0.011 (95% CI: -0.019 to -0.003; p = 0.004), and thus highly significant, when amalgamating within-trial and across-trial information. However, when separating within-trial from across-trial information, the interaction is -0.007 (95% CI: -0.019 to 0.005; p = 0.22), and thus its magnitude and statistical significance are greatly reduced. We recommend that meta-analysts should only use within-trial information to examine individual predictors of treatment effect and that one-stage IPD models should separate within-trial from across-trial information to avoid ecological bias. © 2016 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd.

The purpose of this research was to employ a double-blind placebo control design of iontophoretically administered lidocaine to assess the test-retest reliability and validity of a magnitude matching procedure using a pressure algometer. In studies in the literature on assessment of pain in temporomandibular disorders (TMD), the pressure algometer has been found to be a reliable means of measuring pressure-pain threshold and a sensitive measure of treatment-response in the laboratory. However, previous research with this apparatus has not employed more sophisticated psychophysics. This study employs a magnitude matching psychophysical protocol to obtain a multidimensional pain report. Test-retest reliability over two occasions, 3-8 days apart, was found to be moderate for discriminability (r = .71, p < .01) but poor for response bias (r = .44). The validity study used iontophoresis as an anesthetic in a double-blind placebo and no-treatment control design. Although it was hypothesized that subjects in the anesthesia group would demonstrate reduced discriminability as compared with the control groups, no differences were found among the three groups on this measure. However, differences in response bias were found, with both the placebo control and no-treatment control groups differing from the experimental group but not from each other. Possible explanations and the implications of these findings are discussed.