Search

Stratified medicine utilizes individual-level covariates that are associated with a differential treatment effect, also known as treatment-covariate interactions. When multiple trials are available, meta-analysis is used to help detect true treatment-covariate interactions by combining their data. Meta-regression of trial-level information is prone to low power and ecological bias, and therefore, individual participant data (IPD) meta-analyses are preferable to examine interactions utilizing individual-level information. However, one-stage IPD models are often wrongly specified, such that interactions are based on amalgamating within- and across-trial information. We compare, through simulations and an applied example, fixed-effect and random-effects models for a one-stage IPD meta-analysis of time-to-event data where the goal is to estimate a treatment-covariate interaction. We show that it is crucial to centre patient-level covariates by their mean value in each trial, in order to separate out within-trial and across-trial information. Otherwise, bias and coverage of interaction estimates may be adversely affected, leading to potentially erroneous conclusions driven by ecological bias. We revisit an IPD meta-analysis of five epilepsy trials and examine age as a treatment effect modifier. The interaction is -0.011 (95% CI: -0.019 to -0.003; p = 0.004), and thus highly significant, when amalgamating within-trial and across-trial information. However, when separating within-trial from across-trial information, the interaction is -0.007 (95% CI: -0.019 to 0.005; p = 0.22), and thus its magnitude and statistical significance are greatly reduced. We recommend that meta-analysts should only use within-trial information to examine individual predictors of treatment effect and that one-stage IPD models should separate within-trial from across-trial information to avoid ecological bias. © 2016 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd.

Background Meta‐analyses based on individual participant data (IPD‐MAs) allow more powerful and uniformly consistent analyses as well as better characterisation of subgroups and outcomes, compared to those which are based on aggregate data (AD‐MAs) extracted from published trial reports. However, IPD‐MAs are a larger undertaking requiring greater resources than AD‐MAs. Researchers have compared results from IPD‐MA against results obtained from AD‐MA and reported conflicting findings. We present a methodology review to summarise this empirical evidence . Objectives To review systematically empirical comparisons of meta‐analyses of randomised trials based on IPD with those based on AD extracted from published reports, to evaluate the level of agreement between IPD‐MA and AD‐MA and whether agreement is affected by differences in type of effect measure, trials and participants included within the IPD‐MA and AD‐MA, and whether analyses were undertaken to explore the main effect of treatment or a treatment effect modifier. Search methods An electronic search of the Cochrane Library (includes Cochrane Database of Systematic Reviews, Database of Abstracts of Reviews of Effectiveness, CENTRAL, Cochrane Methodology Register, HTA database, NHS Economic Evaluations Database), MEDLINE, and Embase was undertaken up to 7 January 2016. Potentially relevant articles that were known to any of the review authors and reference lists of retrieved articles were also checked. Selection criteria Studies reporting an empirical comparison of the results of meta‐analyses of randomised trials using IPD with those using AD. Studies were included if sufficient numerical data, comparing IPD‐MA and AD‐MA, were available in their reports. Data collection and analysis Two review authors screened the title and abstract of identified studies with full‐text publications retrieved for those identified as eligible or potentially eligible. A ‘quality’ assessment was done and data were extracted independently by two review authors with disagreements resolved by involving a third author. Data were summarised descriptively for comparisons where an estimate of effect measure and corresponding precision have been provided both for IPD‐MA and for AD‐MA in the study report. Comparisons have been classified according to whether identical effect measures, identical trials and patients had been used in the IPD‐MA and the AD‐MA, and whether the analyses were undertaken to explore the main effect of treatment, or to explore a potential treatment effect modifier. Effect measures were transformed to a standardised scale (z scores) and scatter plots generated to allow visual comparisons. For each comparison, we compared the statistical significance (at the 5% two‐sided level) of an IPD‐MA compared to the corresponding AD‐MA and calculated the number of discrepancies. We examined discrepancies by type of analysis (main effect or modifier) and according to whether identical trials, patients and effect measures had been used by the IPD‐MA and AD‐MA. We calculated the average of differences between IPD‐MA and AD‐MA (z scores, ratio effect estimates and standard errors (of ratio effects)) and 95% limits of agreement. Main results From the 9330 reports found by our searches, 39 studies were eligible for this review with effect estimate and measure of precision extracted for 190 comparisons of IPD‐MA and AD‐MA. We classified the quality of studies as ‘no important flaws’ (29 (74%) studies) or ‘possibly important flaws’ (10 (26%) studies). A median of 4 (interquartile range (IQR): 2 to 6) comparisons were made per study, with 6 (IQR 4 to 11) trials and 1225 (542 to 2641) participants in IPD‐MAs and 7 (4 to 11) and 1225 (705 to 2541) for the AD‐MAs. One hundred and forty‐four (76%) comparisons were made on the main treatment effect meta‐analysis and 46 (24%) made using results from analyses to explore treatment effect modifiers. There is agreement in statistical significance between the IPD‐MA and AD‐MA for 152 (80%) comparisons, 23 of which disagreed in direction of effect. There is disagreement in statistical significance for 38 (20%) comparisons with an excess proportion of IPD‐MA detecting a statistically significant result that was not confirmed with AD‐MA (28 (15%)), compared with 10 (5%) comparisons with a statistically significant AD‐MA that was not confirmed by IPD‐MA. This pattern of disagreement is consistent for the 144 main effect analyses but not for the 46 comparisons of treatment effect modifier analyses. Conclusions from some IPD‐MA and AD‐MA differed even when based on identical trials, participants (but not necessarily identical follow‐up) and treatment effect measures. The average difference between IPD‐MA and AD‐MA in z scores, ratio effect estimates and standard errors is small but limits of agreement are wide and include important differences in both directions. Discrepancies between IPD‐MA and AD‐MA do not appear to increase as the differences between trials and participants increase. Authors' conclusions IPD offers the potential to explore additional, more thorough, and potentially more appropriate analyses compared to those possible with AD. But in many cases, similar results and conclusions can be drawn from IPD‐MA and AD‐MA. Therefore, before embarking on a resource‐intensive IPD‐MA, an AD‐MA should initially be explored and researchers should carefully consider the potential added benefits of IPD.

BACKGROUND: Individual participant data (IPD) from completed clinical trials should be responsibly shared to support efficient clinical research, generate new knowledge and bring benefit to patients. The Medical Research Council (MRC) Hubs for Trials Methodology Research (HTMR) has developed guidance to facilitate the sharing of IPD from publicly funded clinical trials.METHODS: Development of the guidance was completed over four phases which included a focussed review of policy documents, a web-based survey of the UK Clinical Research Collaboration (CRC) Registered Clinical Trials Units (CTU) Network, participation of an expert committee and an open consultation with the UKCRC Registered CTU Network. The project was funded by the MRC HTMR (MR/L004933/1-R39).RESULTS: Good practice principles include: (i) the use of a controlled access approach, using a transparent and robust system to review requests and provide secure data access; (ii) seeking consent for sharing IPD from trial participants in all future clinical trials with adequate assurance that patient privacy and confidentiality can be maintained; and (iii) establishing an approach to resource the sharing of IPD which would include support from trial funders, sponsor organisations and users of IPD. The guidance has been endorsed by Cancer Research UK, MRC Methodology Research Programme Advisory Group, Wellcome Trust and the Executive Group of the UKCRC Registered CTU Network. The National Institute for Health Research (NIHR) has confirmed it is supportive of the application of this guidance.CONCLUSIONS: Implementation of these principles will improve transparency, increase the coherent sharing of IPD from publicly funded trials, and help publicly funded trials to adhere to trial funder and journal requirements for data sharing.

It has been suggested that providing consumers with smaller dishware may prove an effective way of helping people eat less and preventing weight gain, but experimental evidence supporting this has been mixed. The objective of the present work was to examine the current evidence base for whether experimentally manipulated differences in dishware size influence food consumption. We systematically reviewed studies that experimentally manipulated the dishware size participants served themselves at a meal with and measured subsequent food intake. We used inverse variance meta‐analysis, calculating the standardized mean difference (SMD ) in food intake between smaller and larger dishware size conditions. Nine experiments from eight publications were eligible for inclusion. The majority of experiments found no significance difference in food intake when participants ate from smaller vs. larger dishware. With all available data included, analysis indicated a marginal effect of dishware size on food intake, with larger dishware size associated with greater intake. However, this effect was small and there was a large amount of heterogeneity across studies (SMD : −0.18, 95% confidence interval: −0.35, 0.00, I 2 = 77%). Evidence to date does not show that dishware size has a consistent effect on food intake, so recommendations surrounding the use of smaller plates/dishware to improve public health may be premature.

ObjectivesTo demonstrate how individual participant data (IPD) meta-analyses have impacted directly on the design and conduct of trials and highlight other advantages IPD might offer.Study Design and SettingPotential examples of the impact of IPD meta-analyses on trials were identified at an international workshop, attended by individuals with experience in the conduct of IPD meta-analyses and knowledge of trials in their respective clinical areas. Experts in the field who did not attend were asked to provide any further examples. We then examined relevant trial protocols, publications, and Web sites to verify the impacts of the IPD meta-analyses. A subgroup of workshop attendees sought further examples and identified other aspects of trial design and conduct that may inform IPD meta-analyses.ResultsWe identified 52 examples of IPD meta-analyses thought to have had a direct impact on the design or conduct of trials. After screening relevant trial protocols and publications, we identified 28 instances where IPD meta-analyses had clearly impacted on trials. They have influenced the selection of comparators and participants, sample size calculations, analysis and interpretation of subsequent trials, and the conduct and analysis of ongoing trials, sometimes in ways that would not possible with systematic reviews of aggregate data. We identified additional potential ways that IPD meta-analyses could be used to influence trials.ConclusionsIPD meta-analysis could be better used to inform the design, conduct, analysis, and interpretation of trials.