Search

Abstract EU-US data transfers for health research remain a particularly thorny issue in view of the stringent rules of the EU General Data Protection Regulation (GDPR) and the challenges related to US mass surveillance programs, particularly the manner in which US law enforcement and national security agencies can access personal data originating from the EU. Since the entry into force of the GDPR, evidence of impeded collaborations is increasing, particularly in the case of sharing data with US public institutions. The adoption of a new EU-US adequacy decision in July 2023 does not hold the promise for a long-lasting solution due to the risks of being challenged and invalidated – yet again – at the Court of Justice of the EU. As the research community is calling for answers, the new proposal for a European Health Data Space regulation may hold a key to solving some of the existing issues. In this paper, we critically discuss the current rules and outline a possible way forward for transfers between public bodies.

Abstract Background In the behavioral sciences, conducting pilot and/or feasibility studies (PFS) is a key step that provides essential information used to inform the design, conduct, and implementation of a larger-scale trial. There are more than 160 published guidelines, reporting checklists, frameworks, and recommendations related to PFS. All of these publications offer some form of guidance on PFS, but many focus on one or a few topics. This makes it difficult for researchers wanting to gain a broader understanding of all the relevant and important aspects of PFS and requires them to seek out multiple sources of information, which increases the risk of missing key considerations to incorporate into their PFS. The purpose of this study was to develop a consolidated set of considerations for the design, conduct, implementation, and reporting of PFS for interventions conducted in the behavioral sciences. Methods To develop this consolidation, we undertook a review of the published guidance on PFS in combination with expert consensus (via a Delphi study) from the authors who wrote such guidance to inform the identified considerations. A total of 161 PFS-related guidelines, checklists, frameworks, and recommendations were identified via a review of recently published behavioral intervention PFS and backward/forward citation tracking of a well-known PFS literature (e.g., CONSORT Ext. for PFS). Authors of all 161 PFS publications were invited to complete a three-round Delphi survey, which was used to guide the creation of a consolidated list of considerations to guide the design, conduct, and reporting of PFS conducted by researchers in the behavioral sciences. Results A total of 496 authors were invited to take part in the three-round Delphi survey (round 1, N = 46; round 2, N = 24; round 3, N = 22). A set of twenty considerations, broadly categorized into six themes (intervention design, study design, conduct of trial, implementation of intervention, statistical analysis, and reporting) were generated from a review of the 161 PFS-related publications as well as a synthesis of feedback from the three-round Delphi process. These 20 considerations are presented alongside a supporting narrative for each consideration as well as a crosswalk of all 161 publications aligned with each consideration for further reading. Conclusion We leveraged expert opinion from researchers who have published PFS-related guidelines, checklists, frameworks, and recommendations on a wide range of topics and distilled this knowledge into a valuable and universal resource for researchers conducting PFS. Researchers may use these considerations alongside the previously published literature to guide decisions about all aspects of PFS, with the hope of creating and disseminating interventions with broad public health impact.

Abstract Background Selective reporting of results from only well-performing cut-offs leads to biased estimates of accuracy in primary studies of questionnaire-based screening tools and in meta-analyses that synthesize results. Individual participant data meta-analysis (IPDMA) of sensitivity and specificity at each cut-off via bivariate random-effects models (BREMs) can overcome this problem. However, IPDMA is laborious and depends on the ability to successfully obtain primary datasets, and BREMs ignore the correlation between cut-offs within primary studies. Methods We compared the performance of three recent multiple cut-off models developed by Steinhauser et al., Jones et al., and Hoyer and Kuss, that account for missing cut-offs when meta-analyzing diagnostic accuracy studies with multiple cut-offs, to BREMs fitted at each cut-off. We used data from 22 studies of the accuracy of the Edinburgh Postnatal Depression Scale (EPDS; 4475 participants, 758 major depression cases). We fitted each of the three multiple cut-off models and BREMs to a dataset with results from only published cut-offs from each study (published data) and an IPD dataset with results for all cut-offs (full IPD data). We estimated pooled sensitivity and specificity with 95% confidence intervals (CIs) for each cut-off and the area under the curve. Results Compared to the BREMs fitted to the full IPD data, the Steinhauser et al., Jones et al., and Hoyer and Kuss models fitted to the published data produced similar receiver operating characteristic curves; though, the Hoyer and Kuss model had lower area under the curve, mainly due to estimating slightly lower sensitivity at lower cut-offs. When fitting the three multiple cut-off models to the full IPD data, a similar pattern of results was observed. Importantly, all models had similar 95% CIs for sensitivity and specificity, and the CI width increased with cut-off levels for sensitivity and decreased with an increasing cut-off for specificity, even the BREMs which treat each cut-off separately. Conclusions Multiple cut-off models appear to be the favorable methods when only published data are available. While collecting IPD is expensive and time consuming, IPD can facilitate subgroup analyses that cannot be conducted with published data only.

Access to scientific data can enable independent reuse and verification; however, most data are not available and become increasingly irrecoverable over time. This study aimed to retrieve and preserve important datasets from 160 of the most highly-cited social science articles published between 2008–2013 and 2015–2018. We asked authors if they would share data in a public repository—the Data Ark—or provide reasons if data could not be shared. Of the 160 articles, data for 117 (73%, 95% CI [67%–80%]) were not available and data for 7 (4%, 95% CI [0%–12%]) were available with restrictions. Data for 36 (22%, 95% CI [16%–30%]) articles were available in unrestricted form: 29 of these datasets were already available and 7 datasets were made available in the Data Ark. Most authors did not respond to our data requests and a minority shared reasons for not sharing, such as legal or ethical constraints. These findings highlight an unresolved need to preserve important scientific datasets and increase their accessibility to the scientific community.

Just like an army of ants caught in an ant mill, individuals, groups and even whole societies are sometimes caught up in a Death Spiral, a vicious cycle of self-reinforcing dysfunctional behavior characterized by continuous flawed decision making, myopic single-minded focus on one (set of) solution(s), denial, distrust, micromanagement, dogmatic thinking and learned helplessness. We propose the term Death Spiral Effect to describe this difficult-to-break downward spiral of societal decline. Specifically, in the current theory-building review we aim to: (a) more clearly define and describe the Death Spiral Effect; (b) model the downward spiral of societal decline as well as an upward spiral; (c) describe how and why individuals, groups and even society at large might be caught up in a Death Spiral; and (d) offer a positive way forward in terms of evidence-based solutions to escape the Death Spiral Effect. Management theory hints on the occurrence of this phenomenon and offers turn-around leadership as solution. On a societal level strengthening of democracy may be important. Prior research indicates that historically, two key factors trigger this type of societal decline: rising inequalities creating an upper layer of elites and a lower layer of masses; and dwindling (access to) resources. Historical key markers of societal decline are a steep increase in inequalities, government overreach, over-integration (interdependencies in networks) and a rapidly decreasing trust in institutions and resulting collapse of legitimacy. Important issues that we aim to shed light on are the behavioral underpinnings of decline, as well as the question if and how societal decline can be reversed. We explore the extension of these theories from the company/organization level to the society level, and make use of insights from both micro-, meso-, and macro-level theories (e.g., Complex Adaptive Systems and collapsology, the study of the risks of collapse of industrial civilization) to explain this process of societal demise. Our review furthermore draws on theories such as Social Safety Theory, Conservation of Resources Theory, and management theories that describe the decline and fall of groups, companies and societies, as well as offer ways to reverse this trend.

Abstract Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy. A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work. Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.

Abstract There is an ongoing debate on the COVID-19 infection fatality rate (IFR) and the impact of COVID-19 on overall population mortality. Here, we addressed these issues in a community in Germany with a major superspreader event analyzing deaths over time and auditing death certificates in the community.18 deaths that occurred within the first six months of the pandemic had a positive test for SARS-CoV-2. Six out of 18 deaths had non-COVID-19 related causes of death (COD). Individuals with COVID-19 COD typically died of respiratory failure (75%) and tended to have fewer reported comorbidities (p = 0.029). Duration between first confirmed infection and death was negatively associated with COVID-19 being COD (p = 0.04). Repeated seroprevalence essays in a cross-sectional epidemiological study showed modest increases in seroprevalence over time, and substantial seroreversion (30%). IFR estimates accordingly varied depending on COVID-19 death attribution. Careful ascertainment of COVID-19 deaths is important in understanding the impact of the pandemic.

Abstract Background Behavioral interventions are often complex, operate at multiple levels, across settings, and employ a range of behavior change techniques. Collecting and reporting key indicators of initial trial and intervention feasibility is essential to decisions for progressing to larger-scale trials. The extent of reporting on feasibility indicators and how this may have changed over time is unknown. The aims of this study were to (1) conduct a historical scoping review of the reporting of feasibility indicators in behavioral pilot/feasibility studies related to obesity published through 2020, and (2) describe trends in the amount and type of feasibility indicators reported in studies published across three time periods: 1982–2006, 2011–2013, and 2018–2020. Methods A search of online databases (PubMed, Embase, EBSCOhost, Web of Science) for health behavior pilot/feasibility studies related to obesity published up to 12/31/2020 was conducted and a random sample of 600 studies, 200 from each of the three timepoints (1982–2006, 2011–2013, and 2018–2020), was included in this review. The presence/absence of feasibility indicators, including recruitment, retention, participant acceptability, attendance, compliance, and fidelity, were identified/coded for each study. Univariate logistic regression models were employed to assess changes in the reporting of feasibility indicators across time. Results A total of 16,365 unique articles were identified of which 6873 of these were reviewed to arrive at the final sample of 600 studies. For the total sample, 428 (71.3%) studies provided recruitment information, 595 (99.2%) provided retention information, 219 (36.5%) reported quantitative acceptability outcomes, 157 (26.2%) reported qualitative acceptability outcomes, 199 (33.2%) reported attendance, 187 (31.2%) reported participant compliance, 23 (3.8%) reported cost information, and 85 (14.2%) reported treatment fidelity outcomes. When compared to the Early Group (1982–2006), studies in the Late Group (2018–2020) were more likely to report recruitment information (OR=1.60, 95%CI 1.03–2.49), acceptability-related quantitative (OR=2.68, 95%CI 1.76–4.08) and qualitative (OR=2.32, 95%CI 1.48–3.65) outcomes, compliance outcomes (OR=2.29, 95%CI 1.49–3.52), and fidelity outcomes (OR=2.13, 95%CI 1.21, 3.77). Conclusion The reporting of feasibility indicators within behavioral pilot/feasibility studies has improved across time, but key aspects of feasibility, such as fidelity, are still not reported in the majority of studies. Given the importance of behavioral intervention pilot/feasibility studies in the translational science spectrum, there is a need for improving the reporting of feasibility indicators.

Abstract Item 10 of the Edinburgh Postnatal Depression Scale (EPDS) is intended to assess thoughts of intentional self-harm but may also elicit concerns about accidental self-harm. It does not specifically address suicide ideation but, nonetheless, is sometimes used as an indicator of suicidality. The 9-item version of the EPDS (EPDS-9), which omits item 10, is sometimes used in research due to concern about positive endorsements of item 10 and necessary follow-up. We assessed the equivalence of total score correlations and screening accuracy to detect major depression using the EPDS-9 versus full EPDS among pregnant and postpartum women. We searched Medline, Medline In-Process and Other Non-Indexed Citations, PsycINFO, and Web of Science from database inception to October 3, 2018 for studies that administered the EPDS and conducted diagnostic classification for major depression based on a validated semi-structured or fully structured interview among women aged 18 or older during pregnancy or within 12 months of giving birth. We conducted an individual participant data meta-analysis. We calculated Pearson correlations with 95% prediction interval (PI) between EPDS-9 and full EPDS total scores using a random effects model. Bivariate random-effects models were fitted to assess screening accuracy. Equivalence tests were done by comparing the confidence intervals (CIs) around the pooled sensitivity and specificity differences to the equivalence margin of δ = 0.05. Individual participant data were obtained from 41 eligible studies (10,906 participants, 1407 major depression cases). The correlation between EPDS-9 and full EPDS scores was 0.998 (95% PI 0.991, 0.999). For sensitivity, the EPDS-9 and full EPDS were equivalent for cut-offs 7–12 (difference range − 0.02, 0.01) and the equivalence was indeterminate for cut-offs 13–15 (all differences − 0.04). For specificity, the EPDS-9 and full EPDS were equivalent for all cut-offs (difference range 0.00, 0.01). The EPDS-9 performs similarly to the full EPDS and can be used when there are concerns about the implications of administering EPDS item 10. Trial registration: The original IPDMA was registered in PROSPERO (CRD42015024785).

Background. Although preregistration can reduce researcher bias and increase transparency in primary research settings, it is less applicable to secondary data analysis. An alternative method that affords additional protection from researcher bias, which cannot be gained from conventional forms of preregistration alone, is an Explore and Confirm Analysis Workflow (ECAW). In this workflow, a data management organization initially provides access to only a subset of their dataset to researchers who request it. The researchers then prepare an analysis script based on the subset of data, upload the analysis script to a registry, and then receive access to the full dataset. ECAWs aim to achieve similar goals to preregistration, but make access to the full dataset contingent on compliance. The present survey aimed to garner information from the research community where ECAWs could be applied—employing the Avon Longitudinal Study of Parents and Children (ALSPAC) as a case example. Methods. We emailed a Web-based survey to researchers who had previously applied for access to ALSPAC's transgenerational observational dataset. Results. We received 103 responses, for a 9% response rate. The results suggest that—at least among our sample of respondents—ECAWs hold the potential to serve their intended purpose and appear relatively acceptable. For example, only 10% of respondents disagreed that ALSPAC should run a study on ECAWs (versus 55% who agreed). However, as many as 26% of respondents agreed that they would be less willing to use ALSPAC data if they were required to use an ECAW (versus 45% who disagreed). Conclusion. Our data and findings provide information for organizations and individuals interested in implementing ECAWs and related interventions. Preregistration. https://osf.io/g2fw5 Deviations from the preregistration are outlined in electronic supplementary material A.

Abstract Objectives Optimal cutoff thresholds are selected to separate ‘positive’ from ‘negative’ screening results. We evaluated how depression screening tool studies select optimal cutoffs. Methods We included studies from previously conducted meta‐analyses of Patient Health Questionnaire‐9, Edinburgh Postnatal Depression Scale, or Hospital Anxiety and Depression Scale—Depression accuracy. Outcomes included whether an optimal cutoff was selected, method used, recommendations made, and reporting guideline and protocol citation. Results Of 212 included studies, 172 (81%) attempted to identify an optimal cutoff, and 147 of these 172 (85%) reported one or more methods. Methods were heterogeneous with Youden's J (N = 35, 23%) most common. Only 23 of 147 (16%) studies described a rationale for their method. Rationales focused on balancing sensitivity and specificity without describing why desirable. 131 of 172 studies (76%) identified an optimal cutoff other than the standard; most did not make use recommendations (N = 56; 43%) or recommended using a non‐standard cutoff (N = 53; 40%). Only 4 studies cited a reporting guideline, and 4 described a protocol with optimal cutoff selection methods, but none used the protocol method in the published study. Conclusions Research is needed to guide how selection of cutoffs for depression screening tools can be standardized and reflect clinical considerations.

Abstracts Background Pilot/feasibility or studies with small sample sizes may be associated with inflated effects. This study explores the vibration of effect sizes (VoE) in meta-analyses when considering different inclusion criteria based upon sample size or pilot/feasibility status. Methods Searches were to identify systematic reviews that conducted meta-analyses of behavioral interventions on topics related to the prevention/treatment of childhood obesity from January 2016 to October 2019. The computed summary effect sizes (ES) were extracted from each meta-analysis. Individual studies included in the meta-analyses were classified into one of the following four categories: self-identified pilot/feasibility studies or based upon sample size but not a pilot/feasibility study (N ≤ 100, N > 100, and N > 370 the upper 75th of sample size). The VoE was defined as the absolute difference (ABS) between the re-estimations of summary ES restricted to study classifications compared to the originally reported summary ES. Concordance (kappa) of statistical significance of summary ES between the four categories of studies was assessed. Fixed and random effects models and meta-regressions were estimated. Three case studies are presented to illustrate the impact of including pilot/feasibility and N ≤ 100 studies on the estimated summary ES. Results A total of 1602 effect sizes, representing 145 reported summary ES, were extracted from 48 meta-analyses containing 603 unique studies (avg. 22 studies per meta-analysis, range 2–108) and included 227,217 participants. Pilot/feasibility and N ≤ 100 studies comprised 22% (0–58%) and 21% (0–83%) of studies included in the meta-analyses. Meta-regression indicated the ABS between the re-estimated and original summary ES where summary ES ranged from 0.20 to 0.46 depending on the proportion of studies comprising the original ES were either mostly small (e.g., N ≤ 100) or mostly large (N > 370). Concordance was low when removing both pilot/feasibility and N ≤ 100 studies (kappa = 0.53) and restricting analyses only to the largest studies (N > 370, kappa = 0.35), with 20% and 26% of the originally reported statistically significant ES rendered non-significant. Reanalysis of the three case study meta-analyses resulted in the re-estimated ES rendered either non-significant or half of the originally reported ES. Conclusions When meta-analyses of behavioral interventions include a substantial proportion of both pilot/feasibility and N ≤ 100 studies, summary ES can be affected markedly and should be interpreted with caution.

ABSTRACT Amino acids in variable positions of proteins may be correlated, with potential structural and functional implications. Here, we apply exact tests of independence in R × C contingency tables to examine noise-free associations between variable positions of the SARS-CoV-2 spike protein, using as a paradigm sequences from Greece deposited in GISAID (N = 6,683/1,078 full length) for the period 29 February 2020 to 26 April 2021 that essentially covers the first three pandemic waves. We examine the fate and complexity of these associations by network analysis, using associated positions (exact P ≤ 0.001 and Average Product Correction ≥ 2) as links and the corresponding positions as nodes. We found a temporal linear increase of positional differences and a gradual expansion of the number of position associations over time, represented by a temporally evolving intricate web, resulting in a non-random complex network of 69 nodes and 252 links. Overconnected nodes corresponded to the most adapted variant positions in the population, suggesting a direct relation between network degree and position functional importance. Modular analysis revealed 25 k-cliques comprising 3 to 11 nodes. At different k-clique resolutions, one to four communities were formed, capturing epistatic associations of circulating variants (Alpha, Beta, B.1.1.318), but also Delta, which dominated the evolutionary landscape later in the pandemic. Cliques of aminoacidic positional associations tended to occur in single sequences, enabling the recognition of epistatic positions in real-world virus populations. Our findings provide a novel way of understanding epistatic relationships in viral proteins with potential applications in the design of virus control procedures. IMPORTANCE Paired positional associations of adapted amino acids in virus proteins may provide new insights for understanding virus evolution and variant formation. We investigated potential intramolecular relationships between variable SARS-CoV-2 spike positions by exact tests of independence in R × C contingency tables, having applied Average Product Correction (APC) to eliminate background noise. Associated positions (exact P ≤ 0.001 and APC ≥ 2) formed a non-random, epistatic network of 25 cliques and 1–4 communities at different clique resolutions, revealing evolutionary ties between variable positions of circulating variants and a predictive potential of previously unknown network positions. Cliques of different sizes represented theoretical combinations of changing residues in sequence space, allowing the identification of significant aminoacidic combinations in single sequences of real-world populations. Our analytic approach that links network structural aspects to mutational aminoacidic combinations in the spike sequence population offers a novel way to understand virus epidemiology and evolution.

To characterize the impact of major research awards on recipients' subsequent work, we studied Nobel Prize winners in Chemistry, Physiology or Medicine, and Physics and MacArthur Fellows working in scientific fields. Using a case-crossover design, we compared scientists’ citations, publications and citations-per-publication from work published in a 3-year pre-award period to their work published in a 3-year post-award period. Nobel Laureates and MacArthur Fellows received fewer citations for post- than for pre-award work. This was driven mostly by Nobel Laureates. Median decrease was 80.5 citations among Nobel Laureates (p = 0.004) and 2 among MacArthur Fellows (p = 0.857). Mid-career (42–57 years) and senior (greater than 57 years) researchers tended to earn fewer citations for post-award work. Early career researchers (less than 42 years, typically MacArthur Fellows) tended to earn more, but the difference was non-significant. MacArthur Fellows (p = 0.001) but not Nobel Laureates (p = 0.180) had significantly more post-award publications. Both populations had significantly fewer post-award citations per paper (p = 0.043 for Nobel Laureates, 0.005 for MacArthur Fellows, and 0.0004 for combined population). If major research awards indeed fail to increase (and even decrease) recipients' impact, one may need to reassess the purposes, criteria, and impacts of awards to improve the scientific enterprise.

Objectives: To provide a step-by-step, easy-to-understand, practical guide for systematic review and meta-analysis of observational studies.Methods: A multidisciplinary team of researchers with extensive experience in observational studies and systematic review and meta-analysis was established. Previous guidelines in evidence synthesis were considered.Results: There is inherent variability in observational study design, population, and analysis, making evidence synthesis challenging. We provided a framework and discussed basic meta-analysis concepts to assist reviewers in making informed decisions. We also explained several statistical tools for dealing with heterogeneity, probing for bias, and interpreting findings. Finally, we briefly discussed issues and caveats for translating results into clinical and public health recommendations. Our guideline complements “A 24-step guide on how to design, conduct, and successfully publish a systematic review and meta-analysis in medical research” and addresses peculiarities for observational studies previously unexplored.Conclusion: We provided 7 steps to synthesize evidence from observational studies. We encourage medical and public health practitioners who answer important questions to systematically integrate evidence from observational studies and contribute evidence-based decision-making in health sciences.

Clinical trial data-sharing is seen as an imperative for research integrity and is becoming increasingly encouraged or even required by funders, journals, and other stakeholders. However, early experiences with data-sharing have been disappointing because they are not always conducted properly. Health data is indeed sensitive and not always easy to share in a responsible way. We propose 10 rules for researchers wishing to share their data. These rules cover the majority of elements to be considered in order to start the commendable process of clinical trial data-sharing: Rule 1: Abide by local legal and regulatory data protection requirements Rule 2: Anticipate the possibility of clinical trial data-sharing before obtaining funding Rule 3: Declare your intent to share data in the registration step Rule 4: Involve research participants Rule 5: Determine the method of data access Rule 6: Remember there are several other elements to share Rule 7: Do not proceed alone Rule 8: Deploy optimal data management to ensure that the data shared is useful Rule 9: Minimize risks Rule 10: Strive for excellence.

The distribution of effect sizes may offer insights about the research done and reported in a scientific field. We have evaluated 12 412 manually collected correlation effect sizes (Sample 1) and 31 157 computer-extracted correlation effect sizes (Sample 2) published in journals focused on social or developmental psychology. Sample 1 consisted of 243 studies from six journals published in 2010 and 2019. Sample 2 consisted of 5012 papers published in 10 journals between 2010 and 2019. The 25th, 50th and 75th effect size percentiles were 0.08, 0.17 and 0.33, and 0.17, 0.31 and 0.52 in Samples 1 and 2, respectively. Sample 2 percentiles were probably larger because Sample 2 only included effect sizes from the text but not from tables. In text authors may have emphasized larger correlations. Large sample sizes were associated with smaller reported correlations. In Sample 1 about 70% of studies specified a directional hypothesis. In 2010 no papers had power calculations, while in 2019 14% of papers had power calculations. These data offer empirical insights into the distribution of reported correlations and may inform the interpretation of effect sizes. They also demonstrate the importance of computation of statistical power and highlight potential reporting bias.

Abstract Background Convalescent plasma has been widely used to treat COVID-19 and is under investigation in numerous randomized clinical trials, but results are publicly available only for a small number of trials. The objective of this study was to assess the benefits of convalescent plasma treatment compared to placebo or no treatment and all-cause mortality in patients with COVID-19, using data from all available randomized clinical trials, including unpublished and ongoing trials (Open Science Framework, https://doi.org/10.17605/OSF.IO/GEHFX ). Methods In this collaborative systematic review and meta-analysis, clinical trial registries (ClinicalTrials.gov, WHO International Clinical Trials Registry Platform), the Cochrane COVID-19 register, the LOVE database, and PubMed were searched until April 8, 2021. Investigators of trials registered by March 1, 2021, without published results were contacted via email. Eligible were ongoing, discontinued and completed randomized clinical trials that compared convalescent plasma with placebo or no treatment in COVID-19 patients, regardless of setting or treatment schedule. Aggregated mortality data were extracted from publications or provided by investigators of unpublished trials and combined using the Hartung–Knapp–Sidik–Jonkman random effects model. We investigated the contribution of unpublished trials to the overall evidence. Results A total of 16,477 patients were included in 33 trials (20 unpublished with 3190 patients, 13 published with 13,287 patients). 32 trials enrolled only hospitalized patients (including 3 with only intensive care unit patients). Risk of bias was low for 29/33 trials. Of 8495 patients who received convalescent plasma, 1997 died (23%), and of 7982 control patients, 1952 died (24%). The combined risk ratio for all-cause mortality was 0.97 (95% confidence interval: 0.92; 1.02) with between-study heterogeneity not beyond chance (I2 = 0%). The RECOVERY trial had 69.8% and the unpublished evidence 25.3% of the weight in the meta-analysis. Conclusions Convalescent plasma treatment of patients with COVID-19 did not reduce all-cause mortality. These results provide strong evidence that convalescent plasma treatment for patients with COVID-19 should not be used outside of randomized trials. Evidence synthesis from collaborations among trial investigators can inform both evidence generation and evidence application in patient care.

Abstract With the establishment of large biobanks, discovery of single nucleotide variants (SNVs, also known as single nucleotide polymorphisms (SNVs)) associated with various phenotypes has accelerated. An open question is whether genome-wide significant SNVs identified in earlier genome-wide association studies (GWAS) are replicated in later GWAS conducted in biobanks. To address this, we examined a publicly available GWAS database and identified two, independent GWAS on the same phenotype (an earlier, “discovery” GWAS and a later, “replication” GWAS done in the UK biobank). The analysis evaluated 136,318,924 SNVs (of which 6289 reached P

Abstract People receiving COVID-19 vaccines may subsequently markedly increase their previously suppressed exposure risk. A simple model can evaluate the benefit of vaccination to the vaccinated (index) person and others exposed to that person; and calculate the amount of risk compensation required to eliminate all the benefits or to halve the benefit. As shown, 2.5-fold increase in exposure will eliminate the benefit of a vaccine of moderate efficacy (E = 0.6) unless the probability of infection in the population of interest is very high. With very high vaccine efficacy (E = 0.95), substantial benefit is maintained except in situations where there is a very low probability of infection in the population. If the vaccine efficacy decreases to 0.8, the benefit gets eroded easily with modest risk compensation. Risk compensation may markedly affect the benefit of COVID-19 vaccination, especially if vaccine efficacy in real-life or specific high-risk populations (e.g., nursing home residents) is not very high.

Abstract Background The validity of observational studies and their meta-analyses is contested. Here, we aimed to appraise thousands of meta-analyses of observational studies using a pre-specified set of quantitative criteria that assess the significance, amount, consistency, and bias of the evidence. We also aimed to compare results from meta-analyses of observational studies against meta-analyses of randomized controlled trials (RCTs) and Mendelian randomization (MR) studies. Methods We retrieved from PubMed (last update, November 19, 2020) umbrella reviews including meta-analyses of observational studies assessing putative risk or protective factors, regardless of the nature of the exposure and health outcome. We extracted information on 7 quantitative criteria that reflect the level of statistical support, the amount of data, the consistency across different studies, and hints pointing to potential bias. These criteria were level of statistical significance (pre-categorized according to 10−6, 0.001, and 0.05 p-value thresholds), sample size, statistical significance for the largest study, 95% prediction intervals, between-study heterogeneity, and the results of tests for small study effects and for excess significance. Results 3744 associations (in 57 umbrella reviews) assessed by a median number of 7 (interquartile range 4 to 11) observational studies were eligible. Most associations were statistically significant at P < 0.05 (61.1%, 2289/3744). Only 2.6% of associations had P < 10−6, ≥1000 cases (or ≥20,000 participants for continuous factors), P < 0.05 in the largest study, 95% prediction interval excluding the null, and no large between-study heterogeneity, small study effects, or excess significance. Across the 57 topics, large heterogeneity was observed in the proportion of associations fulfilling various quantitative criteria. The quantitative criteria were mostly independent from one another. Across 62 associations assessed in both RCTs and in observational studies, 37.1% had effect estimates in opposite directions and 43.5% had effect estimates differing beyond chance in the two designs. Across 94 comparisons assessed in both MR and observational studies, such discrepancies occurred in 30.8% and 54.7%, respectively. Conclusions Acknowledging that no gold-standard exists to judge whether an observational association is genuine, statistically significant results are common in observational studies, but they are rarely convincing or corroborated by randomized evidence.

Journals exert considerable control over letters, commentaries and online comments that criticize prior research (post-publication critique). We assessed policies (Study One) and practice (Study Two) related to post-publication critique at 15 top-ranked journals in each of 22 scientific disciplines (N = 330 journals). Two-hundred and seven (63%) journals accepted post-publication critique and often imposed limits on length (median 1000, interquartile range (IQR) 500–1200 words) and time-to-submit (median 12, IQR 4–26 weeks). The most restrictive limits were 175 words and two weeks; some policies imposed no limits. Of 2066 randomly sampled research articles published in 2018 by journals accepting post-publication critique, 39 (1.9%, 95% confidence interval [1.4, 2.6]) were linked to at least one post-publication critique (there were 58 post-publication critiques in total). Of the 58 post-publication critiques, 44 received an author reply, of which 41 asserted that original conclusions were unchanged. Clinical Medicine had the most active culture of post-publication critique: all journals accepted post-publication critique and published the most post-publication critique overall, but also imposed the strictest limits on length (median 400, IQR 400–550 words) and time-to-submit (median 4, IQR 4–6 weeks). Our findings suggest that top-ranked academic journals often pose serious barriers to the cultivation, documentation and dissemination of post-publication critique.

ABSTRACT SARS-CoV-2 Omicron variants contain many mutations in its spike receptor-binding domain, the target of all authorized monoclonal antibodies (MAbs). Determining the extent to which Omicron variants reduced MAb susceptibility is critical to preventing and treating COVID-19. We systematically reviewed PubMed and three preprint servers, last updated 11 April 2022, for the in vitro activity of authorized MAbs against the Omicron variants. Fifty-one studies were eligible, including 50 containing Omicron BA.1 susceptibility data and 17 containing Omicron BA.2 susceptibility data. The first two authorized MAb combinations, bamlanivimab/etesevimab and casirivimab/imdevimab, were largely inactive against the Omicron BA.1 and BA.2 variants. In 34 studies, sotrovimab displayed a median 4.0-fold (interquartile range [IQR]: 2.6 to 6.9) reduction in activity against Omicron BA.1, and in 12 studies, it displayed a median 17-fold (IQR: 13 to 30) reduction in activity against Omicron BA.2. In 15 studies, the combination cilgavimab/tixagevimab displayed a median 86-fold (IQR: 27 to 151) reduction in activity against Omicron BA.1, and in six studies, it displayed a median 5.4-fold (IQR: 3.7 to 6.9) reduction in activity against Omicron BA.2. In eight studies against Omicron BA.1 and six studies against Omicron BA.2, bebtelovimab displayed no reduction in activity. Disparate results between assays were common. For authorized MAbs, 51/268 (19.0%) results for wild-type control variants and 78/348 (22.4%) results for Omicron BA.1 and BA.2 variants were more than 4-fold below or 4-fold above the median result for that MAb. Highly disparate results between published assays indicate a need for improved MAb susceptibility test standardization or interassay calibration. IMPORTANCE Monoclonal antibodies (MAbs) targeting the SARS-CoV-2 spike protein are among the most effective measures for preventing and treating COVID-19. However, SARS-CoV-2 Omicron variants contain many mutations in their spike receptor-binding domains, the target of all authorized MAbs. Therefore, determining the extent to which Omicron variants reduced MAb susceptibility is critical to preventing and treating COVID-19. We identified 51 studies that reported the in vitro susceptibility of the two main Omicron variants BA.1 and BA.2 to therapeutic MAbs in advanced clinical development, including eight authorized individual MAbs and three authorized MAb combinations. We estimated the degree to which different MAbs displayed reduced activity against Omicron variants. The marked loss of activity of many MAbs against Omicron variants underscores the importance of developing MAbs that target conserved regions of spike. Highly disparate results between assays indicate the need for improved MAb susceptibility test standardization.

A series of aggressive restrictive measures were adopted around the world in 2020–2022 to attempt to prevent SARS-CoV-2 from spreading. However, it has become increasingly clear the most aggressive (lockdown) response strategies may involve negative side-effects such as a steep increase in poverty, hunger, and inequalities. Several economic, educational, and health repercussions have fallen disproportionately on children, students, young workers, and especially on groups with pre-existing inequalities such as low-income families, ethnic minorities, and women. This has led to a vicious cycle of rising inequalities and health issues. For example, educational and financial security decreased along with rising unemployment and loss of life purpose. Domestic violence surged due to dysfunctional families being forced to spend more time with each other. In the current narrative and scoping review, we describe macro-dynamics that are taking place because of aggressive public health policies and psychological tactics to influence public behavior, such as mass formation and crowd behavior. Coupled with the effect of inequalities, we describe how these factors can interact toward aggravating ripple effects. In light of evidence regarding the health, economic and social costs, that likely far outweigh potential benefits, the authors suggest that, first, where applicable, aggressive lockdown policies should be reversed and their re-adoption in the future should be avoided. If measures are needed, these should be non-disruptive. Second, it is important to assess dispassionately the damage done by aggressive measures and offer ways to alleviate the burden and long-term effects. Third, the structures in place that have led to counterproductive policies should be assessed and ways should be sought to optimize decision-making, such as counteracting groupthink and increasing the level of reflexivity. Finally, a package of scalable positive psychology interventions is suggested to counteract the damage done and improve humanity's prospects.

Abstract Substantial COVID-19 research investment has been allocated to randomized clinical trials (RCTs) on hydroxychloroquine/chloroquine, which currently face recruitment challenges or early discontinuation. We aim to estimate the effects of hydroxychloroquine and chloroquine on survival in COVID-19 from all currently available RCT evidence, published and unpublished. We present a rapid meta-analysis of ongoing, completed, or discontinued RCTs on hydroxychloroquine or chloroquine treatment for any COVID-19 patients (protocol: https://osf.io/QESV4/ ). We systematically identified unpublished RCTs (ClinicalTrials.gov, WHO International Clinical Trials Registry Platform, Cochrane COVID-registry up to June 11, 2020), and published RCTs (PubMed, medRxiv and bioRxiv up to October 16, 2020). All-cause mortality has been extracted (publications/preprints) or requested from investigators and combined in random-effects meta-analyses, calculating odds ratios (ORs) with 95% confidence intervals (CIs), separately for hydroxychloroquine and chloroquine. Prespecified subgroup analyses include patient setting, diagnostic confirmation, control type, and publication status. Sixty-three trials were potentially eligible. We included 14 unpublished trials (1308 patients) and 14 publications/preprints (9011 patients). Results for hydroxychloroquine are dominated by RECOVERY and WHO SOLIDARITY, two highly pragmatic trials, which employed relatively high doses and included 4716 and 1853 patients, respectively (67% of the total sample size). The combined OR on all-cause mortality for hydroxychloroquine is 1.11 (95% CI: 1.02, 1.20; I² = 0%; 26 trials; 10,012 patients) and for chloroquine 1.77 (95%CI: 0.15, 21.13, I² = 0%; 4 trials; 307 patients). We identified no subgroup effects. We found that treatment with hydroxychloroquine is associated with increased mortality in COVID-19 patients, and there is no benefit of chloroquine. Findings have unclear generalizability to outpatients, children, pregnant women, and people with comorbidities.

Abstract Background In this paper, we argue for Gender as a Sociocultural Variable (GASV) as a complement to Sex as a Biological Variable (SABV). Sex (biology) and gender (sociocultural behaviors and attitudes) interact to influence health and disease processes across the lifespan—which is currently playing out in the COVID-19 pandemic. This study develops a gender assessment tool—the Stanford Gender-Related Variables for Health Research—for use in clinical and population research, including large-scale health surveys involving diverse Western populations. While analyzing sex as a biological variable is widely mandated, gender as a sociocultural variable is not, largely because the field lacks quantitative tools for analyzing the influence of gender on health outcomes. Methods We conducted a comprehensive review of English-language measures of gender from 1975 to 2015 to identify variables across three domains: gender norms, gender-related traits, and gender relations. This yielded 11 variables tested with 44 items in three US cross-sectional survey populations: two internet-based (N = 2051; N = 2135) and a patient-research registry (N = 489), conducted between May 2017 and January 2018. Results Exploratory and confirmatory factor analyses reduced 11 constructs to 7 gender-related variables: caregiver strain, work strain, independence, risk-taking, emotional intelligence, social support, and discrimination. Regression analyses, adjusted for age, ethnicity, income, education, sex assigned at birth, and self-reported gender identity, identified associations between these gender-related variables and self-rated general health, physical and mental health, and health-risk behaviors. Conclusion Our new instrument represents an important step toward developing more comprehensive and precise survey-based measures of gender in relation to health. Our questionnaire is designed to shed light on how specific gender-related behaviors and attitudes contribute to health and disease processes, irrespective of—or in addition to—biological sex and self-reported gender identity. Use of these gender-related variables in experimental studies, such as clinical trials, may also help us understand if gender factors play an important role as treatment-effect modifiers and would thus need to be further considered in treatment decision-making.

AbstractResearch policy and planning for a given country may benefit from reliable data on both its scientific workforce as well as the diaspora of scientists for countries with a substantial brain drain. Here we use a systematic approach using Scopus to generate a comprehensive country-level database of all scientists in Greece. Moreover, we expand that database to include also Greek diaspora scientists. The database that we have compiled includes 63,951 scientists who have published at least five papers indexed in Scopus. Of those, 35,116 have an affiliation in Greece. We validate the sensitivity and specificity of the database against different control sets of scientists. We also analyze the scientific disciplines of these scientists according to the Science Metrix classification (174 subfield disciplines) and provide detailed data on each of the 63,951 scientists using multiple citation indicators and a composite thereof. These analyses demonstrate differential concentrations in specific subfields for the local versus the diaspora cohorts, as well as an advantage of the diaspora cohort in terms of citation indicators, especially among top-impact researchers. The approach that we have taken can also be applied to map the scientific workforce of other countries and nations for evaluation, planning, and policy purposes.

Abstract Background Poor recruitment of patients is the predominant reason for early termination of randomized clinical trials (RCTs). Systematic empirical investigations and validation studies of existing recruitment models, however, are lacking. We aim to provide evidence-based guidance on how to predict and monitor recruitment of patients into RCTs. Our specific objectives are the following: (1) to establish a large sample of RCTs (target n = 300) with individual patient recruitment data from a large variety of RCTs, (2) to investigate participant recruitment patterns and study site recruitment patterns and their association with the overall recruitment process, (3) to investigate the validity of a freely available recruitment model, and (4) to develop a user-friendly tool to assist trial investigators in the planning and monitoring of the recruitment process. Methods Eligible RCTs need to have completed the recruitment process, used a parallel group design, and investigated any healthcare intervention where participants had the free choice to participate. To establish the planned sample of RCTs, we will use our contacts to national and international RCT networks, clinical trial units, and individual trial investigators. From included RCTs, we will collect patient-level information (date of randomization), site-level information (date of trial site activation), and trial-level information (target sample size). We will examine recruitment patterns using recruitment trajectories and stratifications by RCT characteristics. We will investigate associations of early recruitment patterns with overall recruitment by correlation and multivariable regression. To examine the validity of a freely available Bayesian prediction model, we will compare model predictions to collected empirical data of included RCTs. Finally, we will user-test any promising tool using qualitative methods for further tool improvement. Discussion This research will contribute to a better understanding of participant recruitment to RCTs, which could enhance efficiency and reduce the waste of resources in clinical research with a comprehensive, concerted, international effort.

Abstract Background There is a clear need for systematic appraisal of models/factors predicting colorectal cancer (CRC) metastasis and recurrence because clinical decisions about adjuvant treatment are taken on the basis of such variables. Methods We conducted an umbrella review of all systematic reviews of observational studies (with/without meta-analysis) that evaluated risk factors of CRC metastasis and recurrence. We also generated an updated synthesis of risk prediction models for CRC metastasis and recurrence. We cross-assessed individual risk factors and risk prediction models. Results Thirty-four risk factors for CRC metastasis and 17 for recurrence were investigated. Twelve of 34 and 4/17 risk factors with p

Abstract Background Despite the enormous financial and humanistic burden of chronic low back pain (CLBP), there is little consensus on what constitutes the best treatment options from a multitude of competing interventions. The objective of this network meta-analysis (NMA) is to determine the relative efficacy and acceptability of primary care treatments for non-specific CLBP, with the overarching aim of providing a comprehensive evidence base for informing treatment decisions. Methods We will perform a systematic search to identify randomised controlled trials of interventions endorsed in primary care guidelines for the treatment of non-specific CLBP in adults. Information sources searched will include major bibliographic databases (MEDLINE, Embase, CENTRAL, CINAHL, PsycINFO and LILACS) and clinical trial registries. Our primary outcomes will be patient-reported pain ratings and treatment acceptability (all-cause discontinuation), and secondary outcomes will be functional ability, quality of life and patient/physician ratings of overall improvement. A hierarchical Bayesian class-based NMA will be performed to determine the relative effects of different classes of pharmacological (NSAIDs, opioids, paracetamol, anti-depressants, muscle relaxants) and non-pharmacological (exercise, patient education, manual therapies, psychological therapy, multidisciplinary approaches, massage, acupuncture, mindfulness) interventions and individual treatments within a class (e.g. NSAIDs: diclofenac, ibuprofen, naproxen). We will conduct risk of bias assessments and threshold analysis to assess the robustness of the findings to potential bias. We will compute the effect of different interventions relative to placebo/no treatment for both short- and long-term efficacy and acceptability. Discussion While many factors are important in selecting an appropriate intervention for an individual patient, evidence for the analgesic effects and acceptability of a treatment are key factors in guiding this selection. Thus, this NMA will provide an important source of evidence to inform treatment decisions and future clinical guidelines. Systematic review registration PROSPERO registry number: CRD42019138115

Abstract Background Among different investigators studying the same exposures and outcomes, there may be a lack of consensus about potential confounders that should be considered as matching, adjustment, or stratification variables in observational studies. Concerns have been raised that confounding factors may affect the results obtained for the alcohol-ischemic heart disease relationship, as well as their consistency and reproducibility across different studies. Therefore, we assessed how confounders are defined, operationalized, and discussed across individual studies evaluating the impact of alcohol on ischemic heart disease risk. Methods For observational studies included in a recent alcohol-ischemic heart disease meta-analysis, we identified all variables adjusted, matched, or stratified for in the largest reported multivariate model (i.e. potential confounders). We recorded how the variables were measured and grouped them into higher-level confounder domains. Abstracts and Discussion sections were then assessed to determine whether authors considered confounding when interpreting their study findings. Results 85 of 87 (97.7%) studies reported multivariate analyses for an alcohol-ischemic heart disease relationship. The most common higher-level confounder domains included were smoking (79, 92.9%), age (74, 87.1%), and BMI, height, and/or weight (57, 67.1%). However, no two models adjusted, matched, or stratified for the same higher-level confounder domains. Most (74/87, 85.1%) articles mentioned or alluded to “confounding” in their Abstract or Discussion sections, but only one stated that their main findings were likely to be affected by residual confounding. There were five (5/87, 5.7%) authors that explicitly asked for caution when interpreting results. Conclusion There is large variation in the confounders considered across observational studies evaluating the impact of alcohol on ischemic heart disease risk and almost all studies spuriously ignore or eventually dismiss confounding in their conclusions. Given that study results and interpretations may be affected by the mix of potential confounders included within multivariate models, efforts are necessary to standardize approaches for selecting and accounting for confounders in observational studies.

Abstract Background Many technology companies, including Airbnb, Amazon, Booking.com, eBay, Facebook, Google, LinkedIn, Lyft, Microsoft, Netflix, Twitter, Uber, and Yahoo!/Oath, run online randomized controlled experiments at scale, namely hundreds of concurrent controlled experiments on millions of users each, commonly referred to as A/B tests. Originally derived from the same statistical roots, randomized controlled trials (RCTs) in medicine are now criticized for being expensive and difficult, while in technology, the marginal cost of such experiments is approaching zero and the value for data-driven decision-making is broadly recognized. Methods and results This is an overview of key scaling lessons learned in the technology field. They include (1) a focus on metrics, an overall evaluation criterion and thousands of metrics for insights and debugging, automatically computed for every experiment; (2) quick release cycles with automated ramp-up and shut-down that afford agile and safe experimentation, leading to consistent incremental progress over time; and (3) a culture of ‘test everything’ because most ideas fail and tiny changes sometimes show surprising outcomes worth millions of dollars annually. Technological advances, online interactions, and the availability of large-scale data allowed technology companies to take the science of RCTs and use them as online randomized controlled experiments at large scale with hundreds of such concurrent experiments running on any given day on a wide range of software products, be they web sites, mobile applications, or desktop applications. Rather than hindering innovation, these experiments enabled accelerated innovation with clear improvements to key metrics, including user experience and revenue. As healthcare increases interactions with patients utilizing these modern channels of web sites and digital health applications, many of the lessons apply. The most innovative technological field has recognized that systematic series of randomized trials with numerous failures of the most promising ideas leads to sustainable improvement. Conclusion While there are many differences between technology and medicine, it is worth considering whether and how similar designs can be applied via simple RCTs that focus on healthcare decision-making or service delivery. Changes – small and large – should undergo continuous and repeated evaluations in randomized trials and learning from their results will enable accelerated healthcare improvements.

Abstract Background Preliminary evaluations of behavioral interventions, referred to as pilot studies, predate the conduct of many large-scale efficacy/effectiveness trial. The ability of a pilot study to inform an efficacy/effectiveness trial relies on careful considerations in the design, delivery, and interpretation of the pilot results to avoid exaggerated early discoveries that may lead to subsequent failed efficacy/effectiveness trials. “Risk of generalizability biases (RGB)” in pilot studies may reduce the probability of replicating results in a larger efficacy/effectiveness trial. We aimed to generate an operational list of potential RGBs and to evaluate their impact in pairs of published pilot studies and larger, more well-powered trial on the topic of childhood obesity. Methods We conducted a systematic literature review to identify published pilot studies that had a published larger-scale trial of the same or similar intervention. Searches were updated and completed through December 31st, 2018. Eligible studies were behavioral interventions involving youth (≤18 yrs) on a topic related to childhood obesity (e.g., prevention/treatment, weight reduction, physical activity, diet, sleep, screen time/sedentary behavior). Extracted information included study characteristics and all outcomes. A list of 9 RGBs were defined and coded: intervention intensity bias, implementation support bias, delivery agent bias, target audience bias, duration bias, setting bias, measurement bias, directional conclusion bias, and outcome bias. Three reviewers independently coded for the presence of RGBs. Multi-level random effects meta-analyses were performed to investigate the association of the biases to study outcomes. Results A total of 39 pilot and larger trial pairs were identified. The frequency of the biases varied: delivery agent bias (19/39 pairs), duration bias (15/39), implementation support bias (13/39), outcome bias (6/39), measurement bias (4/39), directional conclusion bias (3/39), target audience bias (3/39), intervention intensity bias (1/39), and setting bias (0/39). In meta-analyses, delivery agent, implementation support, duration, and measurement bias were associated with an attenuation of the effect size of − 0.325 (95CI − 0.556 to − 0.094), − 0.346 (− 0.640 to − 0.052), − 0.342 (− 0.498 to − 0.187), and − 0.360 (− 0.631 to − 0.089), respectively. Conclusions Pre-emptive avoidance of RGBs during the initial testing of an intervention may diminish the voltage drop between pilot and larger efficacy/effectiveness trials and enhance the odds of successful translation.

Florian Naudet and co-authors discuss strengthening requirements for sharing clinical trial data.

Introduction Inflammatory arthritis (IA) conditions, including rheumatoid arthritis, psoriatic arthritis and axial spondyloarthritis, are characterised by inflammatory infiltration of the joints. Biological disease-modifying antirheumatic drugs (bDMARDs) and targeted synthetic disease-modifying antirheumatic drugs (tsDMARDs), respectively, reduce the effects of proinflammatory cytokines and immune cells to ameliorate disease. However, immunosuppression can be associated with high rates of serious adverse events (SAEs), including serious infections, and maybe an increased risk of malignancies and cardiovascular events. Currently, there is no empirical evidence on the extent to which contextual factors and risk of bias (RoB) domains may modify these harm signals in randomised trials.Methods and analysis We will search MEDLINE (via PubMed) for systematic reviews published since April 2015 and all Cochrane reviews. From these reviews, randomised trials will be eligible if they include patients with an IA condition with at least one group randomly allocated to bDMARD and/or tsDMARD treatments. A predefined form will be used for extracting data on population characteristics (eg, baseline characteristics or eligibility criteria, such as medication background) and specific harm outcome measures, such as number of withdrawals, numbers of patients discontinuing due to adverse events and number of patients having SAEs. RoB in individual trials will be assessed using a modified Cochrane RoB tool. We will estimate the potentially causal harm effects related to the experimental intervention compared with control comparator as risk ratios, and heterogeneity across randomised comparisons will be assessed statistically and evaluated as inconsistency using the I2 Index. Our metaregression analyses will designate population and trial characteristics and each RoB domain as independent variables, whereas the three harm domains will serve as dependent variables.Ethics and dissemination Ethics approval is not required for this study. Results will be disseminated through publication in international peer-reviewed journals.PROSPERO registration number CRD42020171124.

We examined the extent to which the scientific workforce in different fields was engaged in publishing COVID-19-related papers. According to Scopus (data cut, 1 August 2021), 210 183 COVID-19-related publications included 720 801 unique authors, of which 360 005 authors had published at least five full papers in their career and 23 520 authors were at the top 2% of their scientific subfield based on a career-long composite citation indicator. The growth of COVID-19 authors was far more rapid and massive compared with cohorts of authors historically publishing on H1N1, Zika, Ebola, HIV/AIDS and tuberculosis. All 174 scientific subfields had some specialists who had published on COVID-19. In 109 of the 174 subfields of science, at least one in 10 active, influential (top 2% composite citation indicator) authors in the subfield had authored something on COVID-19. Fifty-three hyper-prolific authors had already at least 60 (and up to 227) COVID-19 publications each. Among the 300 authors with the highest composite citation indicator for their COVID-19 publications, most common countries were USA (n = 67), China (n = 52), UK (n = 32) and Italy (n = 18). The rapid and massive involvement of the scientific workforce in COVID-19-related work is unprecedented and creates opportunities and challenges. There is evidence for hyper-prolific productivity.

The objective of this meta-epidemiological study was to explore the impact of attrition rates on treatment effect estimates in randomised trials of chronic inflammatory diseases (CID) treated with biological and targeted synthetic disease-modifying drugs. We sampled trials from Cochrane reviews. Attrition rates and primary endpoint results were retrieved from trial publications; Odds ratios (ORs) were calculated from the odds of withdrawing in the experimental intervention compared to the control comparison groups (i.e., differential attrition), as well as the odds of achieving a clinical response (i.e., the trial outcome). Trials were combined using random effects restricted maximum likelihood meta-regression models and associations between estimates of treatment effects and attrition rates were analysed. From 37 meta-analyses, 179 trials were included, and 163 were analysed (301 randomised comparisons; n = 62,220 patients). Overall, the odds of withdrawal were lower in the experimental compared to control groups (random effects summary OR = 0.45, 95% CI, 0.41-0.50). The corresponding overall treatment effects were large (random effects summary OR = 4.43, 95% CI 3.92-4.99) with considerable heterogeneity across interventions and clinical specialties (I2 = 85.7%). The ORs estimating treatment effect showed larger treatment benefits when the differential attrition was more prominent with more attrition in the control group (OR = 0.73, 95% CI 0.55-0.96). Higher attrition rates from the control arm are associated with larger estimated benefits of treatments with biological or targeted synthetic disease-modifying drugs in CID trials; differential attrition may affect estimates of treatment benefit in randomised trials.

Using a sample of 70,399 published p-values from 192 meta-analyses, we empirically estimate the counterfactual distribution of p-values in the absence of any biases. Comparing observed p-values with counterfactually expected p-values allows us to estimate how many p-values are published as being statistically significant when they should have been published as non-significant. We estimate the extent of selectively reported p-values to range between 57.7% and 71.9% of the significant p-values. The counterfactual p-value distribution also allows us to assess shifts of p-values along the entire distribution of published p-values, revealing that particularly very small p-values (p < 0.001) are unexpectedly abundant in the published literature. Subsample analysis suggests that the extent of selective reporting is reduced in research fields that use experimental designs, analyze microeconomics research questions, and have at least some adequately powered studies.

Abstract Background Until recently a typical rule that has often been used for the endorsement of new medications by the Food and Drug Administration has been the existence of at least two statistically significant clinical trials favoring the new medication. This rule has consequences for the true positive (endorsement of an effective treatment) and false positive rates (endorsement of an ineffective treatment). Methods In this paper, we compare true positive and false positive rates for different evaluation criteria through simulations that rely on (1) conventional p-values; (2) confidence intervals based on meta-analyses assuming fixed or random effects; and (3) Bayes factors. We varied threshold levels for statistical evidence, thresholds for what constitutes a clinically meaningful treatment effect, and number of trials conducted. Results Our results show that Bayes factors, meta-analytic confidence intervals, and p-values often have similar performance. Bayes factors may perform better when the number of trials conducted is high and when trials have small sample sizes and clinically meaningful effects are not small, particularly in fields where the number of non-zero effects is relatively large. Conclusions Thinking about realistic effect sizes in conjunction with desirable levels of statistical evidence, as well as quantifying statistical evidence with Bayes factors may help improve decision-making in some circumstances.

Abstract Background There is growing interest in evaluating differences in healthcare interventions across routinely collected demographic characteristics. However, individual subgroup analyses in randomized controlled trials are often not prespecified, adjusted for multiple testing, or conducted using the appropriate statistical test for interaction, and therefore frequently lack credibility. Meta-analyses can be used to examine the validity of potential subgroup differences by collating evidence across trials. Here, we characterize the conduct and clinical translation of age-treatment subgroup analyses in Cochrane reviews. Methods For a random sample of 928 Cochrane intervention reviews of randomized trials, we determined how often subgroup analyses of age are reported, how often these analyses have a P

Abstract Background Different methodological choices such as inclusion/exclusion criteria and analytical models can yield different results and inferences when meta-analyses are performed. We explored the range of such differences, using several methodological choices for indirect comparison meta-analyses to compare nalmefene and naltrexone in the reduction of alcohol consumption as a case study. Methods All double-blind randomized controlled trials (RCTs) comparing nalmefene to naltrexone or one of these compounds to a placebo in the treatment of alcohol dependence or alcohol use disorders were considered. Two reviewers searched for published and unpublished studies in MEDLINE (August 2017), the Cochrane Library, Embase, and ClinicalTrials.gov and contacted pharmaceutical companies, the European Medicines Agency, and the Food and Drug Administration. The indirect comparison meta-analyses were performed according to different inclusion/exclusion criteria (based on medical condition, abstinence of patients before inclusion, gender, somatic and psychiatric comorbidity, psychological support, treatment administered and dose, treatment duration, outcome reported, publication status, and risk of bias) and different analytical models (fixed and random effects). The primary outcome was the vibration of effects (VoE), i.e. the range of different results of the indirect comparison between nalmefene and naltrexone. The presence of a “Janus effect” was investigated, i.e. whether the 1st and 99th percentiles in the distribution of effect sizes were in opposite directions. Results Nine nalmefene and 51 naltrexone RCTs were included. No study provided a direct comparison between the drugs. We performed 9216 meta-analyses for the indirect comparison with a median of 16 RCTs (interquartile range = 12–21) included in each meta-analysis. The standardized effect size was negative at the 1st percentile (− 0.29, favouring nalmefene) and positive at the 99th percentile (0.29, favouring naltrexone). A total of 7.1% (425/5961) of the meta-analyses with a negative effect size and 18.9% (616/3255) of those with a positive effect size were statistically significant (p

Abstract Background In clinical trials, study designs may focus on assessment of superiority, equivalence, or non-inferiority, of a new medicine or treatment as compared to a control. Typically, evidence in each of these paradigms is quantified with a variant of the null hypothesis significance test. A null hypothesis is assumed (null effect, inferior by a specific amount, inferior by a specific amount and superior by a specific amount, for superiority, non-inferiority, and equivalence respectively), after which the probabilities of obtaining data more extreme than those observed under these null hypotheses are quantified by p-values. Although ubiquitous in clinical testing, the null hypothesis significance test can lead to a number of difficulties in interpretation of the results of the statistical evidence. Methods We advocate quantifying evidence instead by means of Bayes factors and highlight how these can be calculated for different types of research design. Results We illustrate Bayes factors in practice with reanalyses of data from existing published studies. Conclusions Bayes factors for superiority, non-inferiority, and equivalence designs allow for explicit quantification of evidence in favor of the null hypothesis. They also allow for interim testing without the need to employ explicit corrections for multiple testing.

Background: Never before have clinical trials drawn as much public attention as those testing interventions for COVID-19. We aimed to describe the worldwide COVID-19 clinical research response and its evolution over the first 100 days of the pandemic. Methods: Descriptive analysis of planned, ongoing or completed trials by April 9, 2020 testing any intervention to treat or prevent COVID-19, systematically identified in trial registries, preprint servers, and literature databases. A survey was conducted of all trials to assess their recruitment status up to July 6, 2020. Results: Most of the 689 trials (overall target sample size 396,366) were small (median sample size 120; interquartile range [IQR] 60-300) but randomized (75.8%; n=522) and were often conducted in China (51.1%; n=352) or the USA (11%; n=76). 525 trials (76.2%) planned to include 155,571 hospitalized patients, and 25 (3.6%) planned to include 96,821 health-care workers. Treatments were evaluated in 607 trials (88.1%), frequently antivirals (n=144) or antimalarials (n=112); 78 trials (11.3%) focused on prevention, including 14 vaccine trials. No trial investigated social distancing. Interventions tested in 11 trials with >5,000 participants were also tested in 169 smaller trials (median sample size 273; IQR 90-700). Hydroxychloroquine alone was investigated in 110 trials. While 414 trials (60.0%) expected completion in 2020, only 35 trials (4.1%; 3,071 participants) were completed by July 6. Of 112 trials with detailed recruitment information, 55 had recruited

Background: Never before have clinical trials drawn as much public attention as those testing interventions for COVID-19. We aimed to describe the worldwide COVID-19 clinical research response and its evolution over the first 100 days of the pandemic. Methods: Descriptive analysis of planned, ongoing or completed trials by April 9, 2020 testing any intervention to treat or prevent COVID-19, systematically identified in trial registries, preprint servers, and literature databases. A survey was conducted of all trials to assess their recruitment status up to July 6, 2020. Results: Most of the 689 trials (overall target sample size 396,366) were small (median sample size 120; interquartile range [IQR] 60-300) but randomized (75.8%; n=522) and were often conducted in China (51.1%; n=352) or the USA (11%; n=76). 525 trials (76.2%) planned to include 155,571 hospitalized patients, and 25 (3.6%) planned to include 96,821 health-care workers. Treatments were evaluated in 607 trials (88.1%), frequently antivirals (n=144) or antimalarials (n=112); 78 trials (11.3%) focused on prevention, including 14 vaccine trials. No trial investigated social distancing. Interventions tested in 11 trials with >5,000 participants were also tested in 169 smaller trials (median sample size 273; IQR 90-700). Hydroxychloroquine alone was investigated in 110 trials. While 414 trials (60.0%) expected completion in 2020, only 35 trials (4.1%; 3,071 participants) were completed by July 6. Of 112 trials with detailed recruitment information, 55 had recruited

Serious concerns about research quality have catalysed a number of reform initiatives intended to improve transparency and reproducibility and thus facilitate self-correction, increase efficiency and enhance research credibility. Meta-research has evaluated the merits of some individual initiatives; however, this may not capture broader trends reflecting the cumulative contribution of these efforts. In this study, we manually examined a random sample of 250 articles in order to estimate the prevalence of a range of transparency and reproducibility-related indicators in the social sciences literature published between 2014 and 2017. Few articles indicated availability of materials (16/151, 11% [95% confidence interval, 7% to 16%]), protocols (0/156, 0% [0% to 1%]), raw data (11/156, 7% [2% to 13%]) or analysis scripts (2/156, 1% [0% to 3%]), and no studies were pre-registered (0/156, 0% [0% to 1%]). Some articles explicitly disclosed funding sources (or lack of; 74/236, 31% [25% to 37%]) and some declared no conflicts of interest (36/236, 15% [11% to 20%]). Replication studies were rare (2/156, 1% [0% to 3%]). Few studies were included in evidence synthesis via systematic review (17/151, 11% [7% to 16%]) or meta-analysis (2/151, 1% [0% to 3%]). Less than half the articles were publicly available (101/250, 40% [34% to 47%]). Minimal adoption of transparency and reproducibility-related research practices could be undermining the credibility and efficiency of social science research. The present study establishes a baseline that can be revisited in the future to assess progress.