Search

Abstract Background Guideline development on testing is known to be difficult for guideline developers. It requires consideration of various aspects, such as accuracy, purpose of testing, and consequences on management and people-important outcomes. This can be outlined in a test-management pathway. We aimed to create and user-test a step-by-step guide for guideline developers for designing a test-management pathway. Methods Developmental design with a co-creative strategy. We created a draft step-by-step guide, that was user tested in a workshop with 19 experts, and by interviewing 7 guideline panel members. Results Our proposed guide consists of five blocks of signalling questions: patients/population, index test(s), current practice/comparison/control, people-important outcomes, and the link between testing and outcome(s). The user testing led to refinement of the signalling questions, the use of inclusive terminology, and addition of a test-management pathway figure with detailed explanation. Conclusions The step-by-step guide for formulating focused guideline questions regarding healthcare related testing can help in identifying relevant characteristics of the population, tests, and outcomes and to create a test management pathway. This should facilitate the formulation of evidence-based guideline recommendations about healthcare related testing.

Abstract Patients with MYC rearranged (MYC-R) diffuse large B-cell lymphoma (DLBCL) have a poor prognosis. Previously, we demonstrated in a single-arm phase II trial (HOVON-130) that addition of lenalidomide to R-CHOP (R2CHOP) is well-tolerated and yields similar complete metabolic remission rates as more intensive chemotherapy regimens in literature. In parallel with this single-arm interventional trial, a prospective observational screening cohort (HOVON-900) was open in which we identified all newly diagnosed MYC-R DLBCL patients in the Netherlands. Eligible patients from the observational cohort that were not included in the interventional trial served as control group in the present risk-adjusted comparison. R2CHOP treated patients from the interventional trial (n = 77) were younger than patients in the R-CHOP control cohort (n = 56) (median age 63 versus 70 years, p = 0.018) and they were more likely to have a lower WHO performance score (p = 0.013). We adjusted for differences at baseline using 1:1 matching, multivariable analysis, and weighting using the propensity score to reduce treatment-selection bias. These analyses consistently showed improved outcome after R2CHOP with HRs of 0.53, 0.51, and 0.59, respectively, for OS, and 0.53, 0.59, and 0.60 for PFS. Thus, this non-randomized risk-adjusted comparison supports R2CHOP as an additional treatment option for MYC-R DLBCL patients.

A declaração dos Principais Itens para Relatar Revisões Sistemáticas e Meta-análises (PRISMA), publicada em 2009, foi desenvolvida para ajudar revisores sistemáticos a relatar de forma transparente por que a revisão foi feita, os métodos empregados e o que os autores encontraram. Na última década, os avanços na metodologia e terminologia de revisões sistemáticas exigiram a atualização da diretriz. A declaração PRISMA 2020 substitui a declaração de 2009 e inclui novas orientações para relato que refletem os avanços nos métodos para identificar, selecionar, avaliar e sintetizar estudos. A estrutura e apresentação dos itens foram modificadas para facilitar a implementação. Neste artigo, apresentamos a lista de checagem PRISMA 2020 de 27 itens, uma lista de checagem expandida que detalha as recomendações para relato para cada item, a lista de checagem PRISMA 2020 para resumos e os fluxogramas revisados para novas revisões e para atualização de revisões.

Abstract Background Randomized test-treatment studies aim to evaluate the clinical utility of diagnostic tests by providing evidence on their impact on patient health. However, the sample size calculation is affected by several factors involved in the test-treatment pathway, including the prevalence of the disease. Sample size planning is exposed to strong uncertainties in terms of the necessary assumptions, which have to be compensated for accordingly by adjusting prospectively determined study parameters during the course of the study. Method An adaptive design with a blinded sample size recalculation in a randomized test-treatment study based on the prevalence is proposed and evaluated by a simulation study. The results of the adaptive design are compared to those of the fixed design. Results The adaptive design achieves the desired theoretical power, under the assumption that all other nuisance parameters have been specified correctly, while wrong assumptions regarding the prevalence may lead to an over- or underpowered study in the fixed design. The empirical type I error rate is sufficiently controlled in the adaptive design as well as in the fixed design. Conclusion The consideration of a blinded recalculation of the sample size already during the planning of the study may be advisable in order to increase the possibility of success as well as an enhanced process of the study. However, the application of the method is subject to a number of limitations associated with the study design in terms of feasibility, sample sizes needed to be achieved, and fulfillment of necessary prerequisites.

Abstract Background Polygenic risk scores (PRS) could potentially improve breast cancer screening recommendations. Before a PRS can be considered for implementation, it needs rigorous evaluation, using performance measures that can inform about its future clinical value. Objectives To evaluate the prognostic performance of a regression model with a previously developed, prevalence-based PRS and age as predictors for breast cancer incidence in women from the Estonian biobank (EstBB) cohort; to compare it to the performance of a model including age only. Methods We analyzed data on 30,312 women from the EstBB cohort. They entered the cohort between 2002 and 2011, were between 20 and 89 years, without a history of breast cancer, and with full 5-year follow-up by 2015. We examined PRS and other potential risk factors as possible predictors in Cox regression models for breast cancer incidence. With 10-fold cross-validation we estimated 3- and 5-year breast cancer incidence predicted by age alone and by PRS plus age, fitting models on 90% of the data. Calibration, discrimination, and reclassification were calculated on the left-out folds to express prognostic performance. Results A total of 101 (3.33‰) and 185 (6.1‰) incident breast cancers were observed within 3 and 5 years, respectively. For women in a defined screening age of 50–62 years, the ratio of observed vs PRS-age modelled 3-year incidence was 0.86 for women in the 75–85% PRS-group, 1.34 for the 85–95% PRS-group, and 1.41 for the top 5% PRS-group. For 5-year incidence, this was respectively 0.94, 1.15, and 1.08. Yet the number of breast cancer events was relatively low in each PRS-subgroup. For all women, the model’s AUC was 0.720 (95% CI: 0.675–0.765) for 3-year and 0.704 (95% CI: 0.670–0.737) for 5-year follow-up, respectively, just 0.022 and 0.023 higher than for the model with age alone. Using a 1% risk prediction threshold, the 3-year NRI for the PRS-age model was 0.09, and 0.05 for 5 years. Conclusion The model including PRS had modest incremental performance over one based on age only. A larger, independent study is needed to assess whether and how the PRS can meaningfully contribute to age, for developing more efficient screening strategies.

Abstract Background Diagnostic accuracy studies aim to examine the diagnostic accuracy of a new experimental test, but do not address the actual merit of the resulting diagnostic information to a patient in clinical practice. In order to assess the impact of diagnostic information on subsequent treatment strategies regarding patient-relevant outcomes, randomized test-treatment studies were introduced. Various designs for randomized test-treatment studies, including an evaluation of biomarkers as part of randomized biomarker-guided treatment studies, are suggested in the literature, but the nomenclature is not consistent. Methods The aim was to provide a clear description of the different study designs within a pre-specified framework, considering their underlying assumptions, advantages as well as limitations and derivation of effect sizes required for sample size calculations. Furthermore, an outlook on adaptive designs within randomized test-treatment studies is given. Results The need to integrate adaptive design procedures in randomized test-treatment studies is apparent. The derivation of effect sizes induces that sample size calculation will always be based on rather vague assumptions resulting in over- or underpowered study results. Therefore, it might be advantageous to conduct a sample size re-estimation based on a nuisance parameter during the ongoing trial. Conclusions Due to their increased complexity, compared to common treatment trials, the implementation of randomized test-treatment studies poses practical challenges including a huge uncertainty regarding study parameters like the expected outcome in specific subgroups or disease prevalence which might affect the sample size calculation. Since research on adaptive designs within randomized test-treatment studies is limited so far, further research is recommended.

Lower socioeconomic status has been associated with higher colorectal cancer incidence and lower participation in population-based screening with faecal immunochemical testing (FIT) but regional variations in participation may also exist. We analysed differences in participation and yield in colorectal cancer screening by urban density level.Data of all invitees to the Dutch colorectal cancer screening programme in 2014–2015 were included. Primary outcomes were participation (returning FIT), FIT positive predictive value, and screening yield (advanced neoplasia detected in invitees). Differences were explored across five levels of urban density.In total 1,873,639 screening invitees were included. FIT participation was 77.3% in the lowest versus 62.8% in the highest urban areas (RR 1.23; 95%CI 1.23–1.24). FIT positive predictive value was 58.6% in the lowest versus 55.2% in the highest urban areas (RR 1.06; 95% CI 1.04–1.09). Screening yield was also higher in the lowest (2.1%-2.3%) compared to the highest urban areas (1.8%). Compared to socioeconomic status, differences in urban density were associated with larger differences in screening participation.In conclusion, participation is lower and fewer cases of advanced neoplasia are detected in areas with a high urban density in the Dutch colorectal cancer screening programme. Differences in urban density could be used in tailoring regional strategies to target barriers in colorectal cancer screening.

Abstract Background Rapid reviews (RRs) have emerged as an efficient alternative to time-consuming systematic reviews—they can help meet the demand for accelerated evidence synthesis to inform decision-making in healthcare. The synthesis of diagnostic evidence has important methodological challenges. Here, we performed an international survey to identify the current practice of producing RRs for diagnostic tests. Methods We developed and administered an online survey inviting institutions that perform RRs of diagnostic tests from all over the world. Results All participants (N = 25) reported the implementation of one or more methods to define the scope of the RR; however, only one strategy (defining a structured question) was used by ≥90% of participants. All participants used at least one methodological shortcut including the use of a previous review as a starting point (92%) and the use of limits on the search (96%). Parallelization and automation of review tasks were not extensively used (48 and 20%, respectively). Conclusion Our survey indicates a greater use of shortcuts and limits for conducting diagnostic test RRs versus the results of a recent scoping review analyzing published RRs. Several shortcuts are used without knowing how their implementation affects the results of the evidence synthesis in the setting of diagnostic test reviews. Thus, a structured evaluation of the challenges and implications of the adoption of these RR methods is warranted.

Background & Aims: Serum microRNA (miRNA) levels are known to change in non-alcoholic fatty liver disease (NAFLD) and may serve as useful biomarkers. This study aimed to profile miRNAs comprehensively at all NAFLD stages. Methods: We profiled 2,083 serum miRNAs in a discovery cohort (183 cases with NAFLD representing the complete NAFLD spectrum and 10 population controls). miRNA libraries generated by HTG EdgeSeq were sequenced by Illumina NextSeq. Selected serum miRNAs were profiled in 372 additional cases with NAFLD and 15 population controls by quantitative reverse transcriptase PCR. Results: Levels of 275 miRNAs differed between cases and population controls. Fewer differences were seen within individual NAFLD stages, but miR-193a-5p consistently showed increased levels in all comparisons. Relative to NAFL/non-alcoholic steatohepatitis (NASH) with mild fibrosis (stage 0/1), 3 miRNAs (miR-193a-5p, miR-378d, and miR378d) were increased in cases with NASH and clinically significant fibrosis (stages 2–4), 7 (miR193a-5p, miR-378d, miR-378e, miR-320b, miR-320c, miR-320d, and miR-320e) increased in cases with NAFLD activity score (NAS) 5–8 compared with lower NAS, and 3 (miR-193a-5p, miR-378d, and miR-378e) increased but 1 (miR-19b-3p) decreased in steatosis, activity, and fibrosis (SAF) activity score 2–4 compared with lower SAF activity. The significant findings for miR-193a-5p were replicated in the additional cohort with NAFLD. Studies in Hep G2 cells showed that following palmitic acid treatment, miR-193a-5p expression decreased significantly. Gene targets for miR-193a-5p were investigated in liver RNAseq data for a case subgroup (n = 80); liver GPX8 levels correlated positively with serum miR-193a-5p. Conclusions: Serum miR-193a-5p levels correlate strongly with NAFLD activity grade and fibrosis stage. MiR-193a-5p may have a role in the hepatic response to oxidative stress and is a potential clinically tractable circulating biomarker for progressive NAFLD. Lay summary: MicroRNAs (miRNAs) are small pieces of nucleic acid that may turn expression of genes on or off. These molecules can be detected in the blood circulation, and their levels in blood may change in liver disease including non-alcoholic fatty liver disease (NAFLD). To see if we could detect specific miRNA associated with advanced stages of NAFLD, we carried out miRNA sequencing in a group of 183 patients with NAFLD of varying severity together with 10 population controls. We found that a number of miRNAs showed changes, mainly increases, in serum levels but that 1 particular miRNA miR-193a-5p consistently increased. We confirmed this increase in a second group of cases with NAFLD. Measuring this miRNA in a blood sample may be a useful way to determine whether a patient has advanced NAFLD without an invasive liver biopsy.

Abstract Most randomized controlled trials evaluating medical interventions have a pre-specified hypothesis, which is statistically tested against the null hypothesis of no effect. In diagnostic accuracy studies, study hypotheses are rarely pre-defined and sample size calculations are usually not performed, which may jeopardize scientific rigor and can lead to over-interpretation or “spin” of study findings. In this paper, we propose a strategy for defining meaningful hypotheses in diagnostic accuracy studies. Based on the role of the index test in the clinical pathway and the downstream consequences of test results, the consequences of test misclassifications can be weighed, to arrive at minimally acceptable criteria for pre-defined test performance: levels of sensitivity and specificity that would justify the test’s intended use. Minimally acceptable criteria for test performance should form the basis for hypothesis formulation and sample size calculations in diagnostic accuracy studies.

Abstract Background Assessment of diagnostic tests, broadly defined as any element that aids in the collection of additional information for further clarification of a patient’s health status, has increasingly become a critical issue in health policy and decision-making. Diagnostic evidence, including the accuracy of a medical test for a target condition, is commonly appraised using standard systematic review methodology. Owing to the considerable time and resources required to conduct these, rapid reviews have emerged as a pragmatic alternative by tailoring methods according to the decision maker’s circumstances. However, it is not known if streamlining methodological aspects has an impact on the validity of evidence synthesis. Furthermore, due to the particular nature and complexity of the appraisal of diagnostic accuracy, there is need for detailed guidance on how to conduct rapid reviews of diagnostic tests. In this study, we aim to identify the methods currently used by rapid review developers to synthesize evidence on diagnostic test accuracy, as well as to analyze potential shortcomings and challenges related to these methods. Methods We will carry out a two-fold approach: (1) an international survey of professionals working in organizations that develop rapid reviews of diagnostic tests, in terms of the methods and resources used by these agencies when conducting rapid reviews, and (2) semi-structured interviews with senior-level individuals to further explore and validate the findings from the survey and to identify challenges in conducting rapid reviews. We will use STATA 15.0 for quantitative analyses and framework analysis for qualitative analyses. We will ensure protection of data during all stages. Discussion The main result of this research will be a map of methods and resources currently used for conducting rapid reviews of diagnostic test accuracy, as well as methodological shortcomings and potential solutions in diagnostic knowledge synthesis that require further research.

Background: Most studies evaluating the effect of Xpert MTB/RIF testing for tuberculosis (TB) concluded that it did not reduce overall mortality compared to usual care. We conducted a systematic review to assess whether key study design and execution features contributed to earlier identification of patients with TB and decreased pre-treatment loss to follow-up, thereby reducing the potential impact of Xpert MTB/RIF testing. Methods: We searched the Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE, and Scopus for literature published from 1st January 2009 to February 2019. We included all primary intervention studies that had evaluated the effect of Xpert MTB/RIF on mortality compared to usual care in participants with presumptive pulmonary TB. We critically reviewed features of included studies across: Study setting and context, Study population, Participant recruitment and enrolment, Study procedures, and Study follow-up. Results: We included seven randomised and one non-randomised study. All included studies demonstrated relative reductions in overall mortality in the Xpert MTB/RIF arm ranging from 6% to 40%. However, mortality reduction was reported to be statistically significant in two studies. Study features that could explain the lack of observed effect on mortality included: the higher quality of care at study sites; inclusion of patients with a higher pre-test probability of TB leading to higher than expected empirical rates; performance of additional diagnostic testing not done in usual care leading to increased TB diagnosis or empiric treatment initiation; the recruitment of participants likely to return for follow-up; and involvement of study staff in ensuring adherence with care and follow-up. Conclusion: Most studies of Xpert MTB/RIF were designed and conducted in a manner that resulted in more patients being diagnosed and treated for TB, minimising the potential difference in mortality Xpert MTB/RIF testing could have achieved compared to usual care.

Introduction Association between elevated cytokeratin 18 (CK-18) levels and hepatocyte death has made circulating CK-18 a candidate biomarker to differentiate non-alcoholic fatty liver from non-alcoholic steatohepatitis (NASH). Yet studies produced variable diagnostic performance. We aimed to provide summary estimates with increased precision for the accuracy of CK-18 (M30, M65) in detecting NASH and fibrosis among non-alcoholic fatty liver disease (NAFLD) adults. Methods We searched five databases to retrieve studies evaluating CK-18 against a liver biopsy in NAFLD adults. Reference screening, data extraction and quality assessment (QUADAS-2) were independently conducted by two authors. Meta-analyses were performed for five groups based on the CK-18 antigens and target conditions, using one of two methods: linear mixed-effects multiple thresholds model or bivariate logit-normal random-effects model. Results We included 41 studies, with data on 5,815 participants. A wide range of disease prevalence was observed. No study reported a pre-defined cut-off. Thirty of 41 studies provided sufficient data for inclusion in any of the meta-analyses. Summary AUC [95% CI] were: 0.75 [0.69–0.82] (M30) and 0.82 [0.69–0.91] (M65) for NASH; 0.73 [0.57–0.85] (M30) for fibrotic NASH; 0.68 (M30) for significant (F2-4) fibrosis; and 0.75 (M30) for advanced (F3-4) fibrosis. Thirteen studies used CK-18 as a component of a multimarker model. Conclusions For M30 we found lower diagnostic accuracy to detect NASH compared to previous meta-analyses, indicating a limited ability to act as a stand-alone test, with better performance for M65. Additional external validation studies are needed to obtain credible estimates of the diagnostic accuracy of multimarker models.

Abstract Background Despite the growing interest in developing markers for predicting treatment response and optimizing treatment decisions, an appropriate methodology to identify, combine and evaluate such markers has been slow to develop. We propose a step-by-step strategy for analysing data from existing randomised trials with the aim of identifying a multi-marker model for guiding decisions about treatment. Methods We start with formulating the treatment selection problem, continue with defining the treatment threshold, prepare a list of candidate markers, develop the model, apply the model to estimate individual treatment effects, and evaluate model performance in the study group of patients who meet the trial eligibility criteria. In this process, we rely on some well-known techniques for multivariable prediction modelling, but focus on predicting benefit from treatment, rather than outcome itself. We present our approach using data from a randomised trial in which 808 women with multiple pregnancy were assigned to cervical pessary or control, to prevent adverse perinatal outcomes. Overall, cervical pessary did not reduce the risk of adverse perinatal outcomes. Results The treatment threshold was zero. We had a preselected list of 5 potential markers and developed a logistic model including the markers, treatment and all marker-by-treatment interaction terms. The model was well calibrated and identified 35% (95% confidence interval (CI) 32 to 39%) of the trial participants as benefitting from pessary insertion. We estimated that the risk of adverse outcome could be reduced from 13.5 to 8.1% (5.4% risk reduction; 95% CI 2.1 to 8.6%) through model-based selective pessary insertion. The next step is external validation upon existence of independent trial data. Conclusions We suggest revisiting existing trials data to explore whether differences in treatment benefit can be explained by differences in baseline characteristics of patients. This could lead to treatment selection tools which, after validation in comparable existing trials, can be introduced into clinical practice for guiding treatment decisions in future patients.

The prevalence and severity of non-alcoholic fatty liver disease (NAFLD) is increasing, yet adequately validated tests for care paths are limited and non-invasive markers of disease progression are urgently needed. The aim of this work was to summarize the performance of Pro-C3, a biomarker of active fibrogenesis, in detecting significant fibrosis (F ≥ 2), advanced fibrosis (F ≥ 3), cirrhosis (F4) and non-alcoholic steatohepatitis (NASH) in patients with NAFLD. A sensitive search of five databases was performed in July 2021. Studies reporting Pro-C3 measurements and liver histology in adults with NAFLD without co-existing liver diseases were eligible. Meta-analysis was conducted by applying a bivariate random effects model to produce summary estimates of Pro-C3 accuracy. From 35 evaluated reports, eight studies met our inclusion criteria; 1568 patients were included in our meta-analysis of significant fibrosis and 2058 in that of advanced fibrosis. The area under the summary curve was 0.81 (95% CI 0.77–0.84) in detecting significant fibrosis and 0.79 (95% CI 0.73–0.82) for advanced fibrosis. Our results support Pro-C3 as an important candidate biomarker for non-invasive assessment of liver fibrosis in NAFLD. Further direct comparisons with currently recommended non-invasive tests will demonstrate whether Pro-C3 panels can outperform these tests, and improve care paths for patients with NAFLD.

Abstract Background Personalized healthcare relies on the identification of factors explaining why individuals respond differently to the same intervention. Analyses identifying such factors, so called predictors and moderators, have their own set of assumptions and limitations which, when violated, can result in misleading claims, and incorrect actions. The aim of this study was to develop a checklist for critically appraising the results of predictor and moderator analyses by combining recommendations from published guidelines and experts in the field. Methods Candidate criteria for the checklist were retrieved through systematic searches of the literature. These criteria were evaluated for appropriateness using a Delphi procedure. Two Delphi rounds yielded a pilot checklist, which was tested on a set of papers included in a systematic review on reinforced home-based palliative care. The results of the pilot informed a third Delphi round, which served to finalize the checklist. Results Forty-nine appraisal criteria were identified in the literature. Feedback was obtained from fourteen experts from (bio)statistics, epidemiology and other associated fields elicited via three Delphi rounds. Additional feedback from other researchers was collected in a pilot test. The final version of our checklist included seventeen criteria, covering the design (e.g. a priori plausibility), analysis (e.g. use of interaction tests) and results (e.g. complete reporting) of moderator and predictor analysis, together with the transferability of the results (e.g. clinical importance). There are criteria both for individual papers and for bodies of evidence. Conclusions The proposed checklist can be used for critical appraisal of reported moderator and predictor effects, as assessed in randomized or non-randomized studies using individual participant or aggregate data. This checklist is accompanied by a user’s guide to facilitate implementation. Its future use across a wide variety of research domains and study types will provide insights about its usability and feasibility.

Background: Studies evaluating the impact of Xpert MTB/RIF testing for tuberculosis (TB) have demonstrated varied effects on health outcomes with many studies showing inconclusive results. We explored perceptions among diverse stakeholders about studies evaluating the impact of TB diagnostic tests, and identified suggestions for improving these studies. Methods: We used purposive sampling with consideration for differing expertise and geographical balance and conducted in depth semi-structured interviews. We interviewed English-speaking participants, including TB patients, and others involved in research, care or decision-making about TB diagnostics. We used the thematic approach to code and analyse the interview transcripts. Results: We interviewed 31 participants. Our study showed that stakeholders had different expectations with regard to test impact and how it is measured. TB test impact studies were perceived to be important for supporting implementation of tests but there were concerns about the unrealistic expectations placed on tests to improve outcomes in health systems with many influencing factors. To improve TB test impact studies, respondents suggested conducting health system assessments prior to the study; developing clear guidance on the study methodology and interpretation; improving study design by describing questions and interventions that consider the influences of the health-care ecosystem on the diagnostic test; selecting the target population at the health-care level most likely to benefit from the test; setting realistic targets for effect sizes in the sample size calculations; and interpreting study results carefully and avoiding categorisation and interpretation of results based on statistical significance alone. Researchers should involve multiple stakeholders in the design of studies. Advocating for more funding to support robust studies is essential. Conclusion: TB test impact studies were perceived to be important to support implementation of tests but there were concerns about their complexity. Process evaluations of their health system context and guidance for their design and interpretation are recommended.

We aimed to evaluate the performance of the Khorana score in predicting venous thromboembolic events in ambulatory cancer patients. Embase and MEDLINE were searched from January 2008 to June 2018 for studies which evaluated the Khorana score. Two authors independently screened studies for eligibility, extracted data, and assessed risk of bias. Additional data on the 6-month incidence of venous thromboembolism were sought by contacting corresponding authors. The incidence in each Khorana score risk group was estimated with random effects meta-analysis. A total of 45 articles and eight abstracts were included, comprising 55 cohorts enrolling 34,555 ambulatory cancer patients. For 27,849 patients (81%), 6-month follow-up data were obtained. Overall, 19% of patients had a Khorana score of 0 points, 64% a score of 1 or 2 points, and 17% a score of 3 or more points. The incidence of venous thromboembolism in the first six months was 5.0% (95%CI: 3.9-6.5) in patients with a low-risk Khorana score (0 points), 6.6% (95%CI: 5.6-7.7) in those with an intermediate-risk Khorana score (1 or 2 points), and 11.0% (95%CI: 8.8-13.8) in those with a high-risk Khorana score (3 points or higher). Of the patients with venous thromboembolism in the first six months, 23.4% (95%CI: 18.4-29.4) had been classified as high risk according to the Khorana score. In conclusion, the Khorana score can be used to select ambulatory cancer patients at high risk of venous thromboembolism for thromboprophylaxis; however, most events occur outside this high-risk group.

Abstract Background To describe approaches used in systematic reviews of diagnostic test accuracy studies for assessing variability in estimates of accuracy between studies and to provide guidance in this area. Methods Meta-analyses of diagnostic test accuracy studies published between May and September 2012 were systematically identified. Information on how the variability in results was investigated was extracted. Results Of the 53 meta-analyses included in the review, most (n=48; 91 %) presented variability in diagnostic accuracy estimates visually either through forest plots or ROC plots and the majority (n=40; 75 %) presented a test or statistical measure for the variability. Twenty-eight reviews (53 %) tested for variability beyond chance using Cochran’s Q test and 31 (58 %) reviews quantified it with I2. 7 reviews (13 %) presented between-study variance estimates (τ2) from random effects models and 3 of these presented a prediction interval or ellipse to facilitate interpretation. Half of all the meta-analyses specified what was considered a significant amount of variability (n=24; 49 %). Conclusions Approaches to assessing variability in estimates of accuracy varied widely between diagnostic test accuracy reviews and there is room for improvement. We provide initial guidance, complemented by an overview of the currently available approaches.

In ambulatory patients with solid cancer, routine thromboprophylaxis to prevent venous thromboembolism is not recommended. Several risk prediction scores to identify cancer patients at high risk of venous thromboembolism have been proposed, but their clinical usefulness remains a matter of debate. We evaluated and directly compared the performance of the Khorana, Vienna, PROTECHT, and CONKO scores in a multinational, prospective cohort study. Patients with advanced cancer were eligible if they were due to undergo chemotherapy or had started chemotherapy in the previous three months. The primary outcome was objectively confirmed symptomatic or incidental deep vein thrombosis or pulmonary embolism during a 6-month follow-up period. A total of 876 patients were enrolled, of whom 260 (30%) had not yet received chemotherapy. Fifty-three patients (6.1%) developed venous thromboembolism. The c-statistics of the scores ranged from 0.50 to 0.57. At the conventional positivity threshold of 3 points, the scores classified 13–34% of patients as high-risk; the 6-month incidence of venous thromboembolism in these patients ranged from 6.5% (95%CI: 2.8–12) for the Khorana score to 9.6% (95%CI: 6.6–13) for the PROTECHT score. High-risk patients had a significantly increased risk of venous thromboembolism when using the Vienna (subhazard ratio 1.7; 95%CI: 1.0–3.1) or PROTECHT (subhazard ratio 2.1; 95%CI: 1.2–3.6) scores. In conclusion, the prediction scores performed poorly in predicting venous thromboembolism in cancer patients. The Vienna CATS and PROTECHT scores appear to discriminate better between low- and high-risk patients, but further improvements are needed before they can be considered for introduction into clinical practice.

Background Several open science-promoting initiatives have been proposed to improve the quality of biomedical research, including initiatives for assessing researchers’ open science behaviour as criteria for promotion or tenure. Yet there is limited evidence to judge whether the interventions are effective. This review aimed to summarise the literature, identifying open science practices related to researcher assessment, and map the extent of evidence of existing interventions implemented to assess researchers and research impact. Methods A scoping review using the Joanna Briggs Institute Scoping Review Methodology was conducted. We included all study types that described any open science practice-promoting initiatives proposed or implemented to assess researchers and research impact, in health sciences, biomedicine, psychology, and economics. Data synthesis was quantitative and descriptive. Results Among 18,020 identified documents, 27 articles were selectedfor analysis. Most of the publications were in the field of health sciences (n = 10), and were indicated as research culture, perspective, commentary, essay, proceedings of a workshop, research article, world view, opinion, research note, editorial, report, and research policy articles (n = 22). The majority of studies proposed recommendations to address problems regarding threats to research rigour and reproducibility that were multi-modal (n = 20), targeting several open science practices. Some of the studies based their proposed recommendations on further evaluation or extension of previous initiatives. Most of the articles (n = 20) did not discuss implementation of their proposed intervention. Of the 27 included articles, 10 were cited in policy documents, with The Leiden Manifesto being the most cited (104 citations). Conclusion This review provides an overview of proposals to integrate open science into researcher assessment. The more promising ones need evaluation and, where appropriate, implementation. Study registration https://osf.io/ty9m7