Search

Abstract Background How to assess the impact of research is of growing interest to funders, policy makers and researchers mainly to understand the value of investments and to increase accountability. Broadly speaking the term "research impact" refers to the contribution of research activities to achieve desired societal outcomes. The aim of this overview is to identify the most common approaches to research impact assessment, categories of impact and their respective indicators. Methods We systematically searched the relevant literature (PubMed, The Cochrane Library (1990-2009)) and funding agency websites. We included systematic reviews, theoretical and methodological papers, and empirical case-studies on how to evaluate research impact. We qualitatively summarised the included reports, as well the conceptual frameworks. Results We identified twenty-two reports belonging to four systematic reviews and 14 primary studies. These publications reported several theoretical frameworks and methodological approaches (bibliometrics, econometrics, ad hoc case studies). The "payback model" emerged as the most frequently used. Five broad categories of impact were identified: a) advancing knowledge, b) capacity building, c) informing decision-making, d) health benefits, e) broad socio-economic benefits. For each proposed category of impact we summarized a set of indicators whose pros and cons are presented and briefly discussed. Conclusions This overview is a comprehensive, yet descriptive, contribution to summarize the conceptual framework and taxonomy of an heterogeneous and evolving area of research. A shared and comprehensive conceptual framework does not seem to be available yet and its single components (epidemiologic, economic, and social) are often valued differently in different models.

Abstract Background Since September 2005 the International Committee of Medical Journal Editors has required that trials be registered in accordance with the World Health Organization (WHO) minimum dataset, in order to be considered for publication. The objective is to evaluate registries' and individual trial records' compliance with the 2006 version of the WHO minimum data set. Methods A retrospective evaluation of 21 online clinical trial registries (international, national, specialty, pharmaceutical industry and local) from April 2005 to February 2007 and a cross-sectional evaluation of a stratified random sample of 610 trial records from the 21 registries. Results Among 11 registries that provided guidelines for registration, the median compliance with the WHO criteria were 14 out of 20 items (range 6 to 20). In the period April 2005–February 2007, six registries increased their compliance by six data items, on average. None of the local registry websites published guidelines on the trial data items required for registration. Slightly more than half (330/610; 54.1%, 95% CI 50.1% – 58.1%) of trial records completed the contact details criteria while 29.7% (181/610, 95% CI 26.1% – 33.5%) completed the key clinical and methodological data fields. Conclusion While the launch of the WHO minimum data set seemed to positively influence registries with better standardisation of approaches, individual registry entries are largely incomplete. Initiatives to ensure quality assurance of registries and trial data should be encouraged. Peer reviewers and editors should scrutinise clinical trial registration records to ensure consistency with WHO's core content requirements when considering trial-related publications.

Abstract Background In an effort to ensure that all physicians have access to valid and reliable evidence on drug effectiveness, the Italian Drug Agency sponsored a free-access e-learning system, based on Clinical Evidence, called ECCE. Doctors have access to an electronic version and related clinical vignettes. Correct answers to the interactive vignettes provide Continuing Medical Education credits. The aims of this trial are to establish whether the e-learning program (ECCE) increases physicians' basic knowledge about common clinical scenarios, and whether ECCE is superior to the passive diffusion of information through the printed version of Clinical Evidence. Design All Italian doctors naïve to ECCE will be randomised to three groups. Group one will have access to ECCE for Clinical Evidence chapters and vignettes lot A and will provide control data for Clinical Evidence chapters and vignettes lot B; group two vice versa; group three will receive the concise printed version of Clinical Evidence. There are in fact two designs: a before and after pragmatic trial utilising a two by two incomplete block design (group one versus group two) and a classical design (group one and two versus group three). The primary outcome will be the retention of Clinical Evidence contents assessed from the scores for clinical vignettes selected from ECCE at least six months after the intervention. To avoid test-retest effects, we will randomly select vignettes out of lot A and lot B, avoiding repetitions. In order to preserve the comparability of lots, we will select vignettes with similar, optimal psychometric characteristics. Trial registration ISRCTN27453314

Abstract Background Research has shown that many healthcare professionals have problems with guidelines as they would prefer to be given all relevent information relevent to decision-making rather than being told what they should do. This study assesses doctors' judgement of the validity, relevance, clarity and usability of the Italian translation of Clinical Evidence (CE) after its free distribution launched by the Italian Ministry of Health Methods Opinions elicited using a standardised questionnaire delivered either by mail or during educational or professional meetings Results Twenty percent (n = 1350) doctors participated the study. Most of them found CE's content valid, useful and relevant for their clinical practice, and said CE can foster communications among clinicians, particularly among GPs and specialists. Hospital doctors (63%) more often than GPs (48%) read the detailed presentation of individual chapters. Twenty-nine percent said CE brought changes in their clinical practice. Doctors appreciated CE's nature of an evidence-based information compendium and would have not preferred a collection of practice guidelines. Conclusions Overall, the pilot initiative launched by the Italian Ministry of Health seems to have been well received and to support the subsequent decision to make the Italian edition of Clinical Evidence concise available to all doctors practising in the country. Local implementation initiatives should be warranted to favour doctor's use of CE.