Search

Background: By including the needs and perspectives of relevant stakeholders, co-creation is seen as a promising approach for tackling complex public health problems. However, recommendations and guidance on how to plan and implement co-creation are lacking. By identifying and analysing existing implementation and evaluation frameworks for public health, this study aims to offer key recommendations for professional stakeholders and researchers wanting to adopt a co-creation approach to public health interventions. Methods: Firstly, PubMed and CINAHL databases were screened for articles introducing original implementation and evaluation frameworks for public health interventions. Backwards snowballing techniques were applied to the included papers. Secondly, identified frameworks were classified and relevant data extracted, including steps and constructs present in the frameworks. Lastly, recommendations were derived by conducting thematic analysis on the included frameworks. Results: Thirty frameworks were identified and data related to their nature and scope extracted. The frameworks' prominent steps and constructs were also retrieved. Recommendations related to implementation and evaluation in the context of co-creation were included. Conclusion: When engaging in co-creation, we recommend including implementation considerations from an early stage and suggest adopting a systems thinking as a way to explore multiple levels of influence, contextual settings and systems from an early planning stage. We highlight the importance of partnering with stakeholders and suggest applying an evaluation design that is iterative and cyclical, which pays particular attention to the experience of the engaged co-creators. [ABSTRACT FROM AUTHOR]

Clinical research infrastructure is one of the unsung heroes of the scientific response to the current COVID-19 pandemic. The extensive, long-term funding into research support structures, skilled people, and technology allowed the United Kingdom research response to move off the starting blocks at pace by utilizing pre-existing platforms. The increasing focus from funders on evaluating the outcomes and impact of research infrastructure investment requires both a reframing and progression of the current models in order to address the contribution of the underlying support infrastructure. The majority of current evaluation/outcome models focus on a "pipeline" approach using a methodology which follows the traditional research funding route with the addition of quantitative metrics. These models fail to embrace the complexity caused by the interplay of previous investment, the coalescing of project outputs from different funders, the underlying infrastructure investment, and the parallel development across different parts of the system. Research infrastructure is the underpinning foundation of a project-driven research system and requires long-term, sustained funding and capital investment to maintain scientific and technological expertise. Therefore, the short-term focus on quantitative metrics that are easy to collect and interpret and that can be assessed in a roughly 5-year funding cycle needs to be addressed. The significant level of investment in research infrastructure necessitates investment to develop bespoke methodologies that develop fit-for-purpose, longer-term/continual approach(es) to evaluation. Real-world research should reflect real-world evaluation and allow for the accrual of a narrative of value indicators that build a picture of the contribution of infrastructure to research outcomes. The linear approach is not fit for purpose, the research endeavour is a complex, twisted road, and the evaluation approach needs to embrace this complexity through the development of realist approaches and the rapidly evolving data ecosystem. This paper sets out methodological challenges and considers the need to develop bespoke methodological approaches to allow a richer assessment of impact, contribution, attribution, and evaluation of research infrastructure. This paper is the beginning of a conversation that invites the community to "take up the mantle" and tackle the complexity of real-world research translation and evaluation. [ABSTRACT FROM AUTHOR]

Background: Latin America and the Caribbean (LAC) are among those regions most affected by the COVID-19 pandemic worldwide. The COVID-19 pandemic has strained health systems in the region. In this context of severe healthcare resource constraints, there is a need for systematic priority-setting to support decision-making which ensures the best use of resources while considering the needs of the most vulnerable groups. The aim of this paper was to provide a critical description and analysis of how health systems considered priority-setting in the COVID-19 response and preparedness plans of a sample of 14 LAC countries; and to identify the associated research gaps. Methods: A documentary analysis of COVID-19 preparedness and response plans was performed in a sample of 14 countries in the LAC region. We assessed the degree to which the documented priority-setting processes adhered to established quality indicators of effective priority-setting included in the Kapiriri and Martin framework. We conducted a descriptive analysis of the degree to which the reports addressed the quality parameters for each individual country, as well as a cross-country comparison to explore whether parameters varied according to independent variables. Results: While all plans were led and supported by the national governments, most included only a limited number of quality indicators for effective priority-setting. There was no systematic pattern between the number of quality indicators and the country's health system and political contexts; however, the countries that had the least number of quality indicators tended to be economically disadvantaged. Conclusion: This study adds to the literature by providing the first descriptive analysis of the inclusion of priority-setting during a pandemic, using the case of COVID-19 response and preparedness plans in the LAC region. The analysis found that despite the strong evidence of political will and stakeholder participation, none of the plans presented a clear priority-setting process, or used a formal priority-setting framework, to define interventions, populations, geographical regions, healthcare setting or resources prioritized. There is need for case studies that analyse how priority-setting actually occurred during the COVID-19 pandemic and the degree to which the implementation reflected the plans and the parameters of effective priority-setting, as well as the impact of the prioritization processes on population health, with a focus on the most vulnerable groups. [ABSTRACT FROM AUTHOR]

Background: Knowledge translation (KT) is a dynamic and iterative process that includes synthesis, dissemination, exchange and ethically sound application of knowledge to yield beneficial outcomes for society. Effective KT requires researchers to play an active role in promoting evidence uptake. This paper presents a systematised review of evidence on low- and middle-income country (LMIC) researchers' KT capacity, practice and interventions for enhancing their KT practice (support) with the aim of identifying gaps and informing future research and interventions.Methods: An electronic search for peer-reviewed publications focusing on LMIC researchers' KT capacity, practice and support across all academic fields, authored in English and from the earliest records available to February 2019, was conducted using PubMed and Scopus. Selected studies were appraised using the Mixed Methods Appraisal Tool, data pertaining to publication characteristics and study design extracted, and an a priori thematic analysis of reported research findings completed.Results: The search resulted in 334 screened articles, of which 66 met the inclusion criteria. Most (n = 43) of the articles presented original research findings, 22 were commentaries and 1 was a structured review; 47 articles reported on researchers' KT practice, 12 assessed the KT capacity of researchers or academic/research institutions and 9 reported on KT support for researchers. More than half (59%) of the articles focused on sub-Saharan Africa and the majority (91%) on health research. Most of the primary studies used the case study design (41%). The findings suggest that LMIC researchers rarely conduct KT and face a range of barriers at individual and institutional levels that limit their KT practice, including inadequate KT knowledge and skills, particularly for communicating research and interacting with research end-users, insufficient funding, and inadequate institutional guidelines, structures and incentives promoting KT practice. Furthermore, the evidence-base on effective interventions for enhancing LMIC researchers' KT practice is insufficient and largely of weak quality.Conclusions: More high-quality research on researchers' KT capacity, practice and effective KT capacity strengthening interventions is needed. Study designs that extend beyond case studies and descriptive studies are recommended, including better designed evaluation studies, e.g. use of realist approaches, pragmatic trials, impact evaluations, implementation research and participatory action research. [ABSTRACT FROM AUTHOR]

Background: GRADE-CERQual (Confidence in the Evidence from Reviews of Qualitative Research) is a methodological approach to systematically and transparently assess how much confidence decision makers can place in individual review findings from qualitative evidence syntheses. The number of reviews applying GRADE-CERQual is rapidly expanding in guideline and other decision-making contexts. The objectives of this evaluation were, firstly, to describe the uptake of GRADE-CERQual in qualitative evidence synthesis by review authors and, secondly, to assess both reporting of and fidelity to the approach. Methods: The evaluation had two parts. Part 1 was a citation analysis and descriptive overview of the literature citing GRADE-CERQual. Authors worked together to code and chart the citations, first by title and abstract and second by full text. Part 2 was an assessment and analysis of fidelity to, and reporting of, the GRADE-CERQual approach in included reviews. We developed fidelity and reporting questions and answers based on the most recent guidance for GRADE-CERQual and then used NVivo12 to document assessments in a spreadsheet and code full-text PDF articles for any concerns that had been identified. Our assessments were exported to Excel and we applied count formulae to explore patterns in the data. We employed a qualitative content analysis approach in NVivo12 to sub-coding all the data illustrating concerns for each reporting and fidelity criteria. Results: 233 studies have applied the GRADE-CERQual approach, with most (n = 225, 96.5%) in the field of health research. Many studies (n = 97/233, 41.6%) were excluded from full fidelity and reporting assessment because they demonstrated a serious misapplication of GRADE-CERQual, for example interpreting it as a quality appraisal tool for primary studies or reviews. For the remaining studies that applied GRADE-CERQual to assess confidence in review findings, the main areas of reporting concern involved terminology, labelling and completeness. Fidelity concerns were identified in more than half of all studies assessed. Conclusions: GRADE-CERQual is being used widely within qualitative evidence syntheses and there are common reporting and fidelity issues. Most of these are avoidable and we highlight these as gaps in knowledge and guidance for applying the GRADE-CERQual approach. [ABSTRACT FROM AUTHOR]

Given their inherent complexity, we need a better understanding of what is happening inside the "black box" of population health interventions. The theory-driven intervention/evaluation paradigm is one approach to addressing this question. However, barriers related to semantic or practical issues stand in the way of its complete integration into evaluation designs. In this paper, we attempt to clarify how various theories, models and frameworks can contribute to developing a context-dependent theory, helping us to understand the black box of population health interventions and to acknowledge their complexity. To achieve this goal, we clarify what could be referred to as "theory" in the theory-driven evaluation of the interventional system, distinguishing it from other models, frameworks and classical theories. In order to evaluate the interventional system with a theory-driven paradigm, we put forward the concept of interventional system theory (ISyT), which combines a causal theory and an action model. We suggest that an ISyT could guide evaluation processes, whatever evaluation design is applied, and illustrate this alternative method through different examples of studies. We believe that such a clarification can help to promote the use of theories in complex intervention evaluations, and to identify ways of considering the transferability and scalability of interventions. [ABSTRACT FROM AUTHOR]

Background: Health and medical research funding agencies are increasingly interested in measuring the impact of funded research. We present a research impact case study for the first four years of an Australian National Health and Medical Research Council funded Centre of Research Excellence in Cardiovascular Outcomes Improvement (2016-2020). The primary aim of this paper was to explore the application of a research impact matrix to assess the impact of cardiovascular outcomes improvement research.Methods: We applied a research impact matrix developed from a systematic review of existing methodological frameworks used to measure research impact. This impact matrix was used as a bespoke tool to identify and understand various research impacts over different time frames. Data sources included a review of existing internal documentation from the research centre and publicly available information sources, informal iterative discussions with 10 centre investigators, and confirmation of information from centre grant and scholarship recipients.Results: By July 2019, the impact on the short-term research domain category included over 41 direct publications, which were cited over 87 times (median journal impact factor of 2.84). There were over 61 conference presentations, seven PhD candidacies, five new academic collaborations, and six new database linkages conducted. The impact on the mid-term research domain category involved contributions towards the development of a national cardiac registry, cardiovascular guidelines, application for a Medicare Benefits Schedule reimbursement item number, introduction of patient-reported outcome measures into several databases, and the establishment of nine new industry collaborations. Evidence of long-term impacts were described as the development and use of contemporary management for aortic stenosis, a cardiovascular risk prediction model and prevention targets in several data registries, and the establishment of cost-effectiveness for stenting compared to surgery.Conclusions: We considered the research impact matrix a feasible tool to identify evidence of academic and policy impact in the short- to midterm; however, we experienced challenges in capturing long-term impacts. Cost containment and broader economic impacts represented another difficult area of impact to measure. [ABSTRACT FROM AUTHOR]

Communities of Practice (CoPs) are groups of people that interact regularly to deepen their knowledge on a specific topic. Thanks to information and communication technologies, CoPs can involve experts distributed across countries and adopt a 'transnational' membership. This has allowed the strategy to be applied to domains of knowledge such as health policy with a global perspective. CoPs represent a potentially valuable tool for producing and sharing explicit knowledge, as well as tacit knowledge and implementation practices. They may also be effective in creating links among the different 'knowledge holders' contributing to health policy (e.g., researchers, policymakers, technical assistants, practitioners, etc.). CoPs in global health are growing in number and activities. As a result, there is an increasing need to document their progress and evaluate their effectiveness. This paper represents a first step towards such empirical research as it aims to provide a conceptual framework for the analysis and assessment of transnational CoPs in health policy. The framework is developed based on the findings of a literature review as well as on our experience, and reflects the specific features and challenges of transnational CoPs in health policy. It organizes the key elements of CoPs into a logical flow that links available resources and the capacity to mobilize them, with knowledge management activities and the expansion of knowledge, with changes in policy and practice and, ultimately, with an improvement in health outcomes. Additionally, the paper addresses the challenges in the operationalization and empirical application of the framework. [ABSTRACT FROM AUTHOR]

Background: Health researchers and funders are increasingly consulting with stakeholders to set their research agendas but these activities are rarely evaluated. The Cochrane Consumers and Communication Group (CCCG) conducted a priority-setting project for systematic reviews in partnership with stakeholders (consumers/patients, health professionals, policy-makers and others). In this paper, we aim to describe our evaluation of the project's processes and outcomes.Methods: We used a 10-element conceptual framework designed to evaluate processes (e.g. stakeholder engagement, use of explicit process) and outcomes (e.g. improved decision-making quality, stakeholder acceptance and understanding) of health priority-setting. Data sources included empirical data (feedback surveys, project documents and CCCG editorial policies) and CCCG staff reflections. Data were analysed using content analysis.Results: The project met three and partially met two of the process elements, for example, by engaging key stakeholders throughout the project and using pre-determined and transparent methods that offered multiple and meaningful ways to contribute. The project met three and partially met two of the outcome elements. Stakeholders were satisfied with and accepted the process and an additional six Cochrane Review titles aligned with stakeholder priorities are now being conducted in partnership with stakeholders. The project has also directly influenced the editorial work of CCCG, for example, by shifting its organisational focus towards coproduction, and indirectly influenced the work of Cochrane's prioritisation and coproduction activities. Some areas were identified as having room for improvement, for example, there was low participation by people from diverse backgrounds, stakeholders could contribute to most but not all project stages, and there was no formal way for stakeholders to appeal decisions at project end. In the 3 years since its completion, the Cochrane Reviews are nearing completion but none of the reviews have been published.Conclusion: We demonstrated that our priority-setting methods were broadly in line with best practice and the project resulted in many positive outcomes beyond just identifying the top priorities for research. Our evaluation framework and recommendations for future evaluations may be of use to priority-setting researchers planning similar activities. [ABSTRACT FROM AUTHOR]

Background: Cross-sector collaborative partnerships are a vital strategy in efforts to strengthen research-informed policy and practice and may be particularly effective at addressing the complex problems associated with chronic disease prevention. However, there is still a limited understanding of how such partnerships are implemented in practice and how their implementation contributes to outcomes. This paper explores the operationalisation and outcomes of knowledge mobilisation strategies within the Australian Prevention Partnership Centre - a research collaboration between policy-makers, practitioners and researchers.Methods: The Centre's programme model identifies six knowledge mobilisation strategies that are hypothesised to be essential for achieving its objectives. Using a mixed methods approach combining stakeholder interviews, surveys, participant feedback forms and routine process data over a 5-year period, we describe the structures, resources and activities used to operationalise these strategies and explore if and how they have contributed to proximal outcomes.Results: Results showed that Centre-produced research, resources, tools and methods were impacting policy formation and funding. Policy-makers reported using new practical methodologies that were helping them to design, implement, evaluate and obtain funding for scaled-up policies and programmes, and co-creating compelling prevention narratives. Some strategies were better implemented and more impactful than others in supporting these outcomes, with variation in who they worked for. The activities used to effect engagement, capacity-building and partnership formation were mostly generating positive results, but co-production could be enhanced by greater shared decision-making. Considerably more work is needed to successfully operationalise knowledge integration and adaptive learning.Conclusions: Describing how collaborative cross-sector research partnerships are operationalised in practice, and with what effects, can provide important insights into practical strategies for establishing and growing such partnerships and for maximising their contributions to policy. Findings suggest that the Centre has many strengths but could benefit from more inclusive and transparent governance and internal processes that facilitate dialogue about roles, expectations and co-production practices. [ABSTRACT FROM AUTHOR]

Background: Research capacity development (RCD) is considered fundamental to closing the evidence-practice gap, thereby contributing to health, wealth and knowledge for practice. Numerous frameworks and models have been proposed for RCD, but there is little evidence of what works for whom and under what circumstances. There is a need to identify mechanisms by which candidate interventions or clusters of interventions might achieve RCD and contribute to societal impact, thereby proving meaningful to stakeholders.Methods: A realist synthesis was used to develop programme theories for RCD. Structured database searches were conducted across seven databases to identify papers examining RCD in a health or social care context (1998-2013). In addition, citation searches for 10 key articles (citation pearls) were conducted across Google Scholar and Web of Science. Of 214 included articles, 116 reported on specific interventions or initiatives or their evaluation. The remaining 98 articles were discussion papers or explicitly sought to make a theoretical contribution. A core set of 36 RCD theoretical and conceptual papers were selected and analysed to generate mechanisms that map across macro contexts (individual, team, organisational, network). Data were extracted by means of 'If-Then' statements into an Excel spreadsheet. Models and frameworks were deconstructed into their original elements.Results: Eight overarching programme theories were identified featuring mechanisms that were triggered across multiple contexts. Three of these fulfilled a symbolic role in signalling the importance of RCD (e.g. positive role models, signal importance, make a difference), whilst the remainder were more functional (e.g. liberate talents, release resource, exceed sum of parts, learning by doing and co-production of knowledge). Outcomes from one mechanism produced changes in context to stimulate mechanisms in other activities. The eight programme theories were validated with findings from 10 systematic reviews (2014-2017).Conclusions: This realist synthesis is the starting point for constructing an RCD framework shaped by these programme theories. Future work is required to further test and refine these findings against empirical data from intervention studies. [ABSTRACT FROM AUTHOR]

Background: In 2010, the Italian Ministry of Health decided to start the planning process to elaborate the National Plan of Prevention 2010-2012 jointly with the 21 Regions. The National Institute of Health was responsible for supporting regional planners (RPs) by an original participatory approach of a web-based Community of Practice (CoP) to set up their own Regional Plans of Prevention. In this paper, we summarise the theoretical framework adopted, the main phases characterising the lifecycle of the nationwide CoP, the evaluation approach adopted and its findings.Methods: Following the CoP theoretical framework from Wenger, an initial group of RPs were trained on Project Cycle Management as a planning method and thereafter they started interacting on a web-based Moodle platform for 8 months. The CoP evaluation mainly took into account aspects of 'immediate value', such as members interactions within the website, and several quantitative and qualitative tools were used to monitor changes over time. Data were retrieved from Moodle statistics or directly from the RPs by the means of a Knowledge, Attitude and Practice survey, a reaction survey, SWOT analysis and focus groups.Results: The level of individual RPs knowledge increased after the initial course from 55.7% to 75%, attitudes and competence perception about the planning process method also showed an overall favourable change. During the CoP life span, the number of members increased from the original 98 RPs to include up to 600 new members on the basis of spontaneous demand. From April 2010 to January 2011, the 'vital signs' of the CoP were monitored, including RP logins (13,450 total logins and 3744 unique logins), views (27,522) and posts (1606) distributed in 326 forum discussion threads. Data and information retrieved from quantitative and qualitative evaluation approaches proved to be useful for the management and follow-up of the CoP.Conclusions: The CoP experience was successful as 19 out of 20 Regions submitted their Regional Preventive Plan to their Ministry of Health within the due deadline. The CoP has proved to be an approach able to optimise resources and expertise, capitalising and generating new knowledge. However, more efforts should be deployed to define innovative ways to evaluate its values, tangible and intangible, as well as the return of investment. [ABSTRACT FROM AUTHOR]

Background: Implementing evidence-informed population health interventions in new contexts often requires adaptations. While the need to adapt interventions to better fit new contexts is recognised, uncertainties remain regarding why and when to adapt (or not), and how to assess the benefits (or not) of adaptation. The ADAPT Study aims to develop comprehensive guidance on adaptation. This scoping review informs guidance development by mapping and exploring how adaptation has been undertaken in practice, in public health and health services research.Methods: We searched seven databases from January 2000 and October 2018 to identify eligible studies for this scoping review and a related systematic review of adaptation guidance. We mapped the studies of adaptation by coding data from all eligible studies describing the methods, contexts, and interventions considered for adaptation. From this map, we selected a sample of studies for in-depth examination. Two reviewers extracted data independently into seven categories: description, key concepts, types, rationale, processes, evaluation methods, evaluation justification, and accounts of failures and successes.Results: We retrieved 6694 unique records. From 429 records screened at full text, we identified 298 eligible studies for mapping and selected 28 studies for in-depth examination. The majority of studies in our map focused on micro- (i.e., individual-) level interventions (84%), related to transferring an intervention to a new population group within the same country (62%) and did not report using guidance (73%). Studies covered a range of topic areas, including health behaviour (24%), mental health (19%), sexual health (16%), and parenting and family-centred interventions (15%). Our in-depth analysis showed that adaptation is seen to save costs and time relative to developing a new intervention, and to enhance contextual relevance and cultural compatibility. It commonly follows a structured process and involves stakeholders to help with decisions on what to adapt, when, and how.Conclusions: Adaptation has been undertaken on a range of health topics and largely in line with existing guidance. Significant gaps relate to adaptation of macro- (e.g., national-) level interventions, consideration of programme theories, mechanisms and contexts (i.e., a functional view of interventions), nuances around stakeholder involvement, and evaluation of the adapted interventions. Registration Open Science Framework, 2019, osf.io/udzma. [ABSTRACT FROM AUTHOR]

Background: Given the context-specific nature of health research prioritization and the obligation to effectively allocate resources to initiatives that will achieve the greatest impact, evaluation of priority setting processes can refine and strengthen such exercises and their outcomes. However, guidance is needed on evaluation tools that can be applied to research priority setting. This paper describes the adaption and application of a conceptual framework to evaluate a research priority setting exercise operating within the public health sector in Ontario, Canada.Methods: The Nine Common Themes of Good Practice checklist, described by Viergever et al. (Health Res Policy Syst 8:36, 2010) was used as the conceptual framework to evaluate the research priority setting process developed for the Locally Driven Collaborative Projects (LDCP) program in Ontario, Canada. Multiple data sources were used to inform the evaluation, including a review of selected priority setting approaches, surveys with priority setting participants, document review, and consultation with the program advisory committee.Results: The evaluation assisted in identifying improvements to six elements of the LDCP priority setting process. The modifications were aimed at improving inclusiveness, information gathering practices, planning for project implementation, and evaluation. In addition, the findings identified that the timing of priority setting activities and level of control over the process were key factors that influenced the ability to effectively implement changes.Conclusions: The findings demonstrate the novel adaptation and application of the 'Nine Common Themes of Good Practice checklist' as a tool for evaluating a research priority setting exercise. The tool can guide the development of evaluation questions and enables the assessment of key constructs related to the design and delivery of a research priority setting process. [ABSTRACT FROM AUTHOR]

Background: Integrated knowledge translation (IKT) through strategic, continuous engagement with decision-makers represents an approach to bridge research, policy and practice. The Collaboration for Evidence-based Healthcare and Public Health in Africa (CEBHA +), comprising research institutions in Ethiopia, Malawi, Rwanda, South Africa, Uganda and Germany, developed and implemented tailored IKT strategies as part of its multifaceted research on prevention and care of non-communicable diseases and road traffic injuries. The objective of this article is to describe the CEBHA + IKT approach and report on the development, implementation and monitoring of site-specific IKT strategies. Methods: We draw on findings derived from the mixed method IKT evaluation (conducted in 2020–2021), and undertook document analyses and a reflective survey among IKT implementers. Quantitative data were analysed descriptively and qualitative data were analysed using content analysis. The authors used the TIDieR checklist to report results in a structured manner. Results: Preliminary IKT evaluation data (33 interviews with researchers and stakeholders from policy and practice, and 31 survey responses), 49 documents, and eight responses to the reflective survey informed this article. In each of the five African CEBHA + countries, a site-specific IKT strategy guided IKT implementation, tailored to the respective national context, engagement aims, research tasks, and individuals involved. IKT implementers undertook a variety of IKT activities at varying levels of engagement that targeted a broad range of decision-makers and other stakeholders, particularly during project planning, data interpretation, and output dissemination. Throughout the project, the IKT teams continued to tailor IKT strategies informally and modified the IKT approach by responding to ad hoc engagements and involving non-governmental organisations, universities, and communities. Challenges to using systematic, formalised IKT strategies arose in particular with respect to the demand on time and resources, leading to the modification of monitoring processes. Conclusion: Tailoring of the CEBHA + IKT approach led to the inclusion of some atypical IKT partners and to greater responsiveness to unexpected opportunities for decision-maker engagement. Benefits of using systematic IKT strategies included clarity on engagement aims, balancing of existing and new strategic partnerships, and an enhanced understanding of research context, including site-specific structures for evidence-informed decision-making. [ABSTRACT FROM AUTHOR]

Background: This paper reports on an online platform, People's Open Access Education Initiative (Peoples-uni), as a means of enhancing access to master's level public health education for health professionals. Peoples-uni seeks to improve population health in low- and middle-income countries by building public health capacity through e-learning at very low cost. We report here an evaluation of the Peoples-uni programme, conducted within the context of Sustainable Development Goal 4, which seeks to "ensure inclusive and quality education for all and promote lifelong learning" by 2030. The evaluation seeks to address the following three questions: (1) Did Peoples-uni meet its intended goals? (2) What were the different types of impacts that students experienced? (3) What suggestions for future changes in Peoples-uni did students recommend?Methods: A mixed methods evaluation consisted of two parts, namely an online survey and a telephone interview. A total of 119 master's level graduates were invited to participate; responses were obtained from 71 of those invited, giving a response rate of 60%. Respondents were spread across 31 countries. Interviews were conducted with 18 respondents.Results: There was strong evidence that Peoples-uni had achieved its stated goals. Potential impacts on students included knowledge to enhance practice and appreciation of context, enhanced research capacity through knowledge of public health, critical thinking and evidence-based programming, and empowerment of students about the potential of education as a means of improving their lives. Accreditation through future partnerships with local universities was recommended by students.Conclusions: Peoples-uni has been able to deliver a credible public health master's level educational programme, with positive impacts on the students who graduated. Challenges are to find a way to accredit the programme to ensure its sustainability and to see how to take full advantage of the current, and future, graduates to turn this from an education programme into a capacity-building programme with real impact. [ABSTRACT FROM AUTHOR]

Background: Health policy and systems research (HPSR) has influenced Philippine policies, including tobacco control, mental health, and COVID-19. The Department of Health (DOH) Philippines and Philippine Council for Health Research and Development (PCHRD) launched the Advancing Health through Evidence-Assisted Decisions (AHEAD) with HPSR program in 2017, aiming to build a community of researchers and decision-makers committed to evidence production and utilization. Research systems employ grant management processes for transparency and accountability in research funding, preventing waste, fraud, and misuse of funds. Methods: This study evaluated AHEAD-HPSR's grant management using surveys, interviews, and focus groups to document (1) grant administration processes implemented by DOH and PCHRD, and (2) experiences of grantees, program managers, staff, and policymakers. Data were initially analyzed through the USA Grant Accountability Office's Federal Grant Life Cycle, with new themes created as they emerged. The study identified processes and gaps in the research grant life cycle stages: design/redesign, pre-award, award, implementation, closeout, and research dissemination and utilization. Results: Identification of research areas for the grant are identified using national and departmental research priorities. While Calls for Proposals are posted publicly, researchers that have previously worked with policymakers are contacted directly to submit proposals. The evaluation found that research is delayed by bureaucracies in grant administration, particularly in financial reporting and ethics review processes. Complying with the terminal financial report was identified as the most challenging part of the grant process due to immense auditing requirements. Grantees recommend the simplification of bureaucracy for fund release to enable them to focus on research work. Conclusion: This study contributes to the limited literature on health research grant management in developing countries. Valuable information and recommendations were contributed by stakeholders in this evaluation. These are manifestations of a continuing interest and desire to make health policy and systems research in the Philippines more robust and relevant. It is imperative for the program to continually evolve and build systems most applicable to its multidisciplinary context. [ABSTRACT FROM AUTHOR]

Background: To guide efficient investment of limited health resources in sub-Saharan Africa, local researchers need to be involved in, and guide, health system and policy research. While extensive survey and census data are available to health researchers and program officers in resource-limited countries, local involvement and leadership in research is limited due to inadequate experience, lack of dedicated research time and weak interagency connections, among other challenges. Many research-strengthening initiatives host prolonged fellowships out-of-country, yet their approaches have not been evaluated for effectiveness in involvement and development of local leadership in research. Methods: We developed, implemented and evaluated a multi-month, deliverable-driven, survey analysis training based in Rwanda to strengthen skills of five local research leaders, 15 statisticians, and a PhD candidate. Research leaders applied with a specific research question relevant to country challenges and committed to leading an analysis to publication. Statisticians with prerequisite statistical training and experience with a statistical software applied to participate in class-based trainings and complete an assigned analysis. Both statisticians and research leaders were provided ongoing in-country mentoring for analysis and manuscript writing. Results: Participants reported a high level of skill, knowledge and collaborator development from class-based trainings and out-of-class mentorship that were sustained 1 year later. Five of six manuscripts were authored by multi-institution teams and submitted to international peer-reviewed scientific journals, and three-quarters of the participants mentored others in survey data analysis or conducted an additional survey analysis in the year following the training. Conclusions: Our model was effective in utilizing existing survey data and strengthening skills among full-time working professionals without disrupting ongoing work commitments and using few resources. Critical to our success were a transparent, robust application process and time limited training supplemented by ongoing, in-country mentoring toward manuscript deliverables that were led by Rwanda's health research leaders. [ABSTRACT FROM AUTHOR]

Background: From 2014 to 2019, the Centre for Research Excellence in Integrated Quality Improvement (CRE-IQI) was evaluated as an innovation platform focusing on continuous quality improvement in Indigenous Australian primary healthcare. Although social network analysis (SNA) is a recognized method for evaluating the functioning, collaboration and effectiveness of innovation platforms, applied research is limited. This study applies SNA to evaluate the CRE-IQI's functioning as an innovation platform.Methods: Two surveys (2017, 2019) were conducted using social survey and network methods. Survey items covered respondent characteristics, their perceptions of the CRE-IQI's performance, and its impact and sociometric relationships. Members' relationship information was captured for the CRE-IQI at three time points, namely start (retrospectively), midpoint and final year, on three network types (knew, shared information, collaborated). SNA software was used to compute standard network metrics including diameter, density and centrality, and to develop visualizations. Survey and network results were addressed in a workshop held by members to develop improvement strategies.Results: The response rate was 80% in 2017 and 65% in 2019 (n = 49 and 47, respectively). Between 2017 and 2019, respondents' mean ratings of the CRE-IQI's functioning and achievements in meeting its goals were sustained. They perceived the CRE-IQI as multidisciplinary, having effective management and governance, and incorporating Indigenous research leadership, representation and ways of working. Respondents recognized high levels of trust amongst members, rated "good communication and coordination with participants" highly, and "facilitating collaboration" as the CRE's most strongly recognized achievement. In collaboration and information-sharing networks, average path length remained low in 2017 and 2019, indicating good small-world network properties for relaying information. On average, respondents shared information and collaborated with more CRE members in 2017 than 2019. However, in both 2017 and 2019 there were new collaborations and information-sharing outside of direct collaborations. CRE-IQI outcomes included: evidence generation; knowledge transfer and skills development in quality improvement; research capacity-building, career development; mentoring; grant support; development of new projects; health service support; and policy impact.Conclusions: This study shows the utility of network analysis in evaluating the functioning, and collaboration, at the individual, organizational and health system levels, of an innovation platform, and adds to our understanding of factors enabling successful innovation platforms. [ABSTRACT FROM AUTHOR]

Background: WHO publishes public health and clinical guidelines to guide Member States in achieving better health outcomes. Furthermore, WHO's Thirteenth General Programme of Work for 2019-2023 prioritizes strengthening its normative functional role and uptake of normative and standard-setting products, including guidelines at the country level. Therefore, understanding WHO guideline uptake by the Member States, particularly the low- and middle-income countries (LMICs), is of utmost importance for the organization and scholarship.Methods: We conducted a scoping review using a comprehensive search strategy to include published literature in English between 2007 and 2020. The review was conducted between May and June 2021. We searched five electronic databases including CINAHL, the Cochrane Library, PubMed, Embase and Scopus. We also searched Google Scholar as a supplementary source. The review adhered to the PRISMA-ScR (PRISMA extension for scoping reviews) guidelines for reporting the searches, screening and identification of evaluation studies from the literature. A narrative synthesis of the evidence around key barriers and challenges for WHO guideline uptake in LMICs is thematically presented.Results: The scoping review included 48 studies, and the findings were categorized into four themes: (1) lack of national legislation, regulations and policy coherence, (2) inadequate experience, expertise and training of healthcare providers for guideline uptake, (3) funding limitations for guideline uptake and use, and (4) inadequate healthcare infrastructure for guideline compliance. These challenges were situated in the Member States' health systems. The findings suggest that governance was often weak within the existing health systems amongst most of the LMICs studied, as was the guidance provided by WHO's guidelines on governance requirements. This challenge was further exacerbated by a lack of accountability and transparency mechanisms for uptake and implementation of guidelines. In addition, the WHO guidelines themselves were either unclear and were technically challenging for some health conditions; however, WHO guidelines were primarily used as a reference by Member States when they developed their national guidelines.Conclusions: The challenges identified reflect the national health systems' (in)ability to allocate, implement and monitor the guidelines. Historically this is beyond the remit of WHO, but Member States could benefit from WHO implementation guidance on requirements and needs for successful uptake and use of WHO guidelines. [ABSTRACT FROM AUTHOR]

Priority-setting (PS) for health research presents an opportunity for the relevant stakeholders to identify and create a list of priorities that reflects the country's knowledge needs. Zambia has conducted several health research prioritisation exercises that have never been evaluated. Evaluation would facilitate gleaning of lessons of good practices that can be shared as well as the identification of areas of improvement. This paper describes and evaluates health research PS in Zambia from the perspectives of key stakeholders using an internationally validated evaluation framework.Methods: This was a qualitative study based on 28 in-depth interviews with stakeholders who had participated in the PS exercises. An interview guide was employed. Data were analysed using NVIVO 10. Emerging themes were, in turn, compared to the framework parameters.Results: Respondents reported that, while the Zambian political, economic, social and cultural context was conducive, there was a lack of co-ordination of funding sources, partners and research priorities. Although participatory, the process lacked community involvement, dissemination strategies and appeals mechanisms. Limited funding hampered implementation, monitoring and evaluation. Research was largely driven by the research funders.Conclusions: Although there is apparent commitment to health research in Zambia, health research PS is limited by lack of funding, and consistently used explicit and fair processes. The designated national research organisation and the availability of tools that have been validated and pilot tested within Zambia provide an opportunity for focused capacity strengthening for systematic prioritisation, monitoring and evaluation. The utility of the evaluation framework in Zambia could indicate potential usefulness in similar low-income countries. [ABSTRACT FROM AUTHOR]

Background: Over the past decade, the Canadian Institutes of Health Research (CIHR) has implemented multicomponent interventions to increase the uptake of sex and gender in grant applications. Interventions included mandatory reporting on applicant forms, development of resources for applicants and evaluators, and grant review requirements. Here, we aim to inform science policy implementation by describing the 10-year outcomes and lessons learned from these interventions.Methods: This is a prospective longitudinal study. The population is all applicants across 15 investigator-initiated CIHR competitions from 2011 to 2019 and grant evaluators from 2018 to 2019. Quantitative data were derived from applicants' and grant evaluators' mandatory reporting of sex and gender integration in the grants management database. The application was the unit of analysis. Trends in sex and gender uptake in applications were plotted over time, stratified by research area. Univariate logistic regression was used to assess associations between the sex of the applicant and the uptake of sex and gender, and the latter with funding success. Qualitative review of the quality and appropriateness of evaluators' comments informed the development of discipline-specific training to peer review committee members. Feedback was compiled from a subset of evaluators on the perceived usefulness of the educational materials using a brief questionnaire.Results: Since 2011, 39,390 applications were submitted. The proportion that reported integration of sex rose from 22 to 83%, and gender from 12 to 33%. Population health research applications paid the greatest attention to gender (82%). Across every competition, applications with female principal investigators were more likely to integrate sex (odds ratio [OR] 1.60, 95% confidence interval [CI] 1.50-1.63) and gender (OR 2.40, 95% CI 2.29-2.51) than those who identified as male. Since 2018, applications that scored highly for the integration of sex (OR 1.92, 95% CI 1.50-2.50) and gender (OR 2.53, 95% CI 1.83-3.50) were more likely to be funded. Qualitative observations revealed persistent conflation of the terms sex and gender. Eighty-six percent of evaluators appreciated the tailored discipline-specific coaching.Conclusions: A number of policy interventions improved sex and gender uptake in grant applications, with higher success rates observed over time for applications that integrated sex and gender. Other funders' action plans around sex and gender integration may be informed from our experiences of the timing, type and targets of the different interventions, specifically those directed at evaluators. [ABSTRACT FROM AUTHOR]

Background: Performing cancer research relies on substantial financial investment, and contributions in time and effort from patients. It is therefore important that this research has real life impacts which are properly evaluated. The optimal approach to cancer research impact evaluation is not clear. The aim of this study was to undertake a systematic review of review articles that describe approaches to impact assessment, and to identify examples of cancer research impact evaluation within these reviews.Methods: In total, 11 publication databases and the grey literature were searched to identify review articles addressing the topic of approaches to research impact assessment. Information was extracted on methods for data collection and analysis, impact categories and frameworks used for the purposes of evaluation. Empirical examples of impact assessments of cancer research were identified from these literature reviews. Approaches used in these examples were appraised, with a reflection on which methods would be suited to cancer research  impact evaluation going forward.Results: In total, 40 literature reviews were identified. Important methods to collect and analyse data for impact assessments were surveys, interviews and documentary analysis. Key categories of impact spanning the reviews were summarised, and a list of frameworks commonly used for impact assessment was generated. The Payback Framework was most often described. Fourteen examples of impact evaluation for cancer research were identified. They ranged from those assessing the impact of a national, charity-funded portfolio of cancer research to the clinical practice impact of a single trial. A set of recommendations for approaching cancer research impact assessment was generated.Conclusions: Impact evaluation can demonstrate if and why conducting cancer research  is worthwhile. Using a mixed methods, multi-category assessment organised within a framework, will provide a robust evaluation, but the ability to perform this type of assessment may be constrained by time and resources. Whichever approach is used, easily measured, but inappropriate metrics should be avoided. Going forward, dissemination of the results of cancer research impact assessments will allow the cancer research community to learn how to conduct these evaluations. [ABSTRACT FROM AUTHOR]

Background: The Collaboration for Evidence-based Healthcare and Public Health in Africa (CEBHA+) is a research consortium concerned with the prevention, diagnosis and treatment of non-communicable diseases. CEBHA+ seeks to engage policymakers and practitioners throughout the research process in order to build lasting relationships, enhance evidence uptake, and create long-term capacity among partner institutions in Ethiopia, Malawi, Rwanda, South Africa and Uganda in collaboration with two German universities. This integrated knowledge translation (IKT) approach includes the formal development, implementation and evaluation of country specific IKT strategies.Methods: We have conceptualised the CEBHA+ IKT approach as a complex intervention in a complex system. We will employ a comparative case study (CCS) design and mixed methods to facilitate an in-depth evaluation. We will use quantitative surveys, qualitative interviews, quarterly updates, and a policy document analysis to capture the process and outcomes of IKT across the African CEBHA+ partner sites. We will conduct an early stage (early 2020) and a late-stage evaluation (early 2022), triangulate the data collected with various methods at each site and subsequently compare our findings across the five sites.Discussion: Evaluating a complex intervention such as the CEBHA+ IKT approach is complicated, even more so when undertaken across five diverse countries. Despite conceptual, methodological and practical challenges, our comparative case study addresses important evidence gaps: While involving decision-makers in the research process is gaining traction worldwide, we still know very little regarding (i) whether this approach really makes a difference to evidence uptake, (ii) the mechanisms that make IKT successful, and (iii) relevant differences across socio-cultural contexts. The evaluation described here is intended to provide relevant insights on all of these aspects, notably in countries in Sub-Saharan Africa, and is expected to contribute to the science of IKT overall. [ABSTRACT FROM AUTHOR]

Background: Developing research capability and capacity within the healthcare professions is a challenge throughout diverse international settings. Within England, the National Institute for Health Research aimed to address these challenges through the Integrated Clinical Academic (ICA) research careers escalator for nurses, midwives and allied health professionals. Poor academic progression has been identified in the advanced stages of the pathway, though progression from the earlier entry point (Internship) has not previously been investigated. A national evaluation of four completed Internship cohorts was undertaken to explore stakeholder perspectives and progression beyond the Internship programme.Methods: A mixed methods project used sequential qualitative and quantitative data collection phases commencing with two stakeholder focus groups (n = 10); the findings informed the development of an online survey distributed to previous cohorts of interns (n = 104), their managers (n = 12) and academic mentors (n = 36). Eight semi-structured interviews subsequently explored the challenges and opportunities afforded by the internships. Thematic analysis was used to review qualitative data from focus groups and interviews, with survey data analysed and displayed using descriptive statistics. Synthesis of data from each phase is displayed within the four level evaluation framework outlined within the New World Kirkpatrick® Training Evaluation Model.Results: Important regional differences exist yet the internships are highly valued by all stakeholders. Representation varied between different professions, with nursing and some service-based professions poorly represented. All interns successfully completed the programme (n = 104), with evidence of positive impacts on interns, colleagues and patient care. Balancing research commitments with clinical activity was challenging; middle managers were seen as gatekeepers to programme success. Progression to the next stage of the ICA pathway is highly competitive and was achieved by only a quarter of interns; access to mentors outside of the funded programme is vital for a successful transition.Conclusions: The Internship programme succeeds in providing a range of important early experiences in research, though progression beyond the programme is challenging due, in part, to a widening gap between Internship and the next level of the ICA framework. Vital mentorship support to bridge this gap is threatened by a lack of time and funding; therefore, the pursuit of a clinical-academic career will continue to be elusive for many nurses and allied health professionals. A partnership approach to clinical academic support at institutional level is needed with several international models offering alternative strategies for consideration. [ABSTRACT FROM AUTHOR]

Antimicrobial resistance (AMR) has the potential to threaten tens of millions of lives and poses major global economic and development challenges. As the AMR threat grows, it is increasingly important to strengthen the scientific evidence base on AMR policy interventions, to learn from existing policies and programmes, and to integrate scientific evidence into the global AMR response.While rigorous evaluations of AMR policy interventions are the ideal, they are far from the current reality. To strengthen this evidence base, we describe a framework for planning, conducting and disseminating research on AMR policy interventions. The framework identifies challenges in AMR research, areas for enhanced coordination and cooperation with decision-makers, and best practices in the design of impact evaluations for AMR policies.This framework offers a path forward, enabling increased local and global cooperation, and overcoming common limitations in existing research on AMR policy interventions. [ABSTRACT FROM AUTHOR]

Background: Evidence-informed decision-making and better use of scientific information in societal decisions has been an area of development for decades but is still topical. Decision support work can be viewed from the perspective of information collection, synthesis and flow between decision-makers, experts and stakeholders. Open policy practice is a coherent set of methods for such work. It has been developed and utilised mostly in Finnish and European contexts.Methods: An overview of open policy practice is given, and theoretical and practical properties are evaluated based on properties of good policy support. The evaluation is based on information from several assessments and research projects developing and applying open policy practice and the authors' practical experiences. The methods are evaluated against their capability of producing quality of content, applicability and efficiency in policy support as well as how well they support close interaction among participants and understanding of each other's views.Results: The evaluation revealed that methods and online tools work as expected, as demonstrated by the assessments and policy support processes conducted. The approach improves the availability of information and especially of relevant details. Experts are ambivalent about the acceptability of openness - it is an important scientific principle, but it goes against many current research and decision-making practices. However, co-creation and openness are megatrends that are changing science, decision-making and the society at large. Against many experts' fears, open participation has not caused problems in performing high-quality assessments. On the contrary, a key challenge is to motivate and help more experts, decision-makers and citizens to participate and share their views. Many methods within open policy practice have also been widely used in other contexts.Conclusions: Open policy practice proved to be a useful and coherent set of methods. It guided policy processes toward a more collaborative approach, whose purpose was wider understanding rather than winning a debate. There is potential for merging open policy practice with other open science and open decision process tools. Active facilitation, community building and improving the user-friendliness of the tools were identified as key solutions for improving the usability of the method in the future. [ABSTRACT FROM AUTHOR]

Background: There is growing recognition amongst health and medical research funders and researchers that translation of research into policy and practice needs to increase and that more transparency is needed on how impacts are realised. Several approaches are advocated for achieving this, including co-production of research or academic-practitioner research. The Population Health Unit (PHU) within the Hunter New England Local Health District in regional Australia, as an early adopter of this model, has been working to increase the likelihood that its research is translated into community health benefits. With the New South Wales Ministry of Health, the PHU responded to the burden of child overweight and obesity by combining service delivery with research expertise. The 'Good for Kids, Good for Life' (Good for Kids) dissemination trial was developed and implemented in seven community settings in the Hunter region of Australia between 2006 and 2010. This study aims to undertake a retrospective impact assessment to measure the research translation and impact of Good for Kids.Methods: The method will be based upon the application of the Framework to Assess the Impact from Translational health research (FAIT), comprising three core elements, namely quantified metrics, economic assessment and a narrative of the process by which the research in question translates and generates impact.Discussion: Increasingly, funders are interested both in the outcomes resulting from investments in health research and in the expected return on their investments. FAIT was developed specifically for this purpose and its use is anticipated to provide transparency to the pathway to translation and potentially drive increased investment in translational research programmes such as Good for Kids. [ABSTRACT FROM AUTHOR]

Background: Evidence-informed decision-making is essential to ensure that health programs and services are effective and offer value for money; however, barriers to the use of evidence persist. Emerging systems science approaches and advances in technology are providing new methods and tools to facilitate evidence-based decision-making. Simulation modelling offers a unique tool for synthesising and leveraging existing evidence, data and expert local knowledge to examine, in a robust, low risk and low cost way, the likely impact of alternative policy and service provision scenarios. This case study will evaluate participatory simulation modelling to inform the prevention and management of gestational diabetes mellitus (GDM). The risks associated with GDM are well recognised; however, debate remains regarding diagnostic thresholds and whether screening and treatment to reduce maternal glucose levels reduce the associated risks. A diagnosis of GDM may provide a leverage point for multidisciplinary lifestyle modification interventions. This research will apply and evaluate a simulation modelling approach to understand the complex interrelation of factors that drive GDM rates, test options for screening and interventions, and optimise the use of evidence to inform policy and program decision-making. Methods/Design: The study design will use mixed methods to achieve the objectives. Policy, clinical practice and research experts will work collaboratively to develop, test and validate a simulation model of GDM in the Australian Capital Territory (ACT). The model will be applied to support evidence-informed policy dialogues with diverse stakeholders for the management of GDM in the ACT. Qualitative methods will be used to evaluate simulation modelling as an evidence synthesis tool to support evidence-based decision-making. Interviews and analysis of workshop recordings will focus on the participants' engagement in the modelling process; perceived value of the participatory process, perceived commitment, influence and confidence of stakeholders in implementing policy and program decisions identified in the modelling process; and the impact of the process in terms of policy and program change. Discussion: The study will generate empirical evidence on the feasibility and potential value of simulation modelling to support knowledge mobilisation and consensus building in health settings. [ABSTRACT FROM AUTHOR]

Background: Research translation, particularly in the biomedical area, is often discussed but there are few methods that are routinely used to measure it or its impact. Of the impact measurement methods that are used, most aim to provide accountability - to measure and explain what was generated as a consequence of funding research. This case study reports on the development of a novel, conceptual framework that goes beyond measurement. The Framework To Assess the Impact from Translational health research, or FAIT, is a platform designed to prospectively measure and encourage research translation and research impact. A key assumption underpinning FAIT is that research translation is a prerequisite for research impact.Methods: The research impact literature was mined to understand the range of existing frameworks and techniques employed to measure and encourage research translation and research impact. This review provided insights for the development of a FAIT prototype. A Steering Committee oversaw the project and provided the feedback that was used to refine FAIT.Results: The outcome of the case study was the conceptual framework, FAIT, which is based on a modified program logic model and a hybrid of three proven methodologies for measuring research impact, namely a modified Payback method, social return on investment, and case studies or narratives of the process by which research translates and generates impact.Conclusion: As funders increasingly seek to understand the return on their research investments, the routine measurement of research translation and research impact is likely to become mandatory rather than optional. Measurement of research impact on its own is insufficient. There should also be a mechanism attached to measurement that encourages research translation and impact - FAIT was designed for this task. [ABSTRACT FROM AUTHOR]

Background: The interest and activity in measuring and reporting the impact of publicly funded health and medical research has grown rapidly in recent years. Research evaluation typically relies on researchers for much of the information for an impact assessment. However, the acceptability and feasibility of this activity among health researchers is unknown. The aim of this study was to understand the role and opinions of cancer researchers in the growing area of impact evaluation activity, to inform the logistics of a sustainable program of impact evaluation.Methods: A brief anonymous online survey was administered to 95 current and past grant recipients funded through the external grants program at Cancer Council New South Wales. Eleven survey statements were constructed with Likert responses and supplemented with two open-ended questions. The statements covered the conceptual, attitudinal and practical aspects of impact evaluation. The survey targeted researchers from the full spectrum of cancer control research classifications. Descriptive analyses obtained response frequencies and percentages.Results: Forty-five cancer researchers completed the survey (response rate 47%) and 77% were Associate Professors or Professors. Responses were polarised for questions relating to engaging with research end-users, perceived time-pressure to collate data, and pressure to produce research outputs. Some researchers emphasised that quality was an important goal over quantity and warned that collecting impact data created incentives and disincentives for researchers.Conclusion: There was mixed support and acceptance among senior cancer researchers in Australia on their perceived role and engagement with research impact activities. Sole reliance on researchers for collating and reporting impact data may be problematic. Requesting information from researchers could be minimised and confined to final reports and possible verification of externally-led evaluations. [ABSTRACT FROM AUTHOR]

Background Health research capacity strengthening (RCS) projects are often complex and hard to evaluate. In order to inform health RCS evaluation efforts, we aimed to describe and compare key characteristics of existing health RCS evaluation frameworks: their process of development, purpose, target users, structure, content and coverage of important evaluation issues. A secondary objective was to explore what use had been made of the ESSENCE framework, which attempts to address one such issue: harmonising the evaluation requirements of different funders. Methods We identified and analysed health RCS evaluation frameworks published by seven funding agencies between 2004 and 2012, using a mixed methods approach involving structured qualitative analyses of documents, a stakeholder survey and consultations with key contacts in health RCS funding agencies. Results The frameworks were intended for use predominantly by the organisations themselves, and most were oriented primarily towards funders' internal organisational performance requirements. The frameworks made limited reference to theories that specifically concern RCS. Generic devices, such as logical frameworks, were typically used to document activities, outputs and outcomes, but with little emphasis on exploring underlying assumptions or contextual constraints. Usage of the ESSENCE framework appeared limited. Conclusions We believe that there is scope for improving frameworks through the incorporation of more accessible information about how to do evaluation in practice; greater involvement of stakeholders, following evaluation capacity building principles; greater emphasis on explaining underlying rationales of frameworks; and structuring frameworks so that they separate generic and project-specific aspects of health RCS evaluation. The third and fourth of these improvements might assist harmonisation. [ABSTRACT FROM AUTHOR]

Background: The World Health Organization (WHO), like many other organisations around the world, has recognised the need to use more rigorous processes to ensure that health care recommendations are informed by the best available research evidence. This is the last of a series of 16 reviews that have been prepared as background for advice from the WHO Advisory Committee on Health Research to WHO on how to achieve this. Objectives: We reviewed the literature on evaluating guidelines and recommendations, including their quality, whether they are likely to be up-to-date, and their implementation. We also considered the role of guideline developers in undertaking evaluations that are needed to inform recommendations. Methods: We searched PubMed and three databases of methodological studies for existing systematic reviews and relevant methodological research. We did not conduct systematic reviews ourselves. Our conclusions are based on the available evidence, consideration of what WHO and other organisations are doing and logical arguments. Key questions and answers: Our answers to these questions were informed by a review of instruments for evaluating guidelines, several studies of the need for updating guidelines, discussions of the pros and cons of different research designs for evaluating the implementation of guidelines, and consideration of the use of uncertainties identified in systematic reviews to set research priorities. How should the quality of guidelines or recommendations be appraised? • WHO should put into place processes to ensure that both internal and external review of guidelines is undertaken routinely. • A checklist, such as the AGREE instrument, should be used. • The checklist should be adapted and tested to ensure that it is suitable to the broad range of recommendations that WHO produces, including public health and health policy recommendations, and that it includes questions about equity and other items that are particularly important for WHO guidelines. When should guidelines or recommendations be updated? • Processes should be put into place to ensure that guidelines are monitored routinely to determine if they are in need of updating. • People who are familiar with the topic, such as Cochrane review groups, should do focused, routine searches for new research that would require revision of the guideline. • Periodic review of guidelines by experts not involved in developing the guidelines should also be considered. • Consideration should be given to establishing guideline panels that are ongoing, to facilitate routine updating, with members serving fixed periods with a rotating membership. How should the impact of guidelines or recommendations be evaluated? • WHO headquarters and regional offices should support member states and those responsible for policy decisions and implementation to evaluate the impact of their decisions and actions by providing advice regarding impact assessment, practical support and coordination of efforts. • Before-after evaluations should be used cautiously and when there are important uncertainties regarding the effects of a policy or its implementation, randomised evaluations should be used when possible. What responsibility should WHO take for ensuring that important uncertainties are addressed by future research when the evidence needed to inform recommendations is lacking? • Guideline panels should routinely identify important uncertainties and research priorities. This source of potential priorities for research should be used systematically to inform priority-setting processes for global research. [ABSTRACT FROM AUTHOR]

Background: In April 2016, the Rick Hansen Institute (RHI) hosted an innovative, 2-day conference called Praxis 2016. RHI aimed to bring together a diverse group of stakeholders to develop solutions for overcoming the challenges of translating spinal cord injury (SCI) research into practice. To understand the impact of Praxis, RHI funded an independent team to evaluate Praxis at the individual and setting level using the RE-AIM framework. Individual-level findings are published elsewhere. The aim of this evaluation is to report on the impact of Praxis at the setting level in terms of its adoption, implementation and maintenance.Methods: Data sources included interviews with attendees (n = 13) and organisers (n = 9), a fidelity assessment conducted at the conference, and observation notes provided by the evaluation team.Results: Main findings indicated that the Praxis model was adopted by organisers and attendees, implemented by RHI as intended, and has the potential for long-term maintenance. Lessons learned highlighted the importance of including SCI community members throughout the entire process from development to post-conference activities as well as in the research process, the value of facilitation and fostering interactive problem solving, and the need to identify leadership and funds to foster long-term efforts.Conclusions: Developing and implementing a solutions-focused conference that brings together a diverse group of SCI stakeholders was challenging and rewarding for attendees and organisers. Other domains could learn from, adopt and build on the Praxis 2016 approach to address research-to-practice gaps. [ABSTRACT FROM AUTHOR]

Background: Evidence-informed Decision-making in Nutrition and Health (EVIDENT) is an international partnership that seeks to identify information needs in nutrition and health in Africa and build local capacity in knowledge management to help translate the best available evidence into context-appropriate recommendations aligned to the priorities of decision-makers. This study evaluates the extent to which EVIDENT achieved its intended activities, documents the lessons learned and draws on these lessons learned to inform future activities of EVIDENT, as well as in evidence-informed decision-making (EIDM) in nutrition overall.Methods: Purposive and snowball sampling were used to identify participants that were either directly or indirectly involved with EVIDENT. An analytical framework of five key elements was developed to guide data collection from EVIDENT's documentation, in-depth interviews (n = 20), online surveys (n = 26) and a participatory discussion. Interviews were transcribed verbatim and coded in NVivo 11, using deductive thematic content analysis and a phenomenological approach. Online surveys were analysed using Stata 14. Data were triangulated to address both objectives under each element of the analytical framework.Results: EVIDENT succeeded in establishing a collaborative partnership, within which it delivered four short courses in EIDM. This capacity complemented case study activities in four partner African countries where EIDM processes were implemented and assessed. Identified barriers to these processes included little experience in EIDM, difficulties in engaging stakeholders, challenging local environments (e.g. donor influence, bureaucracy, inaccessibility to scientific research, poor internet connectivity), and limited time and funding. However, EVIDENT activities were driven by a local need for EIDM, a sheer interest and commitment to the cause, and the opportunity for the Global North and South to work together and build relationships. Future activities of EVIDENT, and EIDM in nutrition overall, should focus on sustained capacity-building in EIDM processes, leadership and functional skills across the Global South, investment in stakeholder engagement, context-specific EIDM, enhanced communication and linking, and strengthening relationships with existing stakeholder organisations.Conclusions: In its first 3 years, EVIDENT developed and strengthened partnership, capacity and visibility on EIDM in Africa. Innovative and long-term capacity-building, dedicated leadership, further stakeholder engagement and sustainable financing, are needed for future activities of EVIDENT and EIDM in nutrition. [ABSTRACT FROM AUTHOR]

Background: Integrated knowledge translation (IKT) is encouraged in population health intervention research (PHIR) to ensure the co-production of policy-relevant research, yet there is little published literature that reports its implementation and outcomes. The purpose of this study was to describe and evaluate the IKT approach used in a school-based PHIR project to understand how the research informed policy and practice and identify what influenced the IKT process.Methods: A case study approach was used to provide an in-depth description of the IKT process and understand the co-production and application of research evidence. Data were collected through document review, a survey with all elementary school principals in the school board (n = 18) following dissemination of School Reports and interviews with the IKT research team (including two researchers and three knowledge users).Results: Approximately half of the principals reported reading their School Report (52%) and almost all of these principals attributed the partial or full adoption, or implementation, of a new practice as a result of using the information (89%). Key themes related to the IKT process emerged across the interviews, including supportive relationships, role clarity, competing priorities and the complexities of population health interventions.Conclusions: The findings suggest that, while IKT can support policy and practice, it can be challenging to maintain engagement due to differing priorities and role ambiguity. Additional recognition, investment and research would enable better implementation of the approach, thereby bridging the gap between research, policy and practice. [ABSTRACT FROM AUTHOR]