Search

Abstract Research Impact Assessment (RIA) represents one of a suite of policies intended to improve the impact generated from investment in health and medical research (HMR). Positivist indicator-based approaches to RIA are widely implemented but increasingly criticised as theoretically problematic, unfair, and burdensome. This commentary proposes there are useful outcomes that emerge from the process of applying an indicator-based RIA framework, separate from those encapsulated in the metrics themselves. The aim for this commentary is to demonstrate how the act of conducting an indicator-based approach to RIA can serve to optimise the productive gains from the investment in HMR. Prior research found that the issues regarding RIA are less about the choice of indicators/metrics, and more about the discussions prompted and activities incentivised by the process. This insight provides an opportunity to utilise indicator-based methods to purposely optimise the research impact. An indicator-based RIA framework specifically designed to optimise research impacts should: focus on researchers and the research process, rather than institution-level measures; utilise a project level unit of analysis that provides control to researchers and supports collaboration and accountability; provide for prospective implementation of RIA and the prospective orientation of research; establish a line of sight to the ultimate anticipated beneficiaries and impacts; Include process metrics/indicators to acknowledge interim steps on the pathway to final impacts; integrate ‘next’ users and prioritise the utilisation of research outputs as a critical measure; Integrate and align the incentives for researchers/research projects arising from RIA, with those existing within the prevailing research system; integrate with existing peer-review processes; and, adopt a system-wide approach where incremental improvements in the probability of translation from individual research projects, yields higher impact across the whole funding portfolio. Optimisation of the impacts from HMR investment represents the primary purpose of Research Impact policy. The process of conducting an indicator-based approach to RIA, which engages the researcher during the inception and planning phase, can directly contribute to this goal through improvements in the probability that an individual project will generate interim impacts. The research project funding process represents a promising forum to integrate this approach within the existing research system.

Abstract Background A growing body of international literature concurs that comprehensive and complex Advance Care Planning (ACP) programs involving specially qualified or trained healthcare professionals are effective in increasing documentation of Advance Care Directives (ACDs), improving compliance with patients’ wishes and satisfaction with care, and quality of care for patients and their families. Economic analyses of ACDs and ACP have been more sporadic and inconclusive. This study aimed to contribute to the evidence on resource use associated with implementation of ACP and to inform key decision-makers of the resource implications through the conduct of a cost-consequence analysis of the Normalised Advance Care Planning (NACP) trial. Methods The outcomes for the economic evaluation included the number of completed “legally binding” ACDs and the number of completed Conversation Cards (CC). The cost analysis assessed the incremental difference in resource utilisation between Usual Practice and the Intervention. Costs have been categorised into: 1) Contract staff costs; 2) Costs associated with the development of the intervention; 3) Implementation costs; 4) Intervention (delivery) costs; and 5) Research costs. Results The cost incurred for each completed ACD was A$13,980 in the hospital setting and A$1248 in the community setting. The cost incurred for each completed Conversation Card was A$7528 in the hospital setting and A$910 in the community setting. Conclusions The cost-consequence analysis does not support generalisation of the specified intervention within the hospital setting. The trial realised an estimated incremental cost per completed ACD of $1248, within the community setting. This estimate provides an additional benchmark against which decision-makers can assess the value of either 1) this approach towards the realisation of additional completed ACDs; and/or 2) the value of ACP and ACDs more broadly, when this estimate is positioned within the potential health outcomes and downstream health service implications that may arise for people with or without a completed ACD. Trial registration The study was retrospectively registered with the Australian New Zealand Clinical Trials Registry (Trial ID: ACTRN12618001627246 ). The URL of the trial registry record.

Abstract Background Recurrent stroke is a major contributor to stroke-related disability and costs. Improving health-risk behaviours and mental health has the potential to significantly improve recovery, enhance health-related quality of life (HRQoL), independent living, and lower the risk of recurrent stroke. The primary aim will be to test the effectiveness of an online intervention to improve HRQoL among stroke survivors at 6 months’ follow-up. Programme effectiveness on four health behaviours, anxiety and depression, cost-effectiveness, and impact on other hospital admissions will also be assessed. Methods/design An open-label randomised controlled trial is planned. A total of 530 adults will be recruited across one national and one regional stroke registry and block randomised to the intervention or minimal care control group. The intervention group will receive access to the online programme Prevent 2nd Stroke (P2S); the minimal care control group will receive an email with Internet addresses of generic health sites designed for the general population. The primary outcome, HRQoL, will be measured using the EuroQol-5D. A full analysis plan will compare between groups from baseline to follow-up. Discussion A low-cost per user option to supplement current care, such as P2S, has the potential to increase HRQoL for stroke survivors, and reduce the risk of second stroke. Trial registration Australian and New Zealand Clinical Trials Registry, ID: ACTRN12617001205325p. Registered on 17 August 2017.

Abstract Background There is growing recognition that the wider benefits of research (economic, social and health impacts) should be assessed and valued alongside traditional research performance metrics such as peer-reviewed papers. Translation of findings into policy and practice needs to accelerate and pathways to impact need to be better understood. This research protocol outlines a mixed methods study to apply the Framework to Assess the Impact from Translational health research (FAIT) to the Centre for Research Excellence in Stroke Rehabilitation and Brain Recovery (CRE-Stroke). FAIT is purpose-designed to encourage research translation and assess research impact but lacks validation. Methods/Design Phase 1 involves application of the FAIT-modified programme logic model to each CRE-Stroke research stream including identifying process, output and impact metrics, as well as end users of the research. A scoping review will inform potential impacts anticipated from CRE-Stroke. In Phase 2, audit and feedback on achievements against plans will track and encourage research translation. Logic models will be updated to account for changes in the research pathways over time. In Phase 3, three proven methods for measuring research impact – Payback, economic assessment and narratives – will be applied to each research stream and the data triangulated and reported in Phase 4. The feasibility of applying FAIT will also be assessed as part of Phase 3. Discussion Use of prospective, comprehensive research impact frameworks for large interdisciplinary programmes of research is rare. FAIT’s application to CRE-Stroke will provide opportunity for the impact of CRE-Stroke to be assessed and a range of impacts beyond standard academic achievements to be reliably reported. The feasibility of FAIT’s application will also be assessed and, if necessary, refined. The usefulness of FAIT for encouraging research translation will also be described and may prove useful for other programmes looking to implement a research impact framework.

Abstract Background No evaluations of the cost or cost effectiveness of interventions to increase school implementation of food availability policies have been reported. Government and non-government agency decisions regarding the extent of investment required to enhance school implementation of such policies are unsupported by such evidence. This study sought to i) Determine cost and cost-effectiveness of three interventions in improving school implementation of an Australian government healthy canteen policy and; ii) Determine the relative cost-effectiveness of the interventions in improving school implementation of such a policy. Methods An analysis of the cost and cost-effectiveness of three implementation interventions of varying support intensity, relative to usual implementation support conducted during 2013–2015 was undertaken. Secondly, an indirect comparison of the trials was undertaken to determine the most cost-effective of the three strategies. The economic analysis was based on the cost of delivering the interventions by health service delivery staff to increase the proportion of schools ‘adherent’ with the policy. Results The total costs per school were $166,971, $70,926 and $75,682 for the high, medium and low intensity interventions respectively. Compared to usual support, the cost effectiveness ratios for each of the three interventions were: A$2982 (high intensity), A$2627 (medium intensity) and A$4730 (low intensity) per percent increase in proportion of schools reporting ‘adherence’). Indirect comparison between the ‘high’ and ‘medium intensity’ interventions showed no statistically significant difference in cost-effectiveness. Conclusions The results indicate that while the cost profiles of the interventions varied substantially, the cost-effectiveness did not. This result is valuable to policy makers seeking cost-effective solutions that can be delivered within budget.

Abstract Background The question of how to measure, assess and optimise the returns from investment in health and medical research (HMR) is a highly policy-relevant issue. Research Impact Assessment Frameworks (RIAFs) provide a conceptual measurement framework to assess the impact from HMR. The aims of this study were (1) to elicit the views of Medical Research Institutes (MRIs) regarding objectives, definitions, methods, barriers, potential scope and attitudes towards RIAFs, and (2) to investigate whether an assessment framework should represent a retrospective reflection of research impact or a prospective approach integrated into the research process. The wider objective was to inform the development of a draft RIAF for Australia’s MRIs. Methods Purposive sampling to derive a heterogeneous sample of Australian MRIs was used alongside semi-structured interviews with senior executives responsible for research translation or senior researchers affected by research impact initiatives. Thematic analysis of the interview transcriptions using the framework approach was then performed. Results Interviews were conducted with senior representatives from 15 MRIs. Participants understood the need for greater research translation/impact, but varied in their comprehension and implementation of RIAFs. Common concerns included the time lag to the generation of societal impacts from basic or discovery science, and whether impact reflected a narrow commercialisation agenda. Broad support emerged for the use of metrics, case study and economic methods. Support was also provided for the rationale of both standardised and customised metrics. Engendering cultural change in the approach to research translation was acknowledged as both a barrier to greater impact and a critical objective for the assessment process. Participants perceived that the existing research environment incentivised the generation of academic publications and track records, and often conflicted with the generation of wider impacts. The potential to improve the speed of translation through prospective implementation of impact assessment was supported, albeit that the mechanism required development. Conclusion The study found that the issues raised regarding research impact assessment are less about methods and metrics, and more about the research activities that the measurement of research translation and impact may or may not incentivise. Consequently, if impact assessment is to contribute to optimisation of the health gains from the public, corporate and philanthropic investment entrusted to the institutes, then further inquiry into how the assessment process may re-align research behaviour must be prioritised.

Abstract Background Depression is highly prevalent yet often poorly detected and treated among cancer patients. In light of the move towards evidence-based healthcare policy, we have developed a simple tool that can assist policy makers, organisations and researchers to logically think through the steps involved in improving patient outcomes, and to help guide decisions about where to allocate resources. Methods The model assumes that a series of filters operate to determine outcomes and cost-effectiveness associated with depression care for cancer patients, including: detection of depression, provider response to detection, patient acceptance of treatment, and effectiveness of treatment provided. To illustrate the utility of the model, hypothetical data for baseline and four scenarios in which filter outcomes were improved by 15% were entered into the model. Results The model provides outcomes including: number of people successfully treated, total costs per scenario, and the incremental cost-effectiveness ratio per scenario compared to baseline. The hypothetical data entered into the model illustrate the relative effectiveness (in terms of the number of additional incremental successes) and relative cost-effectiveness (in terms of cost per successful outcome and total cost) of making changes at each step or filter. Conclusions The model provides a readily accessible tool to assist decision makers to think through the steps involved in improving depression outcomes for cancer patents. It provides transparent guidance about how to best allocate resources, and highlights areas where more reliable data are needed. The filter model presents an opportunity to improve on current practice by ensuring that a logical approach, which takes into account the available evidence, is applied to decision making.

Abstract Background There is growing recognition amongst health and medical research funders and researchers that translation of research into policy and practice needs to increase and that more transparency is needed on how impacts are realised. Several approaches are advocated for achieving this, including co-production of research or academic-practitioner research. The Population Health Unit (PHU) within the Hunter New England Local Health District in regional Australia, as an early adopter of this model, has been working to increase the likelihood that its research is translated into community health benefits. With the New South Wales Ministry of Health, the PHU responded to the burden of child overweight and obesity by combining service delivery with research expertise. The ‘Good for Kids, Good for Life’ (Good for Kids) dissemination trial was developed and implemented in seven community settings in the Hunter region of Australia between 2006 and 2010. This study aims to undertake a retrospective impact assessment to measure the research translation and impact of Good for Kids. Methods The method will be based upon the application of the Framework to Assess the Impact from Translational health research (FAIT), comprising three core elements, namely quantified metrics, economic assessment and a narrative of the process by which the research in question translates and generates impact. Discussion Increasingly, funders are interested both in the outcomes resulting from investments in health research and in the expected return on their investments. FAIT was developed specifically for this purpose and its use is anticipated to provide transparency to the pathway to translation and potentially drive increased investment in translational research programmes such as Good for Kids.