Search

Abstract Background Translating research, achieving impact, and assessing impact are important aspirations for all research collaboratives but can prove challenging. The Hunter Cancer Research Alliance (HCRA) was funded from 2014 to 2021 to enhance capacity and productivity in cancer research in a regional centre in Australia. This study aimed to assess the impact and benefit of the HCRA to help inform future research investments of this type. Method The Framework to Assess the Impact from Translational health research (FAIT) was selected as the preferred methodology. FAIT incorporates three validated methodologies for assessing impact: 1) Modified Payback; 2) Economic Analysis; and 3) Narrative overview and case studies. All three FAIT methods are underpinned by a Program Logic Model. Data were collected from HCRA and the University of Newcastle administrative records, directly from HCRA members, and website searches. Results In addition to advancing knowledge and providing capacity building support to members via grants, fellowships, scholarships, training, events and targeted translation support, key impacts of HCRA-member research teams included: (i) the establishment of a regional biobank that has distributed over 13,600 samples and became largely self-sustaining; (ii) conservatively leveraging $43.8 M (s.a.$20.5 M - $160.5 M) in funding and support from the initial $9.7 M investment; (iii) contributing to clinical practice guidelines and securing a patent for identification of stem cells for endometrial cell regeneration; (iv) shifting the treatment paradigm for all tumour types that rely on nerve cell innervation, (v) development and implementation of the world’s first real-time patient treatment verification system (Watchdog); (vi) inventing the effective ‘EAT’ psychological intervention to improve nutrition and outcomes in people experiencing radiotherapy for head and neck cancer; (vi) developing effective interventions to reduce smoking rates among priority groups, currently being rolled out to disadvantaged populations in NSW; and (vii) establishing a Consumer Advisory Panel and Consumer Engagement Committee to increase consumer involvement in research. Conclusion Using FAIT methodology, we have demonstrated the significant impact and downstream benefits that can be achieved by the provision of infrastructure-type funding to regional and rural research collaboratives to help address inequities in research activity and health outcomes and demonstrates a positive return on investment.

Abstract Research Impact Assessment (RIA) represents one of a suite of policies intended to improve the impact generated from investment in health and medical research (HMR). Positivist indicator-based approaches to RIA are widely implemented but increasingly criticised as theoretically problematic, unfair, and burdensome. This commentary proposes there are useful outcomes that emerge from the process of applying an indicator-based RIA framework, separate from those encapsulated in the metrics themselves. The aim for this commentary is to demonstrate how the act of conducting an indicator-based approach to RIA can serve to optimise the productive gains from the investment in HMR. Prior research found that the issues regarding RIA are less about the choice of indicators/metrics, and more about the discussions prompted and activities incentivised by the process. This insight provides an opportunity to utilise indicator-based methods to purposely optimise the research impact. An indicator-based RIA framework specifically designed to optimise research impacts should: focus on researchers and the research process, rather than institution-level measures; utilise a project level unit of analysis that provides control to researchers and supports collaboration and accountability; provide for prospective implementation of RIA and the prospective orientation of research; establish a line of sight to the ultimate anticipated beneficiaries and impacts; Include process metrics/indicators to acknowledge interim steps on the pathway to final impacts; integrate ‘next’ users and prioritise the utilisation of research outputs as a critical measure; Integrate and align the incentives for researchers/research projects arising from RIA, with those existing within the prevailing research system; integrate with existing peer-review processes; and, adopt a system-wide approach where incremental improvements in the probability of translation from individual research projects, yields higher impact across the whole funding portfolio. Optimisation of the impacts from HMR investment represents the primary purpose of Research Impact policy. The process of conducting an indicator-based approach to RIA, which engages the researcher during the inception and planning phase, can directly contribute to this goal through improvements in the probability that an individual project will generate interim impacts. The research project funding process represents a promising forum to integrate this approach within the existing research system.

Abstract Background Research impact is an emerging measure of research achievement alongside traditional academic outputs such as publications. We present the results of applying the Framework to Assess the Impact from Translational health research (FAIT) to the Centre for Research Excellence (CRE) in Stroke Rehabilitation and Brain Recovery (CRE-Stroke, 2014–2019) and report on the feasibility and lessons from the application of FAIT to a CRE rather than a discrete research project. Methods Data were gathered via online surveys, in-depth interviews, document analysis and review of relevant websites/databases to report on the three major FAIT methods: the modified Payback Framework, an assessment of costs against monetized consequences, and a narrative account of the impact generated from CRE-Stroke activities. FAIT was applied during the last 4 years of CRE-Stroke operation. Results With an economic investment of AU$ 3.9 million over 5 years, CRE-Stroke delivered a return on investment that included AU$ 18.8 million in leveraged grants, fellowships and consultancies. Collectively, CRE-Stroke members produced 354 publications that were accessed 470,000 times and cited over 7220 times. CRE-Stroke supported 26 PhDs, 39 postdocs and seven novice clinician researchers. There were 59 capacity-building events benefiting 744 individuals including policy-makers and consumers. CRE-Stroke created research infrastructure (including a research register of stroke survivors and a brain biobank), and its global leadership produced international consensus recommendations to influence the stroke research landscape worldwide. Members contributed to the Australian Living Stroke Guidelines: four researchers’ outputs were directly referenced. Based only on the consequences that could be monetized, CRE-Stroke returned AU$ 4.82 for every dollar invested in the CRE. Conclusion This case example in the developing field of impact assessment illustrates how researchers can use evidence to demonstrate and report the impact of and returns on research investment. The prospective application of FAIT by a dedicated research impact team demonstrated impact in broad categories of knowledge-gain, capacity-building, new infrastructure, input to policy and economic benefits. The methods can be used by other research teams to provide comprehensive evidence to governments and other research funders about what has been generated from their research investment but requires dedicated resources to complete.

Abstract Background A regional Australian Primary Health Network (PHN) has been subsidising administrative staff from local general practices to undertake the Medical Practice Assisting (MPA) course as part of its MPA Program. The MPA Program aimed to upskill administrative staff to undertake clinical tasks and fill in for busy or absent Practice Nurses (PNs), freeing up PNs to increase revenue-generating activity, avoiding casual replacement staff wages, and increasing patient throughput. An impact assessment was undertaken to evaluate the impact and estimate the economic costs of the MPA program to the PHN, general practices, and students to inform future uptake of the intervention. Methods The Framework to Assess the Impact of Translational Health Research (FAIT) was utilised. Originally designed to assess the impact of health research, this was its first application to a health services project. FAIT combines three validated methods of impact assessment—Payback, economic analysis and narratives underpinned by a program logic model. Quantified metrics describe the impacts of the program within various “domains of benefit”, the economic model costs the intervention and monetises potential consequences, and the narrative tells the story of the MPA Program and the difference it has made. Data were collected via online surveys from general practitioners (GPs), PNs, practice managers; MPA graduates and PHN staff were interviewed by phone and on Zoom. Results FAIT was effective in evidencing the impacts and economic viability of the MPA Program. GPs and PNs reported greater work satisfaction, PNs reported less stress and reduced workloads and MPA graduates reported higher job satisfaction and greater confidence performing a range of clinical skills. MPA Program economic costs for general practices during candidature, and 12 month post-graduation was estimated at $69,756. With effective re-integration planning, this investment was recoverable within 12 months through increased revenue for practices. Graduates paid appropriately for their new skills also recouped their investment within 24 months. Conclusion Utilisation of MPA graduates varied substantially between practices and COVID-19 impacted on their utilisation. More strategic reintegration of the MPA graduate back into the practice to most effectively utilise their new skillset could optimise potential benefits realised by participating practices.

Abstract Background From 2014 to 2019, the Centre for Research Excellence in Integrated Quality Improvement (CRE-IQI) was evaluated as an innovation platform focusing on continuous quality improvement in Indigenous Australian primary healthcare. Although social network analysis (SNA) is a recognized method for evaluating the functioning, collaboration and effectiveness of innovation platforms, applied research is limited. This study applies SNA to evaluate the CRE-IQI’s functioning as an innovation platform. Methods Two surveys (2017, 2019) were conducted using social survey and network methods. Survey items covered respondent characteristics, their perceptions of the CRE-IQI’s performance, and its impact and sociometric relationships. Members’ relationship information was captured for the CRE-IQI at three time points, namely start (retrospectively), midpoint and final year, on three network types (knew, shared information, collaborated). SNA software was used to compute standard network metrics including diameter, density and centrality, and to develop visualizations. Survey and network results were addressed in a workshop held by members to develop improvement strategies. Results The response rate was 80% in 2017 and 65% in 2019 (n = 49 and 47, respectively). Between 2017 and 2019, respondents’ mean ratings of the CRE-IQI’s functioning and achievements in meeting its goals were sustained. They perceived the CRE-IQI as multidisciplinary, having effective management and governance, and incorporating Indigenous research leadership, representation and ways of working. Respondents recognized high levels of trust amongst members, rated “good communication and coordination with participants” highly, and “facilitating collaboration” as the CRE’s most strongly recognized achievement. In collaboration and information-sharing networks, average path length remained low in 2017 and 2019, indicating good small-world network properties for relaying information. On average, respondents shared information and collaborated with more CRE members in 2017 than 2019. However, in both 2017 and 2019 there were new collaborations and information-sharing outside of direct collaborations. CRE-IQI outcomes included: evidence generation; knowledge transfer and skills development in quality improvement; research capacity-building, career development; mentoring; grant support; development of new projects; health service support; and policy impact. Conclusions This study shows the utility of network analysis in evaluating the functioning, and collaboration, at the individual, organizational and health system levels, of an innovation platform, and adds to our understanding of factors enabling successful innovation platforms.

Abstract Background In hospitals, catheter acquired urinary tract infection causes significant resource waste and discomfort among admitted patients. An intervention for reducing indwelling catheterisations - No-CAUTI - was trialled across four hospitals in New South Wales, Australia. No-CAUTI includes: train-the-trainer workshops, site champions, compliance audits, and point prevalence surveys. The trial showed reductions on usual care catheterisation rates at 4- and 9-month post-intervention. This result was statistically non-significant; and post-intervention catheterisation rates rebounded between 4 and 9 months. However, No-CAUTI showed statistically significant catheterisation decreases for medical wards, female patients and for short-term catheterisations. This study presents a budget impact analysis of a projected five year No-CAUTI roll out across New South Wales public hospitals, from the cost perspective of the New South Wales Ministry of Health. Methods Budget forecasts were made for five year roll outs of: i) No-CAUTI; and ii) usual care, among all public hospitals in New South Wales hosting overnight stays (n=180). The roll out design maintains intervention effectiveness with ongoing workshops, quality audits, and hospital surveys. Forecasts of catheterisations, procedures and treatments were modelled on No-CAUTI trial observations. Costs were sourced from trial records, the Medical Benefits Scheme, the Pharmaceutical Benefits Scheme and public wage awards. Cost and parameter uncertainties were considered with sensitivity scenarios. Results The estimated five-year No-CAUTI roll-out cost was $1.5 million. It had an overall budget saving of $640,000 due to reductions of 100,100 catheterisations, 33,300 urine tests and 6,700 antibiotics administrations. Non-Metropolitan hospitals had a net saving of $1.2 million, while Metropolitan hospitals had a net cost of $0.54 million. Conclusions Compared to usual care, NO-CAUTI is expected to realise overall budget savings and decreases in catheterisations over five years. These findings allow a consideration of the affordability of a wide implementation. Trial registration Registered with the Australian New Zealand Clinical Trials Registry ( ACTRN12617000090314 ). First registered 17 January 2017, retrospectively. First enrolment, 15/11/2016.

Abstract Background Health organisations are increasingly implementing ‘embedded researcher’ models to translate research into practice. This paper examines the impact of an embedded researcher model known as the embedded Economist (eE) Program that was implemented in an Australian Primary Health Network (PHN) located in regional New South Wales, Australia. The site, participants, program aims and design are described. Insights into the facilitators, challenges and barriers to the integration of economic evaluation perspectives into the work of the PHN are provided. Methods The eE Program consisted of embedding a lead health economist on site, supported by offsite economists, part-time, for fifteen weeks to collaborate with PHN staff. Evaluation of the eE at the PHN included qualitative data collection via semi-structured interviews (N= 34), observations (N=8) and a field diary kept by the embedded economists. A thematic analysis was undertaken through the triangulation of this data. Results The eE Program successfully met its aims of increasing PHN staff awareness of the value of economic evaluation principles in decision-making and their capacity to access and apply these principles. There was also evidence that the program resulted in PHN staff applying economic evaluations when commissioning service providers. Evaluation of the eE identified two key facilitators for achieving these results. First, a highly receptive organisational context characterised by a work ethic, and site processes and procedures that were dedicated to improvement. Second was the development of trusted relationships between the embedded economist and PHN staff that was enabled through: the commitment of the economist to bi-directional learning; facilitating access to economic tools and techniques; personality traits (likeable and enthusiastic); and because the eE provided ongoing support for PHN projects beyond the fifteen-week embedding period. Conclusions This study provides the first detailed case description of an embedded health economics program. The results demonstrate how the process, context and relational factors of engaging and embedding the support of a health economist works and why. The findings reinforce international evidence in this area and are of practical utility to the future deployment of such programs.

Abstract Background Implementation of antenatal clinical guideline recommendations for addressing maternal alcohol consumption is sub-optimal. There is a complete absence of evidence of the cost and cost-effectiveness of delivering practice change interventions addressing maternal alcohol consumption amongst women accessing maternity services. The study sought to determine the cost, cost-consequence and cost-effectiveness of developing and delivering a multi-strategy practice change intervention in three sectors of a health district in New South Wales, Australia. Methods The trial-based economic analyses compared the costs and outcomes of the intervention to usual care over the 35-month period of the stepped-wedge trial. A health service provider perspective was selected to focus on the cost of delivering the practice change intervention, rather than the cost of delivering antenatal care itself. All costs are reported in Australian dollars ($AUD, 2019). Univariate and probabilistic sensitivity analyses assessed the effect of variation in intervention effect and costs. Results The total cost of delivering the practice change intervention across all three sectors was $367,646, of which $40,871 (11%) were development costs and $326,774 (89%) were delivery costs. Labour costs comprised 70% of the total intervention delivery cost. A single practice change strategy, ‘educational meetings and educational materials’ contributed 65% of the delivery cost. Based on the trial’s primary efficacy outcome, the incremental cost effectiveness ratio was calculated to be $32,570 (95% CI: $32,566–$36,340) per percent increase in receipt of guideline recommended care. Based on the number of women attending the maternity services during the trial period, the average incremental cost per woman who received all guideline elements was $591 (Range: $329 - $940) . The average cost of the intervention per eligible clinician was $993 (Range: $640-$1928). Conclusion The intervention was more effective than usual care, at an increased cost. Healthcare funders’ willingness to pay for this incremental effect is unknown. However, the strategic investment in systems change is expected to improve the efficiency of the practice change intervention over time. Given the positive trial findings, further research and monitoring is required to assess the sustainability of intervention effectiveness and whether economies of scale, or reduced costs of intervention delivery can be achieved without impact on outcomes. Trial registration The trial was prospectively registered with the Australian and New Zealand Clinical Trials Registry, No. ACTRN12617000882325 (date registered: 16/06/2017).

Abstract Background Physical Activity 4 Everyone (PA4E1) is an evidence-based program effective at increasing adolescent physical activity (PA) and improving weight status. This study aimed to determine a) the effectiveness of an adapted implementation intervention to scale-up PA4E1 at 24-month follow-up, b) fidelity and reach, and c) the cost and cost-effectiveness of the implementation support intervention. Methods A cluster randomised controlled trial using a type III hybrid implementation-effectiveness design in 49 lower socio-economic secondary schools, randomised to a program (n = 24) or control group (n = 25). An adapted implementation intervention consisting of seven strategies was developed to support schools to implement PA4E1 over 24-months. The primary outcome was the proportion of schools implementing at least four of the 7 PA practices, assessed via computer assisted telephone interviews (CATI) with Head Physical Education Teachers. Secondary outcomes included the mean number of PA practices implemented, fidelity and reach, cost and cost-effectiveness. Logistic regression models assessed program effects. Results At baseline, no schools implemented four of the 7 PA practices. At 24-months, significantly more schools in the program group (16/23, 69.6%) implemented at least four of the 7 PA practices than the control group (0/25, 0%) (p 75%). The total cost of the program was $415,112 AUD (2018) ($17,296 per school; $117.30 per student). Conclusions The adapted implementation intervention provides policy makers and researchers with an effective and potentially cost-effective model for scaling-up the delivery of PA4E1 in secondary schools. Further assessment of sustainability is warranted. Trial registration Australian New Zealand Clinical Trials Registry ACTRN12617000681358 prospectively registered 12th May 2017.

Abstract Background Urinary catheters are useful among hospital patients for allowing urinary flows and preparing patients for surgery. However, urinary infections associated with catheters cause significant patient discomfort and burden hospital resources. A nurse led intervention aiming to reduce inpatient catheterisation rates was recently trialled among adult overnight patients in four New South Wales hospitals. It included: ‘train-the trainer’ workshops, site champions, compliance audits and promotional materials. This study is the ‘in-trial’ cost-effectiveness analysis, conducted from the perspective of the New South Wales Ministry of Health. Methods The primary outcome variable was catheterisation rates. Catheterisation and procedure/treatment data were collected in three point prevalence patient surveys: pre-intervention (n = 1630), 4-months (n = 1677), and 9-months post-intervention (n = 1551). Intervention costs were based on trial records while labour costs were gathered from wage awards. Incremental cost effectiveness ratios were calculated for 4- and 9-months post-intervention and tested with non-parametric bootstrapping. Sensitivity scenarios recalculated results after adjusting costs and parameters. Results The trial found reductions in catheterisations across the four hospitals between preintervention (12.0 % (10.4 − 13.5 %), n = 195) and the 4- (9.9 % (8.5 − 11.3 %), n = 166 ) and 9- months (10.2 % (8.7 − 11.7 %) n = 158) post-intervention points. The trend was statistically non-significant (p = 0.1). Only one diagnosed CAUTI case was observed across the surveys. However, statistically and clinically significant decreases in catheterisation rates occurred for medical and critical care wards, and among female patients and short-term catheterisations. Incremental cost effectiveness ratios at 4-months and 9-months post-intervention were $188 and $264. Bootstrapping found reductions in catheterisations at positive costs over at least 72 % of iterations. Sensitivity scenarios showed that cost effectiveness was most responsive to changes in catheterisation rates. Conclusions Analysis showed that the association between the intervention and changes in catheterisation rates was not statistically significant. However, the intervention resulted in statistically significant reductions for subgroups including among short-term catheterisations and female patients. Cost-effectiveness analysis showed that reductions in catheterisations were most likely achieved at positive cost. Trial Registration Registered with the Australian New Zealand Clinical Trials Registry (ACTRN12617000090314). First hospital enrolment, 15/11/2016; last hospital enrolment, 8/12/2016.

Abstract Background Systematic approaches to the inclusion of economic evaluation in national healthcare decision-making are usual. It is less common for economic evaluation to be routinely undertaken at the ‘local-level’ (e.g. in a health service or hospital) despite the largest proportion of health care expenditure being determined at this service level and recognition by local health service decision makers of the need for capacity building in economic evaluation skills. This paper describes a novel program – the embedded Economist (eE) Program. The eE Program aims to increase local health service staff awareness of, and develop their capacity to access and apply, economic evaluation principles in decision making. The eE program evaluation is also described. The aim of the evaluation is to capture the contextual, procedural and relational aspects that assist and detract from the eE program aims; as well as the outcomes and impact from the specific eE projects. Methods The eE Program consists of a embedding a health economist in six health services and the provision of supported education in applied economic evaluation, provided via a community of practice and a university course. The embedded approach is grounded in co-production, embedded researchers and ‘slow science’. The sites, participants, and program design are described. The program evaluation includes qualitative data collection via surveys, semi-structured interviews, observations and field diaries. In order to share interim findings, data are collected and analysed prior, during and after implementation of the eE program, at each of the six health service sites. The surveys will be analysed by calculating frequencies and descriptive statistics. A thematic analysis will be conducted on interview, observation and filed diary data. The Framework to Assess the Impact from Translational health research (FAIT) is utilised to assess the overall impact of the eE Program. Discussion This program and evaluation will contribute to knowledge about how best to build capacity and skills in economic evaluation amongst decision-makers working in local-level health services. It will examine the extent to which participants are able to improve their ability to utilise evidence to inform decisions, avoid waste and improve the value of care delivery.

Abstract Background Despite the known benefits of healthy eating in childhood, few Australian childcare services provide food that is consistent with dietary guidelines. The effectiveness of a web-based menu planning intervention to increase childcare service provision of healthy foods and decrease provision of discretionary foods in long day-care services in Australia was assessed in a randomised controlled trial. Here we consider the costs, consequences, cost-effectiveness and budget impact of the intervention using data collected within the trial. Methods The prospective trial-based economic evaluation involved 54 childcare services across New South Wales (NSW), Australia. Services were randomised to a 12-month intervention or usual care. The intervention involved access to a web-based menu planning and decision support tool and online resources. Effectiveness measures included mean number of food groups, overall menu and individual food group compliance with dietary guidelines, and mean servings of food groups at 12 months. Costs (reported in $AUD, 2017/18) were evaluated from both health sector and societal perspectives. The direct cost to support uptake of the intervention was calculated, as were costs to each childcare service. The incremental cost of the intervention was calculated as the net difference in the cost to undertake menu planning and review plus the direct cost of the intervention. Incremental cost-effectiveness ratios (ICERs) including uncertainty intervals were estimated for differences in costs and effects between intervention and control groups. A relative value index was calculated to determine overall value for money. Results Over the 12 months of the trial, we calculated a difference in cost between usual practice and intervention groups of − $482 (95% UI − $859, − $56). While the measured increase in menu and food group compliance within the trial did not reach statistical significance, there were significant improvements in mean servings of fruit and discretionary food, represented in the cost-consequence analysis. The calculated relative value index of 1.1 suggests that the intervention returns acceptable value for money for the outcomes generated. Conclusion Compared to usual practice, web-based programmes may offer an efficient and sustainable alternative for childcare services to improve the provision of healthy foods to children in their care. Trial registration Australian New Zealand Clinical Trials Registry ACTRN12616000974404

Abstract Background Antenatal clinical practice guidelines recommend routine assessment of women’s alcohol consumption during pregnancy. The delivery of advice and referral when necessary are also recommended. However, evidence suggests there are barriers to the uptake of best-care guidelines. Effective, cost-effective and affordable implementation strategies are needed to ensure the intended benefits of guidelines are realised through addressing identified barriers. This paper describes the protocol for evaluating the efficiency and affordability of a practice change intervention compared to the usual practice in an implementation trial. Methods The effectiveness of the intervention will be evaluated in a stepped-wedge randomised controlled implementation trial, conducted in an Australian setting. An economic evaluation will be conducted alongside the trial to assess intervention efficiency. A budget impact assessment will be conducted to assess affordability. The prospective trial-based economic evaluation will identify, measure and value key resource and outcome impacts arising from the multi-strategy practice change intervention compared with usual practice. The evaluation will comprise (i) cost-consequence analyses, where a scorecard approach will be used to show the costs and benefits given the multiple primary outcomes included in the trial, and (ii) cost-effectiveness analyses, where the primary outcome will be incremental cost per percent increase in participants reporting receipt of antenatal care for maternal alcohol consumption consistent with the guideline recommendations. Intervention affordability will be evaluated using budget impact assessment and will estimate the financial implications of adoption and diffusion of this implementation strategy from the perspective of relevant fundholders. Results will be extrapolated to estimate the cost and cost-effectiveness of rolling out the model of care. Discussion Uptake of clinical guidelines requires practice change support. It is hypothesized that the implementation strategy, if found to be effective, will also be cost-effective, affordable and scalable. This protocol describes the economic evaluation that will address these hypotheses. Trial registration Australian and New Zealand Clinical Trials Registry, ACTRN12617000882325 . Registered on 16 June 2017

Abstract Background ‘Physical Activity 4 Everyone’ (PA4E1) was an efficacious multi-component school-based physical activity (PA) program targeting adolescents. PA4E1 has seven PA practices. It is essential to scale-up, evaluate effectiveness and assess implementation of such programs. Therefore, the aim is to assess the impact of implementation support on school practice uptake of the PA4E1 program at 12 and 24 months. Methods A cluster randomised controlled trial, utilising a type III hybrid implementation-effectiveness design, was conducted in 49 randomly selected disadvantaged Australian Government and Catholic secondary schools. A blinded statistician randomly allocated schools to a usual practice control (n = 25) or the PA4E1 program group (n = 24), with the latter receiving seven implementation support strategies to support school PA practice uptake of the seven practices retained from the efficacy trial. The primary outcome was the proportion of schools adopting at least four of the seven practices, assessed via telephone surveys with Head Physical Education Teachers and analysed using exact logistic regression modelling. This paper reports the 12-month outcomes. Results Schools were recruited from May to November 2017. At baseline, no schools implemented four of the seven practices. At 12 months significantly more schools in the program group had implemented four of the seven practices (16/24, 66.7%) than the control group (1/25, 4%) (OR = 33.0[4.15–1556.4], p 80%). Conclusions Through the application of multiple implementation support strategies, secondary schools were able to overcome commonly known barriers to implement evidence based school PA practices. As such practices have been shown to result in an increase in adolescent PA and improvements in weight status, policy makers and practitioners responsible for advocating PA in schools should consider this implementation approach more broadly when working with schools. Follow-up is required to determine whether practice implementation is sustained. Trial registration Australian New Zealand Clinical Trials Registry ACTRN12617000681358 registered 12th May 2017.

Abstract Background Clinical practice guidelines and policies direct community mental health services to provide preventive care to address chronic disease risks, however, such care is infrequently provided in routine consultations. An alternative model of care is to appoint a clinician to the dedicated role of offering and providing preventive care in an additional consultation: the ‘specialist clinician’ model. Economic evaluations of models of care are needed to determine the cost of adhering to guidelines and policies, and to inform pragmatic service delivery decisions. This study is an economic evaluation of the specialist clinician model; designed to achieve policy concordant preventive care delivery. Methods A retrospective analysis of the incremental costs, cost-effectiveness, and budget impact of a ‘specialist preventive care clinician’ (an occupational therapist) was conducted in a randomised controlled trial, where participants were randomised to receive usual care; or usual care plus the offer of an additional preventive care consultation with the specialist clinician. The study outcome was client acceptance of referrals to two free telephone-based chronic disease prevention services. This is a key care delivery outcome mandated by the local health district policy of the service. The base case analysis assumed the mental health service cost perspective. A budget impact analysis determined the annual budget required to implement the model of care for all clients of the community mental health service over 5 years. Results There was a significantly greater increase from baseline to follow-up in the proportion of intervention participants accepting referrals to both telephone services, compared to usual care. The incremental cost-effectiveness ratio was $347 per additional acceptance of a referral (CI: $263–$494). The annual budget required to implement the model of care for all prospective clients was projected to be $711,446 over 5-years; resulting in 2616 accepted referrals. Conclusions The evaluation provides key information regarding the costs for the mental health service to adhere to policy targets, indicating the model of care involved a low per client cost whilst increasing key preventive care delivery outcomes. Additional modelling is required to further explore its economic benefits. Trial registration ACTRN12616001519448. Registered 3 November 2016, https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=371709 .

Abstract Background A priority area in the field of substance dependence treatment is reducing the rates of relapse. Previous research has demonstrated that telephone delivered continuing care interventions are both clinically and cost effective when delivered as a component of outpatient treatment. This protocol describes a NSW Health funded study that assesses the effectiveness of delivering a telephone delivered continuing care intervention for people leaving residential substance treatment in Australia. Methods/design All participants will be attending residential alcohol and other drug treatment provided by The Salvation Army or We Help Ourselves. The study will be conducted as a randomised controlled trial, where participants will be randomised to one of three treatment arms. The treatment arms will be: (i) 12-session continuing care telephone intervention; (ii) 4-session continuing care telephone intervention, or (iii) continuing care plan only. Baseline assessment batteries and development of the participants’ continuing care plan will be completed prior to participants being randomised to a treatment condition. Research staff blind to the treatment condition will complete follow-up assessments with participants at 3-months and 6-months after they have been discharged from their residential service. Discussion This study will provide comprehensive data on the effect of delivering the continuing care intervention for people exiting residential alcohol and other drug treatment. If shown to be effective, this intervention can be disseminated to improve the rates of relapse among people leaving residential alcohol and other drug treatment. Trial registration Australian New Zealand Clinical Trials Registry, ACTRN12618001231235. Registered on 23rd July 2018. https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=375621&isReview=true

Abstract Background At a population level, small reductions in energy intake have the potential to contribute to a reduction in the prevalence of childhood obesity. In many school systems, there is the potential to achieve a reduction in energy intake through modest improvements in foods packed in children’s school lunchboxes. This study will assess the effectiveness and cost-effectiveness of a multi-component intervention that uses an existing school-based communication application to reduce the kilojoule content from discretionary foods and drinks consumed by children from school lunchboxes whilst at school. Methods A Type I hybrid effectiveness-implementation cluster randomised controlled trial will be conducted in up to 36 primary schools in the Hunter New England, Central Coast and Mid North Coast regions of New South Wales, Australia. Designed using the Behaviour Change Wheel, schools will be randomly allocated to receive either a 5-month (1.5 school terms) multi-component intervention that includes: 1) school lunchbox nutrition guidelines; 2) curriculum lessons; 3) information pushed to parents via an existing school-based communication application and 4) additional parent resources to address common barriers to packing healthy lunchboxes or a control arm (standard school practices). The study will assess both child level dietary outcomes and school-level implementation outcomes. The primary trial outcome, mean energy (kJ) content of discretionary lunchbox foods packed in children’s lunchboxes, will be assessed at baseline and immediately post intervention (5 months or 1.5 school terms). Analyses will be performed using intention to treat principles, assessing differences between groups via hierarchical linear regression models. Discussion This study will be the first fully powered randomised controlled trial internationally to examine the impact of an m-health intervention to reduce the mean energy from discretionary food and drinks packed in the school lunchbox. The intervention has been designed with scalability in mind and will address an important evidence gap which, if shown to be effective, has the potential to be applied at a population level. Trial registration Australian Clinical Trials Registry ACTRN:12618001731280 registered on 17/10/2018. Protocol Version 1.

Abstract Background Recurrent stroke is a major contributor to stroke-related disability and costs. Improving health-risk behaviours and mental health has the potential to significantly improve recovery, enhance health-related quality of life (HRQoL), independent living, and lower the risk of recurrent stroke. The primary aim will be to test the effectiveness of an online intervention to improve HRQoL among stroke survivors at 6 months’ follow-up. Programme effectiveness on four health behaviours, anxiety and depression, cost-effectiveness, and impact on other hospital admissions will also be assessed. Methods/design An open-label randomised controlled trial is planned. A total of 530 adults will be recruited across one national and one regional stroke registry and block randomised to the intervention or minimal care control group. The intervention group will receive access to the online programme Prevent 2nd Stroke (P2S); the minimal care control group will receive an email with Internet addresses of generic health sites designed for the general population. The primary outcome, HRQoL, will be measured using the EuroQol-5D. A full analysis plan will compare between groups from baseline to follow-up. Discussion A low-cost per user option to supplement current care, such as P2S, has the potential to increase HRQoL for stroke survivors, and reduce the risk of second stroke. Trial registration Australian and New Zealand Clinical Trials Registry, ID: ACTRN12617001205325p. Registered on 17 August 2017.

Abstract Background The implementation of interventions at-scale is required to maximise population health benefits. ‘Physical Activity 4 Everyone (PA4E1)’ was a multi-component school-based program targeting adolescents attending secondary schools in low socio-economic areas. An efficacy trial of the intervention demonstrated an increase in students’ mean minutes of moderate-to-vigorous physical activity (MVPA) per day and lower weight gain at low incremental cost. This study aims to assess the effectiveness and cost effectiveness of a multi-component implementation support intervention to improve implementation, at-scale, of the evidence based school physical activity (PA) practices of the PA4E1 program. Impact on student PA levels and adiposity will also be assessed, in addition to the cost of implementation. Methods A cluster randomised controlled trial, utilising an effectiveness-implementation hybrid design, will be conducted in up to 76 secondary schools located in lower socio-economic areas across four health districts in New South Wales (NSW), Australia. Schools will be randomly allocated to a usual practice control arm or a multi-component implementation support intervention to embed the seven school PA practices of the PA4E1 program. The implementation support intervention incorporates seven strategies including executive support, in-School Champion, teacher training, resources, prompts, audit and feedback and access to an external Support Officer. The primary trial outcome will be the proportion of schools meeting at least four of the seven physical activity practices of the program, assessed via surveys with Head Physical Education teachers at 12 and 24-months. Secondary outcomes will be assessed via a nested evaluation of student PA and adiposity at 12-months (Grade 8 students) and 24 months (Grade 9 students) undertaken in 30 schools (15 per group). Resource use associated with the implementation intervention will be measured prospectively. Linear mixed effects regression models will assess program effects on the primary outcome at each follow-up period. Discussion This study is one of few evidence-based multi-component PA programs scaled-up to a large number of secondary schools and evaluated via randomised controlled trial. The use of implementation science theoretical frameworks to implement the evidence-based program and the rigorous evaluation design are strengths of the study. Trial registration Australian New Zealand Clinical Trials Registry ACTRN12617000681358 registered 12th May 2017. Protocol Version 1.

Abstract Background To date, efforts to measure impact have largely focused on health research in high-income countries, reflecting where the majority of health research funding is spent. Nevertheless, there is a growing body of health and medical research being undertaken in low- and middle-income countries (LMICs), supported by both development aid and established research funders. The Framework to Assess the Impact of Translational health research (FAIT) combines three approaches to measuring research impact (Payback, economic assessment and case study narrative). Its aim is to strengthen the focus on translation and impact measurement in health research. FAIT has been used by several Australian research initiatives; however, it has not been used in LMICs. Our aim was to apply FAIT in an LMIC context and evaluate its utility. Methods We retrospectively applied all three FAIT methods to two LMIC studies using available data, supplemented with group discussion and further economic analyses. Results were presented in a scorecard format. Results FAIT helped clarify pathways of impact for the projects and provided new knowledge on areas of impact in several domains, including capacity-building for research, policy development and economic impact. However, there were constraints, particularly associated with calculating the return on investment in the LMIC context. The case study narrative provided a layperson’s summary of the research that helped to explain outcomes and succinctly communicate lessons learnt. Conclusion Use of FAIT to assess the impact of LMIC research was both feasible and useful. We make recommendations related to prospective use, identification of metrics to support use of the Payback framework, and simplification of the economic assessment, which may facilitate further application in LMIC environments.

Abstract Background Diabetes is increasing in incidence, morbidity and treatment costs globally, hence prevention strategies need to be explored. Animal studies and some human data have shown that zinc can improve glycaemic control, but the impact of this effect in a pre-diabetic population remains uncertain. This study is designed to investigate whether zinc gluconate and lifestyle coaching can improve glucose handling and ultimately reduce diabetes incidence in an at-risk pre-diabetic population in Australia. Methods/design The study will be a randomised, placebo-controlled, double-blind clinical trial. The study will be conducted at the Hunter New England Local Health District New South Wales (NSW), Australia. Pre-diabetic (haemoglobin A1c [HbA1c] 5.7–6.4) male and female participants (n = 410) aged 40–70 years will be recruited through the Diabetes Alliance Network, a collaboration of diabetes specialists and general practitioner practices. All participants will be given routine care to encourage healthy lifestyle changes using a telephone coaching service (Get Healthy Information and Coaching Service, NSW Health) and then randomised to receive a supplement, either zinc gluconate (equivalent to 30 mg of elemental zinc) or placebo of identical appearance for 12 months. The identity of the supplements will be blinded to both research personnel and the participants. Participants will be asked to complete medical, lifestyle and dietary surveys and will have baseline and final visits at their general practitioner practice. Primary outcomes will be HbA1c and insulin sensitivity collected at baseline and at 1, 6 and 12 months; secondary outcomes will include fasting blood glucose, fasting cholesterol, blood pressure and body mass index. The primary efficacy endpoint will be judged at 6 months. Discussion This study will generate new evidence about the potential for health coaching, with or without zinc supplementation, to improve glucose handling and ultimately to reduce progression from pre-diabetes to diabetes. Trial registration Australian and New Zealand Clinical Trials Registry, ACTRN12618001120268. Registered on 6 July 2018.

Abstract Background In an attempt to improve children’s physical activity levels governments have introduced policies specifying the minimum time schools are to schedule physical activity each week. Despite this, the majority of schools in many jurisdictions fail to implement these policies. This study will assess the effectiveness of a multi-component implementation strategy on increasing the minutes of planned physical activity scheduled by primary school teachers each week. Methods A cluster randomised controlled trial will be conducted in 62 primary schools in the Hunter New England region of New South Wales, Australia. Schools will be randomly allocated to receive either a multi-component implementation strategy that includes; obtaining executive support, training in-school champions, provision of tools and resources, implementation prompts, reminders and feedback; or usual practice. The study will employ an effectiveness-implementation hybrid design, assessing both policy implementation and individual (student) behavioural outcomes. The primary trial outcome of mean minutes of physical activity scheduled by classroom teachers across the school week will be measured via teacher log-book at baseline and approximately 12 and 18 months post baseline. A nested evaluation of the impact of policy implementation on child physical activity will be undertaken of students in Grades 2 and 3. Analyses will be performed using an intention to treat framework. Linear mixed effects regression models will be used to assess intervention effects on the primary outcome at both follow-up periods. Discussion This study will be the one of the first well powered randomised trials internationally to examine the impact of an implementation strategy for a physical activity policy in primary schools and will address a fundamental research translation gap. Given the dearth of research, the findings will be important in informing future implementation efforts in this setting. Trial registration ANZCTR ACTRN12617001265369 version 1 registered 1st September 2017.

Abstract Background The aim of this systematic literature review is to identify and critique full economic evaluations of interventions for high risk young people with the purpose of informing the design of future rigorous economic evaluations of such intervention programs. Methods A PRISMA compliant search of the literature between 2000 and April 2018 was conducted to identify full economic evaluations of youth focussed interventions for at risk young people. Duplicates were removed and two researchers independently screened the article titles and abstracts according to PICOS criteria for exclusion and inclusion. The remaining full text articles were assessed for eligibility and a quality assessment of the included articles was conducted using the Drummond checklist. Results The database, grey literature and hand searches located 488 studies of interventions for at risk young people. After preliminary screening of titles and abstracts, 104 studies remained for full text examination and 29 empirical studies containing 32 separate economic evaluations were judged eligible for inclusion in the review. These comprised 13 cost-benefit analyses (41%), 17 cost-effectiveness analyses (53%), one cost-utility analysis (3%) and a social return on investment (3%). Three main methodological challenges were identified: 1. attribution of effects; 2. measuring and valuing outcomes; and 3. identifying relevant costs and consequences. Conclusions A cost-benefit analysis would best capture the dynamic nature of a multi-component intervention for high risk young people, incorporating broader intersectoral outcomes and enabling measurement of more domains of risk. Prospective long-term data collection and a strong study design that incorporates a control group contribute to the quality of economic evaluation. Extrapolation of impact into the future is important for this population, in order to account for the time lag in effect of many impacts and benefits arising from youth interventions.

Abstract Background Despite clinical guideline recommendations, implementation of antenatal care addressing alcohol consumption by pregnant women is limited. Implementation strategies addressing barriers to such care may be effective in increasing care provision. The aim of this study is to examine the effectiveness, cost and cost-effectiveness of a multi-strategy practice change intervention in increasing antenatal care addressing the consumption of alcohol by pregnant women. Methods The study will be a randomised, stepped-wedge controlled trial conducted in three sectors in a health district in New South Wales, Australia. Stepped implementation of a practice change intervention will be delivered to sectors in a random order to support the introduction of a model of care for addressing alcohol consumption by pregnant women. A staged process was undertaken to develop the implementation strategies, which comprise of: leadership support, local clinical practice guidelines, electronic prompts and reminders, opinion leaders, academic detailing (audit and feedback), educational meetings and educational materials, and performance monitoring. Repeated cross-sectional outcome data will be gathered weekly across all sectors for the study duration. The primary outcome measures are the proportion of antenatal appointments at ‘booking in’, 27–28 weeks gestation and 35–36 weeks gestation for which women report (1) being assessed for alcohol consumption, (2) being provided with brief advice related to alcohol consumption during pregnancy, (3) receiving relevant care for addressing alcohol consumption during pregnancy, and (4) being assessed for alcohol consumption and receiving relevant care. Data on resources expended during intervention development and implementation will be collected. The proportion of women who report consuming alcohol since knowing they were pregnant will be measured as a secondary outcome. Discussion This will be the first randomised controlled trial to evaluate the effectiveness, cost and cost-effectiveness of implementation strategies in improving antenatal care that addresses alcohol consumption by pregnant women. If positive changes in clinical practice are found, this evidence will support health service adoption of implementation strategies to support improved antenatal care for this recognised risk to the health and wellbeing of the mother and child. Trial registrations Australian and New Zealand Clinical Trials Registry, No. ACTRN12617000882325 (date registered: 16/06/2017).

Abstract Background There is growing recognition that the wider benefits of research (economic, social and health impacts) should be assessed and valued alongside traditional research performance metrics such as peer-reviewed papers. Translation of findings into policy and practice needs to accelerate and pathways to impact need to be better understood. This research protocol outlines a mixed methods study to apply the Framework to Assess the Impact from Translational health research (FAIT) to the Centre for Research Excellence in Stroke Rehabilitation and Brain Recovery (CRE-Stroke). FAIT is purpose-designed to encourage research translation and assess research impact but lacks validation. Methods/Design Phase 1 involves application of the FAIT-modified programme logic model to each CRE-Stroke research stream including identifying process, output and impact metrics, as well as end users of the research. A scoping review will inform potential impacts anticipated from CRE-Stroke. In Phase 2, audit and feedback on achievements against plans will track and encourage research translation. Logic models will be updated to account for changes in the research pathways over time. In Phase 3, three proven methods for measuring research impact – Payback, economic assessment and narratives – will be applied to each research stream and the data triangulated and reported in Phase 4. The feasibility of applying FAIT will also be assessed as part of Phase 3. Discussion Use of prospective, comprehensive research impact frameworks for large interdisciplinary programmes of research is rare. FAIT’s application to CRE-Stroke will provide opportunity for the impact of CRE-Stroke to be assessed and a range of impacts beyond standard academic achievements to be reliably reported. The feasibility of FAIT’s application will also be assessed and, if necessary, refined. The usefulness of FAIT for encouraging research translation will also be described and may prove useful for other programmes looking to implement a research impact framework.

Abstract Background The question of how to measure, assess and optimise the returns from investment in health and medical research (HMR) is a highly policy-relevant issue. Research Impact Assessment Frameworks (RIAFs) provide a conceptual measurement framework to assess the impact from HMR. The aims of this study were (1) to elicit the views of Medical Research Institutes (MRIs) regarding objectives, definitions, methods, barriers, potential scope and attitudes towards RIAFs, and (2) to investigate whether an assessment framework should represent a retrospective reflection of research impact or a prospective approach integrated into the research process. The wider objective was to inform the development of a draft RIAF for Australia’s MRIs. Methods Purposive sampling to derive a heterogeneous sample of Australian MRIs was used alongside semi-structured interviews with senior executives responsible for research translation or senior researchers affected by research impact initiatives. Thematic analysis of the interview transcriptions using the framework approach was then performed. Results Interviews were conducted with senior representatives from 15 MRIs. Participants understood the need for greater research translation/impact, but varied in their comprehension and implementation of RIAFs. Common concerns included the time lag to the generation of societal impacts from basic or discovery science, and whether impact reflected a narrow commercialisation agenda. Broad support emerged for the use of metrics, case study and economic methods. Support was also provided for the rationale of both standardised and customised metrics. Engendering cultural change in the approach to research translation was acknowledged as both a barrier to greater impact and a critical objective for the assessment process. Participants perceived that the existing research environment incentivised the generation of academic publications and track records, and often conflicted with the generation of wider impacts. The potential to improve the speed of translation through prospective implementation of impact assessment was supported, albeit that the mechanism required development. Conclusion The study found that the issues raised regarding research impact assessment are less about methods and metrics, and more about the research activities that the measurement of research translation and impact may or may not incentivise. Consequently, if impact assessment is to contribute to optimisation of the health gains from the public, corporate and philanthropic investment entrusted to the institutes, then further inquiry into how the assessment process may re-align research behaviour must be prioritised.

Abstract Background There is growing recognition amongst health and medical research funders and researchers that translation of research into policy and practice needs to increase and that more transparency is needed on how impacts are realised. Several approaches are advocated for achieving this, including co-production of research or academic-practitioner research. The Population Health Unit (PHU) within the Hunter New England Local Health District in regional Australia, as an early adopter of this model, has been working to increase the likelihood that its research is translated into community health benefits. With the New South Wales Ministry of Health, the PHU responded to the burden of child overweight and obesity by combining service delivery with research expertise. The ‘Good for Kids, Good for Life’ (Good for Kids) dissemination trial was developed and implemented in seven community settings in the Hunter region of Australia between 2006 and 2010. This study aims to undertake a retrospective impact assessment to measure the research translation and impact of Good for Kids. Methods The method will be based upon the application of the Framework to Assess the Impact from Translational health research (FAIT), comprising three core elements, namely quantified metrics, economic assessment and a narrative of the process by which the research in question translates and generates impact. Discussion Increasingly, funders are interested both in the outcomes resulting from investments in health research and in the expected return on their investments. FAIT was developed specifically for this purpose and its use is anticipated to provide transparency to the pathway to translation and potentially drive increased investment in translational research programmes such as Good for Kids.

Abstract Background Urinary tract infection (UTI) as the most common healthcare-associated infection accounts for up to 36% of all healthcare-associated infections. Catheter-associated urinary tract infection (CAUTI) accounts for up to 80% of these. In many instances indwelling urinary catheter (IDC) insertions may be unjustified or inappropriate, creating potentially avoidable and significant patient distress, embarrassment, discomfort, pain and activity restrictions, together with substantial care burden, costs and hospitalisation. Multifaceted interventions combining best practice guidelines with staff engagement, education and monitoring have been shown to be more effective in bringing about practice change than those that focus on a single intervention. This study builds on a nurse-led initiative that identified that significant benefits could be achieved through a systematic approach to implementation of evidence-based practice. Methods The primary aim of the study is to reduce IDC usage rates by reducing inappropriate urinary catheterisation and duration of catheterisation. The study will employ a multiple pre-post control intervention design using a phased mixed method approach. A multifaceted intervention will be implemented and evaluated in four acute care hospitals in NSW, Australia. The study design is novel and strengthened by a phased approach across sites which allows for a built-in control mechanism and also reduces secular effects. Feedback of point prevalence data will be utilised to engage staff and improve compliance. Ward-based champions will help to steward the change and maintain focus. Discussion This study will improve patient safety through implementation and robust evaluation of clinical practice and practice change. It is anticipated that it will contribute to a significant improvement in patient experiences and health care outcomes. The provision of baseline data will provide a platform from which to ensure ongoing improvement and normalisation of best practice. This study will add to the evidence base through enhancing understanding of interventions to reduce CAUTI and provides a prototype for other studies focussed on reduction of hospital acquired harms. Study findings will inform undergraduate and continuing education for health professionals. Trial registration ACTRN12617000090314 . Registered 17 January 2017. Retrospectively registered.

Abstract Background Population-based registries are increasingly used to recruit patient samples for research, however, they have several limitations including low consent and participation rates, and potential selection bias. To improve access to samples for research, the utility of a new model of recruitment termed the ‘Consumer Register’, that allows for direct patient recruitment from hospitals, was examined. This paper reports: (i) consent rates onto the register; (ii) preferred methods and frequency of contact; and (iii) the feasibility of establishing the register, including: (a) cost per person recruited to the register; (b) the differential cost and consent rates of volunteer versus paid data collectors; and (c) participant completion rates. Methods A cross-sectional survey was conducted in five outpatient clinics in Australia. Patients were approached by volunteers or paid data collectors and asked to complete a touch-screen electronic survey. Consenting individuals were asked to indicate their willingness and preferences for enrolment onto a research register. Descriptive statistics were used to examine patient preferences and linear regression used to model the success of volunteer versus paid data collectors. The opportunity and financial costs of establishing the register were calculated. Results A total of 1947 patients (80.6 %) consented to complete the survey, of which, 1486 (76.3 %) completed the questionnaire. Of the completers, the majority (69.4 %, or 1032 participants) were willing to be listed on the register and preferred to be contacted by email (50.3 %). Almost 39 % of completers were willing to be contacted three or more times in a 12 month period. The annual opportunity cost of resources consumed by the register was valued at $37,187, giving an opportunity cost per person recruited to the register of $36. After amortising fixed costs, the annual financial outlay was $23,004 or $22 per person recruited to the register. Use of volunteer data collectors contributed to an annual saving of $14,183, however paid data collectors achieved significantly higher consent rates. Successful enrolment onto the register was completed for 42 % of the sample. Conclusions A Consumer Register is a promising and feasible alternative to population-based registries, with the majority of participants willing to be contacted multiple times via low-resource methods such as email. There is an effectiveness/cost trade off in the use of paid versus volunteer data collectors.

It was highlighted that in the original article [1] author Amanda Baker was erroneously omitted from the authorship during the copy editing stage.