Search

Objectives The sandbox approach, developed in the financial technologies sector, creates an environment to collaboratively develop and test innovative new products, methods and regulatory approaches, separated from business as usual. It has been used in health care to encourage innovation in response to emerging challenges, but, until recently, has not been used in health technology assessment (HTA). This article summarizes our learnings from using the sandbox approach to address three challenges facing HTA organizations and to identify implications for the use of this approach in HTA. Methods We identified three challenging contemporary HTA-related topics to explore in a sandbox environment, away from the pressures and interests of live assessments. We convened a pool of 120 stakeholders and experts to participate in various sandbox activities and ultimately co-develop solutions to help HTA organizations respond to the identified challenges. Results Important general learnings about the potential benefits and implementation of a sandbox approach in HTA were identified. Consequently, we developed recommendations to guide its use, including how to implement an HTA sandbox in an effective way and the types of challenges for which it may be best suited. Conclusions For many HTA organizations, it is difficult to carefully consider emerging challenges and innovate their processes due to risks associated with decision errors and resource limitations. The sandbox approach could reduce these barriers. The potential benefits of addressing HTA challenges in a collaborative safe space are considerable.

Objectives: The sandbox approach, developed in the financial technologies sector, creates an environment to collaboratively develop and test innovative new products, methods and regulatory approaches, separated from business as usual. It has been used in health care to encourage innovation in response to emerging challenges, but, until recently, has not been used in health technology assessment (HTA). This article summarizes our learnings from using the sandbox approach to address three challenges facing HTA organizations and to identify implications for the use of this approach in HTA.Methods: We identified three challenging contemporary HTA-related topics to explore in a sandbox environment, away from the pressures and interests of live assessments. We convened a pool of 120 stakeholders and experts to participate in various sandbox activities and ultimately co-develop solutions to help HTA organizations respond to the identified challenges. Results: Important general learnings about the potential benefits and implementation of a sandbox approach in HTA were identified. Consequently, we developed recommendations to guide its use, including how to implement an HTA sandbox in an effective way and the types of challenges for which it may be best suited. Conclusions: For many HTA organizations, it is difficult to carefully consider emerging challenges and innovate their processes due to risks associated with decision errors and resource limitations. The sandbox approach could reduce these barriers. The potential benefits of addressing HTA challenges in a collaborative safe space are considerable.

Objectives: Economic evaluations (EEs) are commonly used by decision makers to understand the value of health interventions. The Consolidated Health Economic Evaluation Reporting Standards (CHEERS 2022) provide reporting guidelines for EEs. Healthcare systems will increasingly see new interventions that use artificial intelligence (AI) to perform their function. We developed Consolidated Health Economic Evaluation Reporting Standards for Interventions that use AI (CHEERS-AI) to ensure EEs of AI-based health interventions are reported in a transparent and reproducible manner. Methods: Potential CHEERS-AI reporting items were informed by 2 published systematic literature reviews of EEs and a contemporary update. A Delphi study was conducted using 3 survey rounds to elicit multidisciplinary expert views on 26 potential items, through a 9-point Likert rating scale and qualitative comments. An online consensus meeting was held to finalize outstanding reporting items. A digital health patient group reviewed the final checklist from a patient perspective. Results: A total of 58 participants responded to survey round 1, 42, and 31 of whom responded to rounds 2 and 3, respectively. Nine participants joined the consensus meeting. Ultimately, 38 reporting items were included in CHEERS-AI. They comprised the 28 original CHEERS 2022 items, plus 10 new AI-specific reporting items. Additionally, 8 of the original CHEERS 2022 items were elaborated on to ensure AI-specific nuance is reported. Conclusions: CHEERS-AI should be used when reporting an EE of an intervention that uses AI to perform its function. CHEERS-AI will help decision makers and reviewers to understand important AI-specific details of an intervention, and any implications for the EE methods used and cost-effectiveness conclusions.

Background The availability of increasingly advanced and expensive new health technologies puts considerable pressure on publicly financed healthcare systems. Decisions to not—or no longer—reimburse a health technology from public funding may become inevitable. Nonetheless, policymakers are often pressured to amend or revoke negative reimbursement decisions due to the public disagreement that typically follows such decisions. Public disagreement may be reinforced by the publication of pictures of individual patients in the media. Our aim was to assess the effect of depicting a patient affected by a negative reimbursement decision on public disagreement with the decision. Methods We conducted a discrete choice experiment in a representative sample of the public (n = 1008) in the Netherlands and assessed the likelihood of respondents’ disagreement with policymakers’ decision to not reimburse a new pharmaceutical for one of two patient groups. We presented a picture of one of the patients affected by the decision for one patient group and “no picture available” for the other group. The groups were described on the basis of patients’ age, health-related quality of life (HRQOL) and life expectancy (LE) before treatment, and HRQOL and LE gains from treatment. We applied random-intercept logit regression models to analyze the data. Results Our results indicate that respondents were more likely to disagree with the negative reimbursement decision when a picture of an affected patient was presented. Consistent with findings from other empirical studies, respondents were also more likely to disagree with the decision when patients were relatively young, had high levels of HRQOL and LE before treatment, and large LE gains from treatment. Conclusions This study provides evidence for the effect of depicting individual, affected patients on public disagreement with negative reimbursement decisions in healthcare. Policymakers would do wel

IntroductionMeaningful patient involvement in health technology assessment (HTA) is essential in ensuring that the interests of the affected patient population, their families, and the general public are accurately reflected in coverage and reimbursement decisions. Central and Eastern European (CEE) countries are generally at less advanced stages of implementing HTA, which is particularly true for patient involvement activities. As part of the Horizon2020 HTx project, this research aimed to form recommendations for critical barriers to patient involvement in HTA in CEE countries. MethodsBuilt on previous research findings on potential barriers, a prioritisation survey was conducted online with CEE stakeholders. Recommendations for prioritised barriers were formed through a face-to-face workshop by CEE stakeholders and HTx experts. ResultsA total of 105 stakeholders from 13 CEE countries completed the prioritisation survey and identified 12 of the 22 potential barriers as highly important. The workshop had 36 participants representing 9 CEE countries, and 5 Western European countries coming together to discuss solutions in order to form recommendations based on best practices, real-life experience, and transferability aspects. Stakeholder groups involved in both phases included HTA organisation representatives, payers, patients, caregivers, patient organisation representatives, patient experts, health care providers, academic and non-academic researchers, health care consultants and health technology manufacturers/providers. As a result, 12 recommendations were formed specified to the CEE region's context, but potentially useful for a broader geographic audience. ConclusionIn this paper, we present 12 recommendations for meaningful, systematic, and sustainable patient involvement in HTA in CEE countries. Our hope is that engaging more than a hundred CEE stakeholders in the study helped to spread awareness of the importance and potential of patient involvement and tha

BACKGROUND: Artificial intelligence (AI) has attracted much attention because of its enormous potential in healthcare, but uptake has been slow. There are substantial barriers that challenge health technology assessment (HTA) professionals to use AI-generated evidence for decision-making from large real-world databases (e.g., based on claims data). As part of the European Commission-funded HTx H2020 (Next Generation Health Technology Assessment) project, we aimed to put forward recommendations to support healthcare decision-makers in integrating AI into the HTA processes. The barriers, addressed by the paper, are particularly focusing on Central and Eastern European (CEE) countries, where the implementation of HTA and access to health databases lag behind Western European countries.METHODS: We constructed a survey to rank the barriers to using AI for HTA purposes, completed by respondents from CEE jurisdictions with expertise in HTA. Using the results, two members of the HTx consortium from CEE developed recommendations on the most critical barriers. Then these recommendations were discussed in a workshop by a wider group of experts, including HTA and reimbursement decision-makers from both CEE countries and Western European countries, and summarized in a consensus report.RESULTS: Recommendations have been developed to address the top 15 barriers in areas of (1) human factor-related barriers, focusing on educating HTA doers and users, establishing collaborations and best practice sharing; (2) regulatory and policy-related barriers, proposing increasing awareness and political commitment and improving the management of sensitive information for AI use; (3) data-related barriers, suggesting enhancing standardization and collaboration with data networks, managing missing and unstructured data, using analytical and statistical approaches to address bias, using quality assessment tools and quality standards, improving reporting, and developing better con

Aim: Real-world data and real-world evidence (RWE) are becoming more important for healthcare decision making and health technology assessment. We aimed to propose solutions to overcome barriers preventing Central and Eastern European (CEE) countries from using RWE generated in Western Europe. Materials & methods: To achieve this, following a scoping review and a webinar, the most important barriers were selected through a survey. A workshop was held with CEE experts to discuss proposed solutions. Results: Based on survey results, we selected the nine most important barriers. Multiple solutions were proposed, for example, the need for a European consensus, and building trust in using RWE. Conclusion: Through collaboration with regional stakeholders, we proposed a list of solutions to overcome barriers on transferring RWE from Western Europe to CEE countries. Plain language summary: Collecting real-world data and generating real-world evidence from it is becoming more important for making better decisions in healthcare. We investigated the main barriers which prevent using real-world evidence in Central and Eastern Europe, originally generated in Western Europe. After identifying the nine most important barrier, with the help of local experts we proposed solutions to overcome those barriers. Several possible solutions were proposed, many of them highlighting the need for a European consensus on these matters and building trust in new methods. Our results can hopefully serve as a guidance document to help overcome the barriers. Tweetable abstract: Research investigating ways to overcome barriers preventing Central and Eastern European countries from using Western European real-world evidence for healthcare decision making, using a multi-methods approach to create a list of solutions in collaboration with local stakeholders.

Objective: Sodium–glucose cotransporter 2 inhibitors (SGLT2i) have been shown to reduce the risk of cardiovascular complications, which largely drive diabetes’ health and economic burdens. Trial results indicated that SGLT2i are cost effective. However, these findings may not be generalizable to the real-world target population. This study aims to evaluate the cost effectiveness of SGLT2i in a routine care type 2 diabetes population that meets Dutch reimbursement criteria using the MICADO model. Methods: Individuals from the Hoorn Diabetes Care System cohort (N = 15,392) were filtered to satisfy trial inclusion criteria (including EMPA-REG, CANVAS, and DECLARE-TIMI58) or satisfy the current Dutch reimbursement criteria for SGLT2i. We validated a health economic model (MICADO) by comparing simulated and observed outcomes regarding the relative risks of events in the intervention and comparator arm from three trials, and used the validated model to evaluate the long-term health outcomes using the filtered cohorts’ baseline characteristics and treatment effects from trials and a review of observational studies. The incremental cost-effectiveness ratio (ICER) of SGLT2i, compared with care-as-usual, was assessed from a third-party payer perspective, measured in euros (2021 price level), using a discount rate of 4% for costs and 1.5% for effects. Results: From Dutch individuals with diabetes in routine care, 15.8% qualify for the current Dutch reimbursement criteria for SGLT2i. Their characteristics were significantly different (lower HbA1c, higher age, and generally more preexisting complications) than trial populations. After validating the MICADO model, we found that lifetime ICERs of SGLT2i, when compared with usual care, were favorable (< €20,000/QALY) for all filtered cohorts, resulting in an ICER of €5440/QALY using trial-based treatment effect estimates in reimbursed population. Several pragmatic scenarios were tested, the ICERs remained favorable. Conclusio

Introduction Drug reimbursement decisions are often made based on a price set by the manufacturer. In some cases, this price leads to public and scientific debates about whether its level can be justified in relation to its costs, including those related to research and development (R&D) and manufacturing. Such considerations could enter the decision process in collectively financed health care systems. This paper investigates whether manufacturers’ costs in relation to drug prices, or profit margins, are explicitly mentioned and considered by health technology assessment (HTA) organisations. Method An analysis of reimbursement reports for cancer drugs was performed. All relevant Dutch HTA-reports, published between 2017 and 2019, were selected and matched with HTA-reports from three other jurisdictions (England, Canada, Australia). Information was extracted. Additionally, reimbursement reports for three cases of expensive non-oncolytic orphan drugs prominent in pricing debates in the Netherlands were investigated in depth to examine consideration of profit margins. Results A total of 66 HTA-reports concerning 15 cancer drugs were included. None of these reports contained information on manufacturer’s costs or profit margins. Some reports contained general considerations of the HTA organisation which related prices to manufacturers’ costs: six contained a statement on the lack of price setting transparency, one mentioned recouping R&D costs as a potential argument to justify a high price. For the case studies, 21 HTA-reports were selected. One contained a cost-based price justification provided by the manufacturer. None of the other reports contained information on manufacturer’s costs or profit margins. Six reports contained a discussion about lack of transparency. Reports from two jurisdictions contained invitations to justify high prices by demonstrating high costs. Conclusion Despite the attention given to manuf

INTRODUCTION: To make efficient use of available resources, decision-makers in healthcare may assess the costs and (health) benefits of health interventions. For interventions aimed at improving mental health capturing the full health benefits is an important challenge. The Mental Health Quality of Life (MHQoL) instrument was recently developed to meet this challenge. Evaluating the pyschometric properties of this instrument in different contexts remains important. METHODS: A psychometric evaluation of the MHQoL was performed using existing international, cross-sectional data with 7155 respondents from seven European countries (Denmark, France, Germany, Italy, Portugal, The Netherlands, Portugal and the United Kingdom). Reliability was examined by calculating Cronbach's alpha, a measure of internal consistency of the seven MHQoL dimensions, and by examining the association of the MHQoL sum scores with the MHQoL-VAS scores. Construct validity was examined by calculating Spearman's rank correlation coefficients between the MHQoL sum scores and EQ-5D index scores, EQ-VAS scores, EQ-5D anxiety/depression dimension scores, ICECAP-A index scores and PHQ-4 sum scores. RESULTS: The MHQoL was found to have good internal consistency for all seven countries. The MHQoL sum score and the MHQoL-VAS had a high correlation. Spearman's rank correlation coefficients were moderate to very high for all outcomes. CONCLUSION: Our results, based on data gathered in seven European countries, suggest that the MHQoL shows favourable psychometrical characteristics. While further validation remains important, the MHQoL may be a useful instrument in measuring mental health-related quality of life in the Western European context.

Introduction: Drug reimbursement decisions are often made based on a price set by the manufacturer. In some cases, this price leads to public and scientific debates about whether its level can be justified in relation to its costs, including those related to research and development (R&D) and manufacturing. Such considerations could enter the decision process in collectively financed health care systems. This paper investigates whether manufacturers’ costs in relation to drug prices, or profit margins, are explicitly mentioned and considered by health technology assessment (HTA) organisations. Method: An analysis of reimbursement reports for cancer drugs was performed. All relevant Dutch HTA-reports, published between 2017 and 2019, were selected and matched with HTA-reports from three other jurisdictions (England, Canada, Australia). Information was extracted. Additionally, reimbursement reports for three cases of expensive non-oncolytic orphan drugs prominent in pricing debates in the Netherlands were investigated in depth to examine consideration of profit margins. Results: A total of 66 HTA-reports concerning 15 cancer drugs were included. None of these reports contained information on manufacturer’s costs or profit margins. Some reports contained general considerations of the HTA organisation which related prices to manufacturers’ costs: six contained a statement on the lack of price setting transparency, one mentioned recouping R&D costs as a potential argument to justify a high price. For the case studies, 21 HTA-reports were selected. One contained a cost-based price justification provided by the manufacturer. None of the other reports contained information on manufacturer’s costs or profit margins. Six reports contained a discussion about lack of transparency. Reports from two jurisdictions contained invitations to justify high prices by demonstrating high costs. Conclusion: Despite the attention given to manufacturers’ costs in relation to price

Objectives: This study aimed to investigate whether the profit margins of pharmaceuticals would influence the outcome of reimbursement decisions within the Dutch policy context. Methods: We conducted a discrete choice experiment among 58 Dutch decision makers. In 20 choice sets, we asked respondents to indicate which of 2 pharmaceutical treatment options they would select for reimbursement. Options were described using 5 attributes (disease severity, incremental costs per quality-adjusted life-year, health gain, budget impact, and profit margin) with 3 levels each. Additionally, cognitive debriefing questions were presented, and for validation debriefing, interviews were conducted. Choice data were analyzed using mixed logit models, also to calculate marginal effects and choice probabilities. Results: Results indicated that the specified levels of profit margins significantly influenced choices made. Decision makers were less likely to reimburse a product with a higher profit margin. The relative importance of profit margins was lower than that of the included traditional health technology assessment criteria, but not negligible. When asked directly, 61% of respondents indicated that profit margin should play a role in reimbursement decision making, although concerns about feasibility and the connection to price negotiations were voiced. Conclusions: Our results suggest that if available to decision makers the profit margin of pharmaceutical products would influence reimbursement decisions within the Dutch policy context. Higher profit margins would reduce the likelihood of reimbursement. Whether adding profit margin as an additional, explicit criterion to the health technology assessment decision framework would be feasible and desirable is open to further exploration.

Introduction Most collectively funded healthcare systems set limits to their benefit package. Doing so requires judgements which may involve economic evaluations. Performing such evaluations brings methodological challenges, which may be more pronounced in non-pharmaceutical interventions. For example, for medical devices, the validity of assessment results may be limited by learning effects, incremental innovation of the devices and the context-dependency of their outcomes. Objective To review the extent to which "learning effects", "incremental innovation" (related to outcomes) and "context-dependency" are included and/or discussed in peer reviewed economic evaluations on medical devices using Transcatheter Aortic Valve Implementation (TAVI) as an example. Methods A systematic review was performed including full economic evaluations of TAVI for operable patients with aortic stenosis identified using the Pubmed database. Study characteristics, study results and text fragments concerning the aforementioned aspects were extracted. The quality of the studies was assessed using a quality checklist (CHEC-extended). Results Within 207 screened records, 15 studies were identified. Two studies referred to all three aspects, four studies referred to none. "Learning effects" were discussed in five studies, one of which described a method to cope with this challenge. “Incremental innovation” was described in seven studies. Limitations in generalizability of results related to context of care provision were discussed in seven studies. Conclusion The challenges related to economic evaluations of TAVI and their influence on the validity of reported results, are typically only partly discussed and rarely dealt within peer reviewed studies. It is important for better informed policy decisions that this improves.

Deterministic sensitivity analyses (DSA) remain important to interpret the effect of uncertainties in individual parameters on results of cost-effectiveness analyses. Classic DSA methodologies may lead to wrong conclusions due to a lack of or misleading information regarding marginal effects, non-linearity, likelihood and correlations. In addition, tornado diagrams are misleading in some situations. Recent advances in DSA methods have the potential to provide decision makers with more reliable information regarding the effects of uncertainties in individual parameters. This practical application discusses advances to classic DSA methods and their implications. Three methods are discussed: stepwise DSA, distributional DSA and probabilistic DSA. For each method, the technical specifications, options for presenting results, and its implications for decision making are discussed. Options for visualizing DSA results in incremental cost-effectiveness ratios and in incremental net benefits are presented. The use of stepwise DSA increases interpretability of marginal effects and non-linearities in the model, which is especially relevant when arbitrary ranges are implemented. Using the probability distribution of each parameter in distributional DSA provides insight on the likelihood of model outcomes while probabilistic DSA also includes the effects of correlations between parameters. Probabilistic DSA, preferably expressed in incremental net benefit, is the most appropriate method for providing insight on the effect of uncertainty in individual parameters on the estimate of cost effectiveness. However, the opportunities provided by probabilistic DSA may not always be needed for decision making. Other DSA methods, in particular distributional DSA, can sometimes be sufficient depending on model features. Decision makers must determine to which extent they will accept and implement these new and improved DSA methodologies and adjust guidelines accordingly.

Background: When national pharmacoeconomic guidelines are compared, different recommendations are identified on how to identify, measure and value lost productivity, leading to difficulties when comparing lost productivity estimates across countries. From a transferability point of view, the question arises of whether differences between countries regarding lost productivity are the result of using different calculation methods (methodological differences) or of other between-country differences. When lost productivity data differ significantly across countries, the transferability of lost productivity data across countries is hindered. Objective: The objective of this study was to investigate whether country of residence has a significant influence on the quantity of lost productivity among patients with rheumatic disorders. Confounding factors that might differ between countries were corrected for, while the methodology used to identify and measure lost productivity was kept the same. Methods: This question was investigated by means of an online questionnaire filled out by 200 respondents with a rheumatic disorder per country in four European countries, namely the Netherlands, the UK, Germany and France. In addition to those regarding lost productivity, the questionnaire contained questions about patient characteristics, disability insurance, disease characteristics, quality of life and job characteristics as these variables are expected to influence lost productivity in terms of absenteeism and presenteeism. The data were analysed by regression analyses, in which different components - being absent in last 3 months, number of days absent and presenteeism - of lost productivity were the main outcome measures and other variables, such as gender, impact of disease, shift work, job control, partial disability and overall general health, were corrected for. Results: The results showed that country sometimes has a significant influence on lost productivity and that oth

Many countries have national guidelines for performing economic evaluations in healthcare.1 These guidelines should ensure the comparability and quality of such evaluations, which should facilitate making well informed policy decisions regarding reimbursement of interventions. Given the developments in both the methodology and policy context of economic evaluation of healthcare interventions, these guidelines require periodical revision. Recently, the Dutch National Health Care Institute issued new guidance for economic evaluations in healthcare [1]. The new guidelines update and replace three separately published previous guidelines: those for pharmacoeconomic evaluation (latest version 2006), outcomes research (latest version 2008) as well as the Dutch costing manual (latest version 2010). In this editorial, we highlight the distinguishing features of the new Dutch guidelines. Moreover, we highlight which developments, in our opinion, are desirable in coming updates, but are still in development or controversial.

Olena Mandrik,1 Saskia Knies,1,2 Johan L Severens1,3 1Institute of Health Policy and Management, Erasmus University, Rotterdam, 2National Health Care Institute, Diemen, 3iMTA Institute of Medical Technology Assessment, Erasmus University, Rotterdam, the Netherlands Background: An economic value calculation was performed to estimate the lifetime net present value of in vitro fertilization (IVF) in Ukraine, Belarus, and Kazakhstan. Methods: Net lifetime tax revenues were used to represent governmental benefits accruing from a hypothetical cohort of an IVF population born in 2009 using the methodology of generational accounting. Governmental expenses related to this population included social benefits, education and health care, unemployment support, and pensions. Where available, country-specific data referencing official sources were applied. Results: The average health care cost needed to achieve one additional birth from the governmental perspective varied from $2,599 in Ukraine to $5,509 in Belarus. The net present value from the population born using IVF was positive in all countries: for Ukraine ($9,839), Belarus ($21,702), and Kazakhstan ($2,295). The break-even costs of drugs and supplies per IVF procedure is expected to be $3,870, $8,530, and $1,780, respectively. Probabilistic sensitivity analyses based on 5,000 simulations show that the average net present value per person remains positive: $1,894±$7,619, $27,925±$12,407, and $17,229±$24,637 in Ukraine, Belarus, and Kazakhstan, respectively. Conclusion: Financing IVF may represent a good investment in terms of governmental financial returns, even in lower-income countries with state-financed health care systems such as Ukraine, Belarus, and Kazakhstan. Keywords: in vitro fertilization, economic value of life, developing countries

Background: When national pharmacoeconomic guidelines are compared, different recommendations are identified on how to identify, measure and value lost productivity, leading to difficulties when comparing lost productivity estimates across countries. From a transferability point of view, the question arises of whether differences between countries regarding lost productivity are the result of using different calculation methods (methodological differences) or of other between-country differences. When lost productivity data differ significantly across countries, the transferability of lost productivity data across countries is hindered. Objective: The objective of this study was to investigate whether country of residence has a significant influence on the quantity of lost productivity among patients with rheumatic disorders. Confounding factors that might differ between countries were corrected for, while the methodology used to identify and measure lost productivity was kept the same. Methods: This question was investigated by means of an online questionnaire filled out by 200 respondents with a rheumatic disorder per country in four European countries, namely the Netherlands, the UK, Germany and France. In addition to those regarding lost productivity, the questionnaire contained questions about patient characteristics, disability insurance, disease characteristics, quality of life and job characteristics as these variables are expected to influence lost productivity in terms of absenteeism and presenteeism. The data were analysed by regression analyses, in which different components - being absent in last 3 months, number of days absent and presenteeism - of lost productivity were the main outcome measures and other variables, such as gender, impact of disease, shift work, job control, partial disability and overall general health, were corrected for. Results: The results showed that country sometimes has a significant influence on lost productivity and that oth