Search

Background: The first histology-independent marketing authorisation in Europe was granted in 2019. This was the first time that a cancer treatment was approved based on a common biomarker rather than the location in the body at which the tumour originated. This research aims to explore the implications for National Institute for Health and Care Excellence appraisals., Methods: Targeted reviews were undertaken to determine the type of evidence that is likely to be available at the point of marketing authorisation and the analyses required to support National Institute for Health and Care Excellence appraisals. Several challenges were identified concerning the design and conduct of trials for histology-independent products, the greater levels of heterogeneity within the licensed population and the use of surrogate end points. We identified approaches to address these challenges by reviewing key statistical literature that focuses on the design and analysis of histology-independent trials and by undertaking a systematic review to evaluate the use of response end points as surrogate outcomes for survival end points. We developed a decision framework to help to inform approval and research policies for histology-independent products. The framework explored the uncertainties and risks associated with different approval policies, including the role of further data collection, pricing schemes and stratified decision-making., Results: We found that the potential for heterogeneity in treatment effects, across tumour types or other characteristics, is likely to be a central issue for National Institute for Health and Care Excellence appraisals. Bayesian hierarchical methods may serve as a useful vehicle to assess the level of heterogeneity across tumours and to estimate the pooled treatment effects for each tumour, which can inform whether or not the assumption of homogeneity is reasonable. Our review suggests that response end points may not be reliable surrogates for survival end points. However, a surrogate-based modelling approach, which captures all relevant uncertainty, may be preferable to the use of immature survival data. Several additional sources of heterogeneity were identified as presenting potential challenges to National Institute for Health and Care Excellence appraisal, including the cost of testing, baseline risk, quality of life and routine management costs. We concluded that a range of alternative approaches will be required to address different sources of heterogeneity to support National Institute for Health and Care Excellence appraisals. An exemplar case study was developed to illustrate the nature of the assessments that may be required., Conclusions: Adequately designed and analysed basket studies that assess the homogeneity of outcomes and allow borrowing of information across baskets, where appropriate, are recommended. Where there is evidence of heterogeneity in treatment effects and estimates of cost-effectiveness, consideration should be given to optimised recommendations. Routine presentation of the scale of the consequences of heterogeneity and decision uncertainty may provide an important additional approach to the assessments specified in the current National Institute for Health and Care Excellence methods guide., Further Research: Further exploration of Bayesian hierarchical methods could help to inform decision-makers on whether or not there is sufficient evidence of homogeneity to support pooled analyses. Further research is also required to determine the appropriate basis for apportioning genomic testing costs where there are multiple targets and to address the challenges of uncontrolled Phase II studies, including the role and use of surrogate end points., Funding: This project was funded by the National Institute for Health Research (NIHR) Evidence Synthesis programme and will be published in full in Health Technology Assessment ; Vol. 25, No. 76. See the NIHR Journals Library website for further project information.

Background: Breast cancer and its treatment can have an impact on health-related quality of life and survival. Tumour profiling tests aim to identify whether or not women need chemotherapy owing to their risk of relapse., Objectives: To conduct a systematic review of the effectiveness and cost-effectiveness of the tumour profiling tests onco type DX ® (Genomic Health, Inc., Redwood City, CA, USA), MammaPrint ® (Agendia, Inc., Amsterdam, the Netherlands), Prosigna ® (NanoString Technologies, Inc., Seattle, WA, USA), EndoPredict ® (Myriad Genetics Ltd, London, UK) and immunohistochemistry 4 (IHC4). To develop a health economic model to assess the cost-effectiveness of these tests compared with clinical tools to guide the use of adjuvant chemotherapy in early-stage breast cancer from the perspective of the NHS and Personal Social Services., Design: A systematic review and health economic analysis were conducted., Review Methods: The systematic review was partially an update of a 2013 review. Nine databases were searched in February 2017. The review included studies assessing clinical effectiveness in people with oestrogen receptor-positive, human epidermal growth factor receptor 2-negative, stage I or II cancer with zero to three positive lymph nodes. The economic analysis included a review of existing analyses and the development of a de novo model., Results: A total of 153 studies were identified. Only one completed randomised controlled trial (RCT) using a tumour profiling test in clinical practice was identified: Microarray In Node-negative Disease may Avoid ChemoTherapy (MINDACT) for MammaPrint. Other studies suggest that all the tests can provide information on the risk of relapse; however, results were more varied in lymph node-positive (LN+) patients than in lymph node-negative (LN0) patients. There is limited and varying evidence that onco type DX and MammaPrint can predict benefit from chemotherapy. The net change in the percentage of patients with a chemotherapy recommendation or decision pre/post test ranged from an increase of 1% to a decrease of 23% among UK studies and a decrease of 0% to 64% across European studies. The health economic analysis suggests that the incremental cost-effectiveness ratios for the tests versus current practice are broadly favourable for the following scenarios: (1) onco type DX, for the LN0 subgroup with a Nottingham Prognostic Index (NPI) of > 3.4 and the one to three positive lymph nodes (LN1-3) subgroup (if a predictive benefit is assumed); (2) IHC4 plus clinical factors (IHC4+C), for all patient subgroups; (3) Prosigna, for the LN0 subgroup with a NPI of > 3.4 and the LN1-3 subgroup; (4) EndoPredict Clinical, for the LN1-3 subgroup only; and (5) MammaPrint, for no subgroups., Limitations: There was only one completed RCT using a tumour profiling test in clinical practice. Except for onco type DX in the LN0 group with a NPI score of > 3.4 (clinical intermediate risk), evidence surrounding pre- and post-test chemotherapy probabilities is subject to considerable uncertainty. There is uncertainty regarding whether or not onco type DX and MammaPrint are predictive of chemotherapy benefit. The MammaPrint analysis uses a different data source to the other four tests. The Translational substudy of the Arimidex, Tamoxifen, Alone or in Combination (TransATAC) study (used in the economic modelling) has a number of limitations., Conclusions: The review suggests that all the tests can provide prognostic information on the risk of relapse; results were more varied in LN+ patients than in LN0 patients. There is limited and varying evidence that onco type DX and MammaPrint are predictive of chemotherapy benefit. Health economic analyses indicate that some tests may have a favourable cost-effectiveness profile for certain patient subgroups; all estimates are subject to uncertainty. More evidence is needed on the prediction of chemotherapy benefit, long-term impacts and changes in UK pre-/post-chemotherapy decisions., Study Registration: This study is registered as PROSPERO CRD42017059561., Funding: The National Institute for Health Research Health Technology Assessment programme., Competing Interests: Robert C Stein is the chief investigator of the OPTIMA (Optimal Personalised Treatment of breast cancer using Multi-parameter Analysis) trial and has published results from the OPTIMA Prelim study.

Background: Medullary thyroid cancer (MTC) is a rare form of cancer that affects patients' health-related quality of life (HRQoL) and survival. Cabozantinib (Cometriq ® ; Ipsen, Paris, France) and vandetanib (Caprelsa ® ; Sanofi Genzyme, Cambridge, MA, USA) are currently the treatment modality of choice for treating unresectable progressive and symptomatic MTC., Objectives: (1) To evaluate the clinical effectiveness and safety of cabozantinib and vandetanib. (2) To estimate the incremental cost-effectiveness of cabozantinib and vandetanib versus each other and best supportive care. (3) To identify key areas for primary research. (4) To estimate the overall cost of these treatments in England., Data Sources: Peer-reviewed publications (searched from inception to November 2016), European Public Assessment Reports and manufacturers' submissions., Review Methods: A systematic review [including a network meta-analysis (NMA)] was conducted to evaluate the clinical effectiveness and safety of cabozantinib and vandetanib. The economic analysis included a review of existing analyses and the development of a de novo model., Results: The systematic review identified two placebo-controlled trials. The Efficacy of XL184 (Cabozantinib) in Advanced Medullary Thyroid Cancer (EXAM) trial evaluated the efficacy and safety of cabozantinib in patients with unresectable locally advanced, metastatic and progressive MTC. The ZETA trial evaluated the efficacy and safety of vandetanib in patients with unresectable locally advanced or metastatic MTC. Both drugs significantly improved progression-free survival (PFS) more than the placebo ( p  < 0.001). The NMA suggested that, within the symptomatic and progressive MTC population, the effects on PFS were similar (vandetanib vs. cabozantinib: hazard ratio 1.14, 95% credible interval 0.41 to 3.09). Neither trial demonstrated a significant overall survival benefit for cabozantinib or vandetanib versus placebo, although data from ZETA were subject to potential confounding. Both cabozantinib and vandetanib demonstrated significantly better objective response rates and calcitonin (CTN) and carcinoembryonic antigen (CEA) response rates than placebo. Both cabozantinib and vandetanib produced frequent adverse events, often leading to dose interruption or reduction. The assessment group model indicates that, within the EU-label population (symptomatic and progressive MTC), the incremental cost-effectiveness ratios (ICERs) for cabozantinib and vandetanib are > £138,000 per quality-adjusted life-year (QALY) gained. Within the restricted EU-label population (symptomatic and progressive MTC with CEA/CTN doubling times of ≤ 24 months), the ICER for vandetanib is expected to be > £66,000 per QALY gained. The maximum annual budget impact within the symptomatic and progressive population is estimated to be ≈£2.35M for cabozantinib and ≈£5.53M for vandetanib. The costs of vandetanib in the restricted EU-label population are expected to be lower., Limitations: The intention-to-treat populations of the EXAM and ZETA trials are notably different. The analyses of ZETA subgroups may be subject to confounding as a result of differences in baseline characteristics and open-label vandetanib use. Attempts to statistically adjust for treatment switching were unsuccessful. No HRQoL evidence was identified for the MTC population., Conclusions: The identified trials suggest that cabozantinib and vandetanib improve PFS more than the placebo; however, significant OS benefits were not demonstrated. The economic analyses indicate that within the EU-label population, the ICERs for cabozantinib and vandetanib are > £138,000 per QALY gained. Within the restricted EU-label population, the ICER for vandetanib is expected to be > £66,000 per QALY gained., Future Research Priorities: (1) Primary research assessing the long-term effectiveness of cabozantinib and vandetanib within relevant subgroups. (2) Reanalyses of the ZETA trial to investigate the impact of adjusting for open-label vandetanib use using appropriate statistical methods. (3) Studies assessing the impact of MTC on HRQoL., Study Registration: This study is registered as PROSPERO CRD42016050403., Funding: The National Institute for Health Research Health Technology Assessment programme., Competing Interests: Jonathan Wadsley has received personal fees from Sanofi Genzyme (Cambridge, MA, USA), Ipsen (Paris, France), Bayer AG (Leverkusen, Germany), Baxalta (Shire Pharmaceuticals, London, UK), Eisai Co. Ltd (Tokyo, Japan) and Eli Lilly and Company (Basingstoke, UK), grants and personal fees from AstraZeneca plc (Cambridge, UK), non-financial support from Swedish Orphan Biovitrum AB (Stockholm, Sweden), and personal fees and non-financial support from Novartis International AG (Basel, Switzerland) and Celgene Corporation (Summit, NJ, USA) outside the submitted work.

Background: The UK guideline recommends 3-yearly surveillance for patients with intermediate-risk (IR) adenomas. No study has examined whether or not this group has heterogeneity in surveillance needs., Objectives: To examine the effect of surveillance on colorectal cancer (CRC) incidence; assess heterogeneity in risk; and identify the optimum frequency of surveillance, the psychological impact of surveillance, and the cost-effectiveness of alternative follow-up strategies., Design: Retrospective multicentre cohort study., Setting: Routine endoscopy and pathology data from 17 UK hospitals ( n  = 11,944), and a screening data set comprising three pooled cohorts ( n  = 2352), followed up using cancer registries., Subjects: Patients with IR adenoma(s) (three or four small adenomas or one or two large adenomas)., Primary Outcomes: Advanced adenoma (AA) and CRC detected at follow-up visits, and CRC incidence after baseline and first follow-up., Methods: The effects of surveillance on long-term CRC incidence and of interval length on findings at follow-up were examined using proportional hazards and logistic regression, adjusting for patient, procedural and polyp characteristics. Lower-intermediate-risk (LIR) subgroups and higher-intermediate-risk (HIR) subgroups were defined, based on predictors of CRC risk. A model-based cost-utility analysis compared 13 surveillance strategies. Between-group analyses of variance were used to test for differences in bowel cancer worry between screening outcome groups ( n  = 35,700). A limitation of using routine hospital data is the potential for missed examinations and underestimation of the effect of interval and surveillance., Results: In the hospital data set, 168 CRCs occurred during 81,442 person-years (pys) of follow-up [206 per 100,000 pys, 95% confidence interval (CI) 177 to 240 pys]. One surveillance significantly lowered CRC incidence, both overall [hazard ratio (HR) 0.51, 95% CI 0.34 to 0.77] and in the HIR subgroup ( n  = 9265; HR 0.50, 95% CI 0.34 to 0.76). In the LIR subgroup ( n  = 2679) the benefit of surveillance was less clear (HR 0.62, 95% CI 0.16 to 2.43). Additional surveillance lowered CRC risk in the HIR subgroup by a further 15% (HR 0.36, 95% CI 0.20 to 0.62). The odds of detecting AA and CRC at first follow-up (FUV1) increased by 18% [odds ratio (OR) 1.18, 95% CI 1.12 to 1.24] and 32% (OR 1.32, 95% CI 1.20 to 1.46) per year increase in interval, respectively, and the odds of advanced neoplasia at second follow-up increased by 22% (OR 1.22, 95% CI 1.09 to 1.36), after adjustment. Detection rates of AA and CRC remained below 10% and 1%, respectively, with intervals to 3 years. In the screening data set, 32 CRCs occurred during 25,745 pys of follow-up (124 per 100,000 pys, 95% CI 88 to 176 pys). One follow-up conferred a significant 73% reduction in CRC incidence (HR 0.27, 95% CI 0.10 to 0.71). Owing to the small number of end points in this data set, no other outcome was significant. Although post-screening bowel cancer worry was higher in people who were offered surveillance, worry was due to polyp detection rather than surveillance. The economic evaluation, using data from the hospital data set, suggested that 3-yearly colonoscopic surveillance without an age cut-off would produce the greatest health gain., Conclusions: A single surveillance benefited all IR patients by lowering their CRC risk. We identified a higher-risk subgroup that benefited from further surveillance, and a lower-risk subgroup that may require only one follow-up. A surveillance interval of 3 years seems suitable for most IR patients. These findings should be validated in other studies to confirm whether or not one surveillance visit provides adequate protection for the lower-risk subgroup of intermediate-risk patients., Study Registration: Current Controlled Trials ISRCTN15213649., Funding: The National Institute for Health Research Health Technology Assessment programme.

Background: Ulcerative colitis (UC) is the most common form of inflammatory bowel disease in the UK. UC can have a considerable impact on patients' quality of life. The burden for the NHS is substantial., Objectives: To evaluate the clinical effectiveness and safety of interventions, to evaluate the incremental cost-effectiveness of all interventions and comparators (including medical and surgical options), to estimate the expected net budget impact of each intervention, and to identify key research priorities., Data Sources: Peer-reviewed publications, European Public Assessment Reports and manufacturers' submissions. The following databases were searched from inception to December 2013 for clinical effectiveness searches and from inception to January 2014 for cost-effectiveness searches for published and unpublished research evidence: MEDLINE, EMBASE, Cumulative Index to Nursing and Allied Health Literature, The Cochrane Library including the Cochrane Systematic Reviews Database, Cochrane Controlled Trials Register, Database of Abstracts of Reviews of Effects, the Health Technology Assessment database and NHS Economic Evaluation Database; ISI Web of Science, including Science Citation Index, and the Conference Proceedings Citation Index-Science and Bioscience Information Service Previews. The US Food and Drug Administration website and the European Medicines Agency website were also searched, as were research registers, conference proceedings and key journals., Review Methods: A systematic review [including network meta-analysis (NMA)] was conducted to evaluate the clinical effectiveness and safety of named interventions. The health economic analysis included a review of published economic evaluations and the development of a de novo model., Results: Ten randomised controlled trials were included in the systematic review. The trials suggest that adult patients receiving infliximab (IFX) [Remicade(®), Merck Sharp & Dohme Ltd (MSD)], adalimumab (ADA) (Humira(®), AbbVie) or golimumab (GOL) (Simponi(®), MSD) were more likely to achieve clinical response and remission than those receiving placebo (PBO). Hospitalisation data were limited, but suggested more favourable outcomes for ADA- and IFX-treated patients. Data on the use of surgical intervention were sparse, with a potential benefit for intervention-treated patients. Data were available from one trial to support the use of IFX in paediatric patients. Safety issues identified included serious infections, malignancies and administration site reactions. Based on the NMA, in the induction phase, all biological treatments were associated with statistically significant beneficial effects relative to PBO, with the greatest effect associated with IFX. For patients in response following induction, all treatments except ADA and GOL 100 mg at 32-52 weeks were associated with beneficial effects when compared with PBO, although these were not significant. The greatest effects at 8-32 and 32-52 weeks were associated with 100 mg of GOL and 5 mg/kg of IFX, respectively. For patients in remission following induction, all treatments except ADA at 8-32 weeks and GOL 50 mg at 32-52 weeks were associated with beneficial effects when compared with PBO, although only the effect of ADA at 32-52 weeks was significant. The greatest effects were associated with GOL (at 8-32 weeks) and ADA (at 32-52 weeks). The economic analysis suggests that colectomy is expected to dominate drug therapies, but for some patients, colectomy may not be considered acceptable. In circumstances in which only drug options are considered, IFX and GOL are expected to be ruled out because of dominance, while the incremental cost-effectiveness ratio for ADA versus conventional treatment is approximately £50,300 per QALY gained., Limitations: The health economic model is subject to several limitations: uncertainty associated with extrapolating trial data over a lifetime horizon, the model does not consider explicit sequential pathways of non-biological treatments, and evidence relating to complications of colectomy was identified through consideration of approaches used within previous models rather than a full systematic review., Conclusions: Adult patients receiving IFX, ADA or GOL were more likely to achieve clinical response and remission than those receiving PBO. Further data are required to conclusively demonstrate the effect of interventions on hospitalisation and surgical outcomes. The economic analysis indicates that colectomy is expected to dominate medical treatments for moderate to severe UC., Study Registration: This study is registered as PROSPERO CRD42013006883., Funding: The National Institute for Health Research Health Technology Assessment programme.

Background: As part of the National Institute for Health and Care Excellence (NICE) single technology appraisal (STA) process, independent Evidence Review Groups (ERGs) critically appraise the company submission. During the critical appraisal process the ERG may undertake analyses to explore uncertainties around the company's model and their implications for decision-making. The ERG reports are a central component of the evidence considered by the NICE Technology Appraisal Committees (ACs) in their deliberations., Objective: The aim of this research was to develop an understanding of the number and type of exploratory analyses undertaken by the ERGs within the STA process and to understand how these analyses are used by the NICE ACs in their decision-making., Methods: The 100 most recently completed STAs with published guidance were selected for inclusion in the analysis. The documents considered were ERG reports, clarification letters, the first appraisal consultation document and the final appraisal determination. Over 400 documents were assessed in this study. The categories of types of exploratory analyses included fixing errors, fixing violations, addressing matters of judgement and the ERG-preferred base case. A content analysis of documents (documentary analysis) was undertaken to identify and extract relevant data, and narrative synthesis was then used to rationalise and present these data., Results: The level and type of detail in ERG reports and clarification letters varied considerably. The vast majority (93%) of ERG reports reported one or more exploratory analyses. The most frequently reported type of analysis in these 93 ERG reports related to the category 'matters of judgement', which was reported in 83 (89%) reports. The category 'ERG base-case/preferred analysis' was reported in 45 (48%) reports, the category 'fixing errors' was reported in 33 (35%) reports and the category 'fixing violations' was reported in 17 (18%) reports. The exploratory analyses performed were the result of issues raised by an ERG in its critique of the submitted economic evidence. These analyses had more influence on recommendations earlier in the STA process than later on in the process., Limitations: The descriptions of analyses undertaken were often highly specific to a particular STA and could be inconsistent across ERG reports and thus difficult to interpret., Conclusions: Evidence Review Groups frequently conduct exploratory analyses to test or improve the economic evaluations submitted by companies as part of the STA process. ERG exploratory analyses often have an influence on the recommendations produced by the ACs., Future Work: More in-depth analysis is needed to understand how ERGs make decisions regarding which exploratory analyses should be undertaken. More research is also needed to fully understand which types of exploratory analyses are most useful to ACs in their decision-making., Funding: The National Institute for Health Research Health Technology Assessment programme.

Background: High fractions of exhaled nitric oxide (FeNO) in the breath of patients with symptoms of asthma are correlated with high levels of eosinophils and indicate that a patient is likely to respond to inhaled corticosteroids. This may have a role in the diagnosis and management of asthma., Objective: To assess the diagnostic accuracy, clinical effectiveness and cost-effectiveness of the hand-held electrochemical devices NIOX MINO(®) (Aerocrine, Solna, Sweden), NIOX VERO(®) (Aerocrine) and NObreath(®) (Bedfont Scientific, Maidstone, UK) for the diagnosis and management of asthma., Data Sources: Systematic searches were carried out between March 2013 and April 2013 from database inception. Databases searched included MEDLINE, EMBASE, the Cochrane Database of Systematic Reviews, the Database of Abstracts of Reviews of Effects, Science Citation Index Expanded and Conference Proceedings Citation Index - Science. Trial registers such as ClinicalTrials.gov and the metaRegister of Controlled Trials were also searched in March 2013. All searches were updated in September 2013., Review Methods: A rapid review was conducted to assess the equivalence of hand-held and chemiluminescent FeNO monitors. Systematic reviews of diagnostic accuracy and management efficacy were conducted. A systematic review of economic analyses was also conducted and two de novo health economic models were developed. All three reviews were undertaken according to robust high-quality methodology., Results: The rapid review (27 studies) found varying levels of agreement between monitors (Bland-Altman 95% limits of agreement up to ±10 parts per billion), with better agreement at lower FeNO values. Correlation was good (generally r > 0.9). The diagnostic accuracy review identified 22 studies in adults (all ages) and four in children. No studies used NObreath or NIOX VERO and seven used NIOX MINO. Estimates of diagnostic accuracy varied widely. FeNO used in combination with another test altered diagnostic accuracy only slightly. High levels of heterogeneity precluded meta-analysis. Limited observations included that FeNO may be more reliable and useful as a rule-in than as a rule-out test; lower cut-off values in children and in smokers may be appropriate; and FeNO may be less reliable in the elderly. The management review identified five randomised controlled trials in adults, one in pregnant asthmatics and seven in children. Despite clinical heterogeneity, exacerbation rates were lower in all studies but not generally statistically significantly so. Effects on inhaled corticosteroid (ICS) use were inconsistent, possibly because of differences in management protocols, differential effectiveness in adults and children and differences in population severity. One UK diagnostic model and one management model were identified. Aerocrine also submitted diagnostic and management models. All had significant limitations including short time horizons and the selective use of efficacy evidence. The de novo diagnostic model suggested that the expected difference in quality-adjusted life-year (QALY) gains between diagnostic options is likely to be very small. Airway hyper-responsiveness by methacholine challenge test is expected to produce the greatest QALY gain but with an expected incremental cost-effectiveness ratio (ICER) compared with FeNO (NObreath) in combination with bronchodilator reversibility of £1.125M per QALY gained. All remaining options are expected to be dominated. The de novo management model indicates that the ICER of guidelines plus FeNO monitoring using NObreath compared with guidelines alone in children is expected to be approximately £45,200 per QALY gained. Within the adult subgroup, FeNO monitoring using NObreath compared with guidelines alone is expected to have an ICER of approximately £2100 per QALY gained. The results are particularly sensitive to assumptions regarding changes in ICS use over time, the number of nurse visits for FeNO monitoring and duration of effect., Conclusions: Limitations of the evidence base impose considerable uncertainty on all analyses. Equivalence of devices was assumed but not assured. Evidence for diagnosis is difficult to interpret in the context of inserting FeNO monitoring into a diagnostic pathway. Evidence for management is also inconclusive, but largely consistent with FeNO monitoring resulting in fewer exacerbations, with a small or zero reduction in ICS use in adults and a possible increased ICS use in children or patients with more severe asthma. It is unclear which specific management protocol is likely to be most effective. The economic analysis indicates that FeNO monitoring could have value in diagnostic and management settings. The diagnostic model indicates that FeNO monitoring plus bronchodilator reversibility dominates many other diagnostic tests. FeNO-guided management has the potential to be cost-effective, although this is largely dependent on the duration of effect. The conclusions drawn from both models require strong technical value judgements with respect to several aspects of the decision problem in which little or no empirical evidence exists. There are many potential directions for further work, including investigations into which management protocol is best and long-term follow-up in both diagnosis and management studies., Study Registration: This study is registered as PROSPERO CRD42013004149., Funding: The National Institute for Health Research Health Technology Assessment programme.

Background: It is estimated that only 39% of men and 29% of women in England achieve the levels of physical activity that are recommended to protect health and prevent disease. One approach to addressing this problem has been the development of exercise referral schemes (ERSs), in which health professionals refer patients to external exercise providers. These schemes have been widely rolled out across the UK despite concerns that they may not produce sustained changes in levels of physical activity and, therefore, may not be cost-effective interventions. The evidence to determine clinical effectiveness and cost-effectiveness was evaluated in 2009. This review seeks to update this earlier work by incorporating new evidence and re-examining the cost-effectiveness., Objectives: To assess the clinical effectiveness and cost-effectiveness of ERSs compared with usual care., Design: Exhaustive searches of relevant electronic databases and journals were undertaken to identify new studies evaluating ERSs using a randomised controlled trial (RCT) design. RCTs that incorporated a qualitative evaluation of the intervention were identified in order to explore the barriers and facilitators to the uptake of and adherence to ERSs. Data were extracted using a previously designed tool and study quality assessed for potential bias. Where data could be pooled, meta-analyses were carried out. Qualitative analysis was also undertaken using a thematic approach. The cost-effectiveness was evaluated using a Markov structure which estimated the likelihood of becoming physically active and the subsequent risk reduction on coronary heart disease (CHD), stroke and type 2 diabetes mellitus. The model adopts a lifetime horizon, and a NHS and Personal Social Services perspective was taken with discounting at 1.5% for both costs and benefits., Results: The search identified one new RCT and one new qualitative study. The new data were pooled with existing data from the 2011 review by Pavey et al. [Pavey TG, Anokye N, Taylor AH, Trueman P, Moxham T, Fox KR, et al. The clinical effectiveness and cost-effectiveness of exercise referral schemes: a systematic review and economic evaluation. Health Technol Assess 2011;15(44)] to give a total of eight studies with 5190 participants. The proportion of individuals achieving 90-150 minutes of at least moderate-intensity activity per week at 6-12 months' follow-up was greater for ERSs than usual care (relative risk 1.12; 95% confidence interval 1.04 to 1.20). Older patients and those referred for CHD risk factors appeared to be more likely than others to increase their levels of physical activity. Qualitative evidence suggests that interventions enabling the development of social support networks are beneficial in promoting uptake and adherence. Exercise referral gained 0.003 quality-adjusted life-years (QALYs) at an additional cost of £225 per person. The estimated mean incremental cost-effectiveness ratio (ICER) in the probabilistic sensitivity analysis was £76,276. In the univariate sensitivity analysis the results were very sensitive (ICERs ranged from < £30,000 to > £100,000) to changes in the effect of ERSs on physical activity uptake and the duration of the protective effects and the direct health-related quality-of-life gains attributable to physical activity., Conclusions: Exercise referral schemes result in a small improvement in the number of people who increase their levels of physical activity. The cost-effectiveness analysis indicates that the ICER for ERSs compared with usual care is around £76,000 per QALY, although the cost-effectiveness of ERSs is subject to considerable uncertainty., Study Registration: This study is registered as PROSPERO CRD42013005200., Funding: National Institute for Health Research Health Technology Assessment programme.

Background: National Institute for Health and Care Excellence (NICE) clinical guidelines (CGs) make recommendations across large, complex care pathways for broad groups of patients. They rely on cost-effectiveness evidence from the literature and from new analyses for selected high-priority topics. An alternative approach would be to build a model of the full care pathway and to use this as a platform to evaluate the cost-effectiveness of multiple topics across the guideline recommendations., Objectives: In this project we aimed to test the feasibility of building full guideline models for NICE guidelines and to assess if, and how, such models can be used as a basis for cost-effectiveness analysis (CEA)., Data Sources: A 'best evidence' approach was used to inform the model parameters. Data were drawn from the guideline documentation, advice from clinical experts and rapid literature reviews on selected topics. Where possible we relied on good-quality, recent UK systematic reviews and meta-analyses., Review Methods: Two published NICE guidelines were used as case studies: prostate cancer and atrial fibrillation (AF). Discrete event simulation (DES) was used to model the recommended care pathways and to estimate consequent costs and outcomes. For each guideline, researchers not involved in model development collated a shortlist of topics suggested for updating. The modelling teams then attempted to evaluate options related to these topics. Cost-effectiveness results were compared with opinions about the importance of the topics elicited in a survey of stakeholders., Results: The modelling teams developed simulations of the guideline pathways and disease processes. Development took longer and required more analytical time than anticipated. Estimates of cost-effectiveness were produced for six of the nine prostate cancer topics considered, and for five of eight AF topics. The other topics were not evaluated owing to lack of data or time constraints. The modelled results suggested 'economic priorities' for an update that differed from priorities expressed in the stakeholder survey., Limitations: We did not conduct systematic reviews to inform the model parameters, and so the results might not reflect all current evidence. Data limitations and time constraints restricted the number of analyses that we could conduct. We were also unable to obtain feedback from guideline stakeholders about the usefulness of the models within project time scales., Conclusions: Discrete event simulation can be used to model full guideline pathways for CEA, although this requires a substantial investment of clinical and analytic time and expertise. For some topics lack of data may limit the potential for modelling. There are also uncertainties over the accessibility and adaptability of full guideline models. However, full guideline modelling offers the potential to strengthen and extend the analytical basis of NICE's CGs. Further work is needed to extend the analysis of our case study models to estimate population-level budget and health impacts. The practical usefulness of our models to guideline developers and users should also be investigated, as should the feasibility and usefulness of whole guideline modelling alongside development of a new CG., Funding: This project was funded by the Medical Research Council and the National Institute for Health Research through the Methodology Research Programme [grant number G0901504] and will be published in full in Health Technology Assessment; Vol. 17, No. 58. See the NIHR Journals Library website for further project information.

Background: Cystic fibrosis (CF) is an inherited condition characterised by the abnormal transport of chloride ions across transporting epithelia. This leads to the production of thick sticky mucus in the lungs, pancreas, liver, intestine and reproductive tract, and an increase in the salt content in sweat. Among other problems, people with CF experience recurrent respiratory infections and have difficulties digesting food. CF affects over 9000 individuals in the UK. CF shortens life expectancy and adversely affects quality of life. In 2010, CF was recorded as the cause of 103 deaths in England and Wales., Objective: To evaluate the clinical effectiveness and cost-effectiveness of colistimethate sodium dry powder for inhalation (DPI) (Colobreathe(®), Forest Laboratories) and tobramycin DPI (TOBI Podhaler(®), Novartis Pharmaceuticals) for the treatment of Pseudomonas aeruginosa lung infection in CF., Data Sources: Electronic databases were searched in February and March 2011 [MEDLINE, MEDLINE In-Process & Other Non-Indexed citations, EMBASE, The Cochrane Library databases, Cumulative Index to Nursing and Allied Health Literature (CINAHL), Web of Science, Conference Proceedings Citation Index (CPCI) and Bioscience Information Service (BIOSIS) Previews]. Relevant databases were searched for ongoing and unpublished studies, and bibliographies of relevant systematic reviews and the manufacturers' submissions were also hand-searched., Review Methods: A systematic review of the clinical effectiveness and cost-effectiveness of colistimethate sodium DPI and tobramycin DPI for the treatment of chronic P. aeruginosa lung infection in CF was conducted. Existing economic evidence within the literature was reviewed and a de novo health economic model was also developed., Results: Three randomised controlled trials (RCTs) were included in the clinical effectiveness review. Both colistimethate sodium DPI and tobramycin DPI were reported to be non-inferior to nebulised tobramycin for the outcome forced expiratory volume in first second percentage predicted (FEV1%). It was not possible to draw any firm conclusions as to the relative efficacy of colistimethate sodium DPI compared with tobramycin DPI. The economic analysis suggests that colistimethate sodium DPI produces fewer quality-adjusted life-years (QALYs) than nebulised tobramycin. Given the incremental discounted lifetime cost of tobramycin DPI compared with nebulised tobramycin, it highly unlikely that tobramycin DPI has an incremental cost-effectiveness ratio that is better than £30,000 per QALY gained., Limitation: The uncertainty surrounding the short-term evidence base inevitably results in uncertainty surrounding the long-term clinical effectiveness and cost-effectiveness of colistimethate sodium DPI., Conclusions: Both DPI formulations have been shown to be non-inferior to nebulised tobramycin as measured by FEV1%. The results of these trials should be interpreted with caution owing to the means by which the results were analysed, the length of follow-up, and concerns about the ability of FEV1% to accurately represent changes in lung health. Although the increase in QALYs is expected to be lower with colistimethate sodium DPI than with nebulised tobramycin, a price for this intervention had not been agreed at the time of the assessment. Depending on the price of colistimethate sodium DPI, this results either in a situation whereby colistimethate sodium DPI is dominated by nebulised tobramycin or in one whereby the incremental cost-effectiveness of nebulised tobramycin compared with colistimethate sodium DPI is in the range of £24,000-277,000 per QALY gained. The economic analysis also suggests that, given its price, it is unlikely that tobramycin DPI has a cost-effectiveness ratio of < £30,000 per QALY gained when compared with nebulised tobramycin. A RCT to assess the longer-term (≥ 12 months) efficacy of colistimethate sodium DPI and tobramycin DPI in comparison with nebulised treatments would be beneficial. Such a study should include the direct assessment of HRQoL using a relevant preference-based instrument. Future studies should ensure that the European Medicines Agency guidelines are adhered to. In addition, high-quality research concerning the relationship between forced expiratory volume in first second % (FEV1%) predicted or other measures of lung function and survival/health-related quality of life (HRQoL) would be useful., Study Registration: PROSPERO CRD42011001350., Funding: The National Institute for Health Research Health Technology Assessment programme.

Background: Colorectal cancer is the third most commonly diagnosed cancer in the UK after breast and lung cancer. People with metastatic disease who are sufficiently fit are usually treated with active chemotherapy as first- or second-line therapy. Recently, targeted agents have become available including anti-epidermal growth factor receptor (EGFR) agents, for example cetuximab and panitumumab, and anti-vascular endothelial growth factor (VEGF) receptor agents, for example bevacizumab., Objective: To investigate the clinical effectiveness and cost-effectiveness of panitumumab monotherapy and cetuximab (mono- or combination chemotherapy) for Kirsten rat sarcoma (KRAS) wild-type (WT) patients, and bevacizumab in combination with non-oxaliplatin chemotherapy, for the treatment of metastatic colorectal cancer after first-line chemotherapy., Data Sources: The assessment comprises a systematic review of clinical effectiveness and cost-effectiveness studies, a review and critique of manufacturer submissions and a de novo cohort-based economic analysis. For the assessment of effectiveness, a literature search was conducted in a range of electronic databases, including MEDLINE, EMBASE and The Cochrane Library, from 2005 to November 2010., Review Methods: Studies were included if they were randomised controlled trials (RCTs) or systematic reviews of RCTs of cetuximab, bevacizumab or panitumumab in participants with EGFR-expressing metastatic colorectal cancer with KRAS WT status that has progressed after first-line chemotherapy (for cetuximab and panitumumab) or participants with metastatic colorectal cancer that has progressed after first-line chemotherapy (bevacizumab). All steps in the review were performed by one reviewer and checked independently by a second. Synthesis was mainly narrative. An economic model was developed focusing on third-line and subsequent lines of treatment. Costs and benefits were discounted at 3.5% per annum. Probabilistic and univariate deterministic sensitivity analyses were performed., Results: The searches identified 7745 titles and abstracts. Two clinical trials (reported in 12 papers) were included. No data were available for bevacizumab in combination with non-oxaliplatin-based chemotherapy in previously treated patients. Neither of the included studies had KRAS status performed prospectively, but the studies did report retrospective analyses of the results for the KRAS WT subgroups. Third-line treatment with cetuximab plus best supportive care or panitumumab plus best supportive care appears to have statistically significant advantages over treatment with best supportive care alone in patients with KRAS WT status. For the economic evaluation, five studies met the inclusion criteria. The base-case incremental cost-effectiveness ratio (ICER) for KRAS WT patients for cetuximab compared with best supportive care is £98,000 per quality-adjusted life-year (QALY), for panitumumab compared with best supportive care is £150,000 per QALY and for cetuximab plus irinotecan compared with best supportive care is £88,000 per QALY. All ICERs are sensitive to treatment duration., Limitations: In the specific populations of interest, there is a lack of evidence on bevacizumab, cetuximab and cetuximab plus irinotecan used second line and on bevacizumab and cetuximab plus irinotecan used third line. For cetuximab plus irinotecan treatment for KRAS WT people, there is no direct evidence on progression-free survival, overall survival and duration of treatment., Conclusions: Although cetuximab and panitumumab appear to be clinically beneficial for KRAS WT patients compared with best supportive care, they are likely to represent poor value for money when judged by cost-effectiveness criteria currently used in the UK. It would be useful to conduct a RCT for patients with KRAS WT status receiving cetuximab plus irinotecan., Funding: The National Institute for Health Research Health Technology Assessment programme.

Background: In older age, reduction in physical function can lead to loss of independence, the need for hospital and long-term nursing or residential home care, and premature death. Home-visiting programmes for older people, carried out by nurses and other health-care professionals (e.g. occupational therapists and physiotherapists), aim to positively affect health and functional status, and may promote independent functioning of older people., Objective: The main research question addressed by this assessment is 'What is the clinical effectiveness and cost-effectiveness of home-based, nurse-led health promotion intervention for older people in the UK?', Data Sources: A comprehensive literature search was undertaken across 12 different databases and research registries from the year 2001 onwards (including MEDLINE, MEDLINE in Process & Other Non-Indexed Citations, EMBASE, Science Citation Index Expanded, Cochrane Database of Systematic Reviews, Cochrane Central Register of Controlled Trials, NHS Health Economic Evaluation Database, Health Technology Assessment Database, Database of Abstracts of Reviews of Effects, Cumulative Index to Nursing and Allied Health Literature). Published systematic reviews were also hand searched to identify other trials previously published., Review Methods: Potentially relevant studies were sifted by one reviewer, and inclusion decisions were agreed among the broader research team. The methodological quality of included studies was assessed using the Cochrane Risk of Bias tool. The results of included studies were synthesised using narrative and statistical methods. A separate systematic search was undertaken to identify existing health economic analyses of home-based, nurse-led health promotion programmes. Included studies were critically appraised using a published checklist. Owing to resource constraints, a de novo health economic model was not developed., Results: Eleven studies were included in the systematic review of clinical effectiveness. There was considerable heterogeneity among the studies with respect to the nature of the intervention, the nurses delivering the programmes and the populations in which the interventions were assessed. Overall, the quality of the included studies was good: all but one of the included studies were judged to be at medium or low risk of bias. Meta-analysis of eight studies suggested a statistically significant mortality benefit for the home-based health promotion groups, whereas a meta-analysis of four studies suggested non-significant benefits in terms of fewer falls in the intervention groups than in the control groups. Positive outcomes for home-based, nurse-led health promotion interventions were also reported within individual studies across several other outcomes. Only three economic studies met the criteria for inclusion in the review of cost-effectiveness. This evidence base consists of one non-randomised cost minimisation analysis and two economic evaluations undertaken alongside randomised controlled trials. Two of these studies involved an intervention targeted specifically at patients with a known underlying incurable disease, whereas the third study examined the clinical effectiveness and cost-effectiveness of early discharge in patients with a range of conditions, including fractures, neurological conditions and cardiorespiratory conditions. Each study indicated some likelihood that home-based, nurse-led health promotion may offer cost savings to the NHS and associated sectors, such as social services. However, one study did not report any comparison of health outcomes and instead simply assumed equivalence between the intervention and comparator groups, whereas the other two studies suggested at best a negligible incremental benefit in terms of preference-based health-related quality-of-life measures., Limitations: The evidence base for clinical effectiveness is subject to considerable heterogeneity. The UK economic evidence base is limited to three studies., Conclusions: On the basis of the evidence included in this systematic review, home-based, nurse-led health promotion may offer clinical benefits across a number of important health dimensions. However, it is generally unclear from the available studies which components of this type of complex intervention contribute towards individual aspects of benefit for older people. Given the limitations of the current evidence base, it remains unclear whether or not home-based health promotion interventions offer good value for money for the NHS and associated sectors. Given the considerable uncertainties in the available evidence base, it is difficult to isolate the key areas in which future research would be valuable or the exact study design required. Although this report does not identify specific studies that should be undertaken, it does set out a number of key considerations for the design of future research in this area., Study Registration: PROSPERO number: CRD42012002133.

Background: Colorectal cancer (CRC) is the third most common cancer in the UK: incidence increases with age, median age at diagnosis being over 70 years. Approximately 25% of cases occur in individuals with a family history of CRC, including 5% caused by familial adenomatous polyposis (FAP) or hereditary non-polyposis CRC (HNPCC). Most develop from adenomatous polyps arising from the intestine lining. Individuals with these polyps undergo polypectomy and are invited for endoscopic surveillance. Screening via faecal occult blood testing has been rolled out across the UK., Objectives: To evaluate the clinical effectiveness and cost-effectiveness of drug and micronutrient interventions for the prevention of CRC and/or adenomatous polyps. Interventions considered include: non-steroidal anti-inflammatory drugs (NSAIDs), including aspirin and cyclo-oxygenase-2 (COX-2) inhibitors; folic acid; calcium; vitamin D and antioxidants (including vitamin A, vitamin C, vitamin E, selenium and beta-carotene). Chemoprevention was assessed in the general population, in individuals at increased risk of CRC, and in individuals with FAP or HNPCC., Data Sources: A systematic review identified randomised controlled trials (RCTs) assessing drug and nutritional agents for the prevention of CRC or adenomatous polyps. A separate search identified qualitative studies relating to individuals' views, attitudes and beliefs about chemoprevention. MEDLINE, MEDLINE In-Process & Other Non-Indexed Citations, EMBASE, CINAHL, the Cochrane Database of Systematic Reviews, Cochrane CENTRAL Register of Controlled Trials, DARE, NHS-EED (NHS Economic Evaluation Database), HTA database, Science Citation Index, BIOSIS previews and the Current Controlled Trials research register were searched in June 2008. Data were extracted by one reviewer and checked by a second., Review Methods: The synthesis methods used were systematic review and meta-analysis for RCTs and qualitative framework synthesis for qualitative studies. A health economic model was developed to assess the cost-effectiveness of chemoprevention for two populations with different levels of risk of developing CRC: the general population and an intermediate-risk population., Results: The search identified 44 relevant RCTs and six ongoing studies. A small study of aspirin in FAP patients produced no statistically significant reduction in polyp number but a possible reduction in polyp size. There was a statistically significant 21% reduction in risk of adenoma recurrence [relative risk (RR) 0.79, 95% confidence interval (CI) 0.68 to 0.92] in an analysis of aspirin versus no aspirin in individuals with a history of adenomas or CRC. In the general population, a significant 26% reduction in CRC incidence was demonstrated in studies with a 23-year follow-up (RR 0.74, 95% CI 0.57 to 0.97). Non-aspirin NSAID use in FAP individuals produced a non-statistically significant reduction in adenoma incidence after 4 years of treatment and follow-up and reductions in polyp number and size. In individuals with a history of adenomas there was a statistically significant 34% reduction in adenoma recurrence risk (RR 0.66, 95% CI 0.60 to 0.72) and a statistically significant 55% reduction in advanced adenoma incidence (RR 0.45, 95% CI 0.35 to 0.58). No studies assessed the effect of non-aspirin NSAIDs in the general population. There were no studies of folic acid in individuals with FAP or HNPCC. There was no significant effect of folic acid versus placebo on adenoma recurrence (RR 1.16, 95% CI 0.97 to 1.39) or advanced adenoma incidence in individuals with a history of adenomas. In the general population there was no significant effect of folic acid on risk of CRC (RR 1.13, 95% CI 0.77 to 1.64), although studies were of relatively short duration. Calcium use by FAP patients produced no significant reduction in polyp number or disease progression. In individuals with a history of adenomas there was a statistically significant 18% reduction in risk of adenoma recurrence (RR 0.82, 95% CI 0.69 to 0.98) and a non-significant reduction in risk of advanced adenomas (RR 0.77, 95% CI 0.50 to 1.17). In the general population there was no significant effect of calcium on risk of CRC (RR 1.08, 95% CI 0.87 to 1.34), although studies were of relatively short duration. There were no studies of antioxidant use in individuals with FAP or HNPCC, and in individuals with a history of adenomas no statistically significant differences in relative risk of adenoma recurrence were found. In the general population there was no difference in incidence of CRC (RR 1.00, 95% CI 0.88 to 1.13) with antioxidant use compared with no antioxidant use. Twenty studies reported qualitative findings concerning chemoprevention. People are more likely to use NSAIDs if there is a strong perceived need. Perceptions of risk and benefit also influence decision-making and use. People have fewer concerns about using antioxidants or other supplements, but their perception of the benefits of these agents is less well-defined. The model analysis suggested that the most cost-effective age-range policy in the general population would be to provide chemoprevention to all individuals within the general population from age 50 to 60 years. The use of aspirin in addition to screening within the general population is likely to result in a discounted cost per life-year gained of around 10,000 pounds and a discounted cost per quality-adjusted life-year (QALY) gained of around 23,000 pounds compared with screening alone. In the intermediate-risk group the most economically viable age-range policy would be to provide chemoprevention to individuals following polypectomy aged 61 to 70 years. Calcium is likely to have a discounted cost per QALY gained of around 8000 pounds compared with screening alone. Although aspirin in addition to screening should be more effective and less costly than screening alone, under the current assumptions of benefits to harms of aspirin and calcium, aspirin is expected to be extendedly dominated by calcium., Limitations: Whilst a number of studies were included in the review, the duration of follow-up was generally insufficient to detect an effect on cancer incidence. Given the uncertainties and ambiguities in the evidence base, the results of the health economic analysis should be interpreted with caution., Conclusions: Aspirin and celecoxib may reduce recurrence of adenomas and incidence of advanced adenomas in individuals with an increased risk of CRC and calcium may reduce recurrence of adenomas in this group. COX-2 inhibitors may decrease polyp number in patients with FAP. There is some evidence for aspirin reducing the incidence of CRC in the general population. Both aspirin and NSAIDs are associated with adverse effects so it will be important to consider the risk-benefit ratio before recommending these agents for chemoprevention. The economic analysis suggests that chemoprevention has the potential to represent a cost-effective intervention, particularly when targeted at intermediate-risk populations following polypectomy.

Background: Health policy decisions must be relevant, evidence-based and transparent. Decision-analytic modelling supports this process but its role is reliant on its credibility. Errors in mathematical decision models or simulation exercises are unavoidable but little attention has been paid to processes in model development. Numerous error avoidance/identification strategies could be adopted but it is difficult to evaluate the merits of strategies for improving the credibility of models without first developing an understanding of error types and causes., Objectives: The study aims to describe the current comprehension of errors in the HTA modelling community and generate a taxonomy of model errors. Four primary objectives are to: (1) describe the current understanding of errors in HTA modelling; (2) understand current processes applied by the technology assessment community for avoiding errors in development, debugging and critically appraising models for errors; (3) use HTA modellers' perceptions of model errors with the wider non-HTA literature to develop a taxonomy of model errors; and (4) explore potential methods and procedures to reduce the occurrence of errors in models. It also describes the model development process as perceived by practitioners working within the HTA community., Data Sources: A methodological review was undertaken using an iterative search methodology. Exploratory searches informed the scope of interviews; later searches focused on issues arising from the interviews. Searches were undertaken in February 2008 and January 2009. In-depth qualitative interviews were performed with 12 HTA modellers from academic and commercial modelling sectors., Review Methods: All qualitative data were analysed using the Framework approach. Descriptive and explanatory accounts were used to interrogate the data within and across themes and subthemes: organisation, roles and communication; the model development process; definition of error; types of model error; strategies for avoiding errors; strategies for identifying errors; and barriers and facilitators., Results: There was no common language in the discussion of modelling errors and there was inconsistency in the perceived boundaries of what constitutes an error. Asked about the definition of model error, there was a tendency for interviewees to exclude matters of judgement from being errors and focus on 'slips' and 'lapses', but discussion of slips and lapses comprised less than 20% of the discussion on types of errors. Interviewees devoted 70% of the discussion to softer elements of the process of defining the decision question and conceptual modelling, mostly the realms of judgement, skills, experience and training. The original focus concerned model errors, but it may be more useful to refer to modelling risks. Several interviewees discussed concepts of validation and verification, with notable consistency in interpretation: verification meaning the process of ensuring that the computer model correctly implemented the intended model, whereas validation means the process of ensuring that a model is fit for purpose. Methodological literature on verification and validation of models makes reference to the Hermeneutic philosophical position, highlighting that the concept of model validation should not be externalized from the decision-makers and the decision-making process. Interviewees demonstrated examples of all major error types identified in the literature: errors in the description of the decision problem, in model structure, in use of evidence, in implementation of the model, in operation of the model, and in presentation and understanding of results. The HTA error classifications were compared against existing classifications of model errors in the literature. A range of techniques and processes are currently used to avoid errors in HTA models: engaging with clinical experts, clients and decision-makers to ensure mutual understanding, producing written documentation of the proposed model, explicit conceptual modelling, stepping through skeleton models with experts, ensuring transparency in reporting, adopting standard housekeeping techniques, and ensuring that those parties involved in the model development process have sufficient and relevant training. Clarity and mutual understanding were identified as key issues. However, their current implementation is not framed within an overall strategy for structuring complex problems., Limitations: Some of the questioning may have biased interviewees responses but as all interviewees were represented in the analysis no rebalancing of the report was deemed necessary. A potential weakness of the literature review was its focus on spreadsheet and program development rather than specifically on model development. It should also be noted that the identified literature concerning programming errors was very narrow despite broad searches being undertaken., Conclusions: Published definitions of overall model validity comprising conceptual model validation, verification of the computer model, and operational validity of the use of the model in addressing the real-world problem are consistent with the views expressed by the HTA community and are therefore recommended as the basis for further discussions of model credibility. Such discussions should focus on risks, including errors of implementation, errors in matters of judgement and violations. Discussions of modelling risks should reflect the potentially complex network of cognitive breakdowns that lead to errors in models and existing research on the cognitive basis of human error should be included in an examination of modelling errors. There is a need to develop a better understanding of the skills requirements for the development, operation and use of HTA models. Interaction between modeller and client in developing mutual understanding of a model establishes that model's significance and its warranty. This highlights that model credibility is the central concern of decision-makers using models so it is crucial that the concept of model validation should not be externalized from the decision-makers and the decision-making process. Recommendations for future research would be studies of verification and validation; the model development process; and identification of modifications to the modelling process with the aim of preventing the occurrence of errors and improving the identification of errors in models.

This paper presents a summary of the submission's evidence for the clinical effectiveness and cost-effectiveness of varenicline for smoking cessation included four studies of varenicline (one of which was commercial-in-confidence) and a meta-analysis of varenicline versus nicotine replacement therapy (NRT), bupropion and placebo. Two controlled trials of 12 weeks of varenicline versus sustained-release bupropion and placebo suggested that varenicline results in a statistically significant improvement in the odds of quitting at 12 weeks [odds ratio (OR) for quit rate during last 4 weeks of the study: 1.90-1.93 (p < 0.001) varenicline versus bupropion; 3.85 (p < 0.001) varenicline versus placebo). The ORs for sustained abstinence (weeks 9-52) for varenicline versus bupropion were 1.77 (p = 0.004) and 1.46 (p = 0.057), and for varenicline versus placebo were 2.66-3.09 (p < 0.01). A placebo-controlled maintenance trial examined whether a further 12 weeks of varenicline would maintain the rate of abstinence among those successfully treated on one 12-week course [OR = 2.48 at week 24 for varenicline versus placebo (p < 0.001)]. The meta-analysis suggested that varenicline was superior to placebo and bupropion at 1 year and 3 months. Based on indirect comparisons, varenicline was reported to be superior to NRT when compared with placebo or all controls at 1 year and 3 months. The submission presented a state transition model to estimate the incremental cost-effectiveness of varenicline compared with bupropion, NRT and placebo. The model suggests that varenicline dominates bupropion, NRT and placebo.Treatment efficacy was based on a pooled analysis of 1-year quit rates from the varenicline clinical trials. Assuming a willingness-to-pay threshold range of 20,000-30,000 pounds per quality-adjusted life-year gained, the probabilistic sensitivity analysis suggests that the probability that varenicline produces the greatest amount of net benefit is 0.70. Weaknesses of the manufacturer's submission include the assumption that only a single quit attempt using a single smoking cessation intervention is made, the presence of multiple computational errors and a limited sensitivity analysis. In conclusion, varenicline is likely to be clinically and cost-effective for smoking cessation assuming that each user makes a single quit attempt. The key area of uncertainty concerns the long-term experience of subjects who have remained abstinent from smoking beyond 12 months. The guidance issued by the National Institute for Health and Clinical Excellence in July 2007 states that varenicline is recommended within its licensed indications as an option for smokers who have expressed a desire to quit smoking and that varenicline should normally be prescribed only as part of a programme of behavioral support.

Objectives: To evaluate the clinical effectiveness and incremental cost-effectiveness of amantadine, oseltamivir and zanamivir for seasonal and post-exposure prophylaxis of influenza., Data Sources: A MEDLINE search strategy was used and searches were carried out in July 2007., Review Methods: An independent health economic model was developed based on a review of existing cost-effectiveness models and clinical advice.The model draws together a broad spectrum of evidence relating to the costs and consequences associated with influenza and its prevention. Where direct evidence concerning the effectiveness of prophylaxis within specific model subgroups was lacking, the model uses estimates from mixed subgroups or extrapolates from other mutually exclusive subgroups., Results: Twenty-six published references relating to 22 randomised controlled trials (RCTs) were included in the clinical effectiveness review, along with one unpublished report. Eight, six and nine RCTs were included for amantadine, oseltamivir and zanamivir respectively. The study quality was variable and gaps in the evidence base limited the assessment of the clinical effectiveness of the interventions. For seasonal prophylaxis, there was limited evidence for the efficacy of amantadine in preventing symptomatic, laboratory-confirmed influenza (SLCI) in healthy adults [relative risk (RR) 0.40, 95% confidence interval (CI) 0.08-2.03]. Oseltamivir was effective in preventing SLCI, particularly when used in at-risk elderly subjects (RR 0.08, 95% CI 0.01-0.63). The preventative efficacy of zanamivir was most notable in at-risk adults and adolescents (RR 0.17, 95% CI 0.07-0.44), and healthy and at-risk elderly subjects (RR 0.20, 95% CI 0.02-1.72). For post-exposure prophylaxis, data on the use of amantadine were again limited: in adolescents an RR of 0.10 (95% CI 0.03-0.34) was reported for the prevention of SLCI. Oseltamivir was effective in households of mixed composition (RR 0.19, 95% CI 0.08-0.45). The efficacy of zanamivir in post-exposure prophylaxis within households was also reported (RR 0.21, 95% CI 0.13-0.33). Interventions appeared to be well tolerated. Limited evidence was available for the effectiveness of the interventions in preventing complications and hospitalisation and in minimising length of illness and time to return to normal activities. No clinical effectiveness data were identified for health-related quality of life or mortality outcomes. With the exception of at-risk children, the incremental cost-utility of seasonal influenza prophylaxis is expected to be in the range 38,000-428,000 pounds per QALY gained (depending on subgroup). The cost-effectiveness ratios for oseltamivir and zanamivir as post-exposure prophylaxis are expected to be below 30,000 pounds per QALY gained in healthy children, at-risk children, healthy elderly and at-risk elderly individuals. Despite favourable clinical efficacy estimates, the incorporation of recent evidence of viral resistance to amantadine led to it being dominated in every economic comparison., Conclusions: All three interventions showed some efficacy for seasonal and post-exposure prophylaxis. However, weaknesses and gaps in the clinical evidence base are directly relevant to the interpretation of the health economic model and rendered the use of advanced statistical analyses inappropriate. These data limitations should be borne in mind in interpreting the findings of the review.

Objectives: To evaluate three technologies for the management of advanced colorectal cancer: (1) first-line irinotecan combination [with 5-fluorouracil (5-FU)] or second-line monotherapy; (2) first- or second-line oxaliplatin combination (again, with 5-FU); and (3) raltitrexed, where 5-FU is inappropriate. To examine the role of irinotecan and oxaliplatin in reducing the extent of incurable disease before curative surgery (downstaging)., Sources: Ten electronic bibliographic databases covering the period up to August 2004., Methods: Searches identified existing studies of the effectiveness and economics of the technologies and any studies that evaluated any of the indications outlined above were included. Data were extracted and assessed generic components of methodological quality. Survival outcomes were meta-analysed., Results: Seventeen trials were found, of varying methodological quality. Compared with 5-FU, first-line irinotecan improved overall survival (OS) by 2-4 months (p=0.0007), progression-free survival (PFS) by 2-3 months (p<0.00001) and response rates (p<0.001). It offered a different toxicity profile and no quality of life (QoL) advantage. However, second-line irinotecan compared with 5-FU improved OS by 2 months (p=0.035) and PFS by 1 month (p=0.03), and provided a better partial response rate, but with more toxicities and no QoL advantage. Compared with second-line best supportive care, irinotecan improved OS by 2 months (p=0.0001), had a different toxicity profile and maintained baseline QoL longer, but with no overall difference. The addition of oxaliplatin to second-line 5-FU is associated with a borderline significant improvement in overall survival (p<0.07); a significantly higher response rate (<0.0001); and more serious toxicities. There is no evidence for a significant difference in QoL. Schedules with treatment breaks may not reduce clinical effectiveness but reduce toxicity. The addition of oxaliplatin to second-line 5-FU also saw no improvement in OS (p<0.07), better PFS (by 2.1 months, p=0.0001), an 8.9% higher response rate (p<0.0001), more toxicities and no QoL advantage. There was no significant difference in OS or PFS between first-line irinotecan and oxaliplatin combinations except when 5-FU was delivered by bolus injection, when oxaliplatin provided better OS (p=0.032) and response rates (p=0.032), but not PFS (p=0.169). The regimens had different toxicity profiles and neither conferred a QoL advantage. When compared to 5-FU, raltitrexed is associated with no significant difference in overall or progression-free survival; no significant difference in response rates; more vomiting and nausea, but less diarrhoea and mucositis; no significant difference in, or worse QoL. Raltitrexed treatment was cut short in two out of four included trials due to excess toxic deaths. 5-FU followed by irinotecan was inferior to any other sequence. First-line irinotecan/5-FU combination improved OS and PFS, although further unplanned therapy exaggerated the OS effect size. Staged combination therapy (combination oxaliplatin followed by combination irinotecan or vice versa) provided the best OS and PFS, although there was no head-to-head comparison against other treatment plans. In the only trial to use three active chemotherapies in any staged combination, median OS was over 20 months. In another study, the longest median OS from a treatment plan using two active agents was 16.2 months. Where irinotecan or oxaliplatin were used with 5-FU to downstage people with unresectable liver metastases, studies consistently showed response rates of around 50%. Resection rates ranged from 9 to 35% with irinotecan and from 7 to 51% with oxaliplatin. In the one study that compared the regimens, oxaliplatin enabled more resections (p=0.02). Five-year OS rates of 5-26% and disease-free survival rates of 3-11% were reported in studies using oxaliplatin. Alone or in combination, 5-FU was more effective and less toxic when delivered by continuous infusion. Existing economic models were weak because of the use of unplanned second-line therapies in their trial data: the survival benefits in patients on such trials cannot be uniquely attributed to the allocated therapy. Consequently, the economic analyses are either limited to the use of PES (at best, a surrogate outcome) or are subject to confounding. Weaknesses in cost components, the absence of direct in-trial utility estimates and the limited use of sensitivity analysis were identified. Improvements to the methodologies used in existing economic studies are presented. Using data from two trials that planned treatment sequences, an independent economic evaluation of six plans compared with first-line 5-FU followed on progression by second-line irinotecan monotherapy (NHS standard treatment) is presented. 5-FU followed on progression by irinotecan combination cost 13,174 pounds per life-year gained (LYG) and 10,338 pounds per quality-adjusted life-year (QALY) gained. Irinotecan combination followed on progression by additional second-line therapies was estimated to cost 12,418 pounds per LYG and 13,630 pounds per QALY gained. 5-FU followed on progression by oxaliplatin combination was estimated to cost 23,786 pounds per LYG and 31,556 pounds per QALY gained. Oxaliplatin combination followed on progression by additional second-line therapies was estimated to cost 43,531 pounds per LYG and 67,662 pounds per QALY gained. Evaluations presented in this paragraph should be interpreted with caution owing to missing information on the costs of salvage therapies in the trial from which data were drawn. Irinotecan combination followed on progression by oxaliplatin combination cost 12,761 pounds per LYG and 16,663 pounds per QALY gained. Oxaliplatin combination followed on progression by irinotecan combination cost 16,776 pounds per LYG and 21,845 pounds per QALY gained. The evaluation suggests that these two sequences have a cost-effectiveness profile that is favourable in comparison to other therapies currently funded by the NHS. However, the differences in OS observed between the two trials from which data were taken may be a result of heterogeneous patient populations, unbalanced protocol-driven intensity biases or other differences between underlying health service delivery systems., Conclusions: Treatment with three active therapies appears most clinically effective and cost-effective. NHS routine data could be used to validate downstaging findings and a meta-analysis using individual patient-level data is suggested to validate the optimal treatment sequence.

Objectives: To undertake a structured review and critical appraisal of methods for the model-based cost-utility analysis of screening programmes. Also to develop guidelines and an assessment checklist of good practice in the development of screening models., Data Sources: Major electronic databases of healthcare and operational research literatures were searched up to June 2003., Review Methods: Searches of the literature were undertaken to identify applied and methodological studies of economic evaluations of healthcare screening programmes. All applied screening models were also reviewed in three broad disease areas (cancer, cardiovascular disease and diabetes), as well as antenatal screening. A second-level review focused on particular aspects of the modelling process through case study assessments of screening models for three specific disease areas (colorectal cancer, abdominal aortic aneurysms and antenatal screening for haemoglobinopathies). A separate literature review of studies reporting the utility effects of screening was also undertaken. Guidelines and an assessment checklist for good practice for screening modelling were developed., Results: Few relevant methodological studies were identified, and no studies reporting direct empirical comparisons of alternative methodologies were retrieved. From the review of disease-based screening models, it was apparent that many alternative modelling methods had been applied, including some relatively new approaches that had not been widely disseminated. Natural history modelling is the preferred approach. Alternative modelling approaches were generally only used to extrapolate the observed effects of screening and were unsuitable for evaluating unobserved screening options. More complex model structures may incorporate important additional aspects of the disease natural history, although any benefits should outweigh the consequences of additional unobservable input parameters and increased complexity in implementing the model. No direct comparisons of more detailed and less detailed screening model structures informed areas in which more realistic representations of the disease process may be most beneficial, so only general aspects of good practice could be defined. Two structural aspects that were not well handled by existing screening models included post-diagnosis disease progression and screening uptake. Most models described the former using historical mortality rates, rather than treatment models that are representative of current treatment patterns for different stages of the disease. Constant screening uptake rates were applied to all screening programmes and attendance was not linked to disease incidence or progression. Evidence exists to inform a more detailed representation of screening uptake. The most commonly applied modelling techniques were cohort Markov models and individual sampling simulation models. Individual sampling simulation models may provide more flexibility in their representation of a screening decision problem, but any benefits should outweigh the consequences of the need to assess both variability and uncertainty. Complex mathematical models describing input parameters as continuous variables have analysed the cost-effectiveness of screening; these require further development to estimate the cost-utility of screening directly, or to inform a more detailed representation of the preclinical section of a natural history model (with a traditional state-based model describing pathways' post-clinical presentation). Calibration is a common aspect of screening models, whereby models are fitted to observed data describing outputs of the model in order to populate unobserved input parameters. The review concluded that the estimation of a reference case input parameter set is not recommended., Conclusions: The review of methods for the model-based cost-utility analysis of screening programmes identified the natural history modelling approach as the preferred general method of evaluation for screening programmes. State transition models have generally been used to represent disease natural histories, with individual sampling models more prevalent than in treatment intervention evaluations. No comparative methodological studies were identified, so no empirical data were available to inform the relative merits of alternative methodologies. The defined guidelines and assessment checklist are informed, therefore, by theoretical interpretations of the impact of alternative approaches to different components of the modelling process when applied to the cost-utility analysis of screening programmes. Further research is needed into methods with the potential to improve the accuracy of screening models, and to respond to the needs of model users.

Objectives: To assess the clinical effectiveness and cost-effectiveness of bevacizumab and cetuximab in the treatment of individuals with metastatic colorectal cancer (CRC)., Data Sources: Searches of main electronic databases were conducted in April and May 2005., Review Methods: For the assessment of bevacizumab, trials were included if they recruited participants with untreated metastatic CRC for first-line treatment. Only trials comparing bevacizumab in combination with irinotecan and/or established fluorouracil (5-FU)-containing or releasing regimens given as first-line therapy were included. For the assessment of cetuximab, trials were included if they recruited participants with epidermal growth-factor receptor-expressing metastatic CRC who had previously failed irinotecan-including therapy. Independent cost-effectiveness models of bevacizumab and cetuximab were developed using survival modelling methods., Results: Adding bevacizumab to irinotecan in combination with 5-FU/folic acid (FA) plus irinotecan resulted in a statistically significant increase in median overall survival (OS) of 4.7 months. Adding bevacizumab to 5-FU/FA resulted in a non-significant increase in median OS of 3.7 months within one study and 7.7 months in another. Adding bevacizumab to irinotecan, fluorouracil and leucovorin (IFL) resulted in a statistically significant increase in median progression-free survival (PFS) of 4.4 months. Adding bevacizumab to 5-FU/FA resulted in a statistically significant increase in median PFS of 3.7 months, and a statistically significant increase in time to disease progression of 3.8 months compared to FU/FA alone. An overall tumour response rate of 44.8% was reported for bevacizumab plus IFL compared to 34.8% for IFL plus placebo. This addition was statistically significant. The addition of bevacizumab to 5-FU/FA resulted in a significant difference in tumour response rate within one study, but not another. Bevacizumab in combination with IFL or 5-FU/FA was observed to result in an increase of grade 3/4 adverse events. The independent health economic assessment suggests that the cost-effectiveness of bevacizumab plus IFL is unlikely to be better than pound 46,853 per life-year gained (LYG); the cost-utility of bevacizumab plus IFL is unlikely to be better than pound 62,857 per quality-adjusted life-year (QALY) gained. The cost-effectiveness of bevacizumab plus 5-FU/FA versus 5-FU/FA is unlikely to be better than pound 84,607 per LYG; the cost-utility of bevacizumab plus 5-FU/FA versus 5-FU/FA is unlikely to be better than pound 88,658 per QALY gained. A Phase II trial reported a median OS duration of 8.6 months for patients receiving cetuximab plus irinotecan, plus a median time to progression of 4.1 months, a tumour response rate of 22.9% and suggested that treatment with cetuximab in combination with irinotecan is associated with significantly more adverse events (any grade 3 or grade 4 adverse event) than cetuximab monotherapy. The single arm study of cetuximab plus irinotecan reported a median OS duration of 8.4 months, a median time to progression of 2.9 months and a tumour response rate of 15.2%. The cost-effectiveness model suggested that the expected survival duration of patients receiving cetuximab plus irinotecan is 0.79 years (9.5 months) when the proposed continuation rule is applied. In order for cetuximab plus irinotecan to achieve a cost-utility ratio of pound 30,000 per QALY gained, treatment with cetuximab plus irinotecan must provide an additional 0.65 life years (7.8 months) over treatment with active/best supportive care, implying that survival in the active/best supportive care group must be 0.14 life years (1.7 months) or less., Conclusions: The trials indicate that bevacizumab in combination with 5-FU/FA, and bevacizumab in combination with IFL, is clinically effective in comparison to standard chemotherapy options for the first-line treatment of metastatic CRC. The health economic analysis suggests that the marginal cost-utility of bevacizumab plus IFL versus IFL is unlikely to be better than pound 62,857 per QALY gained, and the marginal cost-utility of bevacizumab plus 5-FU/FA versus 5-FU/FA is unlikely to be better than pound 88,658 per QALY gained. There is no direct evidence to demonstrate whether cetuximab in combination with irinotecan improves health-related quality of life or OS in comparison to active/best supportive care or oxaliplatin plus 5-FU/FA, although the evidence on tumour response rates suggests that cetuximab plus irinotecan has some clinical activity. While it is difficult to suggest whether cetuximab represents value for money, indirect comparisons suggest that the incremental cost-utility of cetuximab plus irinotecan is unlikely to be better than pound 30,000 per QALY gained. This review highlights a number of areas for further research, including clarifying the true impact of first-line bevacizumab in combination with irinotecan and/or infusional 5-FU/FA, without subsequent bevacizumab treatment following disease progression, on OS in patients with metastatic CRC who are representative of the typical population of CRC patients in England and Wales. Further research concerning the impact of therapies on health-related quality of life is essential.

Objectives: To assess the clinical and cost-effectiveness of oxaliplatin in combination with 5-fluorouracil/leucovorin (5-FU/LV), and capecitabine monotherapy (within their licensed indications), as adjuvant therapies in the treatment of patients with Stage III (Dukes' C) colon cancer after complete surgical resection of the primary tumour, as compared with adjuvant chemotherapy with an established fluorouracil-containing regimen., Data Sources: Ten electronic bibliographic databases were searched from inception to January 2005. Searches were supplemented by hand searching relevant articles, sponsor and other submissions of evidence to the National Institute of Health and Clinical Excellence and conference proceedings., Review Methods: A systematic review and meta-analysis (where appropriate) of clinical efficacy evidence and a cost-effectiveness review and economic modelling were carried out. Marginal costs, life years gained and cost-effectiveness acceptability curves were estimated. Probabilistic sensitivity analysis was used to generate information on the likelihood that each of the interventions was optimal., Results: Three randomised active-controlled trials, of varying methodological quality, were included in the review. The MOSAIC trial and NSABP C-07 study considered the addition of oxaliplatin to adjuvant treatment (albeit administered in different 5-FU/LV regimens) and the X-ACT study compared oral capecitabine with bolus 5-FU/LV alone. A review of the available evidence indicated that in patients with Stage III colon cancer, oxaliplatin in combination with an infusional de Gramont schedule of 5-FU/LV (FOLFOX4) was more effective in preventing and delaying disease recurrence than infusional 5-FU/LV alone (de Gramont regimen). Serious adverse events and treatment discontinuations due to toxicity were more evident with oxaliplatin-based regimens (FOLFOX4 and FLOX regimen) than infusional or bolus 5-FU/LV alone (de Gramont and Roswell Park regimen). Oral capecitabine was at least equivalent in disease-free survival to the bolus Mayo Clinic 5-FU/LV regimen for patients with resected Stage III colon cancer. Although, the safety and tolerability profile of capecitabine was superior to that of the Mayo Clinic 5-FU/LV regimen, it has not been evaluated in comparison with other less toxic 5-FU/LV regimens currently in common use in the UK. Based on the assumptions and survival analysis methods used, the cost-effectiveness analysis using economic modelling estimated that capecitabine was a dominating strategy and resulted in a cost-saving of approximately pound 3320 per patient in comparison with the Mayo Clinic 5-FU/LV regimen, while also providing an additional 0.98 quality-adjusted life-years (QALYs) over a 50-year model time horizon. Oxaliplatin in combination with 5-FU/LV (FOLFOX4 regimen) is estimated to cost an additional pound 2970 per QALY gained when compared with the de Gramont 5-FU/LV regimen and demonstrated superior survival outcomes with marginal costs. The uncertainty analysis suggests that both interventions have a high probability of being cost-effective at a threshold of both pound 20,000 and pound 30,000. An indirect comparison of the FOLFOX4 and Mayo Clinic 5-FU/LV regimens suggests that the use of FOLFOX4 in place of the Mayo Clinic 5-FU/LV regimen would cost an additional pound 5777 per QALY gained. An incremental cost-effectiveness ratio (ICER) is estimated to be approximately pound 13,000 per QALY gained from treatment with FOLFOX4 compared with capecitabine. However, if the Mayo Clinic and the de Gramont 5-FU/LV regimens are assumed to be equivalent in terms of effectiveness, the ICER of FOLFOX4 in comparison with capecitabine may be greater than pound 30,000 per QALY., Conclusions: The evidence suggests that both capecitabine and FOLFOX4 are clinically effective and cost-effective in comparison with 5-FU/LV regimens (Mayo Clinic and de Gramont schedules). Further research is suggested into the effectiveness, tolerability, patient acceptability and costs of different oxaliplatin/fluoropyrimidine schedules in the adjuvant setting; the effects of treatment duration on efficacy; adverse events; resource data collection strategies and reporting of summary statistics; subgroups benefiting most from adjuvant chemotherapy; and methods for estimating mean survival.

Objectives: To develop methods for performing expected value of perfect information (EVPI) analysis in computationally expensive models and to report on the developments on the health economics of interferon-beta and glatiramer acetate in the management of multiple sclerosis (MS) using this methodological framework., Data Sources: Electronic databases and Internet resources, reference lists of relevant articles., Review Methods: A methodological framework was developed for undertaking EVPI analysis for complex models. The framework identifies conditions whereby EVPI may be calculated numerically, where the one-level algorithm sufficiently approximates the two-level algorithm, and whereby metamodelling techniques may accurately approximate the original simulation model. Metamodelling techniques, including linear regression, neural networks and Gaussian processes (GP), were systematically reviewed and critically appraised. Linear regression metamodelling, GP metamodelling and the one-level EVPI approximation were used to estimate partial EVPIs using the ScHARR MS cost-effectiveness model., Results: The review of metamodelling approaches suggested that in general the simpler techniques such as linear regression may be easier to implement, as they require little specialist expertise although may provide only limited predictive accuracy. More complex methods such as Gaussian process metamodelling and neural networks tend to use less-restrictive assumptions concerning the relationship between the model inputs and net benefits, and therefore may permit greater accuracy in estimating EVPIs. Assuming independent treatment efficacy, the 'per patient' EVPI for all uncertainty parameters within the ScHARR MS model is 8855 British pounds. This leads to a population EVPI of 86,208,936 British pounds, which represents the upper estimate for the overall EVPI over 10 years. Assuming all treatment efficacies are perfectly correlated, the overall per patient EVPI is 4271 British pounds. This leads to a population EVPI of 41,581,273 British pounds, which represents the lower estimate for the overall EVPI over 10 years. The partial EVPI analysis, undertaken using both the linear regression metamodel and Gaussian process metamodel clearly, suggests that further research is indicated on the long-term impact of these therapies on disease progression, the proportion of patients dropping off therapy and the relationship between the EDSS, quality of life and costs of care., Conclusions: The applied methodology points towards using more sophisticated metamodelling approaches in order to obtain greater accuracy in EVPI estimation. Programming requirements, software availability and statistical accuracy should be considered when choosing between metamodelling techniques. Simpler, more accessible techniques are open to greater predictive error, whilst sophisticated methodologies may enhance accuracy within non-linear models, but are considerably more difficult to implement and may require specialist expertise. These techniques have been applied in only a limited number of cases hence their suitability for use in EVPI analysis has not yet been demonstrated. A number of areas requiring further research have been highlighted. Further clinical research is required concerning the relationship between the EDSS, costs of care and health outcomes, the rates at which patients drop off therapy and in particular the impact of disease-modifying therapies on the progression of MS. Further methodological research is indicated concerning the inclusion of epidemiological population parameters within the sensitivity analysis; the development of criteria for selecting a metamodelling approach; the application of metamodelling techniques within health economic models and in the specific application to EVI analyses; and the use of metamodelling for EVSI and ENBS analysis.

Objectives: To identify the role of modelling in planning and prioritising trials. The review focuses on modelling methods used in the construction of disease models and on methods for their analysis and interpretation., Data Sources: Searches were initially developed in MEDLINE and then translated into other databases., Review Methods: Systematic reviews of the methodological and case study literature were undertaken. Search strategies focused on the intersection between three domains: modelling, health technology assessment and prioritisation., Results: The review found that modelling can extend the validity of trials by: generalising from trial populations to specific target groups; generalising to other settings and countries; extrapolating trial outcomes to the longer term; linking intermediate outcome measures to final outcomes; extending analysis to the relevant comparators; adjusting for prognostic factors in trials; and synthesising research results. The review suggested that modelling may offer greatest benefits where the impact of a technology occurs over a long duration, where disease/technology characteristics are not observable, where there are long lead times in research, or for rapidly changing technologies. It was also found that modelling can inform the key parameters for research: sample size, trial duration and population characteristics. One-way, multi-way and threshold sensitivity analysis have been used in informing these aspects but are flawed. The payback approach has been piloted and while there have been weaknesses in its implementation, the approach does have potential. Expected value of information analysis is the only existing methodology that has been applied in practice and can address all these issues. The potential benefit of this methodology is that the value of research is directly related to its impact on technology commissioning decisions, and is demonstrated in real and absolute rather than relative terms; it assesses the technical efficiency of different types of research. Modelling is not a substitute for data collection. However, modelling can identify trial designs of low priority in informing health technology commissioning decisions., Conclusions: Good practice in undertaking and reporting economic modelling studies requires further dissemination and support, specifically in sensitivity analyses, model validation and the reporting of assumptions. Case studies of the payback approach using stochastic sensitivity analyses should be developed. Use of overall expected value of perfect information should be encouraged in modelling studies seeking to inform prioritisation and planning of health technology assessments. Research is required to assess if the potential benefits of value of information analysis can be realised in practice; on the definition of an adequate objective function; on methods for analysing computationally expensive models; and on methods for updating prior probability distributions.