Search

In the context of survival analysis, calibration refers to the agreement between predicted probabilities and observed event rates or frequencies of the outcome within a given duration of time. We aimed to describe and evaluate methods for graphically assessing the calibration of survival models. We focus on hazard regression models and restricted cubic splines in conjunction with a Cox proportional hazards model. We also describe modifications of the Integrated Calibration Index, of E50 and of E90. In this context, this is the average (respectively, median or 90th percentile) absolute difference between predicted survival probabilities and smoothed survival frequencies. We conducted a series of Monte Carlo simulations to evaluate the performance of these calibration measures when the underlying model has been correctly specified and under different types of model mis‐specification. We illustrate the utility of calibration curves and the three calibration metrics by using them to compare the calibration of a Cox proportional hazards regression model with that of a random survival forest for predicting mortality in patients hospitalized with heart failure. Under a correctly specified regression model, differences between the two methods for constructing calibration curves were minimal, although the performance of the method based on restricted cubic splines tended to be slightly better. In contrast, under a mis‐specified model, the smoothed calibration curved constructed using hazard regression tended to be closer to the true calibration curve. The use of calibration curves and of these numeric calibration metrics permits for a comprehensive comparison of the calibration of competing survival models.

Background This study was conducted to validate a pretreatment (i.e. prior to neoadjuvant chemoradiotherapy) pathological staging system in the resection specimen after neoadjuvant chemoradiotherapy for esophageal cancer. The study investigated the prognostic value of pretreatment pathological T and N categories (prepT and prepN categories) in both an independent and a combined patient cohort. Methods Patients with esophageal cancer treated with neoadjuvant chemotherapy and esophagectomy between 2012 and 2015 were included. PrepT and prepN categories were estimated based on the extent of tumor regression and regressional changes of lymph nodes in the resection specimen. The difference in Akaike’s information criterion (ΔAIC) was used to assess prognostic performance. PrepN and ypN categories were combined to determine the effect of nodal sterilization on prognosis. A multivariable Cox regression model was used to identify combined prepN and ypN categories as independent prognostic factors. Results The prognostic strength of the prepT category was better than the cT and ypT categories (ΔAIC 7.7 vs. 3.0 and 2.9, respectively), and the prognostic strength of the prepN category was better than the cN category and similar to the ypN category (ΔAIC 29.2 vs. − 1.0 and 27.9, respectively). PrepN + patients who became ypN0 had significantly worse survival than prepN0 patients (2-year overall survival 69% vs. 86% in 137 patients; p = 0.044). Similar results were found in a combined cohort of 317 patients (2-year overall survival 62% vs. 85%; p = 0.002). Combined prepN/ypN stage was independently associated with overall survival. Conclusions These results independently confirm the prognostic value of prepTNM staging. PrepTNM staging is of additional prognostic value to cTNM and ypTNM. PrepN0/ypN0 patients have a better survival than prepN +/ypN0 patients.

The Platelets for Neonatal Thrombocytopenia (PlaNeT-2) trial reported an unexpected overall benefit of a prophylactic platelet transfusion threshold of 25 × 109/L compared with 50 × 109/L for major bleeding and/or mortality in preterm neonates (7% absolute-risk reduction). However, some neonates in the trial may have experienced little benefit or even harm from the 25 × 109/L threshold. We wanted to assess this heterogeneity of treatment effect in the PlaNet-2 trial, to investigate whether all preterm neonates benefit from the low threshold. We developed a multivariate logistic regression model in the PlaNet-2 data to predict baseline risk of major bleeding and/or mortality for all 653 neonates. We then ranked the neonates based on their predicted baseline risk and categorized them into 4 risk quartiles. Within these quartiles, we assessed absolute-risk difference between the 50 × 109/L- and 25 × 109/L-threshold groups. A total of 146 neonates died or developed major bleeding. The internally validated C-statistic of the model was 0.63 (95% confidence interval, 0.58-0.68). The 25 × 109/L threshold was associated with absolute-risk reduction in all risk groups, varying from 4.9% in the lowest risk group to 12.3% in the highest risk group. These results suggest that a 25 × 109/L prophylactic platelet count threshold can be adopted in all preterm neonates, irrespective of predicted baseline outcome risk. Future studies are needed to improve the predictive accuracy of the baseline risk model. This trial was registered at www.isrctn.com as #ISRCTN87736839.

Background This study aimed to develop an easy to use prediction model to predict the risk of having a total of 1 to 2, >= 3, or >= 4 positive axillary lymph nodes (LNs), for patients with sentinel lymph node (SLN) positive breast cancer. Methods Data of 911 SLN positive breast cancer patients were used for model development. The model was validated externally in an independent population of 180 patients with SLN positive breast cancer. Results Final pathology after ALND showed additional positive LN for 259 (28%) of the patients. A total of 726 (81%) out of 911 patients had a total of 1 to 2 positive nodes, whereas 175 (19%) had >= 3 positive LNs. The model included three predictors: the tumor size (in mm), the presence of a negative SLN, and the size of the SLN metastases (in mm). At external validation, the model showed a good discriminative ability (area under the curve = 0.82; 95% confidence interval = 0.74-0.90) and good calibration over the full range of predicted probabilities. Conclusion This new and validated model predicts the extent of nodal involvement in node-positive breast cancer and will be useful for counseling patients regarding their personalized axillary treatment.

ObjectivesDevelop simple and valid models for predicting mortality and need for intensive care unit (ICU) admission in patients who present at the emergency department (ED) with suspected COVID-19.DesignRetrospective.SettingSecondary care in four large Dutch hospitals.ParticipantsPatients who presented at the ED and were admitted to hospital with suspected COVID-19. We used 5831 first-wave patients who presented between March and August 2020 for model development and 3252 second-wave patients who presented between September and December 2020 for model validation.Outcome measuresWe developed separate logistic regression models for in-hospital death and for need for ICU admission, both within 28 days after hospital admission. Based on prior literature, we considered quickly and objectively obtainable patient characteristics, vital parameters and blood test values as predictors. We assessed model performance by the area under the receiver operating characteristic curve (AUC) and by calibration plots.ResultsOf 5831 first-wave patients, 629 (10.8%) died within 28 days after admission. ICU admission was fully recorded for 2633 first-wave patients in 2 hospitals, with 214 (8.1%) ICU admissions within 28 days. A simple model—COVID outcome prediction in the emergency department (COPE)—with age, respiratory rate, C reactive protein, lactate dehydrogenase, albumin and urea captured most of the ability to predict death. COPE was well calibrated and showed good discrimination for mortality in second-wave patients (AUC in four hospitals: 0.82 (95% CI 0.78 to 0.86); 0.82 (95% CI 0.74 to 0.90); 0.79 (95% CI 0.70 to 0.88); 0.83 (95% CI 0.79 to 0.86)). COPE was also able to identify patients at high risk of needing ICU admission in second-wave patients (AUC in two hospitals: 0.84 (95% CI 0.78 to 0.90); 0.81 (95% CI 0.66 to 0.95)).ConclusionsCOPE is a simple tool that is well able to predict mortality and need for ICU admission in patients who present to the ED with suspected COVID-19 and may help patients and doctors in decision making.

Aims To evaluate the impact of chronic obstructive pulmonary disease (COPD) on 10-year all-cause death and the treatment effect of CABG versus PCI on 10-year all-cause death in patients with three-vessel disease (3VD) and/or left main coronary artery disease (LMCAD) and COPD. Methods Patients were stratified according to COPD status and compared with regard to clinical outcomes. Ten-year all-cause death was examined according to the presence of COPD and the revascularization strategy. Results COPD status was available for all randomized 1800 patients, of whom, 154 had COPD (8.6%) at the time of randomization. Regardless of the revascularization strategy, patients with COPD had a higher risk of 10-year all-cause death, compared with those without COPD (43.1% vs. 24.9%; hazard ratio [HR]: 2.03; 95% confidence interval [CI]: 1.56–2.64; p p = 0.858), whereas among those without COPD, CABG had a significantly lower risk of 10-year all-cause death (22.7% vs. 27.1%; HR: 0.81; 95% CI: 0.67–0.99, p = 0.041). There was no significant differential treatment effect of CABG versus PCI on 10-year all-cause death between patients with and without COPD (pinteraction = 0.544). Conclusions COPD was associated with a higher risk of 10-year all-cause death after revascularization for complex coronary artery disease. The presence of COPD did not significantly modify the beneficial effect of CABG versus PCI on 10-year all-cause death. Trial registration: SYNTAX: ClinicalTrials.gov reference: NCT00114972. SYNTAX Extended Survival: ClinicalTrials.gov reference: NCT03417050 Graphic abstract

AIMS We aimed to update the logistic clinical SYNTAX score to predict 2��year all-cause mortality after contemporary percutaneous coronary intervention (PCI). METHODS AND RESULTS We analyzed 15,883 patients in the GLOBAL LEADERS study who underwent PCI. The logistic clinical SYNTAX model was updated after imputing missing values by refitting the original model (refitted original model) and fitting an extended new model (new model, with, selection based on the Akaike Information Criterion). External validation was performed in 10,100 patients having PCI at Fu Wai hospital. Chronic obstructive pulmonary disease, prior stroke, current smoker, hemoglobin level, and white blood cell count were identified as additional independent predictors of 2��year all-cause mortality and included into the new model. The c-indexes of the original, refitted original and the new model in the derivation cohort were 0.74 (95% CI 0.72-0.76), 0.75 (95% CI 0.73-0.77), and 0.78 (95% CI 0.76-0.80), respectively. The c-index of the new model was lower in the validation cohort than in the derivation cohort, but still showed improved discriminative ability of the newly developed model (0.72; 95% CI 0.67-0.77) compared to the refitted original model (0.69; 95% CI 0.64-0.74). The models overestimated the observed 2��year all-cause mortality of 1.11% in the Chinese external validation cohort by 0.54 percentage points, indicating the need for calibration of the model to the Chinese patient population. CONCLUSIONS The new model of the logistic clinical SYNTAX score better predicts 2��year all-cause mortality after PCI than the original model. The new model could guide clinical decision making by risk stratifying patients undergoing PCI.

OBJECTIVE: Approximately 84 million people in the US have pre-diabetes, but only a fraction of them receive proven effective therapies to prevent type 2 diabetes. We estimated the value of prioritizing individuals at highest risk of progression to diabetes for treatment, compared to non-targeted treatment of individuals meeting inclusion criteria for the Diabetes Prevention Program (DPP). METHODS: Using microsimulation to project outcomes in the DPP trial population, we compared two interventions to usual care: (1) lifestyle modification and (2) metformin administration. For each intervention, we compared targeted and non-targeted strategies, assuming either limited or unlimited program capacity. We modeled the individualized risk of developing diabetes and projected diabetic outcomes to yield lifetime costs and quality-adjusted life expectancy, from which we estimated net monetary benefits (NMB) for both lifestyle and metformin versus usual care. RESULTS: Compared to usual care, lifestyle modification conferred positive benefits and reduced lifetime costs for all eligible individuals. Metformin’s NMB was negative for the lowest population risk quintile. By avoiding use when costs outweighed benefits, targeted administration of metformin conferred a benefit of $500 per person. If only 20% of the population could receive treatment, when prioritizing individuals based on diabetes risk, rather than treating a 20% random sample, the difference in NMB ranged from $14,000 to $20,000 per person. CONCLUSIONS: Targeting active diabetes prevention to patients at highest risk could improve health outcomes and reduce costs compared to providing the same intervention to a similar number of patients with pre-diabetes without targeted selection.

OBJECTIVE: To develop an electronic health record (EHR)-based risk tool that provides point-of-care estimates of diabetes risk to support targeting interventions to patients most likely to benefit.PATIENTS AND METHODS: A risk prediction model was developed and validated in a large observational database of patients with an index visit date between January 1, 2012, and December 31, 2016, with treatment effect estimates from risk-based reanalysis of clinical trial data. The risk model development cohort included 1.1 million patients with prediabetes from the OptumLabs Data Warehouse (OLDW); the validation cohort included a distinct sample of 1.1 million patients in OLDW. The randomly assigned clinical trial cohort included 3081 people from the Diabetes Prevention Program (DPP) study.RESULTS: Eleven variables reliably obtainable from the EHR were used to predict diabetes risk. This model validated well in the OLDW (C statistic = 0.76; observed 3-year diabetes rate was 1.8% (95% confidence interval [CI], 1.7 to 1.9) in the lowest-risk quarter and 19.6% (19.4 to 19.8) in the highest-risk quarter). In the DPP, the hazard ratio (HR) for lifestyle modification was constant across all levels of risk (HR, 0.43; 95% CI, 0.35 to 0.53), whereas the HR for metformin was highly risk dependent (HR, 1.1; 95% CI, 0.61 to 2.0 in the lowest-risk quarter vs HR, 0.45; 95% CI, 0.35 to 0.59 in the highest-risk quarter). Fifty-three percent of the benefits of population-wide dissemination of the DPP lifestyle modification and 73% of the benefits of population-wide metformin therapy can be obtained by targeting the highest-risk quarter of patients.CONCLUSION: The Tufts-Predictive Analytics and Comparative Effectiveness DPP Risk model is an EHR-compatible tool that might support targeted diabetes prevention to more efficiently realize the benefits of the DPP interventions.

BackgroundAn intensive lifestyle modification program or metformin pharmacotherapy reduced the risk of developing diabetes in patients at high risk, but are not widely used in the 88 million American adults with prediabetes.ObjectiveDevelop an electronic health record (EHR)-based risk tool that provides point-of-care estimates of diabetes risk to support targeting interventions to patients most likely to benefit.DesignCross-design synthesis: risk prediction model developed and validated in large observational database, treatment effect estimates from risk-based reanalysis of clinical trial data.SettingOutpatient clinics in US.PatientsRisk model development cohort: 1.1 million patients with prediabetes from the OptumLabs Data Warehouse (OLDW); validation cohort: distinct sample of 1.1 million patients in OLDW. Randomized clinical trial cohort: 3081 people from the Diabetes Prevention Program (DPP) study.InterventionsRandomization in the DPP: 1) an intensive program of lifestyle modification; 2) standard lifestyle recommendations plus 850 mg metformin twice daily; or 3) standard lifestyle recommendations plus placebo twice daily.ResultsEleven variables reliably obtainable from the EHR were used to predict diabetes risk. This model validated well in the OLDW (c-statistic = 0.76; observed 3-year diabetes rate was 1.8% in lowest-risk quarter and 19.6% in highest-risk quarter). In the DPP, the hazard ratio for lifestyle modification was constant across all levels of risk (HR = 0.43, 95% CI 0.35 – 0.53); while the HR for metformin was highly risk-dependent (HR HR = 1.1 [95% CI: 0.61 - 2.0] in the lowest-risk quarter vs. HR=0.45 [95% CI: 0.35 0.59] in the highest risk quarter). Fifty-three percent of the benefits of population-wide dissemination of the DPP lifestyle modification, and 76% of the benefits of population-wide metformin therapy can be obtained targeting the highest risk quarter of patients.LimitationsDifferences in variable definitions and in missingness across observational and trial settings may introduce estimation error in risk-based treatment effects.ConclusionAn EHR-compatible risk model might support targeted diabetes prevention to more efficiently realize the benefits of the DPP interventions.

Background and aimCOVID-19 is putting extraordinary pressure on emergency departments (EDs). To support decision making in the ED, we aimed to develop a simple and valid model for predicting mortality and need for intensive care unit (ICU) admission in suspected COVID-19 patients.MethodsFor model development, we retrospectively collected data of patients who were admitted to 4 large Dutch hospitals with suspected COVID-19 between March and August 2020 (first wave of the pandemic). Based on prior literature we considered quickly and objectively obtainable patient characteristics, vital parameters and blood test values as predictors. Logistic regression analyses with post-hoc uniform shrinkage was used to obtain predicted probabilities of in-hospital death and of the need for ICU admission, both within 28 days after hospital admission. We assessed model performance (Area Under the ROC curve (AUC); calibration plots) with temporal validation in patients who presented between September and December 2020 (second wave). We used multiple imputation to account for missing values.ResultsThe development data included 5,831 patients, of whom 629 (10.8%) died and 5,070 (86.9%) were discharged within 28 days after admission. ICU admission was fully recorded for 2,633 first wave patients in 2 hospitals, with 214 (8%) ICU admissions within 28 days. A simple model – COVID Outcome Prediction in the Emergency Department (COPE) – with age, respiratory rate, C-reactive protein, lactic dehydrogenase, albumin and urea captured most of the ability to predict death. COPE was well-calibrated and showed good discrimination in 3,252 second wave patients (AUC in 4 hospitals: 0.82 [0.78; 0.86]; 0.82 [0.74; 0.90]; 0.79 [0.70; 0.88]; 0.83 [0.79; 0.86]). COPE was also able to identify patients at high risk of needing IC in 706 second wave patients with complete information on ICU admission (AUC: 0.84 [0.78; 0.90]; 0.81 [0.66; 0.95]). The models are implemented in web-based and mobile applications.ConclusionCOPE is a simple tool that is well able to predict mortality and need for ICU admission for patients who present to the ED with suspected COVID-19 and may help patients and doctors in decision making.CONTRIBUTION TO THE LITERATUREWhat is already known on this topicPrediction models have the potential to support decision making about hospital admission of patients presenting to the emergency department with suspected COVID-19Most currently available models that were independently assessed contain a high risk of biasPromising models were developed in different patient selections and included predictors that are not quickly and objective obtainable in emergency departmentsWhat this study addsA simple and objective tool (“COPE”) is well able to predict mortality and need for ICU admission for patients who present to the ED with suspected COVID-19COPE may support ED physicians to identify high-risk patients – i.e. those at high risk of deterioration and/or death – requiring treatment in the ICU, intermediate-risk patients requiring admission to the clinical ward, and low-risk patients who can potentially be sent home

Background In patients with complex coronary artery disease (CAD), women favored coronary artery bypass grafting surgery (CABG) compared to percutaneous coronary intervention (PCI) at 5 years in the SYNTAX trial, whereas mortality rates after PCI and CABG were not different in men. On the other hand, poor outcomes of women undergoing PCI were not observed in the PRECOMBAT and BEST trials. The long-term optimal revascularization strategy according to gender has not been fully evaluated. Purpose In the SYNTAX Extended Survival (SYNTAXES) study, no significant difference existed in all-cause death between PCI and CABG at 10 years. This study aimed to assess treatment effect of PCI and CABG for 10-year all-cause death according to gender. Methods The SYNTAXES study evaluated vital status up to 10 years in 1,800 patients with de novo three-vessel disease (3VD) and/or left main coronary artery disease (LMCAD) randomized to treatment with CABG or PCI in the SYNTAX trial, and the pre-specified primary endpoint was all-cause death at 10 years. In this prespecified analysis, all-cause death at 10 years according to gender in patients undergoing PCI or CABG was evaluated. Results Of 1800 patients, 402 (22.3%) were women and 1398 (77.7%) were men. In women, the rate of mortality was significantly higher in the PCI arm at 5 years than in the CABG arm (19.3% vs. 10.3%; Log-rank p=0.010, Figure A), but the rates of mortality were not different at 10 years between the PCI and CABG arms (33.0% vs. 32.5%; Log-rank p=0.600, Figure A). In men, the mortality rate tended to be higher in the PCI arm at 10 years than in the CABG arm (27.0% vs. 22.5%; Log-rank p=0.082, Figure B), although the mortality rates were not different at 5 years between the PCI and CABG arms (12.4% vs. 12.3%; Log-rank p=0.957, Figure B). Conclusion The efficacy of CABG observed at 5 years disappeared at 10 years in women, whereas the efficacy of CABG became apparent after 5 years in men. Figure 1 Funding Acknowledgement Type of funding source: Public Institution(s). Main funding source(s): Erasmus University Medical Centre, Rotterdam, Netherlands, reference: MEC-2016-716

Background Recent evidence suggests that there is often substantial variation in the benefits and harms across a trial population. We aimed to identify regression modeling approaches that assess heterogeneity of treatment effect within a randomized clinical trial. Methods We performed a literature review using a broad search strategy, complemented by suggestions of a technical expert panel. Results The approaches are classified into 3 categories: 1) Risk-based methods (11 papers) use only prognostic factors to define patient subgroups, relying on the mathematical dependency of the absolute risk difference on baseline risk; 2) Treatment effect modeling methods (9 papers) use both prognostic factors and treatment effect modifiers to explore characteristics that interact with the effects of therapy on a relative scale. These methods couple data-driven subgroup identification with approaches to prevent overfitting, such as penalization or use of separate data sets for subgroup identification and effect estimation. 3) Optimal treatment regime methods (12 papers) focus primarily on treatment effect modifiers to classify the trial population into those who benefit from treatment and those who do not. Finally, we also identified papers which describe model evaluation methods (4 papers). Conclusions Three classes of approaches were identified to assess heterogeneity of treatment effect. Methodological research, including both simulations and empirical evaluations, is required to compare the available methods in different settings and to derive well-informed guidance for their application in RCT analysis.

Aims Adequate risk prediction can optimize the clinical management in adult congenital heart disease (ACHD). We aimed to update and subsequently validate a previously developed ACHD risk prediction model. Methods and results A prediction model was developed in a prospective cohort study including 602 moderately or severely complex ACHD patients, enrolled as outpatients at a tertiary centre in the Netherlands (2011–2013). Multivariable Cox regression was used to develop a model for predicting the 1-year risks of death, heart failure (HF), or arrhythmia (primary endpoint). The Boston ACHD Biobank study, a prospectively enrolled cohort (n = 749) of outpatients who visited a referral centre in Boston (2012–2017), was used for external validation. The primary endpoint occurred in 153 (26%) and 191 (28%) patients in the derivation and validation cohorts over median follow-up of 5.6 and 2.3 years, respectively. The final model included 5 out of 14 pre-specified predictors with the following hazard ratios; New York Heart Association class ≥II: 1.92 [95% confidence interval (CI) 1.28–2.90], cardiac medication 2.52 (95% CI 1.72–3.69), ≥1 reintervention after initial repair: 1.56 (95% CI 1.09–2.22), body mass index: 1.04 (95% CI 1.01–1.07), log2 N-terminal pro B-type natriuretic peptide (pmol/L): 1.48 (95% CI 1.32–1.65). At external validation, the model showed good discrimination (C-statistic 0.79, 95% CI 0.74–0.83) and excellent calibration (calibration-in-the-large = −0.002; calibration slope = 0.99). Conclusion These data support the validity and applicability of a parsimonious ACHD risk model based on five readily available clinical variables to accurately predict the 1-year risk of death, HF, or arrhythmia. This risk tool may help guide appropriate care for moderately or severely complex ACHD.

Background: Longitudinal bleeding risk scores have been validated in patients treated with dual antiplatelet therapy (DAPT) following percutaneous coronary intervention. How these scores apply to the population of patients with acute coronary syndrome (ACS) treated without revascularization remains unknown. The objective was to evaluate and compare the performances of the PRECISE-DAPT, PARIS, and DAPT (bleeding component) bleeding risk scores in the medically managed patients with ACS treated with DAPT. Methods and Results: TRILOGY ACS (Targeted Platelet Inhibition to Clarify the Optimal Strategy to Medically Manage Acute Coronary Syndromes) was a double-blind, placebo-controlled randomized trial conducted from 2008 to 2012 over a median follow-up of 17.0 months in 966 sites (52 countries). High-risk patients with unstable angina or non–ST-segment–elevation myocardial infarction who did not undergo revascularization were randomized to prasugrel or clopidogrel. The PRECISE-DAPT, PARIS, and DAPT (bleeding component) risk scores were applied in the TRILOGY ACS population to evaluate their performance to predict adjudicated non–coronary artery bypass grafting–related GUSTO (Global Use of Strategies to Open Occluded Coronary Arteries) severe/life-threatening/moderate and TIMI (Thrombolysis in Myocardial Infarction) major/minor bleeding with time-dependent c-indices. Among the 9326 participants, median age was 66 years (interquartile range, 59–74 years), and 3650 were females (39.1%). A total of 158 (1.69%) GUSTO severe/life-threatening/moderate and 174 (1.87%) TIMI major/minor non–coronary artery bypass grafting bleeding events occurred. The c-indices (95% CI) of the PRECISE-DAPT, PARIS, and DAPT (bleeding component) scores through 12 months were 0.716 (0.677–0.758), 0.693 (0.658–0.733), and 0.674 (0.637–0.713), respectively, for GUSTO bleeding and 0.624 (0.582–0.666), 0.612 (0.578–0.651), and 0.608 (0.571–0.649), respectively, for TIMI bleeding. There was no significant difference in the c-indices of each score based upon pairwise comparisons. Conclusions: Among medically managed patients with ACS treated with DAPT, the performances of the PRECISE-DAPT, PARIS, and DAPT (bleeding component) scores were reasonable and similar to their performances in the derivation percutaneous coronary intervention populations. Bleeding risk scores may be used to predict longitudinal bleeding risk in patients with ACS treated with DAPT without revascularization and help support shared decision making. Registration: URL: https://www.clinicaltrials.gov ; Unique identifier: NCT00699998.

PURPOSE: Based on recent advances in the management of patients with sentinel node (SN)-positive melanoma, we aimed to develop prediction models for recurrence, distant metastasis (DM) and overall mortality (OM).METHODS: The derivation cohort consisted of 1080 patients with SN-positive melanoma from nine European Organization for Research and Treatment of Cancer (EORTC) centres. Prognostic factors for recurrence, DM and OM were studied with Cox regression analysis. Significant factors were incorporated in the models. Performance was assessed by discrimination (c-index) and calibration in cross-validation across centres. The models were externally validated using a prospective cohort consisting of 705 German patients with SN-positive: 473 trial participants of the German Dermatologic Cooperative Oncology Group study (DeCOG-SLT) and 232 screened patients. A nomogram was developed for graphical presentation.RESULTS: The final model for recurrence and the calibrated models for DM and OM included ulceration, age, SN tumour burden and Breslow thickness. The models showed reasonable calibration. The c-index for the recurrence, DM and OM model was 0.68, 0.70 and 0.70, respectively, and 0.70, 0.72 and 0.74, respectively, in external validation. The EORTC-DeCOG model identified a robust low-risk group, with all identified low-risk patients (approximately 4% of the entire population) having a 5-year recurrence probability of CONCLUSIONS: The EORTC-DeCOG nomogram provides an adequate prognostic tool for patients with SN-positive melanoma, without the need for CLND. It showed consistent results across validation. The nomogram could be used for patient counselling and might aid in adjuvant therapy decision-making.

How did a percutaneous coronary intervention/coronary artery bypass grafting randomised trial generate the anatomic syntax score?

OBJECTIVES The SYNTAX score II (SSII) was developed from the SYNTAX trial to predict the 4-year all-cause mortality after left main or multivessel disease revascularization and to facilitate the decision-making process. The SSII provides the following treatment recommendations: (i) coronary artery bypass grafting (CABG) or percutaneous coronary intervention (PCI) (equipoise risk), (ii) CABG preferred (excessive risk for PCI) or (iii) PCI preferred (excessive risk for CABG). We sought to externally validate SSII and to investigate the impact of not abiding by the SSII recommendations in the randomized EXCEL trial of PCI versus CABG for left main disease. METHODS The calibration plot of predicted versus observed 4-year mortality was constructed from individual values of SSII in EXCEL. To assess overestimation versus underestimation of predicted mortality risk, an optimal fit regression line with slope and intercept was determined. Prospective treatment recommendations based on SSII were compared with actual treatments and all-cause mortality at 4 years. RESULTS SSII variables were available from EXCEL trial in 1807/1905 (95%) patients. For the entire cohort, discrimination was possibly helpful (C statistic = 0.670). SSII-predicted all-cause mortality at 4 years overestimated the observed mortality, particularly in the highest-risk percentiles, as confirmed by the fit regression line [intercept 2.37 (1.51–3.24), P = 0.003; slope 0.67 (0.61–0.74), P CONCLUSIONS In the EXCEL trial of patients with left main disease, the SSII-predicted 4-year mortality overestimated the 4-year observed mortality with a possibly helpful discrimination. Non-compliance with SSII CABG treatment recommendations (i.e. randomized to PCI) was associated with higher 4-year all-cause mortality.

Objective: To compare overall survival in patients with esophageal adenocarcinoma who underwent transhiatal esophagectomy (THE) with limited lymphadenectomy or transthoracic esophagectomy (TTE) with extended lymphadenectomy with or without neoadjuvant chemoradiotherapy (nCRT). Background: The application of neoadjuvant therapy might change the association between the extent of lymphadenectomy and survival in patients with esophageal adenocarcinoma. This may influence the choice of surgical approach in patients treated with nCRT. Methods: Patients with potentially curable subcarinal esophageal adenocarcinoma treated with surgery alone or nCRT followed by surgery in 7 centers were included. The effect of surgical approach on overall survival, differentiated by the addition or omission of nCRT, was analyzed using a multivariable Cox regression model that included well-known prognostic factors and factors that might have influenced the choice of surgical approach. Results: In total, 701 patients were included, of whom 318 had TTE with extended lymphadenectomy and 383 had THE with limited lymphadenectomy. TTE had differential effects on survival (P for interaction = 0.02), with a more favorable prognostic effect in patients who were treated with surgery alone [hazard ratio (HR) = 0.77, 95% confidence interval (CI) 0.58-1.03]. This association was statistically significant in a subgroup of patients with 1 to 8 positive lymph nodes in the resection specimen (HR = 0.62, 95% CI 0.43-0.90). The favorable prognostic effect of TTE over THE was absent in the nCRT and surgery group (HR = 1.16, 95% CI 0.80-1.66) and in the subgroup of nCRT patients with 1 to 8 positive lymph nodes in the resection specimen (HR = 1.00, 95% CI 0.61-1.68). Conclusions: Compared to surgery alone, the addition of nCRT may reduce the need for TTE with extended lymphadenectomy to improve long-term survival in patients with esophageal adenocarcinoma.

Heterogeneity of treatment effect (HTE) refers to the nonrandom variation in the magnitude or direction of a treatment effect across levels of a covariate, as measured on a selected scale, against a clinical outcome. In randomized controlled trials (RCTs), HTE is typically examined through a subgroup analysis that contrasts effects in groups of patients defined "1 variable at a time" (for example, male vs. female or old vs. young). The authors of this statement present guidance on an alternative approach to HTE analysis, "predictive HTE analysis." The goal of predictive HTE analysis is to provide patient-centered estimates of outcome risks with versus without the intervention, taking into account all relevant patient attributes simultaneously. The PATH (Predictive Approaches to Treatment effect Heterogeneity) Statement was developed using a multidisciplinary technical expert panel, targeted literature reviews, simulations to characterize potential problems with predictive approaches, and a deliberative process engaging the expert panel. The authors distinguish 2 categories of predictive HTE approaches: a "risk-modeling" approach, wherein a multivariable model predicts the risk for an outcome and is applied to disaggregate patients within RCTs to define risk-based variation in benefit, and an "effect-modeling" approach, wherein a model is developed on RCT data by incorporating a term for treatment assignment and interactions between treatment and baseline covariates. Both approaches can be used to predict differential absolute treatment effects, the most relevant scale for clinical decision making. The authors developed 4 sets of guidance: criteria to determine when risk-modeling approaches are likely to identify clinically important HTE, methodological aspects of risk-modeling methods, considerations for translation to clinical practice, and considerations and caveats in the use of effect-modeling approaches. The PATH Statement, together with its explanation and elaboration document, may guide future analyses and reporting of RCTs.

The PATH (Predictive Approaches to Treatment effect Heterogeneity) Statement was developed to promote the conduct of, and provide guidance for, predictive analyses of heterogeneous treatment effects (HTE) in clinical trials. The goal of predictive HTE analysis is to provide patient-centered estimates of outcome risks with versus without the intervention, taking into account all relevant patient attributes simultaneously, to support more personalized clinical decision making than can be made based only on an overall average treatment effect. We distinguished two categories of predictive HTE approaches (a “risk modeling” and an “effect modeling” approach) and developed four sets of guidance statements: 1) criteria to determine when risk modeling approaches are likely to identify clinically meaningful HTE; 2) methodological aspects of risk modeling methods; 3) considerations for translation to clinical practice; and 4) considerations and caveats in the use of effect modeling approaches. We also discuss limitations of these methods and enumerate research priorities for advancing methods designed to generate more personalized evidence. This explanation and elaboration document describes the intent and rationale of each recommendation; and discusses related analytic considerations, caveats, and reservations.

Background Pneumonectomy in lung cancer treatment is associated with considerable morbidity and mortality. Its use is reserved only for patients in whom a complete oncological resection by (sleeve) lobectomy is not possible. It is unclear whether a patients’ risk of receiving a pneumonectomy is equally distributed. This study examined between-hospital variation of pneumonectomy use for primary lung cancer in the Netherlands. Methods Data from the Dutch Lung Cancer Audit for Surgery from 2012 to 2016 were used to study the use of pneumonectomy for primary lung cancer in the Netherlands. Using multivariable logistic regression, factors associated with pneumonectomy use were identified and the expected number of pneumonectomies per hospital was determined. Subsequently, the observed/expected ratio (O/E ratio) per hospital was calculated to study between-hospital differences. Results Of the 8446 included patients, 659 (7.8%) underwent a pneumonectomy with a mean postoperative mortality of 7.1% (n = 47). Factors associated with receiving a pneumonectomy were age, gender, cardiac and pulmonary comorbidities, tumor side, size and histopathology. The pneumonectomy use in the Netherlands varied considerably between hospitals (IQR 5.5–10.1%). Three hospitals out of 51 performed significantly less pneumonectomies than expected (O/E ratio O/E ratio > 1.7). In the latter group, severe complications were more frequent, taking other influencing factors into account (OR 1.51, 95% CI 1.05–2.19). Conclusions There is a considerable between-hospital variation in pneumonectomy use in lung cancer treatment. To further optimize surgical lung cancer care, we suggest center-specific feedback on pneumonectomy use and the development of a risk-adjusted pneumonectomy indicator.

Background: Time-to-first-event analysis considers only the first event irrespective of its severity. There are several methods to assess trial outcomes beyond time-to-first-event analysis, such as analyzing total events and ranking outcomes. In the GLOBAL LEADERS study, time-to-first-event analysis did not show superiority of ticagrelor monotherapy following one-month dual antiplatelet therapy (DAPT) after percutaneous coronary intervention to conventional 12-month DAPT followed by aspirin monotherapy in the reduction of the primary composite end point of all-cause mortality or new Q-wave myocardial infarction. This study sought to explore various analytical approaches in assessing total ischemic and bleeding events after percutaneous coronary intervention in the GLOBAL LEADERS study. Methods and Results: Total ischemic and bleeding events were defined as all-cause mortality, any stroke, any myocardial infarction, any revascularization, or Bleeding Academic Research Consortium grade 2 or 3 bleeding. We used various analytical approaches to analyze the benefit of ticagrelor monotherapy over conventional DAPT. For ischemic and bleeding events at 2 years after percutaneous coronary intervention, ticagrelor monotherapy demonstrated a 6% risk reduction, compared with conventional 12-month DAPT in time-to-first-event analysis (hazard ratio, 0.94 [95% CI, 0.88–1.01]; log-rank P =0.10). In win ratio analysis, win ratio was 1.05 (95% CI, 0.97–1.13; P =0.20). Negative binomial regression and Andersen-Gill analyses which include repeated events showed statistically significant advantage for ticagrelor monotherapy (rate ratio, 0.92 [95% CI, 0.85–0.99; P =0.020] and hazard ratio, 0.92 [95% CI, 0.85–0.99; P =0.028], respectively), although in weighted composite end point analysis, the hazard ratio was 0.93 (95% CI, 0.84–1.04; log-rank P =0.22). Conclusions: Statistical analyses considering repeated events or event severity showed that ticagrelor monotherapy consistently reduced ischemic and bleeding events by 5% to 8%, compared with conventional 1-year DAPT. Applying multiple statistical methods could emphasize the multiple facets of a trial and result in accurate and more appropriate analyses. Considering the recurrence of ischemic and bleeding events, ticagrelor monotherapy appeared to be beneficial after percutaneous coronary intervention. Registration: URL: https://www.clinicaltrials.gov ; Unique identifier: NCT01813435.

The originally-proposed PRECISE-DAPT score is a 5-item risk score supporting decision-making for dual antiplatelet therapy1 duration after PCI. It is unknown if a simplified version of the score based on 4 factors (age, hemoglobin, creatinine clearance, prior bleeding), and lacking white-blood cell count, retains potential to guide DAPT duration. The 4-item PRECISE-DAPT was used to categorize 10,081 patients who were randomized to short (3-6 months) or long (12-24 months) DAPT regimen according to high (HBR defined by PRECISE-DAPT ≥25 points) or non-high bleeding risk (PRECISE-DAPT

Aims: The aim of this study was the external validation of the updated logistic clinical SYNTAX score for two-year all-cause mortality after PCI in the GLOBAL LEADERS trial. Methods and results: The GLOBAL LEADERS trial was an investigator-initiated, prospective randomised, multicentre, open-label trial comparing two strategies of antiplatelet therapy in 15,991 patients undergoing PCI. As a predefined analysis, we studied the first 4,006 consecutive patients enrolled between July 2013 and April 2014 for whom the anatomic SYNTAX scores were calculated by an independent core lab. The updated logistic clinical SYNTAX score was available in 3,271 patients. Patients were divided into quintiles according to the score. The C-statistic of the updated logistic clinical SYNTAX score for twoyear all-cause mortality was 0.71 (95% confidence interval [CI]: 0.64-0.77). The updated logistic clinical SYNTAX score identified patients at very high risk for two-year all-cause mortality after PCI. Although it systematically overestimated two-year all-cause mortality, predicted and observed two-year all-cause mortality in the majority of the patients (four out of five quintiles) were in agreement. Conclusions: Overall discrimination for two-year all-cause mortality of the updated logistic clinical SYNTAX score is either borderline acceptable or possibly helpful. Calibration in the majority of patients is appropriate. The score is potentially useful in selecting enriched high-risk populations.

Background ACEF score has been shown to have predictive ability in the patients undergoing percutaneous coronary intervention (PCI). The ACEF II score has recently been developed to predict short-term mortality after cardiac surgery. We compared the predictive ability of the ACEF and ACEF II scores to predict mortality after PCI in the all-comers population. Methods The ACEF and ACEF II scores were calculated in 15,968 patients enrolled in the GLOBAL LEADERS study. Discrimination and calibration were assessed for outcomes after PCI. Recalibration of the regression model by updating the intercept and slope were performed to adjust the original ACEF model to the PCI setting. In a stratified approach, patients were divided into quintiles according to the score. Outcomes were compared between quintiles. Results The ACEF and ACEF II score were available in 14,941 and 14,355 patients respectively. Discrimination for 30-day all-cause mortality was acceptable for both scores (C-statistic ACEF 0.75 and ACEF II 0.77). For 2-year all-cause mortality, the discrimination of ACEF score was acceptable (C-statistic 0.72) while the discrimination of ACEF II score was moderate (C-statistic 0.69). Both scores identified patients at high risk of mortality but overestimated all-cause mortality at 30 days in all quintiles. After recalibration, agreement between predicted and observed 30-day all-cause mortality in both scores are close to the identity line. Conclusions The ACEF II model did not improve the predictive ability of the ACEF score. Recalibrated ACEF model can be used to estimated all-cause mortality rate at 30 days after PCI.

Background Discriminative ability is an important aspect of prediction model performance, but challenging to assess in clustered (e.g., multicenter) data. Concordance (c)-indexes may be too extreme within small clusters. We aimed to define a new approach for the assessment of discriminative ability in clustered data. Methods We assessed discriminative ability of a prediction model for the binary outcome mortality after traumatic brain injury within centers of the CRASH trial. With multilevel logistic regression analysis, we estimated cluster-specific calibration slopes which we used to obtain the recently proposed calibrated model-based concordance (c-mbc) within each cluster. We compared the c-mbc with the naïve c-index in centers of the CRASH trial and in simulations of clusters with varying calibration slopes. Results The c-mbc was less extreme in distribution than the c-index in 19 European centers (internal validation; n = 1716) and 36 non-European centers (external validation; n = 3135) of the CRASH trial. In simulations, the c-mbc was biased but less variable than the naïve c-index, resulting in lower root mean squared errors. Conclusions The c-mbc, based on multilevel regression analysis of the calibration slope, is an attractive alternative to the c-index as a measure of discriminative ability in multicenter studies with patient clusters of limited sample size. Electronic supplementary material The online version of this article (10.1186/s41512-019-0055-8) contains supplementary material, which is available to authorized users.

Background Dual antiplatelet therapy (DAPT) with aspirin plus a P2Y12 inhibitor prevents ischaemic events after coronary stenting, but increases bleeding. Guidelines support weighting bleeding risk before the selection of treatment duration, but no standardised tool exists for this purpose. Methods A total of 14 963 patients treated with DAPT after coronary stenting—largely consisting of aspirin and clopidogrel and without indication to oral anticoagulation—were pooled at a single-patient level from eight multicentre randomised clinical trials with independent adjudication of events. Using Cox proportional hazards regression, we identified predictors of out-of-hospital Thrombosis in Myocardial Infarction (TIMI) major or minor bleeding stratified by trial, and developed a numerical bleeding risk score. The predictive performance of the novel score was assessed in the derivation cohort and validated in patients treated with percutaneous coronary intervention from the PLATelet inhibition and patient Outcomes (PLATO) trial (n=8595) and BernPCI registry (n=6172). The novel score was assessed within patients randomised to different DAPT durations (n=10 081) to identify the effect on bleeding and ischaemia of a long (12–24 months) or short (3–6 months) treatment in relation to baseline bleeding risk. Findings The PRECISE-DAPT score (age, creatinine clearance, haemoglobin, white-blood-cell count, and previous spontaneous bleeding) showed a c-index for out-of-hospital TIMI major or minor bleeding of 0·73 (95% CI 0·61–0·85) in the derivation cohort, and 0·70 (0·65–0·74) in the PLATO trial validation cohort and 0·66 (0·61–0·71) in the BernPCI registry validation cohort. A longer DAPT duration significantly increased bleeding in patients at high risk (score ≥25), but not in those with lower risk profiles (pinteraction=0·007), and exerted a significant ischaemic benefit only in this latter group. Interpretation The PRECISE-DAPT score is a simple five-item risk score, which provides a standardised tool for the prediction of out-of-hospital bleeding during DAPT. In the context of a comprehensive clinical evaluation process, this tool can support clinical decision making for treatment duration. Funding None.

Background: In 1 of 3 patients with initial lymph node-positive (cN(+)) breast cancer, neoadjuvant chemotherapy (NAC) results in an axillary pathologic complete response (ax-pCR). This urges the need for a less-invasive axillary staging method. Recently introduced less-invasive procedures have been insufficient in accurately identifying ax-pCR. Therefore, we propose a novel less-invasive axillary staging procedure: the Radioactive Iodine Seed localization in the Axilla with the Sentinel node procedure (RISAS), a combination of the procedure of marking axillary lymph nodes with radioactive iodine seeds (MARI) and sentinel lymph node biopsy (SLNB). Patients and Methods: In the present open single-arm multicenter validation study, 225 cN(+) (biopsy-proven) patients will undergo the RISAS procedure, in which a positive lymph node is marked by an iodine-125 seed before NAC. After NAC completion, this iodine-125 seed-marked lymph node is removed, together with any additional sentinel lymph nodes. The RISAS procedure is subsequently followed by completion axillary lymph node dissection (ALND). The RISAS lymph nodes will be compared with the lymph nodes from the completion ALND specimen. The primary endpoint is accuracy of the RISAS procedure. The identification rate, false-negative rate, negative predictive value, and possible concordance between the MARI and SLNB will be reported. Conclusion: The present prospective multicenter RISAS trial will enable us to validate the combination of MARI and SLNB for assessing the axillary response to NAC in cN(+) patients. If RISAS proves to be an accurate axillary staging procedure, ALND could safely be abandoned in the case of ax-pCR confirmed using the RISAS procedure.

Background. The benefits and costs of a treatment are typically heterogeneous across individual patients. Randomized clinical trials permit the examination of individualized treatment benefits over the trial horizon but extrapolation to lifetime horizon usually involves combining trial-based individualized estimates of short-term risk reduction with less detailed (less granular) population life tables. However, the underlying assumption of equal post-trial life expectancy for low- and high-risk patients of the same sex and age is unrealistic. We aimed to study the influence of unequal granularity between models of short-term risk reduction and life expectancy on individualized estimates of cost-effectiveness of aggressive thrombolysis for patients with an acute myocardial infarction. Methods. To estimate life years gained, we multiplied individualized estimates of short-term risk reduction either with less granular and with equally granular post-trial life expectancy estimates. Estimates of short-term risk reduction were obtained from GUSTO trial data (30,510 patients) using logistic regression analysis with treatment, sex, and age as predictor variables. Life expectancy estimates were derived from sex- and age-specific US life tables. Results. Based on sex- and age-specific, short-term risk reductions but average population life expectancy (less granularity), we found that aggressive thrombolysis was cost-effective (incremental cost-effectiveness ratio below $50,000) for women above age 49 y and men above age 53 y (92% and 69% of the population, respectively). Considering sex- and age-specific short-term mortality risk reduction and correspondingly sex- and age-specific life expectancy (equal granularity), aggressive thrombolysis was cost-effective for men above age 45 y and women above age 50 y (95% and 76% of the population, respectively). Conclusions. Failure to model short-term risk reduction and life expectancy at an equal level of granularity may bias our estimates of individualized cost-effectiveness and misallocate resources.

Objective Validation of clinical prediction models traditionally refers to the assessment of model performance in new patients. We studied different approaches to geographic and temporal validation in the setting of multicenter data from two time periods. Study Design and Setting We illustrated different analytic methods for validation using a sample of 14,857 patients hospitalized with heart failure at 90 hospitals in two distinct time periods. Bootstrap resampling was used to assess internal validity. Meta-analytic methods were used to assess geographic transportability. Each hospital was used once as a validation sample, with the remaining hospitals used for model derivation. Hospital-specific estimates of discrimination (c-statistic) and calibration (calibration intercepts and slopes) were pooled using random-effects meta-analysis methods. I 2 statistics and prediction interval width quantified geographic transportability. Temporal transportability was assessed using patients from the earlier period for model derivation and patients from the later period for model validation. Results Estimates of reproducibility, pooled hospital-specific performance, and temporal transportability were on average very similar, with c-statistics of 0.75. Between-hospital variation was moderate according to I 2 statistics and prediction intervals for c-statistics. Conclusion This study illustrates how performance of prediction models can be assessed in settings with multicenter data at different time periods.

Objectives The study sought to validate the SYNTAX (Synergy Between Percutaneous Coronary Intervention With Taxus and Cardiac Surgery) score II mortality prediction model after percutaneous coronary intervention (PCI) or coronary artery bypass grafting in a large pooled population of patients with multivessel coronary disease (MVD) and/or unprotected left main disease (UPLMD) enrolled in the PRECOMBAT (Bypass Surgery Versus Angioplasty Using Sirolimus-Eluting Stent in Patients With Left Main Coronary Artery Disease) and BEST (Artery Bypass Surgery and Everolimus-Eluting Stent Implantation in the Treatment of Patients with Multivessel Coronary Artery Disease) randomized controlled trials. Background For patients with MVD and/or UPLMD, the choice of the best revascularization strategy remains challenging. Methods Pooled individual patient-level data from PRECOMBAT and BEST were used to assess calibration and discrimination of the SYNTAX score II prediction model for all-cause mortality after PCI and coronary artery bypass grafting at 4-year follow-up. The study population comprised 1,480 patients (600 with UPLMD, 880 with MVD). Results The overall incidence of all-cause mortality was 6.1% after a median follow-up period of 4.9 years. Validation plots showed good model calibration overall and across treatment groups but tended to overestimate all-cause mortality in the highest risk quintiles of patients in the whole population and the PCI arm. The SYNTAX score II showed moderate discrimination ability for the whole population (C index = 0.685) but better for patients receiving PCI than CABG (C index = 0.718 vs. 0.662 in patients with UPLMD, C index = 0.700 vs. 0.661 in those with MVD). Observed all-cause mortality was higher when the treatment received was at variance with that recommended by the model and similar when it was concordant. Conclusions The SYNTAX score II has good calibration but only moderate discrimination ability for long-term mortality prediction in this randomized population. This score provides an important tool to help guide the heart team’s decision-making process regarding the selection of the best revascularization strategy for patients with MVD and/or UPLMD. (Bypass Surgery Versus Angioplasty Using Sirolimus-Eluting Stent in Patients With Left Main Coronary Artery Disease, NCT00422968; Bypass Surgery Versus Everolimus-Eluting Stent Implantation for Multivessel Coronary Artery Disease, NCT00997828)

Concordance measures are frequently used for assessing the discriminative ability of risk prediction models. The interpretation of estimated concordance at external validation is difficult if the case-mix differs from the model development setting. We aimed to develop a concordance measure that provides insight into the influence of case-mix heterogeneity and is robust to censoring of time-to-event data. We first derived a model-based concordance (mbc) measure that allows for quantification of the influence of case-mix heterogeneity on discriminative ability of proportional hazards and logistic regression models. This mbc can also be calculated including a regression slope that calibrates the predictions at external validation (c-mbc), hence assessing the influence of overall regression coefficient validity on discriminative ability. We derived variance formulas for both mbc and c-mbc. We compared the mbc and the c-mbc with commonly used concordance measures in a simulation study and in two external validation settings. The mbc was asymptotically equivalent to a previously proposed resampling-based case-mix corrected c-index. The c-mbc remained stable at the true value with increasing proportions of censoring, while Harrell's c-index and to a lesser extent Uno's concordance measure increased unfavorably. Variance estimates of mbc and c-mbc were well in agreement with the simulated empirical variances. We conclude that the mbc is an attractive closed-form measure that allows for a straightforward quantification of the expected change in a model's discriminative ability due to case-mix heterogeneity. The c-mbc also reflects regression coefficient validity and is a censoring-robust alternative for the c-index when the proportional hazards assumption holds. Copyright © 2016 John Wiley & Sons, Ltd.

The performance of previously developed models to predict non sentinel lymph node status for patients with sentinel lymph node-positive breast cancer was poor for our Dutch patient population. Therefore, a new model was developed that includes only 2 predictors, is easy to use, and can help clinicians and their patients in the clinical decision-making regarding axillary treatment. Background: Multiple predictive systems have previously been developed to identify the sentinel lymph node (SLN)-positive patients at low risk of additional axillary non-SLN involvement and for whom completion axillary lymph node dissection (ALND) could be avoided. However, previous studies showed that these tools had poor performance in Dutch patients with breast cancer, probably owing to variations in pathology settings and differences in population characteristics. The aim of the present study was to develop a predictive tool for the risk of non-SLN involvement in a Dutch population with SLN-positive breast cancer. Materials and Methods: The data from 513 patients with SLN-positive breast cancer at 10 participating hospitals, who had undergone ALND from January 2007 to December 2008 were studied. The uni- and multivariable associations of predictors for non-SLN metastases were analyzed, and a predictive model was developed. The discriminatory ability of the model was measured by the area under the receiver operating characteristic curve (AUC) and the agreement between predicted probabilities and observed frequencies was visualized by a calibration plot. Results: A predictive model was developed that included the 2 strongest predictors: the size of the SLN metastases in millimeters and the presence of a negative sentinel lymph node. The model showed good discriminative ability (AUC, 0.75) and good calibration over the complete range of predicted probabilities. Conclusion: We have developed a tool to predict additional non-SLN metastases in Dutch patients with SLN-positive breast cancer that is easy to use in daily clinical breast cancer practice. (C) 2016 Elsevier Inc. All rights reserved.

Background Complex percutaneous coronary intervention (PCI) is associated with higher ischemic risk, which can be mitigated by long-term dual antiplatelet therapy (DAPT). However, concomitant high bleeding risk (HBR) may be present, making it unclear whether short- or long-term DAPT should be prioritized. Objectives This study investigated the effects of ischemic (by PCI complexity) and bleeding (by PRECISE-DAPT [PREdicting bleeding Complications in patients undergoing stent Implantation and SubsequEnt Dual AntiPlatelet Therapy] score) risks on clinical outcomes and on the impact of DAPT duration after coronary stenting. Methods Complex PCI was defined as ≥3 stents implanted and/or ≥3 lesions treated, bifurcation stenting and/or stent length >60 mm, and/or chronic total occlusion revascularization. Ischemic and bleeding outcomes in high (≥25) or non-high ( Results Among 14,963 patients from 8 randomized trials, 3,118 underwent complex PCI and experienced a higher rate of ischemic, but not bleeding, events. Long-term DAPT in non-HBR patients reduced ischemic events in both complex (absolute risk difference: −3.86%; 95% confidence interval: −7.71 to +0.06) and noncomplex PCI strata (absolute risk difference: −1.14%; 95% confidence interval: −2.26 to −0.02), but not among HBR patients, regardless of complex PCI features. The bleeding risk according to the Thrombolysis In Myocardial Infarction scale was increased by long-term DAPT only in HBR patients, regardless of PCI complexity. Conclusions Patients who underwent complex PCI had a higher risk of ischemic events, but benefitted from long-term DAPT only if HBR features were not present. These data suggested that when concordant, bleeding, more than ischemic risk, should inform decision-making on the duration of DAPT.

The use of evidence from clinical trials to support decisions for individual patients is a form of “reference class forecasting”: implicit predictions for an individual are made on the basis of outcomes in a reference class of “similar” patients treated with alternative therapies. Evidence based medicine has generally emphasized the broad reference class of patients qualifying for a trial. Yet patients in a trial (and in clinical practice) differ from one another in many ways that can affect the outcome of interest and the potential for benefit. The central goal of personalized medicine, in its various forms, is to narrow the reference class to yield more patient specific effect estimates to support more individualized clinical decision making. This article will review fundamental conceptual problems with the prediction of outcome risk and heterogeneity of treatment effect (HTE), as well as the limitations of conventional (one-variable-at-a-time) subgroup analysis. It will also discuss several regression based approaches to “predictive” heterogeneity of treatment effect analysis, including analyses based on “risk modeling” (such as stratifying trial populations by their risk of the primary outcome or their risk of serious treatment-related harms) and analysis based on “effect modeling” (which incorporates modifiers of relative effect). It will illustrate these approaches with clinical examples and discuss their respective strengths and vulnerabilities.

Objectives: We aimed to compare the performance of different regression modeling approaches for the prediction of heterogeneous treatment effects. Study Design and Setting: We simulated trial samples (n = 3,600; 80% power for a treatment odds ratio of 0.8) from a superpopulation (N = 1,000,000) with 12 binary risk predictors, both without and with six true treatment interactions. We assessed predictions of treatment benefit for four regression models: a “risk model” (with a constant effect of treatment assignment) and three “effect models” (including interactions of risk predictors with treatment assignment). Three novel performance measures were evaluated: calibration for benefit (i.e., observed vs. predicted risk difference in treated vs. untreated), discrimination for benefit, and prediction error for benefit. Results: The risk modeling approach was well-calibrated for benefit, whereas effect models were consistently overfit, even with doubled sample sizes. Penalized regression reduced miscalibration of the effect models considerably. In terms of discrimination and prediction error, the risk modeling approach was superior in the absence of true treatment effect interactions, whereas penalized regression was optimal in the presence of true treatment interactions. Conclusion: A risk modeling approach yields models consistently well calibrated for benefit. Effect modeling may improve discrimination for benefit in the presence of true interactions but is prone to overfitting. Hence, effect models—including only plausible interactions—should be fitted using penalized regression.

Introduction: Most heart failure (HF) clinical prediction models (CPMs] have not been independently externally validated. We sought to test the performance of HF models in a diverse population using a systematic approach. Methods: A systematic review identified CPMs predicting outcomes for patients with HF. Individual patient data from 5 large publicaly available clinical trials enrolling patients with chronic HF were matched to published CPMs based on similarity in populations and available outcome and predictor variables in the clinical trial databases. CPM performance was evaluated for discrimination (c-statistic, % relative change in c-statistic) and calibration (Harrell’s E and E 90 , the mean and the 90% quantile of the error distribution from the smoothed loess observed value) for the original and recalibrated models. Results: Out of 135 HF CPMs reviewed, we identified 45 CPM-trial pairs including 13 unique CPMs. The outcome was mortality for all of the models with a trial match. During external validations, median c-statistic was 0.595 (IQR 0.563 to 0.630) with a median relative decrease in the c-statistic of -57 % (IQR, -49% to -71%) compared to the c-statistic reported in the derivation cohort. Overall, the median Harrell’s E was 0.09 (IQR, 0.04 to 0.135) and E 90 was 0.11 (IQR, 0.07 to 0.21). Recalibration of the intercept and slope led to substantially improved calibration with median change in Harrell’s E of -35% [IQR 0 to -75%] for the intercept and -56% [IQR -17% to -75%] for the intercept and slope. Refitting model covariates improved the median c-statistic by 38% to 0.629 [IQR 0.613 to 0.649]. Conclusion: For HF CPMs, independent external validations demonstrate that CPMs perform significantly worse than originally presented; however with significant heterogeneity. Recalibration of the intercept and slope improved model calibration. These results underscore the need to carefully consider the derivation cohort characteristics when using published CPMs.