Search

The SAVVY project aims to improve the analyses of adverse events (AEs) in clinical trials through the use of survival techniques appropriately dealing with varying follow-up times and competing events (CEs). This paper summarizes key features and conclusions from the various SAVVY papers. Through theoretical investigations using simulations and in an empirical study including randomized clinical trials from several sponsor organisations, biases from ignoring varying follow-up times or CEs are investigated. The bias of commonly used estimators of the absolute and relative AE risk is quantified. Furthermore, we provide a cursory assessment of how pertinent guidelines for the analysis of safety data deal with the features of varying follow-up time and CEs. SAVVY finds that for both, avoiding bias and categorization of evidence with respect to treatment effect on AE risk into categories, the choice of the estimator is key and more important than features of the underlying data such as percentage of censoring, CEs, amount of follow-up, or value of the gold-standard. The choice of the estimator of the cumulative AE probability and the definition of CEs are crucial. SAVVY recommends using the Aalen-Johansen estimator (AJE) with an appropriate definition of CEs whenever the risk for AEs is to be quantified. There is an urgent need to improve the guidelines of reporting AEs so that incidence proportions or one minus Kaplan-Meier estimators are finally replaced by the AJE with appropriate definition of CEs., Comment: 17 pages, 1 Figure, 4 Tables. arXiv admin note: text overlap with arXiv:2008.07881

As a reaction to the pandemic of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), a multitude of clinical trials for the treatment of SARS-CoV-2 or the resulting corona disease (COVID-19) are globally at various stages from planning to completion. Although some attempts were made to standardize study designs, this was hindered by the ferocity of the pandemic and the need to set up trials quickly. We take the view that a successful treatment of COVID-19 patients (i) increases the probability of a recovery or improvement within a certain time interval, say 28 days; (ii) aims to expedite favourable events within this time frame; and (iii) does not increase mortality over this time period. On this background we discuss the choice of endpoint and its analysis. Furthermore, we consider consequences of this choice for other design aspects including sample size and power and provide some guidance on the application of adaptive designs in this particular context.

Analyses of adverse events (AEs) are an important aspect of the evaluation of experimental therapies. The SAVVY (Survival analysis for AdVerse events with Varying follow-up times) project aims to improve the analyses of AE data in clinical trials through the use of survival techniques appropriately dealing with varying follow-up times, censoring, and competing events (CE). In an empirical study including seventeen randomized clinical trials the effect of varying follow-up times, censoring, and competing events on comparisons of two treatment arms with respect to AE risks is investigated. The comparisons of relative risks (RR) of standard probability-based estimators to the gold-standard Aalen-Johansen estimator or hazard-based estimators to an estimated hazard ratio (HR) from Cox regression are done descriptively, with graphical displays, and using a random effects meta-analysis on AE level. The influence of different factors on the size of the bias is investigated in a meta-regression. We find that for both, avoiding bias and categorization of evidence with respect to treatment effect on AE risk into categories, the choice of the estimator is key and more important than features of the underlying data such as percentage of censoring, CEs, amount of follow-up, or value of the gold-standard RR. There is an urgent need to improve the guidelines of reporting AEs so that incidence proportions are finally replaced by the Aalen-Johansen estimator - rather than by Kaplan-Meier - with appropriate definition of CEs. For RRs based on hazards, the HR based on Cox regression has better properties than the ratio of incidence densities.

The SAVVY project aims to improve the analyses of adverse event (AE) data in clinical trials through the use of survival techniques appropriately dealing with varying follow-up times and competing events (CEs). Although statistical methodologies have advanced, in AE analyses often the incidence proportion, the incidence density, or a non-parametric Kaplan-Meier estimator (KME) are used, which either ignore censoring or CEs. In an empirical study including randomized clinical trials from several sponsor organisations, these potential sources of bias are investigated. The main aim is to compare the estimators that are typically used in AE analysis to the Aalen-Johansen estimator (AJE) as the gold-standard. Here, one-sample findings are reported, while a companion paper considers consequences when comparing treatment groups. Estimators are compared with descriptive statistics, graphical displays and with a random effects meta-analysis. The influence of different factors on the size of the bias is investigated in a meta-regression. Comparisons are conducted at the maximum follow-up time and at earlier evaluation time points. CEs definition does not only include death before AE but also end of follow-up for AEs due to events possibly related to the disease course or the treatment. Ten sponsor organisations provided 17 trials including 186 types of AEs. The one minus KME was on average about 1.2-fold larger than the AJE. Leading forces influencing bias were the amount of censoring and of CEs. As a consequence, the average bias using the incidence proportion was less than 5%. Assuming constant hazards using incidence densities was hardly an issue provided that CEs were accounted for. There is a need to improve the guidelines of reporting risks of AEs so that the KME and the incidence proportion are replaced by the AJE with an appropriate definition of CEs.

Safety analyses in terms of adverse events (AEs) are an important aspect of benefit-risk assessments of therapies. Compared to efficacy analyses AE analyses are often rather simplistic. The probability of an AE of a specific type is typically estimated by the incidence proportion, sometimes the incidence density or the Kaplan-Meier estimator are proposed. But these analyses either do not account for censoring, rely on a too restrictive parametric model, or ignore competing events. With the non-parametric Aalen-Johansen estimator as the gold-standard, these potential sources of bias are investigated in a data example from oncology and in simulations, both in the one-sample and in the two-sample case. As the estimators may have large variances at the end of follow-up, the estimators are not only compared at the maximal event time but also at two quantiles of the observed times. To date, consequences for safety comparisons have hardly been investigated in the literature. The impact of using different estimators for group comparisons is unclear, as, for example, the ratio of two both underestimating or overestimating estimators may or may not be comparable to the ratio of the gold-standard estimator. Therefore, the ratio of the AE probabilities is also calculated based on different approaches. By simulations investigating constant and non-constant hazards, different censoring mechanisms and event frequencies, we show that ignoring competing events is more of a problem than falsely assuming constant hazards by use of the incidence density and that the choice of the AE probability estimator is crucial for group comparisons., Comment: 27 pages, 5 figures

The assessment of safety is an important aspect of the evaluation of new therapies in clinical trials, with analyses of adverse events being an essential part of this. Standard methods for the analysis of adverse events such as the incidence proportion, i.e. the number of patients with a specific adverse event out of all patients in the treatment groups, do not account for both varying follow-up times and competing risks. Alternative approaches such as the Aalen-Johansen estimator of the cumulative incidence function have been suggested. Theoretical arguments and numerical evaluations support the application of these more advanced methodology, but as yet there is to our knowledge only insufficient empirical evidence whether these methods would lead to different conclusions in safety evaluations. The Survival analysis for AdVerse events with VarYing follow-up times (SAVVY) project strives to close this gap in evidence by conducting a meta-analytical study to assess the impact of the methodology on the conclusion of the safety assessment empirically. Here we present the rationale and statistical concept of the empirical study conducted as part of the SAVVY project. The statistical methods are presented in unified notation and examples of their implementation in R and SAS are provided.

The analysis of adverse events (AEs) is a key component in the assessment of a drug's safety profile. Inappropriate analysis methods may result in misleading conclusions about a therapy's safety and consequently its benefit-risk ratio. The statistical analysis of AEs is complicated by the fact that the follow-up times can vary between the patients included in a clinical trial. This paper takes as its focus the analysis of AE data in the presence of varying follow-up times within the benefit assessment of therapeutic interventions. Instead of approaching this issue directly and solely from an analysis point of view, we first discuss what should be estimated in the context of safety data, leading to the concept of estimands. Although the current discussion on estimands is mainly related to efficacy evaluation, the concept is applicable to safety endpoints as well. Within the framework of estimands, we present statistical methods for analysing AEs with the focus being on the time to the occurrence of the first AE of a specific type. We give recommendations which estimators should be used for the estimands described. Furthermore, we state practical implications of the analysis of AEs in clinical trials and give an overview of examples across different indications. We also provide a review of current practices of health technology assessment (HTA) agencies with respect to the evaluation of safety data. Finally, we describe problems with meta-analyses of AE data and sketch possible solutions.