Search

Postmarket safety surveillance is an integral part of mass vaccination programs. Typically relying on sequential analysis of real-world health data as they accrue, safety surveillance is challenged by sequential multiple testing and by biases induced by residual confounding in observational data. The current standard approach based on the maximized sequential probability ratio test (MaxSPRT) fails to satisfactorily address these practical challenges and it remains a rigid framework that requires prespecification of the surveillance schedule. We develop an alternative Bayesian surveillance procedure that addresses both aforementioned challenges using a more flexible framework. To mitigate bias, we jointly analyze a large set of negative control outcomes that are adverse events with no known association with the vaccines in order to inform an empirical bias distribution, which we then incorporate into estimating the effect of vaccine exposure on the adverse event of interest through a Bayesian hierarchical model. To address multiple testing and improve on flexibility, at each analysis timepoint, we update a posterior probability in favor of the alternative hypothesis that vaccination induces higher risks of adverse events, and then use it for sequential detection of safety signals. Through an empirical evaluation using six US observational healthcare databases covering more than 360 million patients, we benchmark the proposed procedure against MaxSPRT on testing errors and estimation accuracy, under two epidemiological designs, the historical comparator and the self-controlled case series. We demonstrate that our procedure substantially reduces Type 1 error rates, maintains high statistical power and fast signal detection, and provides considerably more accurate estimation than MaxSPRT. Given the extensiveness of the empirical study which yields more than 7 million sets of results, we present all results in a public R ShinyApp. As an effort to promote open science, we provide full implementation of our method in the open-source R package EvidenceSynthesis., (© 2023 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd.)

Background and Objective: Spontaneous reporting systems (SRSs) remain the cornerstone of post-marketing drug safety surveillance despite their well-known limitations. Judicious use of other available data sources is essential to enable better detection, strengthening and validation of signals. In this study, we investigated the potential of electronic healthcare records (EHRs) to be used alongside an SRS as an independent system, with the aim of improving signal detection., Methods: A signal detection strategy, focused on a limited set of adverse events deemed important in pharmacovigilance, was performed retrospectively in two data sources-(1) the Exploring and Understanding Adverse Drug Reactions (EU-ADR) database network and (2) the EudraVigilance database-using data between 2000 and 2010. Five events were considered for analysis: (1) acute myocardial infarction (AMI); (2) bullous eruption; (3) hip fracture; (4) acute pancreatitis; and (5) upper gastrointestinal bleeding (UGIB). Potential signals identified in each system were verified using the current published literature. The complementarity of the two systems to detect signals was expressed as the percentage of the unilaterally identified signals out of the total number of confirmed signals. As a proxy for the associated costs, the number of signals that needed to be reviewed to detect one true signal (number needed to detect [NND]) was calculated. The relationship between the background frequency of the events and the capability of each system to detect signals was also investigated., Results: The contribution of each system to signal detection appeared to be correlated with the background incidence of the events, being directly proportional to the incidence in EU-ADR and inversely proportional in EudraVigilance. EudraVigilance was particularly valuable in identifying bullous eruption and acute pancreatitis (71 and 42 % of signals were correctly identified from the total pool of known associations, respectively), while EU-ADR was most useful in identifying hip fractures (60 %). Both systems contributed reasonably well to identification of signals related to UGIB (45 % in EudraVigilance, 40 % in EU-ADR) but only fairly for signals related to AMI (25 % in EU-ADR, 20 % in EudraVigilance). The costs associated with detection of signals were variable across events; however, it was often more costly to detect safety signals in EU-ADR than in EudraVigilance (median NNDs: 7 versus 5)., Conclusion: An EHR-based system may have additional value for signal detection, alongside already established systems, especially in the presence of adverse events with a high background incidence. While the SRS appeared to be more cost effective overall, for some events the costs associated with signal detection in the EHR might be justifiable.

Aim: Electronic healthcare record (EHR)-based surveillance systems are increasingly being developed to support early detection of safety signals. It is unknown what the power of such a system is for surveillance among children and adolescents. In this paper we provide estimates of the number and classes of drugs, and incidence rates (IRs) of events, that can be monitored in children and adolescents (0-18 years)., Methods: Data were obtained from seven population-based EHR databases in Denmark, Italy, and the Netherlands during the period 1996-2010. We estimated the number of drugs for which specific adverse events can be monitored as a function of actual drug use, minimally detectable relative risk (RR) and IRs for 10 events., Results: The population comprised 4 838 146 individuals (25 575 132 person years (PYs)), who were prescribed 2170 drugs (1 610 631 PYs drug-exposure). Half of the total drug-exposure in PYs was covered by only 18 drugs (0.8%). For a relatively frequent event like upper gastrointestinal bleeding there were 39 drugs for which an association with a RR ≥4, if present, could be investigated. The corresponding number of drugs was eight for a rare event like anaphylactic shock., Conclusion: Drug use in children is rare and shows little variation. The number of drugs with enough exposure to detect rare adverse events in children and adolescents within an EHR-based surveillance system such as EU-ADR is limited. Use of additional sources of paediatric drug exposure information and global collaboration are imperative in order to optimize EHR data for paediatric safety surveillance., (© 2015 The British Pharmacological Society.)

Background: Electronic reporting and processing of suspected adverse drug reactions (ADRs) is increasing and has facilitated automated screening procedures. It is crucial for healthcare professionals to understand the nature and proper use of data available in pharmacovigilance practice., Objectives: To (a) compare performance of EU-ADR [electronic healthcare record (EHR) exemplar] and FAERS [spontaneous reporting system (SRS) exemplar] databases in detecting signals using "positive" and "negative" drug-event reference sets; and (b) evaluate the impact of timing bias on sensitivity thresholds by comparing all data to data restricted to the time before a warning/regulatory action., Methods: Ten events with known positive and negative reference sets were selected. Signals were identified when respective statistics exceeded defined thresholds. Main outcome measure Performance metrics, including sensitivity, specificity, positive predictive value and accuracy were calculated. In addition, the effect of regulatory action on the performance of signal detection in each data source was evaluated., Results: The sensitivity for detecting signals in EHR data varied depending on the nature of the adverse events and increased substantially if the analyses were restricted to the period preceding the first regulatory action. Across all events, using data from all years, a sensitivity of 45-73 % was observed for EU-ADR and 77 % for FAERS. The specificity was high and similar for EU-ADR (82-96 %) and FAERS (98 %). EU-ADR data showed range of PPV (78-91 %) and accuracy (78-72 %) and FAERS data yielded a PPV of 97 % with 88 % accuracy., Conclusion: Using all cumulative data, signal detection in SRS data achieved higher specificity and sensitivity than EHR data. However, when data were restricted to time prior to a regulatory action, performance characteristics changed in a manner consistent with both the type of data and nature of the ADR. Further research focusing on prospective validation of is necessary to learn more about the performance and utility of these databases in modern pharmacovigilance practice.

Background: Data mining in spontaneous reporting databases has shown that drug-induced liver injury is infrequently reported in children., Objectives: Our objectives were to (i) identify drugs potentially associated with acute liver injury (ALI) in children and adolescents using electronic healthcare record (EHR) data; and (ii) to evaluate the significance and novelty of these associations., Methods: We identified potential cases of ALI during exposure to any prescribed/dispensed drug for individuals <18 years old from the EU-ADR network, which includes seven databases from three countries, covering the years 1996-2010. Several new methods for signal detection were applied to identify all statistically significant associations between drugs and ALI. A drug was considered statistically significantly associated with ALI, using all other time as a reference category, if the 95% CI lower band of the relative risk was >1 and in the presence of at least three exposed cases of ALI. Potentially new signals were distinguished from already known associations concerning ALI (whether in adults and/or in the paediatric population) through manual review of published literature and drug product labels., Results: The study population comprised 4,838,146 individuals aged <18 years, who contributed an overall 25,575,132 person-years of follow-up. Within this population, we identified 1,015 potential cases of ALI. Overall, 20 positive drug-ALI associations were detected. The associations between ALI and domperidone, flunisolide and human insulin were considered as potentially new signals. Citalopram and cetirizine have been previously described as hepatotoxic in adults but not in children, while all remaining associations were already known in both adults and children., Conclusions: Data mining of multiple EHR databases for signal detection confirmed known associations between ALI and several drugs, and identified some potentially new signals in children that require further investigation through formal epidemiologic studies. This study shows that EHRs may complement traditional spontaneous reporting systems for signal detection and strengthening.

Purpose: To provide estimates of the number and types of drugs that can be monitored for safety surveillance using electronic healthcare databases., Methods: Using data from eight European databases (administrative claims, medical records) and in the context of a cohort study, we determined the amount of drug exposure required for signal detection across varying magnitudes of relative risk (RR). We provide estimates of the number and types of drugs that can be monitored as a function of actual use, minimal detectable RR, and empirically derived incidence rates for the following adverse events: (i) acute myocardial infarction; (ii) acute renal failure; (iii) anaphylactic shock; (iv) bullous eruptions; (v) rhabdomyolysis; and (vi) upper gastrointestinal bleeding. We performed data simulation to see how expansion of database size would influence the capabilities of such system., Results: Data from 1,947,452 individuals (59,594,132 person-years follow-up) who used 2,289 drugs in the EU-ADR network show that for a frequent event such as acute myocardial infarction, there are 531 drugs (23% of total) for which an association with RR = 2, if present, can be investigated. For a rare event such as rhabdomyolysis, there are 19 drugs (1%) for which an association of same magnitude can be investigated., Conclusion: Active surveillance using healthcare data-based networks for signal detection is feasible, although the leverage to do so may be low for infrequently used drugs and for rare outcomes. Extending database network size to include data from heterogeneous populations and increasing follow-up time are warranted to maximize leverage of these surveillance systems., (Copyright © 2012 John Wiley & Sons, Ltd.)

Purpose: There is a growing interest in using longitudinal observational databases for drug safety signal detection, but most of the existing statistical methods are tailored towards spontaneous reporting. Here a sequential set of methods for detecting and filtering drug safety signals in longitudinal databases is presented., Method: Longitudinal GPS (LGPS) is a modification of the Gamma Poisson Shrinker (GPS) that uses person time rather than case counts for the estimation of the expected number of events. Longitudinal Evaluation of Observational Profiles of Adverse events Related to Drugs (LEOPARD) is a method that can be used to automatically discard false drug-event associations caused by protopathic bias or misclassification of the dates of the adverse events by comparing prior event prescription rates to post event prescription rates. LEOPARD can generate a single test statistic, or a visualization that can be used for more qualitative information on the relationship between drug and event. Both methods were evaluated using data simulated using the Observational medical dataset SIMulator (OSIM), including the dataset used in the Observational Medical Outcomes Partnership (OMOP) cup, a recent public competition for signal detection methods. The Mean Average Precision (MAP) was used for performance measurement., Results: On the OMOP cup data, LGPS achieved a MAP of 0.245, and the combination of LGPS and LEOPARD achieved a MAP of 0.260, the highest score in the competition., Conclusions: The sequential use of LGPS and LEOPARD have proven to be a useful novel set of methods for drug safety signal detection on longitudinal health records., (Copyright © 2010 John Wiley & Sons, Ltd.)

The EU-ADR project aims to exploit different European electronic healthcare records (EHR) databases for drug safety signal detection. In this paper we report the preliminary results concerning the comparison of signal detection between EU-ADR network and two spontaneous reporting databases, the Food and Drug Administration and World Health Organization databases. EU-ADR data sources consist of eight databases in four countries (Denmark, Italy, Netherlands, and United Kingdom) that are virtually linked through distributed data network. A custom-built software (Jerboa©) elaborates harmonized input data that are produced locally and generates aggregated data which are then stored in a central repository. Those data are subsequently analyzed through different statistics (i.e. Longitudinal Gamma Poisson Shrinker). As potential signals, all the drugs that are associated to six events of interest (bullous eruptions - BE, acute renal failure - ARF, acute myocardial infarction - AMI, anaphylactic shock - AS, rhabdomyolysis - RHABD, and upper gastrointestinal bleeding - UGIB) have been detected via different data mining techniques in the two systems. Subsequently a comparison concerning the number of drugs that could be investigated and the potential signals detected for each event in the spontaneous reporting systems (SRSs) and EU-ADR network was made. SRSs could explore, as potential signals, a larger number of drugs for the six events, in comparison to EU-ADR (range: 630-3,393 vs. 87-856), particularly for those events commonly thought to be potentially drug-induced (i.e. BE: 3,393 vs. 228). The highest proportion of signals detected in SRSs was found for BE, ARF and AS, while for ARF, and UGIB in EU-ADR. In conclusion, it seems that EU-ADR longitudinal database network may complement traditional spontaneous reporting system for signal detection, especially for those adverse events that are frequent in general population and are not commonly thought to be drug-induced. The methodology for signal detection in EU-ADR is still under development and testing phase.