Optimization of a quantitative signal detection algorithm for spontaneous reports of adverse events post immunization.

ABSTRACT Purpose To optimize the efficiency of signal detection by maximizing the proportion of true positive (TP) signals among signals detected by a disproportionality algorithm. Methods We compared 176 different combinations of stratification factors, sex (S), age (A), region (R) and year of report (Y), and cut-off values of a Multi-Item Gamma Poisson Schrinker (MGPS) algorithm. Spontaneous adverse event reports of eight vaccines from the GlaxoSmithKline Biologicals safety database were used. Defining events listed in the Product Information as proxy of true safety signals, we compared each algorithm performance in terms of positive predictive value (PPV). For each vaccine, each algorithm was ranked according to PPV. Median rank and overall PPV were computed across vaccines. Results For a standard cut-off of 2, the optimal stratification factors differed by vaccine and led to a set of algorithms with a median rank of 34.5 (PPV = 0.22; 34 TP). Keeping the original SARY stratification led to optimal cut-offs that differed by vaccine and a set of algorithms with a median rank of 1.75 (PPV = 0.20; 142 TP). The optimal combination of cut-off and stratification led to different algorithms by vaccine with a median rank of 1 (PPV = 0.19; 139 TP). The best unique algorithm parameterization across vaccines was 0.8-SARY (cut-off-stratification), with a median rank of 3 (PPV = 0.20; 195 TP). The original 2-SARY was one of the worst algorithms, with a median rank of 150.75 (PPV = 0.13; 8 TP). Conclusion Within the scope of this study, a unique MGPS algorithm across vaccines with the original full stratification but a lowered cut-off provided major performance improvement. Copyright © 2012 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]