Review
Quantitative signal detection using spontaneous ADR reporting
A. Bate MA PhD,
S. J. W. Evans BA MSc,
Corresponding Author
A. Bate MA PhD
WHO Collaborating Centre for International Drug Monitoring, Uppsala, Sweden
Department of Information Systems and Computing, Brunel University, West London, UK
WHO Collaborating Centre for International Drug Monitoring, PO Box 1051, S-75140 Uppsala, Sweden.Search for more papers by this authorS. J. W. Evans BA MSc
London School of Hygiene and Tropical Medicine, Keppel Street, University of London, UK
Search for more papers by this authorA. Bate MA PhD,
S. J. W. Evans BA MSc,
Corresponding Author
A. Bate MA PhD
WHO Collaborating Centre for International Drug Monitoring, Uppsala, Sweden
Department of Information Systems and Computing, Brunel University, West London, UK
WHO Collaborating Centre for International Drug Monitoring, PO Box 1051, S-75140 Uppsala, Sweden.Search for more papers by this authorS. J. W. Evans BA MSc
London School of Hygiene and Tropical Medicine, Keppel Street, University of London, UK
Search for more papers by this authorAbstract
Quantitative methods are increasingly used to analyse spontaneous reports. We describe the core concepts behind the most common methods, the proportional reporting ratio (PRR), reporting odds ratio (ROR), information component (IC) and empirical Bayes geometric mean (EBGM). We discuss the role of Bayesian shrinkage in screening spontaneous reports, the importance of changes over time in screening the properties of the measures. Additionally we discuss three major areas of controversy and ongoing research: stratification, method evaluation and implementation. Finally we give some suggestions as to where emerging research is likely to lead. Copyright © 2009 John Wiley & Sons, Ltd.
REFERENCES
- 1 Rawlins MD. Spontaneous reporting of adverse drug reactions. II. Uses. Br J Clin Pharmacol 1988; 26: 7–11.
- 2 Edwards IR, Biriell C. Harmonisation in pharmacovigilance. Drug Saf 1994; 10: 93–102.
- 3 Norwood PK, Sampson AR. A statistical methodology for postmarketing surveillance of adverse drug reaction reports. Stat Med 1988; 7: 1023–1030.
- 4 Praus M, Schindel F, Fescharek R, et al. Alert systems for post-marketing surveillance of adverse drug reactions. Stat Med 1993; 12: 2383–2393.
- 5 Lindquist M, Edwards IR. Risks of non-sedating antihistamines. Lancet 1997; 349: 1322.
- 6 Finney DJ. Systematic signalling of adverse reactions to drugs. Methods Inf Med 1974; 13: 1–10.
- 7 Hauben M. Application of an empiric Bayesian data mining algorithm to reports of pancreatitis associated with atypical antipsychotics. Pharmacotherapy 2004; 24: 1122–1129.
- 8 Lindquist M, Stahl M, Bate A, et al. A retrospective evaluation of a data mining approach to aid finding new adverse drug reaction signals in the WHO international database. Drug Saf 2000; 23: 533–542.
- 9 Alvarez-Requejo A, Carvajal A, Begaud B, et al. Under-reporting of adverse drug reactions. Estimate based on a spontaneous reporting scheme and a sentinel system. Eur J Clin Pharmacol 1998; 54: 483–488.
- 10 Evans SJ, Waller PC, Davis S. Use of proportional reporting ratios (PRRs) for signal generation from spontaneous adverse drug reaction reports. Pharmacoepidemiol Drug Saf 2001; 10: 483–486.
- 11 Moore N, Kreft-Jais C, Haramburu F, et al. Reports of hypoglycaemia associated with the use of ACE inhibitors and other drugs: a case/non-case study in the French pharmacovigilance system database. Br J Clin Pharmacol 1997; 44: 513–518.
- 12 van Puijenbroek EP, Bate A, Leufkens HG, et al. A comparison of measures of disproportionality for signal detection in spontaneous reporting systems for adverse drug reactions. Pharmacoepidemiol Drug Saf 2002; 11: 3–10.
- 13 Bate A, Lindquist M, Edwards IR, et al. A Bayesian neural network method for adverse drug reaction signal generation. Eur J Clin Pharmacol 1998; 54: 315–321.
- 14 DuMouchel W. Bayesian data mining in large frequency tables, with an application to the FDA Spontaneous Reporting System. Am Stat 1999; 53: 177–190.
- 15 Gould AL. Practical pharmacovigilance analysis strategies. Pharmacoepidemiol Drug Saf 2003; 12: 559–574.
- 16 Szarfman A, Machado SG, O'Neill RT. Use of screening algorithms and computer systems to efficiently signal higher-than-expected combinations of drugs and events in the US FDA's spontaneous reports database. Drug Saf 2002; 25: 381–392.
- 17
Evans SJ.
Statistical methods of signal detection. In
Pharmacovigilance,
RD Mann,
EB Andrews (eds).
2002;
John Wiley and Sons Ltd.: Chichester, UK,
273–279.
10.1002/0470853093.ch20 Google Scholar
- 18 Stahl M, Lindquist M, Edwards IR, et al. Introducing triage logic as a new strategy for the detection of signals in the WHO Drug Monitoring Database. Pharmacoepidemiol Drug Saf 2004; 13: 355–363.
- 19 Rodriguez EM, Staffa JA, Graham DJ. The role of databases in drug postmarketing surveillance. Pharmacoepidemiol Drug Saf 2001; 10: 407–410.
- 20 Waller P. Dealing with uncertainty in drug safety: lessons for the future from sertindole. Pharmacoepidemiol Drug Saf 2003; 12: 283–287; discussion. 289–290.
- 21 Moore N, Hall G, Sturkenboom M, et al. Biases affecting the proportional reporting ratio (PPR) in spontaneous reports pharmacovigilance databases: the example of sertindole. Pharmacoepidemiol Drug Saf 2003; 12: 271–281.
- 22 van der Heijden PG, van Puijenbroek EP, van Buuren S, et al. On the assessment of adverse drug reactions from spontaneous reporting systems: the influence of under-reporting on odds ratios. Stat Med 2002; 21: 2027–2044.
- 23 Mantel N, Haenszel W. Statistical aspects of the analysis of data from retrospective studies of disease. J Nat Cancer Inst 1959: 719–749.
- 24 Noren GN, Bate A, Orre R, et al. Extending the methods used to screen the WHO drug safety database towards analysis of complex associations and improved accuracy for rare events. Stat Med 2006; 25: 3740–3757.
- 25 Hopstadius J, Norén GN, Bate A, et al. Impact of stratification in adverse drug reaction surveillance. Drug Saf 2008; 31: 1035–1048.
- 26 Bate A, Edwards I, Lindquist M, et al. Violation of homogeneity: a methodological issue in the use of data mining tools: the authors' reply. Drug Saf 2003; 26: 363–366.
- 27 Evans SJW. Stratification for spontaneous report databases. Drug Saf 2008; 31: 1049–1052.
- 28 Woo EJ, Ball R, Burwen DR, et al. Effects of stratification on data mining in the US Vaccine Adverse Event Reporting System (VAERS). Drug Saf 2008; 31: 667–674.
- 29 Hauben M, Reich L, Chung S. Postmarketing surveillance of potentially fatal reactions to oncology drugs: potential utility of two signal-detection algorithms. Eur J Clin Pharmacol 2004; 60: 747–750.
- 30 Hauben M, Reich L. Safety related drug-labelling changes: findings from two data mining algorithms. Drug Saf 2004; 27: 735–744.
- 31 Roux E, Thiessard F, Fourrier A, et al. Evaluation of statistical association measures for the automatic signal generation in pharmacovigilance. IEEE Trans Inf Technol Biomed 2005; 9: 518–527.
- 32 Rolka H, Bracy D, Russell C, et al. Using simulation to assess the sensitivity and specificity of a signal detection tool for multidimensional public health surveillance data. Stat Med 2005; 24: 551–562.
- 33 Olsson S, Edwards IR. Tachycardia during cisapride treatment. Br Med J 1992; 305: 748–749.
- 34 Inman WHW, Kubota K. Tachycardia during cisapride treatment. Br Med J 1992; 305: 1019.
- 35 Wysowski DK, Corken A, Gallo-Torres H, et al. Postmarketing reports of QT prolongation and ventricular arrhythmia in association with cisapride and food and drug administration regulatory actions. Am J Gastroenterol 2001; 96: 1698–1703.
- 36 Sanz EJ, De-las-Cuevas C, Kiuru A, et al. Selective serotonin reuptake inhibitors in pregnant women and neonatal withdrawal syndrome: a database analysis. Lancet 2005; 365: 482–487.
- 37 Coulter DM, Bate A, Meyboom RH, et al. Antipsychotic drugs and heart muscle disorder in international pharmacovigilance: data mining study. Br Med J 2001; 322: 1207–1209.
- 38 Jacobs P, Conforti A, Wood L, et al. Immune agranulocytosis and clarithromycin. Hematology 2004; 9: 291–296.
- 39 Zhou W, Pool V, DeStefano F, et al. A potential signal of Bell's palsy after parenteral inactivated influenza vaccines: reports to the Vaccine Adverse Event Reporting System (VAERS)—United States. 1991–2001. Pharmacoepidemiol Drug Saf 2004; 13: 505–510.
- 40 Niu MT, Erwin DE, Braun MM. Data mining in the US Vaccine Adverse Event Reporting System (VAERS): early detection of intussusception and other events after rotavirus vaccination. Vaccine 2001; 19: 4627–4634.
- 41 Lindquist M. Data quality management in pharmacovigilance. Drug Saf 2004; 27: 857–870.
- 42 Bright RA, Nelson RC. Automated support for pharmacovigilance: a proposed system. Pharmacoepidemiol Drug Saf 2002; 11: 121–125.
- 43 Bate A. The use of a Bayesian Confidence Propagation Neural Network in Pharmacovigilance. In Clinical Pharmacology, 2003; Umeå University: Ume.
- 44 Rothman KJ, Lanes S, Sacks ST. The reporting odds ratio and its advantages over the proportional reporting ratio. Pharmacoepidemiol Drug Saf 2004; 13: 519–523.
- 45 Waller P, van Puijenbroek E, Egberts A, et al. The reporting odds ratio versus the proportional reporting ratio: 'deuce'. Pharmacoepidemiol Drug Saf 2004; 13: 525- 526; discussion 527–528.
- 46 Lindquist M, Edwards IR, Bate A, et al. From association to alert—a revised approach to International Signal Analysis. Pharmacoepidemiol Drug Saf 1999; 8: S15–S25.
- 47
DuMouchel W,
Pregibon D.
Empirical Bayes screening for multi-item associations.
Proceedings of the Seventh ACM SIGKDD International conference on Knowledge Discovery and Data Mining
2001:
67–76.
10.1145/502512.502526 Google Scholar
- 48 Evans SJW. Statistics: analysis and, presentation of safety data. In Stephens' Detection of New Adverse Drug Reactions, J Talbot, P Waller (eds). 2004; John Wiley & Sons Ltd.: London, 301–328.
- 49 Purcell P, Barty S. Statistical techniques for signal generation: the Australian experience. Drug Saf 2002; 25: 415–421.
- 50 Brown EG. Effects of coding dictionary on signal generation: a consideration of use of MedDRA compared with WHO-ART. Drug Saf 2002; 25: 445–452.
- 51 Yokotsuka M, Aoyama M, Kubota K. The use of a medical dictionary for regulatory activities terminology (MedDRA) in prescription-event monitoring in Japan (J-PEM). Int J Med Inf 2000; 57: 139–153.
- 52 Bousquet C, Lagier G, Lillo-Le Louet A, et al. Appraisal of the MedDRA conceptual structure for describing and grouping adverse drug reactions. Drug Saf 2005; 28: 19–34.
- 53 Orre R, Bate A, Noren GN, et al. A Bayesian recurrent neural network approach for finding dependencies in large incomplete data sets. Int J Neural Syst 2005; 15: 207–222.
- 54 Edwards IR, Lindquist M, Wiholm BE, et al. Quality criteria for early signals of possible adverse drug reactions. Lancet 1990; 336: 156–158.
- 55 Natsch S, Vinks MH, Voogt AK, et al. Anaphylactic reactions to proton-pump inhibitors. Ann Pharmacother 2000; 34: 474–476.
- 56 Meyboom RH, Lindquist M, Egberts AC, et al. Signal selection and follow-up in pharmacovigilance. Drug Saf 2002; 25: 459–465.
- 57
Waller PC,
Lee EH.
Responding to drug safety issues.
Pharmacoepidemiol Drug Saf
1999;
8:
535–553.
10.1002/(SICI)1099-1557(199912)8:7<535::AID-PDS456>3.0.CO;2-D CAS PubMed Web of Science® Google Scholar
- 58 Amery WK, Post-marketing drug safety management: a pharmaceutical industry perspective. Int J Risk Saf Med 1994; 5: 67–270.
- 59 Bate A, Lindquist M, Orre R, et al. Data-mining analyses of pharmacovigilance signals in relation to relevant comparison drugs. Eur J Clin Pharmacol 2002; 58: 483–490.
- 60 van Puijenbroek EP, Egberts AC, Heerdink ER, et al. Detecting drug–drug interactions using a database for spontaneous adverse drug reactions: an example with diuretics and non-steroidal anti-inflammatory drugs. Eur J Clin Pharmacol 2000; 56: 733–738.
- 61 Orre R, Lansner A, Bate A, et al. Bayesian neural networks with confidence estimations applied to data mining. Comput Stat Data Anal 2000; 34: 473–493.
- 62 Noren GN, Sundberg R, Bate A, et al. A statistical methodology for drug–drug interaction surveillance. Stat Med 2008; 27: 3057–3070.
- 63 Bate A, Noren N, Orre R, et al. Pattern detection for celecoxib and rofecoxib in the WHO Database (Abstract). Pharmacoepidemiol Drug Saf 2004: 323.
- 64 Noren GN, Orre R, Bate A, et al. Duplicate detection in adverse drug reaction surveillance. Data Min Knowl Discov 2007; 14: 305–328.
- 65 Heeley E, Wilton LV, Shakir SA. Automated signal generation in prescription-event monitoring. Drug Saf 2002; 25: 423–432.
- 66 Anonymous, Appendix 3: Pilot Data Report. 2003, National Patient Safety Agency: London. http://www.npsa.nhs.uk/admin/publications/docs/businessplan.pdf
- 67 Brown JS, Kulldorff M, Chan KA, et al. Early detection of adverse drug events within population-based health networks: application of sequential testing methods. Pharmacoepidemiol Drug Saf 2007; 16: 1275–1284.
- 68 Brown JS, Kulldorff M, Petronis K, et al. Early adverse drug event signal detection within population-based health networks using sequential methods: key methodologic considerations. Pharmacoepidemiol Drug Saf 2009; doi:10.1002/pds.1706.
- 69 Norén GN, Bate A, Hopstadius J, et al. Temporal pattern discovery for trends and transient effects: its application to patient records, In Fourteenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Las Vegas, 2008.
- 70 Bate A, Edwards IR, Edwards J, et al. Knowledge finding in IMS Disease Analyser Mediplus UK database—effective data mining in longitudinal patient safety data. Drug Saf 2004; 27: 917–918.
- 71 van Puijenbroek EP, van Grootheest K, Diemont WL, et al. Determinants of signal selection in a spontaneous reporting system for adverse drug reactions. Br J Clin Pharmacol 2001; 52: 579–586.
- 72
Edwards IR,
Fucik H.
Impact and credibility of the WHO adverse reaction signals.
Drug Inf J
1996;
30:
461–464.
10.1177/009286159603000216 Google Scholar
- 73 Stahl M, Edwards I, Bowring G, et al. Assessing the impact of drug safety signals from the WHO Database presented in 'SIGNAL': results from a questionnaire of National Pharmacovigilance Centres. Drug Saf 2003; 26: 721–727.
- 74 Waller PC, Evans SJ. A model for the future conduct of pharmacovigilance. Pharmacoepidemiol Drug Saf 2003; 12: 17–29.
Citing Literature
