A pattern discovery framework for adverse event evaluation and inference in spontaneous reporting systems
Marianthi Markatou
T. J. Watson Research Center, IBM, Yorktown Heights, NY, USA
Department of Biostatistics, School of Public Health and Health Professions, SUNY Buffalo, Buffalo, NY, 14216 USA
Search for more papers by this authorCorresponding Author
Robert Ball
Office of Biostatistics and Epidemiology, Center for Biologics Evaluation and Research, Food and Drug Administration, Rockville, MD, 20852 USA
Robert Ball ([email protected])Search for more papers by this authorMarianthi Markatou
T. J. Watson Research Center, IBM, Yorktown Heights, NY, USA
Department of Biostatistics, School of Public Health and Health Professions, SUNY Buffalo, Buffalo, NY, 14216 USA
Search for more papers by this authorCorresponding Author
Robert Ball
Office of Biostatistics and Epidemiology, Center for Biologics Evaluation and Research, Food and Drug Administration, Rockville, MD, 20852 USA
Robert Ball ([email protected])Search for more papers by this authorAbstract
Safety of medical products is a major public health concern. We present a critical discussion of the currently used analytical tools for mining spontaneous reporting systems (SRS) to identify safety signals after use of medical products. We introduce a pattern discovery framework for the analysis of SRS. The terminology ‘pattern discovery’ is borrowed from the engineering and artificial intelligence literature and signifies that the basis of the proposed framework is the medical case, formalizing the cognitive paradigm known to clinicians who evaluate individual patients and individual case safety reports submitted to SRS. The fundamental contribution of this approach is a strong probabilistic component that may account for selection and other biases and facilitates rigorous modeling and inference. We discuss somewhat in depth the concept of signal in pharmacovigilance and connect it with the concept of a pattern; we illustrate this conceptual framework using the example of anaphylaxis. Finally, we propose a research agenda in statistics, informatics, and pharmacovigilance practices needed to advance the pattern discovery framework in both the short and long terms.
REFERENCES
- 1D. C. Classen, L. S. Pestotnik, R. S. Evans, J. F. Lloyd, and J. P. Buke, Adverse drug events in hospitalized patients. Excess length of stay, extra costs, and attributable mortality, JAMA 277 (1997), 301–306.
- 2D. J. Cullen, B. J. Sweitzer, D. W. Bates, E. Burdick, A. Edmondson, and L. L. Leape, Preventable adverse drug events in hospitalized patients: a comparative study of intensive care and general care units, Crit Care Med 25 (1997), 1289–1297.
- 3D. J. Cullen, D. W. Bates, S. D. Small, J. B. Cooper, A. R. Nemeskal, and L. L. Leape, The incident reporting system does not detect adverse events: a problem for quality improvement, Jt. Comm. J. Qual. Improv 21 (1995), 541–548.
- 4G. Peter and M. G. Myers, Intussusception, rotavirus and oral vaccines: summary of a workshop, Pediatrics 110 (2002), e67.
- 5D. W. Bates, N. Spell, D. J. Cullen, E. Burdick, N. Laird, L. A. Petersen, S. D. Small, B. J. Sweitzer, L. L. Leape, The cost of adverse drug events in hospitalized patients. Adverse Drug Events Prevention Study Group JAMA 277 (1997), 307–311.
- 6D. W. Bates, D. J. Cullen, N. Laird, L. A. Petersen, S. D. Small, D. Servi, G. Laffel, B. J. Sweitzer, B. F. Shea, R. Hallisey, M. Vander Vliet, R. Nemeskal, L. L. Leape, D. Bates, P. Hojnowski-Diaz, S. Petrycki, M. Cotugno, H. Patterson, M. Hickey, S. Kleefield, J. Cooper, E. Kinneally, H. J. Demonaco, M. Dempsey Clapp, T. Gallivan, J. Ives, K. Porter, B. T. Thompson, J. R. Hackman, A. Edmondson, Incidence of adverse drug events and potential adverse drug events. Implications for prevention. Adverse Drug Events Study Group JAMA 274 (1995), 29–34.
- 7W. K. Amery, Why there is a need for pharmacovigilance, Pharmacoepidemiol Drug Saf 8 (1999), 61–64.
10.1002/(SICI)1099-1557(199901/02)8:1<61::AID-PDS395>3.0.CO;2-A CAS PubMed Web of Science® Google Scholar
- 8C. Chuang-Stein, N. R. Mohberg, and D. M. Musselman, Organization and analysis of safety data using a multivariate approach, Stat Med 11 (1992), 1075–1089.
- 9C. Chuang-Stein, Safety analysis in controlled clinical trials, Drug Inf Assoc J (1998), 32 1363S–1372S.
- 10B. J. Crowe, H. A. Xia, J. A. Berlin, D. J. Watson, H. Shi, S. L. Lin, J. Kuebler, R. C. Schriver, N. C. Santanello, G. Rochester, J. B. Porter, M. Oster, D. V. Mehrotra, Z. Li, E. C. King, E. S. Harpus, and D. B. Hall, Recommendations for safety planning, data collection, evaluation and reporting during drug, biologic and vaccine development: a report of the safety planning, evaluation, and reporting team, Clin Trials 6 (2009) 430–440.
- 11M. A. Robb, J. A. Racoosin, R. E. Sherman, T. P. Gross, R. Ball, M. E. Reichman, K. Midthun, J. Woodcock, FDA's sentinel initiative: expanding the horizons of medical product safety, Pharmacoepidemiol Drug Saf 21(S1) (2012), 9–11.
- 12M. B. Rennels, U. D. Parashar, R. C. Holman, C. T. Le, H.-G. Chang, R. I. Glass, Lack of an apparent association between intussusception and wild or vaccine rotavirus infection, Pediatr Infect Dis J 17 (1998), 924–925.
- 13T. V. Murphy, P. M. Gargiullo, M. S. Massoudi, D. B. Nelson, A. O. Jumaan, C. A. Okoro, L. R. Zanardi, S. Setia, E. Fair, C. W. LeBaron, M. Wharton, J. R. Livengood, (Rotavirus Intussusception Investigation Team), Intussusception among infants given an oral rotavirus vaccine, New Engl J Med 344 (2001), 564–572.
- 14D. J. Hand and R. J. Bolton, Pattern discovery and detection: a unified statistical methodology, J Appl Stat 31(8) (2004), 885–924.
- 15L. Huang, J. Zalkikar, and R. C. Tiwari, A likelihood ratio test based method for signal detection with application to FDA's drug safety data, J Am Stat Assoc 106 (2011), 1230–1241.
- 16W. DuMouchel, Bayesian data mining in large frequency tables, with an application to the FDA spontaneous reporting system (with discussion), Am Stat 53 (1999), 177–202.
- 17H. Cao, G. B. Melton, M. Markatou, and G. Hripcsak, Use abstracted patient-specific features to assist an information-theoretic measurement to assess similarity between medical cases, J Biomed Inf 41 (2008) 882–888.
- 18M. Markatou, P. Kurrupumullage Don, J. Hu, F. Wang, J. Sun, R. Sorrentino, S. Ebadollahi, Case-based reasoning in comparative effectiveness research, IBM J Res Dev 56 (2012), 4:1–4:12.
- 19 Food and Drug Administration, Guidance to Industry: Good Pharmacovigilance Practices and Pharmacoepidemiologic Assessment, 2005, http://www.fda.gov/cder/guidance/index.html.
- 20M. Hauben, D. Madigan, C. M. Gerrits, L. Walsh, E. P. van Puijenbroek, The role of data mining in pharmacovigilance, Expert Opin Drug Saf 4 (2005) 929–948
- 21H. Cao, G. Hripcsak, and M. Markatou, A statistical methodology for analyzing co-occurrence data from a large sample, J Biomed Inf 40 (2007), 343–352.
- 22S. W. J. Evans, P. C. Waller, and S. Davis, Use of proportional reporting ratios (PRRs) for signal generation from spontaneous adverse drug reaction reports, Pharmacoepidemiol Drug Saf 10 (2001), 483–486.
- 23R. Orre, A. Bate, N. G. Noren, E. Swahn, S. Arnborg, and R. I. Edwards, A Bayesian recurrent neural network for unsupervised pattern recognition in large incomplete data sets, Int J Neural Syst 15 (2005), 207–222.
- 24A. J. Forster, A. Jennings, C. Chow, C. van Walraven, A systematic review to evaluate the accuracy of electronic adverse drug event detection, J Am Med Inform Assoc 19 (2012) 31–38.
- 25M. Lindquist, The need for definitions in pharmacovigilance, Drug Saf 30 (2007), 825–830.
- 26A. Bate, E. G. Brown, S. A. Goldman, and M. Hauben, Terminological challenges in safety surveillance, Drug Saf 35 (2012), 79–84.
- 27P. B. Ryan, D. Madigan, P. E. Stang, J. M. Overhage, J. A. Racoosin, and A. G. Hartzema, Empirical assessment of methods for risk identification in healthcare data: results from the experiments of the observational medical outcomes partnership, Stat Med (2012) 31 4401–4415.
- 28C. Reich, P. B. Ryan, P. E. Stang, M. Rocca, Evaluation of alternative standardized terminologies for medical conditions within a network of observational healthcare databases, J Biomed Inf 45 (2012) 689–696.
- 29R. Harpaz, W. DuMouchel, N. H. Shah, D. Madigan, P. Ryan, and C. Friedman, Novel data mining methodologies for adverse drug event discovery and analysis, Clin Pharmacol Ther 91(6) (2012), 1010–1021.
- 30R. Harpaz, W. DuMouchel, P. LePendu, A. Bauer-Mehren, P. Ryan, and N. H. Shah, Performance of pharmacovigilance signal detection algorithms for the FDA adverse event reporting system, Clin Pharmacol Ther 93(6) (2013) 539–546.
- 31O. Caster, G. N. Noren, D. Madigan, and A. Bate, A logistic regression in signal detection: another piece added to the puzzle, Clin Pharmacol Ther 94(3) (2013), 312.
- 32J. Hopstadius, G. N. Noren, A. Bate, and J. R. Edwards, Impact of stratification on adverse drug reaction surveillance, Drug Saf 31(11) (2008), 1035–1048.
- 33I. Ahmed, F. Thiessard, G. Miremont-Salame, B. Begaud, and P. Tubert-Bitter, Pharmacovigilance data mining with methods based on false discovery rates: a comparative simulation study, Clin Pharmacol Ther 88 (2010), 492–498.
- 34H. Rolka, D. Bracy, C. Russell, D. Fram, and R. Ball, Using simulations to assess the sensitivity and specificity of a signal detection tool for multidimensional public health surveillance data, Stat Med 24 (2005), 551–562.
- 35P. W. Ewan, Anaphylaxis, BMJ 316 (1998), 1442–1445.
- 36C. Ponvert, P. Scheinmann, Vaccine allergy and pseudo-allergy, Eur J Dermatol 13 (2003) 10–15 .
- 37J. U. Ruggeberg, M. S. Gold, J.-M. Bayas, M. D. Blum, J. Bonhoeffer, S. Friedlander, G. de Souza Brito, U. Heininger, B. Imoukhuede, A. Khamesipour, M. Erlewyn-Lajeunesse, S. Martin, M. Makela, P. Nell, V. Pool, N. Simpson, Anaphylaxis: case definition and guidelines for data collection, analysis and presentation of immunization safety data, Vaccine 25 (2007), 5675–5684.
- 38S. I. Wasserman, Anaphylaxis, In Clinical Immunology. Principles and Practice, vol. 46, R. R. Rich, T. A. Fleisher, W. T. Shearer, and W. T. Kotzin, eds. Toronto, Mosby, 2001, 1–11.
- 39A. Madaan, D. E. Maddox, Vaccine allergy: diagnosis and management, Immunol Allergy Clin North Am, 23 (2003), 555–588.
- 40 FDA, Adverse Events Reporting System (AERS), http://www.fda.gov/Drugs/GuidanceComplianceRegulatoryInformation/Surveillance/AdverseDrugEffects/default.htm.
- 41 FDA and CDC Vaccine Adverse Events Reporting System (VAERS), http://vaers.hhs.gov/data/index.
- 42 Eudravigilance, EMA. http://eudravigilance.ema.europa.eu/human/index.asp.
- 43 Japanese and Medical Devices Agency Individual Case Safety Report database, http://www.pmda.go.jp/english/service/precautions.html
- 44 The WHO Upsalla Monitoring Center database, VigiBase, http://www.who-umc.org.
- 45 International Conference on Harmonization of Technical Requirements for Registration of Pharmaceuticals for Human Use, Maintenance of the ICH Guideline on Clinical Safety Data Management: Data Elements for Transmission of Individual Case Safety Reports E2b(R2), http://www.ich.org/products/guidelines/efficacy/article/efficacy-guidelines.html.
- 46T. Botsis, M. D. Nguyen, E. J. Woo, M. Markatou, and R. Ball, Text mining for the vaccine adverse event reporting system: medical text classification using informative feature selection, J Am Med Inf Assoc 18 (2011), 631–638.
- 47T. Botsis, J. E. Woo, and R. Ball, Application of information retrieval approaches to case classification in the vaccine adverse event reporting system, Drug Saf 36 (2013) 573–582.
- 48R. Ball and T. Botsis, Can network analysis improve pattern recognition among adverse events following immunization reported to VAERS?, Clin Pharmacol Ther 90 (2011), 271–278.
- 49T. Botsis and R. Ball, Network analysis of possible anaphylaxis cases reported to the US Vaccine Adverse Event Reporting System after H1N1 influenza vaccine, Stud Health Technol Inf 169 (2011), 564–568.
- 50A. L. Barabasi and R. Albert, Emergence of scaling in random networks, Science 286 (1999), 509–512.
- 51F. Papadopoulos, M. Kitsak, M. A. Serrano, M. Boguna, and D. Krioukov, Popularity versus similarity in growing networks, Nature 489 (2012), 537–540.
- 52 CIOMS VIII, Practical Aspects of Signal Detection in Pharmacovigilance, Geneva, CIOMS, 2010.
- 53M. Hauben and J. K. Aronson, Defining “signal” and its subtypes in pharmacovigilance based on a systematic review of previous definitions, Drug Saf 32 (2009), 99–110.
- 54J. Einmahl, J. Li, and R. Liu, Thresholding events of extreme in simultaneous monitoring of multiple risks, Journal of the American Statistical Association 104 (2009), 982–992.
- 55T. B. Agbabiaka, J. Savoric, and E. Ernst, Methods for causality assessment of adverse drug reactions: a systematic review, Drug Saf 31 (2008) 21–37.
- 56A. Tversky, Features of similarity, Psychol Rev 84 (1977), 327–352.
- 57D. W. Goodall, A new similarity index based on probability, Biometrics 22 882–907.
- 58J. C. Gower, A general coefficient of similarity and some of its properties, Biometrics 27 (1971), 857–874.
- 59E. Levina and P. J. Bickel, The earth mover's distance is the Mallows distance. Some insights from statistics In Proceedings of ICCV2001, Vancouver, Canada, 2001, 251–256.
- 60M.-F. Balcan, A. Blum, and N. Srebro, A theory of learning with similarity functions, Mach Learn 72 (2008), 89–112.
- 61M.-F. Balcan, A. Blum, and N. Srebro, Improved guarantees for learning via similarity functions, In Proceedings of the 21st Annual Conference on Learning Theory (COLT), 2008, 287–298.
- 62A. Panagiotelis, C. Czado, and H. Joe, Pair copula constructions for multivariate discrete data, J Am Stat Assoc 107 (2012), 1063–1072.
- 63D. G. Clayton, A model for association in bivariate life tables and its application in epidemiological studies of familial tendency in chronic disease incidence, Biometrika 65 (1978), 141–151.
- 64D. Oakes, Bivariate survival models induced by frailties, J Am Stat Assoc 84 (1989), 487–493.
- 65D. M. Zimmer and P. K. Trivedi, Using trivariate copulas to model sample selection and treatment effects: application to family health care demand, J Bus Econ Stat 24 (2006), 63–76.
- 66P. Tubert, B. Begaud, J.-C. Pere, F. Haramburu, and J. Lelouch, Power and weaknesses of spontaneous reporting: a probabilistic approach, J Clin Epidemiol 45 (1992), 283–286.
- 67J. Scott, T. Botsis, R. Ball, Simulating adverse event spontaneous reporting systems as preferential attachment networks: application to the vaccine adverse event reporting system, Appl Clin Inf 5(1) (2014), 206–218.
- 68I. Ahmed, C. Dalmasso, F. Haramburu, F. Thiessard, P. Broet, and P. Tubert-Bitter, False discovery rate estimation for frequentist pharmacovigilance signal detection methods, Biometrics 66 (2010), 301–309.
- 69J. B. Copas and H. G. Li, Inference for non-random samples (with discussion), J R Stat Soc Ser B 59 (1997), 55–95.
- 70P. McCullagh, Sampling bias and logistic models (with discussion), J R Stat Soc Ser B 70 (2008), 643–677.
- 71T. M. F. Smith, On the validity of inferences from non-random sample, J R Stat Soc Ser A 146 (1983), 394–403.
- 72J. J. Heckman, Sample selection bias as a specification error, Econometrica 47 (1979), 153–161.
- 73C. F. Manski, Anatomy of the selection problem, J Hum Resour 24 (1989), 343–360.
- 74B. L. De Stavola and D. R. Cox, On the consequences of overstratification, Biometrika 95 (2008), 992–996.
- 75P. R. Rosenbaum and D. B. Rubin, The central role of the propensity score in observational studies for causal effects, Biometrika 70 (1983), 41–55.
- 76P. C. Austin, The performance of different propensity-score methods for estimating differences in proportions (risk differences or absolute risk reductions) in observational studies, Stat Med 29 (2010) 2137–2148.
- 77D. Madigan, P. B. Ryan, M. Schuemie, P. E. Stang, M. J. Overhage, A. G. Hartzema, M. A. Suchard, W. DuMouchel, and J. A. Berlin, Evaluating the impact of database heterogeneity on observational study results, Am J Epidemiol 178 (2013), 645–651. DOI: 10.1093/aje/kwt010.