Evaluating Strength of Evidence of Pediatric Otolaryngology Research Literature: A 20-Year Review
Corresponding Author
Jennifer A. Silver MD
Department of Otolaryngology-Head and Neck Surgery, McGill University, Montreal, Québec, Canada
Send correspondence to Jennifer A. Silver, MD, Department of Otolaryngology-Head and Neck Surgery, McGill University Health Center, 1001 Boulevard Decarie, Montreal, Quebac H4A 3J1, Canada. E-mail: [email protected]
Search for more papers by this authorJeffrey C. Yeung MD, MAS, FRCSC
Department of Otolaryngology-Head and Neck Surgery, McGill University, Montreal, Québec, Canada
Search for more papers by this authorDeema Almutawa MD
Department of Otolaryngology-Head and Neck Surgery, McGill University, Montreal, Québec, Canada
Search for more papers by this authorRachel Szwimer MSc
Faculty of Medicine, McGill University, Montreal, Quebec, Canada
Search for more papers by this authorLily H. P. Nguyen MD, MSc, MPHE, FRCSC
Department of Otolaryngology-Head and Neck Surgery, McGill University, Montreal, Québec, Canada
Search for more papers by this authorCorresponding Author
Jennifer A. Silver MD
Department of Otolaryngology-Head and Neck Surgery, McGill University, Montreal, Québec, Canada
Send correspondence to Jennifer A. Silver, MD, Department of Otolaryngology-Head and Neck Surgery, McGill University Health Center, 1001 Boulevard Decarie, Montreal, Quebac H4A 3J1, Canada. E-mail: [email protected]
Search for more papers by this authorJeffrey C. Yeung MD, MAS, FRCSC
Department of Otolaryngology-Head and Neck Surgery, McGill University, Montreal, Québec, Canada
Search for more papers by this authorDeema Almutawa MD
Department of Otolaryngology-Head and Neck Surgery, McGill University, Montreal, Québec, Canada
Search for more papers by this authorRachel Szwimer MSc
Faculty of Medicine, McGill University, Montreal, Quebec, Canada
Search for more papers by this authorLily H. P. Nguyen MD, MSc, MPHE, FRCSC
Department of Otolaryngology-Head and Neck Surgery, McGill University, Montreal, Québec, Canada
Search for more papers by this authorEditor's Note: This Manuscript was accepted for publication on November 05, 2021.
The authors have no funding, financial relationships, or conflicts of interest to disclose.
Abstract
Objectives/Hypothesis
Quantity and quality of Otolaryngology-Head and Neck Surgery (OTL-HNS) research are increasing, yet patterns within Pediatric OTL-HNS publications are unknown. This study examines trends in the level of evidence of pediatric OTL-HNS articles over a 20-year period to quantify the growth and characterize contributing factors.
Study Design
Review article.
Methods
A retrospective review was conducted on 12 peer-reviewed OTL-HNS journals at three time-points: 1996, 2006, and 2016. Pediatric-specific OTL-HNS journals were selected; all were among the top 10 highest impact factor journals, with one pediatric-specific and one Canadian journal. Publication details, author characteristics, and study focus were collected. Papers were classified based on the Oxford Centre for Evidence-Based Medicine Levels of Evidence by two independent reviewers.
Results
Of the 1,733 articles reviewed, 727 met inclusion criteria. A greater absolute number of pediatric OTL-HNS articles were published over the years studied: from 95 in 1996 to 359 in 2016 (P < .001). As well, the absolute number of high-quality studies has increased over the study period, from 28 articles in 1996 to 100 articles in 2016. However, the relative percentage of high-quality papers remained stable between 27.9% and 32.2% with an average of 29.7% (P = .89). Higher impact factor journals did not tend to publish higher-quality pediatric OTL-HNS articles (P = .48).
Conclusions
Over the past 20 years, there is no appreciable improvement in the proportion of high-quality publications in pediatric OTL-HNS; however, there is an overall greater number of high-quality papers within OTL-HNS literature. These findings likely relate to challenges of research within pediatric surgical specialties.
Level of Evidence
NA Laryngoscope, 132:1869–1876, 2022
Supporting Information
| Filename | Description |
|---|---|
| lary29945-sup-0001-AppendixA.docxWord 2007 document , 12.9 KB | Appendix S1. List of included journals. |
| lary29945-sup-0002-AppendixB.docxWord 2007 document , 13.6 KB | Appendix S2. Medline search. |
| lary29945-sup-0003-AppendixC.docxWord 2007 document , 15.4 KB | Appendix S3. Journal Breakdown, by year and by quality. |
Please note: The publisher is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.
BIBLIOGRAPHY
- 1Tsay MY, Yang YH. Bibliometric analysis of the literature of randomized controlled trials. J Med Libr Assoc 2005; 93: 450–458.
- 2Hoffmann T, Erueti C, Thorning S, Glasziou P. The scatter of research: cross sectional comparison of randomised trials and systematic reviews across specialties. BMJ 2012; 344:e3223.
- 3Bagues M, Sylos-Labini M, Zinovyeva N. A walk on the wild side: “predatory” journals and information asymmetries in scientific evaluations. Res Policy 2019; 48: 462–477.
- 4Chung KH, Cox RAK, Kim KA. On the relation between intellectual collaboration and intellectual output: evidence from the finance academe. Q Rev Econ Finance 2009; 49: 893–916.
10.1016/j.qref.2008.08.001 Google Scholar
- 5Kehlet H, Wilmore DW. Evidence-based surgical care and the evolution of fast-track surgery. Ann Surg 2008; 248: 189–198.
- 6Wasserman JM, Wynn R, Bash TS, Rosenfeld RM. Levels of evidence in otolaryngology journals. Otolaryngol Head Neck Surg 2006; 134: 717–723.
- 7Rosenfeld RM. Clinical research in otolaryngology journals. Arch Otolaryngol Head Neck Surg 1991; 117: 164–170.
- 8Shin JJ, Rauch SD, Wasserman J, Coblens O, Randolph GW. Evidence-based medicine in otolaryngology, part 2: the current state of affairs. Otolaryngol Head Neck Surg 2011; 144: 331–337.
- 9Stewart MG. Reporting levels of evidence. Laryngoscope 2014; 124: 358.
- 10Silver JA, Mascarella M, Tali G, et al. Levels of evidence in rhinology and skull base surgery research. Otolaryngol Head Neck Surg 2021; 165: 477–482.
- 11Eggerstedt M, Shay AD, Brown HJ, et al. An update on level of evidence trends in facial plastic surgery research. Facial Plast Surg Aesthet Med 2020; 22: 105–109.
- 12Socolovsky LD, Krishnan P, Ramraj R, et al. Quality of otology and neurotology research in otolaryngology journals. Otol Neurotol 2020; 41: 1021–1026.
- 13 OCEBM Levels of Evidence Working Group. The Oxford 2011 Levels of Evidence. Oxford, UK: Oxford Centre for Evidence-Based Medicine; 2011.
- 14 OCEBM Levels of Evidence Working Group. The 2011 Oxford CEBM Levels of Evidence (Introductory Document). Periodical [Serial Online]. Oxford, UK: Oxford Centre for Evidence-Based Medicine; 2011.
- 15 OCEBM Levels of Evidence Working Group. Explanation of the 2011 Oxford Centre for Evidence-Based Medicine (OCEBM) Levels of Evidence (Background Document). Periodical [Serial Online]. Oxford, UK: Oxford Centre for Evidence-Based Medicine; 2011.
- 16 StataCorp. Stata Statistical Software: Release 15. College Station. TX: StataCorp LLC; 2017.
- 17Gates GA. So where's the evidence? Otolaryngol Head Neck Surg 1999; 120: 619–620.
- 18Maran AG, Molony NC, Armstrong MW, Ah-See K. Is there an evidence base for the practice of ENT surgery? Clin Otolaryngol Allied Sci 1997; 22: 152–157.
- 19Ah-See KW, Molony NC, Maran AG. Trends in randomized controlled trials in ENT: a 30-year review. J Laryngol Otol 1997; 111: 611–613.
- 20Yao F, Singer M, Rosenfeld RM. Randomized controlled trials in otolaryngology journals. Otolaryngol Head Neck Surg 2007; 137: 539–544.
- 21Merani S, Switzer N, Kayssi A, Blitz M, Ahmed N, Shapiro AM. Research productivity of residents and surgeons with formal research training. J Surg Educ 2014; 71: 865–870.
- 22Xu CC, Cote DWJ, Chowdhury RH, Morrissey AT, Ansari K. Trends in level of evidence in facial plastic surgery research. Plast Reconstr Surg 2011; 127: 1499–1504.
- 23Finkel ML, Brown HA, Gerber LM, Supino PG. Teaching evidence-based medicine to medical students. Med Teach 2003; 25: 202–204.
- 24Sugrue CM, Joyce CW, Sugrue RM, Carroll SM. Trends in the level of evidence in clinical hand surgery research. Hand 2016; 11: 211–215.
- 25Hollenberg AM, Bernstein DN, Baldwin AL, Beltejar MJ, Rubery PT, Mesfin A. Trends and characteristics of spine research from 2006 to 2015: a review of spine articles in a high impact general orthopedic journal. Spine 2020; 45: 141–147.
- 26Zaidi R, Abbassian A, Cro S, et al. Levels of evidence in foot and ankle surgery literature: progress from 2000 to 2010? J Bone Joint Surg Am 2012; 94: e1121–e1110.
- 27Judy RP, Shin JJ, McCrum C, Ayeni OR, Samuelsson K, Musahl V. Level of evidence and authorship trends of clinical studies in knee surgery, sports traumatology, arthroscopy, 1995–2015. Knee Surg Sports Traumatol Arthrosc 2018; 26: 9–14.
- 28Cashin MS, Kelley SP, Douziech JR, Varghese RA, Hamilton QP, Mulpuri K. The levels of evidence in pediatric orthopaedic journals: where are we now? J Pediatr Orthop 2011; 31: 721–725.
- 29Joseph PD, Craig JC, Caldwell PH. Clinical trials in children. Br J Clin Pharmacol 2015; 79: 357–369.
- 30Field MJ, Behrman RE. Ethical Conduct of Clinical Research Involving Children. Washington, DC: The National Academies; 2004.
- 31Haynes RB, McKibbon KA, Fitzgerald D, Guyatt GH, Walker CJ, Sackett DL. How to keep up with the medical literature: II. Deciding which journals to read regularly. Ann Intern Med 1986; 105: 309–312.
- 32Bornmann L, Daniel HD. What do citation counts measure? A review of studies on citing behavior. J Doc 2008; 64: 45–80.
- 33Venekamp RP, Sanders S, Glasziou PP, Del Mar CB, Rovers MM. Antibiotics for acute otitis media in children. Cochrane Database Syst Rev 2013; 1:CD000219.
- 34Donnelly MJ, Quraishi MS, McShane DP. ENT and general practice: a study of paediatric ENT problems seen in general practice and recommendations for general practitioner training in ENT in Ireland. Ir J Med Sci 1995; 164: 209–211.
- 35Hu A, Sardesai MG, Meyer TK. A need for otolaryngology education among primary care providers. Med Educ Online 2012; 17: 17350.
- 36Hannaford PC, Simpson JA, Bisset AF, Davis A, McKerrow W, Mills R. The prevalence of ear, nose and throat problems in the community: results from a national cross-sectional postal survey in Scotland. Fam Pract 2005; 22: 227–233.
- 37Shetty KR, Komshian SR, Devaiah A, Levi JR. Review of pediatric otolaryngology clinical trials: past trends and future opportunities. Laryngoscope 2020; 130: 2719–2724.
- 38Ashurst JV, Danishyar A. Acute Otitis Media. Treasure Island (FL): StatPearls; 2020.
- 39Qiu L. A study of interdisciplinary research collaboration. Res Eval 1992; 2: 169–175.
10.1093/rev/2.3.169 Google Scholar
- 40Aldieri L, Guida G, Kotsemir M, Vinci CP. An investigation of impact of research collaboration on academic performance in Italy. Qual Quant 2019; 53: 2003–2040.
10.1007/s11135-019-00853-1 Google Scholar
- 41Aldieri L, Kotsemir M, Vinci CP. The impact of research collaboration on academic performance: an empirical analysis for some European countries. Socioecon Plann Sci 2018; 62: 13–30.
- 42Fu JY, Zhang X, Zhao YH, Tong HF, Chen DZ, Huang MH. Bibliometric analysis of acupuncture research fronts and their worldwide distribution over three decades. Afr J Tradit Complement Altern Med 2017; 14: 257–273.
- 43Chuang K-Y, Ho Y-S. An evaluation based on highly cited publications in Taiwan. Curr Sci 2015; 108: 933–941.
- 44Camp M, Escott BG. Authorship proliferation in the orthopaedic literature. J Bone Joint Surg Am 2013; 95:e44.
- 45Wislar JS, Flanagin A, Fontanarosa PB, Deangelis CD. Honorary and ghost authorship in high impact biomedical journals: a cross sectional survey. BMJ 2011; 343:d6128.
- 46Rennie D, Flanagin A. Authorship! Authorship!: guests, ghosts, grafters, and the two-sided coin. JAMA 1994; 271: 469–471.
- 47Vučković-Dekić L. Multiauthorship and false authorship: why worrying about this? Srp Arh Celok Lek 2014; 142: 637–640.
- 48Pintér A. Changing trends in authorship patterns in the JPS: publish or perish. J Pediatr Surg 2013; 48: 412–417.
- 49Garner RM, Hirsch JA, Albuquerque FC, Fargen KM. Bibliometric indices: defining academic productivity and citation rates of researchers, departments and journals. J Neurointerv Surg 2018; 10: 102–106.
- 50Roldan-Valadez E, Salazar-Ruiz SY, Ibarra-Contreras R, Rios C. Current concepts on bibliometrics: a brief review about impact factor, Eigenfactor score, CiteScore, SCImago journal rank, source-normalised impact per paper, H-index, and alternative metrics. Ir J Med Sci 2019; 188: 939–951.
- 51Grech V, Rizk DEE. Increasing importance of research metrics: journal impact factor and H-index. Int Urogynecol J 2018; 29: 619–620.
- 52Joshi ND, Lieber B, Wong K, Al-Alam E, Agarwal N, Diaz V. Social media in neurosurgery: using ResearchGate. World Neurosurg 2019; 127: e950–e956.
- 53Brooke BS, Nathan H, Pawlik TM. Trends in the quality of highly cited surgical research over the past 20 years. Ann Surg 2009; 249: 162–167.
- 54Brooke BS, Ghaferi AA, Kibbe MR. Effective use of reporting guidelines to improve the quality of surgical research. JAMA Surg 2021; 156: 515–516.
- 55Begg C, Cho M, Eastwood S, et al. Improving the quality of reporting of randomized controlled trials. The CONSORT Statement. JAMA 1996; 276: 637–639.
- 56Simera I, Moher D, Hirst A, Hoey J, Schulz KF, Altman DG. Transparent and accurate reporting increases reliability, utility, and impact of your research: reporting guidelines and the EQUATOR network. BMC Med 2010; 8: 24.
- 57Sener A, Anderson CC, Auger FA, et al. Best practices for enhancing surgical research: a perspective from the Canadian Association of Chairs of surgical research. Can J Surg 2019; 62: 488–498.
- 58Leitch KK, Walker PM. Surgeon compensation and motivation. Arch Surg 2000; 135: 708–712.
- 59Keswani SG, Moles CM, Morowitz M, et al. The future of basic science in academic surgery: identifying barriers to success for surgeon-scientists. Ann Surg 2017; 265: 1053–1059.
- 60Rangel SJ, Efron B, Moss RL. Recent trends in National Institutes of Health funding of surgical research. Ann Surg 2002; 236: 277–286.
Citing Literature
