Quality in Health Professions Education Research
Bridget C. O'Brie
University of California San Francisco, San Francisco, California, USA
Search for more papers by this authorEliot L. Rees
Keele University, Staffordshire, UK
University College London, London, UK
Search for more papers by this authorBridget C. O'Brie
University of California San Francisco, San Francisco, California, USA
Search for more papers by this authorEliot L. Rees
Keele University, Staffordshire, UK
University College London, London, UK
Search for more papers by this authorCharlotte Rees
The University of Newcastle, Callaghan, New South Wales, Australia
Search for more papers by this authorLynn Monrouxe
The University of Sydney, Camperdown, New South Wales, Australia
Search for more papers by this authorBridget O'Brien
University of California San Francisco, San Francisco, California, USA
Search for more papers by this authorSummary
This chapter discusses different ways of conceptualising quality in health professions education research (HPER). It explains a framework for quality in HPER. The chapter describes the difference between reporting guidelines and critical appraisal tools. It also discusses the strengths and weaknesses of different tools for reporting and critical appraisal. The chapter also explains the quality of a research report using appropriate tools and provides feedback to report authors. Conducting quality research requires time, resources, and training — all of which have been identified as lacking due to insufficient funding for HPER. Positioning is important when interpreting the findings of a study and describing its implications. Relevance considers the relationship between a study's anticipated implications and important contextual factors such as audience, setting, current events, and priorities. Internal coherence is an important marker of quality as it ‘makes clear the conventions of the research to support interpretations’.
References
- Charmaz K . Constructing Grounded Theory , 2nd ed. Thousand Oaks, Calif .: SAGE ; 2014 .
- Palermo C , Reidlinger DP , Rees CE . Internal coherence matters: lessons for nutrition and dietetics research . Nutr Diet . 2021 ; 78 ( 3 ): 252 – 267 .
- Ringsted C , Hodges B , Scherpbier A . ‘The research compass’: an introduction to research in medical education: AMEE Guide no. 56 . Med Teach . 2011 ; 33 ( 9 ): 695 – 709 .
- Watling CJ , Lingard L . Grounded theory in medical education research: AMEE Guide no 70 . Med Teach . 2012 ; 34 ( 10 ): 850 – 861 .
- Artino AR , La Rochelle JS , Dezee KJ , et al. Developing questionnaires for educational research: AMEE Guide no. 87 . Med Teach . 2014 ; 36 ( 6 ): 463 – 474 .
- Durning SJ , Carline JD . Review Criteria for Research Manuscripts , 2nd ed. Washington, DC : Association of American Medical Colleges ; 2015 .
- Critical Appraisal Skills Programme . CASP Checklist. https://casp-uk.net/casp-tools-checklists (accessed 1 September 2022 ).
- Bligh J , Brice J . What is the value of good medical education research? Med Educ . 2008 ; 42 ( 7 ): 652 – 653 .
- Dornan T , Peile E , Spencer J . On ‘evidence’ . Med Educ . 2008 ; 42 ( 3 ): 232 – 234 .
- Dornan T , Peile E , Spencer J . In defence of the existing strengths of medical education research . Med Educ . 2009 ; 43 ( 4 ): 391 .
- Ellaway RH . Journal standards . Adv Health Sci Educ . 2022 ; 27 ( 1 ): 1 – 5 .
- Eva KW . Broadening the debate about quality in medical education research . Med Educ . 2009 ; 43 ( 4 ): 294 – 296 .
- Majid U , Vanstone M . Appraising qualitative research for evidence syntheses: a compendium of quality appraisal tools . Qual Health Res . 2018 ; 28 ( 13 ): 2115 – 2131 .
- Monrouxe LV , Rees CE . Picking up the gauntlet: constructing medical education as a social science . Med Educ . 2009 ; 43 ( 3 ): 196 – 198 .
-
Pigott TD
,
Tocci C
,
Ryan AM
, et al.
Introduction – quality of research evidence in education: how do we know?
Rev Res Educ
.
2021
;
45
(
1
):
vii
–
xii
.
10.3102/0091732X211001824 Google Scholar
- Thistlethwaite J , Davies H , Dornan T , et al. What is evidence? Reflections on the AMEE symposium, Vienna, August 2011 . Med Teach . 2012 ; 34 ( 6 ): 454 – 457 .
- Todres M , Stephenson A , Jones R . Medical education research remains the poor relation . BMJ . 2007 ; 335 ( 7615 ): 333 – 335 .
- Varpio L , Driessen E , Maggio L , et al. Advice for authors from the editors of Perspectives on Medical Education: getting your research published . Perspect Med Educ . 2018 ; 7 ( 6 ): 343 – 347 .
- West DC , Miller KH , Artino AR . Foreword: characteristics of RIME papers that make the cut . Acad Med . 2016 ; 91 ( 11 ): Si – Siii .
- [No authors listed]. Good advice from the deputy editors of Medical Education: the sequel . Med Educ . 2022 ; 56 ( 5 ): 468 – 469 .
- Bordage G . Conceptual frameworks to illuminate and magnify . Med Educ . 2009 ; 43 ( 4 ): 312 – 319 .
- Cook DA , Beckman TJ , Bordage G . Quality of reporting experimental studies in medical education: a systematic review . Med Educ . 2007 ; 41 ( 8 ): 737 – 745 .
- Meyer HS , Durning SJ , Sklar DP , et al. Making the first cut: an analysis of academic medicine editors' reasons for not sending manuscripts out for external peer review . Acad Med . 2018 ; 93 ( 3 ): 464 – 470 .
- Norman G . Data dredging, salami-slicing, and other successful strategies to ensure rejection: twelve tips on how to not get your paper published . Adv Health Sci Educ . 2014 ; 19 ( 1 ): 1 – 5 .
- Regehr G . It's NOT rocket science: rethinking our metaphors for research in health professions education . Med Educ . 2010 ; 44 ( 1 ): 31 – 39 .
- Lingard LA . Joining a conversation: problem/gap/hook heuristic . Perspect Med Educ . 2015 ; 4 ( 5 ): 252 – 253 .
- Norman G . RCT = results confounded and trivial: the perils of grand educational experiments . Med Educ . 2003 ; 37 ( 7 ): 582 – 584 .
- Norman G . Is experimental research passé . Adv Health Sci Educ . 2010 ; 15 ( 3 ): 297 – 301 .
- Archer J , McManus C , Woolf K , et al. Without proper research funding, how can medical education be evidence based? BMJ . 2015 ; 350 : h3445 .
- Gruppen LD , Durning SJ . Needles and haystacks: finding funding for medical education research . Acad Med . 2016 ; 91 ( 4 ): 480 – 484 .
- Eva KW . Publishing during COVID-19: lessons for health professions education research . Med Educ . 2021 ; 55 ( 3 ): 278 – 280 .
- Roberts LW , Coverdale J . Editorial decision making for Academic Medicine, 2021 . Acad Med . 2021 ; 96 ( 1 ): 1 – 4 .
- Sethi A , Schofield S , McAleer S , et al. The influence of postgraduate qualifications on educational identity formation of healthcare professionals . Adv Health Sci Educ . 2018 ; 23 ( 3 ): 567 – 585 .
- Tekian A . Doctoral programs in health professions education . Med Teach . 2014 ; 36 ( 1 ): 73 – 81 .
- ten Cate O . Health professions education scholarship: the emergence, current status, and future of a discipline in its own right . FASEB Bioadv . 2021 ; 3 ( 7 ): 510 – 522 .
- Thomas A , Bussières A . Leveraging knowledge translation and implementation science in the pursuit of evidence informed health professions education . Adv Health Sci Educ . 2021 ; 26 ( 3 ): 1157 – 1171 .
- Maggio LA , Costello JA , Norton C , et al. Knowledge syntheses in medical education: a bibliometric analysis . Perspect Med Educ . 2021 ; 10 ( 2 ): 79 – 87 .
- Varpio L , Paradis E , Uijtdehaage S , et al. The distinctions between theory, theoretical framework, and conceptual framework . Acad Med . 2020 ; 95 ( 7 ): 989 – 994 .
- Young M , LaDonna K , Varpio L , et al. Focal length fluidity: research questions in medical education research and scholarship . Acad Med . 2019 ; 94 ( 11S ): S1 – S4 .
- Lingard LA . Writing an effective literature review. Part I: mapping the gap . Perspect Med Educ . 2018 ; 7 ( 1 ): 47 – 49 .
- Miller KH , O'Brien B , Karani R . Foreword: the role of relevance in medical education research . Acad Med . 2018 ; 93 ( 11S ): Si – Siii .
- Pangaro L , McGaghie W . Chapter 8: Relevance . In SJ Durning , JD Carline , eds. Review Criteria for Research Manuscripts , 2nd ed. Washington, DC : Association of American Medical Colleges ; 2015 : 25 – 27 .
- The American Educational Research Association (AERA). Professional ethics. AERA; 2011 . https://www.aera.net/About-AERA/AERA-Rules-Policies/Professional-Ethics (accessed 29 August 2022 ).
- British Educational Research Association (BERA) . Ethical Guidelines for Educational Research , 4th ed. London : BERA ; 2018 . https://www.bera.ac.uk/researchers-resources/publications/ethical-guidelines-for-educational-research-2018 (accessed 29 August 2022 ).
- World Conference on Research Integrity . Singapore Statement on Research Integrity . Singapore : World Conference on Research Integrity ; 2010 . https://wcrif.org/guidance/singapore-statement (accessed 29 August 2022 ).
- Artino AR , Driessen EW , Maggio LA . Ethical shades of gray: international frequency of scientific misconduct and questionable research practices in health professions education . Acad Med . 2019 ; 94 ( 1 ): 76 – 84 .
- Maggio LA , Artino AR , Watling CJ , et al. Exploring researchers' perspectives on authorship decision making . Med Educ . 2019 ; 53 ( 12 ): 1253 – 1262 .
- Uijtdehaage S , Mavis B , Durning SJ . Whose paper is it anyway? Authorship criteria according to established scholars in health professions education . Acad Med . 2018 ; 93 ( 8 ): 1171 – 1175 .
- International Committee of Medical Journal Editors (ICMJE) . Defining the role of authors and contributors. ICMJE ; 2022 . https://www.icmje.org/recommendations/browse/roles-and-responsibilities/defining-the-role-of-authors-and-contributors.html (accessed 29 August 2022 ).
- National Health and Medical Research Council (NHMRC) . Ethical conduct in research with Aboriginal and Torres Strait Islander peoples and communities: guidelines for researchers and stakeholders. Australian Government: Canberra; 2018 . https://www.nhmrc.gov.au/about-us/resources/ethical-conduct-research-aboriginal-and-torres-strait-islander-peoples-and-communities (accessed 29 August 2022 ).
- National Health and Medical Research Council (NHMRC) . Australian Research Council (ARC), and Universities Australia. National Statement on Ethical Conduct in Human Research; Australia : NHMRC ; 2018 . https://www.nhmrc.gov.au/about-us/publications/national-statement-ethical-conduct-human-research-2007-updated-2018#block-views-block-file-attachments-content-block-1 (accessed 26 October 2022 ).
- Rees EL , Burton O , Asif A , et al. A method for the madness: an international survey of health professions education authors' journal choice . Perspect Med Educ . 2022 ; 11 ( 3 ): 165 – 172 .
- Kemp C , van Herwerden L , Molloy E , et al. How do students offer value to organisations through work integrated learning? A qualitative study using social exchange theory . Adv Health Sci Educ . 2021 ; 26 ( 3 ): 1075 – 1093 .
- O'Brien BC , Zapata J , Chang A , et al. Bridging medical education goals and health system outcomes: an instrumental case study of pre-clerkship students' improvement projects . Perspect Med Educ . 2022 ; 11 ( 4 ): 179 – 186 .
- Carter SM , Little M . Justifying knowledge, justifying method, taking action: epistemologies, methodologies, and methods in qualitative research . Qual Health Res . 2007 ; 17 ( 10 ): 1316 – 1328 .
- Johnston J , Bennett D , Kajamaa A . How to… get started with theory in education . Clin Teach . 2018 ; 15 ( 4 ): 294 – 297 .
-
Varpio L
,
Martimianakis MA
,
Mylopoulos M
.
Qualitative research methodologies: embracing methodological borrowing, shifting and importing
. In
J Cleland
,
S Durning
, eds.
Researching Medical Education
.
Oxford
:
Wiley Blackwell
;
2015
:
245
–
256
.
10.1002/9781118838983.ch21 Google Scholar
- Gill TG , Gill TR . What is research rigor? Lessons for a transdiscipline . Inf Sci: The Int J Emerg Transdiscipline . 2020 ; 23 : 47 – 76 .
- Buetow S , Zawaly K . Rethinking researcher bias in health research . J Eval Clin Pract . 2022 ; 28 ( 5 ): 843 – 846 .
- Hopkins RM , Regehr G , Pratt DD . A framework for negotiating positionality in phenomenological research . Med Teach . 2017 ; 39 ( 1 ): 20 – 25 .
- Olmos-Vega FM , Stalmeijer RE , Varpio L , et al. A practical guide to reflexivity in qualitative research: AMEE Guide no. 149 . Med Teach . 2022 ; 1 – 11 . https://pubmed.ncbi.nlm.nih.gov/35389310 . Epub ahead of print.
- Varpio L , O'Brien B , Rees CE , et al. The applicability of generalisability and bias to health professions education's research . Med Educ . 2021 ; 55 ( 2 ): 167 – 173 .
- Stenfors T , Kajamaa A , Bennett D . How to … assess the quality of qualitative research . Clin Teach . 2020 ; 17 ( 6 ): 596 – 599 .
- Lingard LA , Watling C . Story, Not Study: 30 Brief Lessons to Inspire Health Researchers as Writers . Cham : Springer ; 2020 .
- Coverdale JH , Roberts LW , Balon R , et al. Writing for academia: getting your research into print: AMEE Guide no. 74 . Med Teach . 2013 ; 35 ( 2 ): e926 – 934 .
- Asif A , Burton O . Comic Sans or common sense? Graphic design for clinical teachers . Clin Teach . 2021 ; 18 ( 6 ): 583 – 589 .
- EQUATOR Network . Enhancing the QUAlity and Transparency Of Health Research. Reporting Guidelines for Main Study Types; https://www.equator-network.org (accessed 29 August 2022 ).
- Ginsburg S , Lynch M , Walsh CM . A fine balance: how authors strategize around journal submission . Acad Med . 2018 ; 93 ( 8 ): 1176 – 1181 .
- Fleming JI , Wilson SE , Hart SA , et al. Open accessibility in education research: enhancing the credibility, equity, impact, and efficiency of research . Educ Psychol . 2021 ; 56 ( 2 ): 110 – 121 .
- Reed MS. The Research Impact Handbook , 2nd ed. Aberdeenshire : Fast Track Impact ; 2018 .
- Research Excellence Framework (REF) . Guidance on submissions. Annex C. UK : REF; 2021 . https://www.ref.ac.uk/media/1447/ref-2019_01-guidance-on-submissions.pdf (accessed 22 March 2023 ).
-
Dillman DA
,
Smyth JD
,
Christian LM
.
Internet, Phone, Mail, and Mixed-Mode Surveys: The Tailored Design Method
,
4th
ed.
Hoboken
:
Wiley
;
2014
.
10.1002/9781394260645 Google Scholar
- Bunton SA , Sandberg SF . Case study research in health professions education . Acad Med . 2016 ; 91 ( 12 ): e3 .
- Cleland J , MacLeod A , Ellaway RH . The curious case of case study research . Med Educ . 2021 ; 55 ( 10 ): 1131 – 1141 .
- Yazan B . Three approaches to case study methods in education: Yin, Merriam, and Stake . Qual Rep . 2015 ; 20 ( 2 ): 134 – 152 .
- Buccheri RK , Sharifi C . Critical appraisal tools and reporting guidelines for evidence-based practice . Worldviews Evid Based Nurs . 2017 ; 14 ( 6 ): 463 – 472 .
- Critical Appraisal Skills Programme (CASP) . Critical appraisal. https://casp-uk.net/glossary/critical-appraisal (accessed 1 September 2022 ).
- Altman DG , Simera I. A history of the evolution of guidelines for reporting medical research: the long road to the EQUATOR Network . J Roy Soc Med . 2016 ; 109 ( 2 ): 67 – 77 .
- Critical appraisal tools: introduction. Duquesne University; https://guides.library.duq.edu/critappraise (accessed 3 September 2022 ).
- O'Brien BC , West CP , Coverdale JH , et al. On the use and value of reporting guidelines in health professions education research . Acad Med . 2020 ; 95 ( 11 ): 1619 – 1622 .
- Altman DG , Schulz KF , Moher D , et al. The revised CONSORT statement for reporting randomized trials: explanation and elaboration . Ann Intern Med . 2001 ; 134 ( 8 ): 663 – 694 .
- von Elm E , Altman DG , Egger M , et al. The strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies . PLoS Med . 2007 ; 4 ( 10 ): e296 .
- Page MJ , McKenzie JE , Bossuyt PM , et al. Updating guidance for reporting systematic reviews: development of the PRISMA 2020 statement . J Clin Epidemiol . 2021 ; 134 : 103 – 112 .
- Gordon M , Gibbs T . STORIES statement: publication standards for healthcare education evidence synthesis . BMC Med . 2014 ; 12 : 143 .
- Husereau D , Drummond M , Petrou S , et al. CHEERS good reporting practices task force. consolidated health economic evaluation reporting standards (CHEERS)—explanation and elaboration: a report of the ISPOR health economic evaluation publication guidelines good reporting practices task force . Value Health . 2013 ; 16 ( 2 ): 231 – 250 .
- Cook DA , Reed DA . Appraising the quality of medical education research methods: the Medical Education Research Study Quality Instrument and the Newcastle–Ottawa Scale-Education . Acad Med . 2015 ; 90 ( 8 ): 1067 – 1076 .
- Kmet LM , Lee RC , Cook LS . HTA Initiative #13 Standard quality assessment criteria for evaluating primary research papers from a variety of fields. Alberta Heritage Foundation; 2004 .
- Wong G , Greenhalgh T , Westhorp G , et al. RAMESES publication standards: meta-narrative reviews . BMC Med . 2013 ; 11 : 20 .
- Wong G , Westhorp G , Manzano A , et al. RAMESES II reporting standards for realist evaluations . BMC Med . 2016 ; 14 ( 1 ): 96 .
- O'Brien BC , Harris IB , Beckman TJ , et al. Standards for reporting qualitative research: a synthesis of recommendations . Acad Med . 2014 ; 89 ( 9 ): 1245 – 1251 .
- Tong A , Sainsbury P , Craig J . Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups . Int J Qual Health Care . 2007 ; 19 ( 6 ): 349 – 357 .
- Tong A , Flemming K , McInnes E , et al. Enhancing transparency in reporting the synthesis of qualitative research: ENTREQ . BMC Med Res Methodol . 2012 ; 12 : 181 .
- Goodman D , Ogrinc G , Davies L , et al. Explanation and elaboration of the SQUIRE (Standards for Quality Improvement Reporting Excellence) guidelines V.2.0: examples of SQUIRE elements in the healthcare improvement literature . BMJ Qual Safe . 2016 ; 25 ( 12 ): e27 .
- Ogrinc G , Armstrong GE , Dolansky MA , et al. SQUIRE-EDU (Standards for Quality Improvement Reporting Excellence in Education): publication guidelines for educational improvement . Acad Med . 2019 ; 94 ( 10 ): 1461 – 1470 .
- Abelson J , Li K , Wilson G , et al. Supporting quality public and patient engagement in health system organizations: development and usability testing of the Public and Patient Engagement Evaluation Tool . Health Exp . 2016 ; 19 ( 4 ): 817 – 827 .
- Staniszewska S , Brett J , Simera I , et al. GRIPP2 reporting checklists: tools to improve reporting of patient and public involvement in research . BMJ . 2017 ; 358 : j3453 .
- National Health and Medical Research Council (NHMRC) . Statement on consumer and community involvement in health and medical research. https://www.nhmrc.gov.au/about-us/publications/statement-consumer-and-community-involvement-health-and-medical-research (accessed 22 March 2023 ).
- O'Cathain A , Murphy E , Nicholl J . The quality of mixed methods studies in health services research . J Health Serv Res Policy . 2008 ; 13 ( 2 ): 92 – 98 .
- Hong QN , Pluye P , Fàbregues S , et al. Mixed Methods Appraisal Tool (MMAT), version 2018. Registration of Copyright (#1148552); Industry Canada : Canadian Intellectual Property Office ; 2018 .
-
Bradbury H
,
Lewis R
,
Embury DC
.
Education action research: with and for the next generation
. In
CA Mertler
, ed.
The Wiley Handbook of Action Research in Education
.
Hoboken
:
John Wiley & Sons, Inc.
;
2019
:
7
–
28
.
10.1002/9781119399490.ch1 Google Scholar
- Evans D. Hierarchy of evidence: a framework for ranking evidence evaluating healthcare interventions . J Clin Nurs . 2003 ; 12 ( 1 ): 77 – 84 .
- Greenhalgh T. How to read a paper: getting your bearings (deciding what the paper is about) . BMJ . 1997 ; 315 ( 7102 ): 243 – 246 .
- Guyatt GH , Sackett DL , Sinclair JC , et al. Users' guides to the medical literature. IX. A method for grading health care recommendations . JAMA . 1995 ; 274 ( 22 ): 1800 – 1804 .
- Barbour RS . Checklist for improving rigour in qualitative research: a case of the tail wagging the dog ? BMJ . 2001 ; 322 ( 7294 ): 1115 – 1117 .
- Morse J . Why the Qualitative Health Research (QHR) review process does not use checklists . Qual Health Res . 2021 ; 31 ( 5 ): 819 – 821 .
- Wharton T . Rigor, transparency, and reporting social science research: why guidelines don't have to kill your story . Res Soc Work Pract . 2017 ; 27 ( 4 ): 487 – 493 .
- Barbour RS , Barbour M . Evaluating and synthesizing qualitative research: the need to develop a distinctive approach . J Eval Clin Pract . 2003 ; 9 ( 2 ): 179 – 186 .
- Varpio L , Ajjawi R , Monrouxe LV , et al. Shedding the cobra effect: problematising thematic emergence, triangulation, saturation and member checking . Med Educ . 2017 ; 51 ( 1 ): 40 – 50 .
- Haile ZT . Critical appraisal tools and reporting guidelines . J Hum Lact . 2022 ; 38 ( 1 ): 21 – 27 .
- Eva KW . Altruism as enlightened self-interest: how helping others through peer review helps you . Med Educ . 2021 ; 55 ( 8 ): 880 – 882 .
- Dumenco L , Engle DL , Goodell K , et al. Expanding group peer review: a proposal for medical education scholarship . Acad Med . 2017 ; 92 ( 2 ): 147 – 149 .
- Richards BF , Cardell EM , Chow CJ , et al. Discovering the benefits of group peer review of submitted manuscripts . Teach Learn Med . 2020 ; 32 ( 1 ): 104 – 109 .
- Azer SA , Ramani S , Peterson R . Becoming a peer reviewer to medical education journals . Med Teach . 2012 ; 34 ( 9 ): 698 – 704 .
- Durning SJ , Sklar DP , Driessen EW , et al. “This manuscript was a complete waste of time”: reviewer etiquette matters . Acad Med . 2019 ; 94 ( 6 ): 744 – 745 .
- Eva KW. The reviewer is always right: peer review of research in medical education . Med Educ . 2009 ; 43 ( 1 ): 2 – 4 .
- Yarris LM , Gottlieb M , Scott K , et al. Academic Primer Series: key papers about peer review . West J Emerg Med . 2017 ; 18 ( 4 ): 721 – 728 .