Breakout Session
Free Access
Evaluating Educational Interventions in Emergency Medicine
Nicole M. Deiorio MD,
Michael T. Fitch MD, PhD,
Julianna Jung MD,
Susan B. Promes MD,
Lorraine G. Thibodeau MD,
Wendy L. Woolley DO,
Michael A. Gisondi MD,
Larry D. Gruppen PhD,
Corresponding Author
Nicole M. Deiorio MD
Department of Emergency Medicine, Oregon Health and Science University, Portland, OR
Address for correspondence and reprints: Nicole M. Deiorio, MD; e-mail: [email protected].Search for more papers by this authorMichael T. Fitch MD, PhD
Department of Emergency Medicine, Wake Forest University, Salem, NC
Search for more papers by this authorJulianna Jung MD
Department of Emergency Medicine, Johns Hopkins University, Baltimore, MD
Search for more papers by this authorSusan B. Promes MD
Department of Emergency Medicine, University of California San Francisco, San Francisco, CA
Search for more papers by this authorLorraine G. Thibodeau MD
Department of Emergency Medicine, Albany Medical College, Albany, NY
Search for more papers by this authorWendy L. Woolley DO
Department of Emergency Medicine, Albany Medical College, Albany, NY
Search for more papers by this authorMichael A. Gisondi MD
Department of Emergency Medicine, Northwestern University, Chicago, IL
Search for more papers by this authorLarry D. Gruppen PhD
Department of Medical Education, University of Michigan, Ann Arbor, MI
Search for more papers by this authorNicole M. Deiorio MD,
Michael T. Fitch MD, PhD,
Julianna Jung MD,
Susan B. Promes MD,
Lorraine G. Thibodeau MD,
Wendy L. Woolley DO,
Michael A. Gisondi MD,
Larry D. Gruppen PhD,
Corresponding Author
Nicole M. Deiorio MD
Department of Emergency Medicine, Oregon Health and Science University, Portland, OR
Address for correspondence and reprints: Nicole M. Deiorio, MD; e-mail: [email protected].Search for more papers by this authorMichael T. Fitch MD, PhD
Department of Emergency Medicine, Wake Forest University, Salem, NC
Search for more papers by this authorJulianna Jung MD
Department of Emergency Medicine, Johns Hopkins University, Baltimore, MD
Search for more papers by this authorSusan B. Promes MD
Department of Emergency Medicine, University of California San Francisco, San Francisco, CA
Search for more papers by this authorLorraine G. Thibodeau MD
Department of Emergency Medicine, Albany Medical College, Albany, NY
Search for more papers by this authorWendy L. Woolley DO
Department of Emergency Medicine, Albany Medical College, Albany, NY
Search for more papers by this authorMichael A. Gisondi MD
Department of Emergency Medicine, Northwestern University, Chicago, IL
Search for more papers by this authorLarry D. Gruppen PhD
Department of Medical Education, University of Michigan, Ann Arbor, MI
Search for more papers by this authorConsensus conference participants: Diane Birnbaumer, Chad Braneeki, David Cone, Lauren Conlon, Robert Cooney, Nicole Deiorio, Rakesh Engineer, Michael Fitch, Jenna Fredette, Gus Garmel, Michael Gisondi, Jim Gordon, Veronica Greer, David Griffen, Larry Gruppen, Lindsay Harmon, Keli Kwok, Tricia Lanter, John Marshall, John Montgomery, Rahul Patwari, Susan Promes, Janet Riddle, Jesse Rideout, David Salzman, Sneha Shah, Sarah Stahmer, Alison Suarez, Demian Szyld, Taku Taira, Laurie Thibodeau, Lindsey Tilt, Sal Vicarino, Jason Wagner, Grant Wei, Brian Weitzman, Moshe Weizberg, Sarah Williams, Millie Willy, and Stella Yiu.
This paper reports on a workshop session of the 2012 Academic Emergency Medicine consensus conference, “Education Research in Emergency Medicine: Opportunities, Challenges, and Strategies for Success,” May 9, 2012, Chicago, IL.
The authors have no relevant financial information or potential conflicts of interest to disclose.
Abstract
This article presents the proceedings of the 2012 Academic Emergency Medicine consensus conference breakout group charged with identifying areas necessary for future research regarding effectiveness of educational interventions for teaching emergency medicine (EM) knowledge, skills, and attitudes outside of the clinical setting. The objective was to summarize both medical and nonmedical education literature and report the consensus formation methods and results. The authors present final statements to guide future research aimed at evaluating the best methods for understanding and developing successful EM curricula using all types of educational interventions.
References
- 1Kirkpatrick DL, Kirkpatrick JD. Evaluating Training Programs, 3rd ed. San Francisco, CA: Berrett-Koehler Publishers, 2006.
- 2Norcini J, Burch V. Workplace-based assessment as an educational tool: AMEE Guide No. 31. Med Teach. 2007; 29: 855–71.
- 3Norcini JJ. Work-based assessment. BMJ. 2003; 326: 753–5.
- 4Berk RA. Using the 360 degrees multisource feedback model to evaluate teaching and professionalism. Med Teach. 2009; 31: 1073–80.
- 5Miller GE. The assessment of clinical skills/competence/performance. Acad Med. 1990; 65: S63–7.
- 6Eva KW, Regehr G. Self-assessment in the health professions: a reformulation and research agenda. Acad Med. 2005; 80: S46–54.
- 7McLoughlin C, Lee MJ. Social Software and Participatory Learning: Pedagogical Choices with Technology Affordances in the Web 2.0 Era. Proce-edings Ascilite Singapore 2007. Available at: http://www.ascilite.org.au/conferences/singapore07/procs/mcloughlin.pdf. Accessed Sep 15, 2012.
- 8Kamel Boulos MN, Wheeler S. The emerging Web 2.0 social software: an enabling suite of sociable technologies in health and health care education. Health Info Libr J. 2007; 24: 2–23.
- 9Van De Belt TH, Engelen LJ, Berben SA, Schoonhoven L. Definition of Health 2.0 and Medicine 2.0: a systematic review. J Med Internet Res. 2010; 12: e18.
- 10Bahner DP, Adkins E, Patel N, Donley C, Nagel R, Kman NE. How we use social media to supplement a novel curriculum in medical education. Med Teach. 2012; 34: 439–44.
- 11Chu SK, Woo M, King RB, Choi S, Cheng M, Koo P. Examining the application of Web 2.0 in medical-related organisations. Health Info Libr J. 2012; 29: 47–60.
- 12George DR, Dellasega C. Use of social media in graduate-level medical humanities education: two pilot studies from Penn State College of Medicine. Med Teach. 2011; 33: e429–34.
- 13Sandars J, Homer M, Pell G, Crocker T. Web 2.0 and social software: the medical student way of e-learning. Med Teach. 2008; 30: 308–12.
- 14Lemley T, Burnham JF. Web 2.0 tools in medical and nursing school curricula. J Med Libr Assoc. 2009; 97: 50–2.
- 15Moloney JF, Oakley B. Scaling online education: increasing access to higher education. J Asynchron Learn Netw. 2010: 14: 55–70. Available at: http://www.eric.ed.gov/ERICWebPortal/contentdelivery/servlet/ERICServlet?accno=EJ909842. Accessed Sep 15, 2012.
- 16 U.S. Department of Education. Evaluation of Evidence-based Practices in Online Learning: A Meta-analysis and Review of Online Learning Studies. Available at: http://www2.ed.gov/rschstat/eval/tech/evidence-based-practices/finalreport.pdf. Accessed Sep 15, 2012.
- 17Larson CK, Sung CH. Student performance: online versus blended versus face-to-face. J Asynchron Learn Netw. 2009; 13: 31–42.
- 18Fasse R, Humbert J, Rappold R. Rochester Institute of Technology: analyzing student success. J Asynchron Learn Netw. 2009; 13: 37–48.
- 19Lovell K, Vignare K. MSU medical colleges blended learning for first year science courses: uniting pedagogy to maximize experience and real world limitations. J Asynchron Learn Netw. 2009; 13: 55–63.
- 20Howlett D, Vincent T, Gainsborough N, et al. Integration of a case-based online module into an undergraduate curriculum: what is involved and is it effective? E-Learning. 2009; 6: 372–84.
10.2304/elea.2009.6.4.372 Google Scholar
- 21Burnette K, Ramundo M, Stevenson M, Beeson MS. Evaluation of a web-based asynchronous pediatric emergency medicine learning tool for residents and medical students. Acad Emerg Med. 2009; 16(Suppl 2): S46–50.
- 22Gisondi MA, Lu DW, Yen M, et al. Adaptation of EPEC-EM curriculum in a residency with asynchronous learning. West J Emerg Med. 2010; 11: 491–9.
- 23Lund A, Lam K, Parks P. Disaster medicine online: evaluation of an online, modular, interactive, asynchronous curriculum. CJEM. 2002; 4: 408–13.
- 24Sadosty AT, Goyal DG, Gene Hern H Jr, Kilian BJ, Beeson MS. Alternatives to the conference status quo: summary recommendations from the 2008 CORD Academic Assembly Conference Alternatives workgroup. Acad Emerg Med. 2009; 16(Suppl 2): S25–31.
- 25Ko PY, Scott JM, Mihai A, Grant WD. Comparison of a modified longitudinal simulation-based advanced cardiovascular life support to a traditional advanced cardiovascular life support curriculum in third-year medical students. Teach Learn Med. 2011; 23: 324–30.
- 26Papageorgiou A, Miles S, Fromage M, Kemmy J, Leinster SJ. Cross-sectional evaluation of a longitudinal consultation skills course at a new UK medical school. BMC Med Educ. 2011; 11: e55.
- 27Hirsh D, Gaufberg E, Ogur B, et al. Educational outcomes of the Harvard Medical School-Cambridge integrated clerkship: a way forward for medical education. Acad Med. 2012; 87: 643–50.
- 28Ogur B, Hirsh D, Krupat E, Bor D. The Harvard Medical School-Cambridge integrated clerkship: an innovative model of clinical education. Acad Med. 2007; 82: 397–404.
- 29Poncelet A, Bokser S, Calton B, et al. Development of a longitudinal integrated clerkship at an academic medical center. Med Educ Online. 2011; 16:doi: 10.3402/meo.v16i0.5939.
- 30Norris TE, Schaad DC, DeWitt D, Ogur B, Hunt DD, Consortium of Longitudinal Integrated Clerkships. Longitudinal integrated clerkships for medical students: an innovation adopted by medical schools in Australia, Canada, South Africa, and the United States. Acad Med. 2009l; 84: 902–7.
- 31Drees BM, Arnold L, Jonas HS. The University of Missouri-Kansas City School of Medicine: thirty-five years of experience with a nontraditional approach to medical education. Acad Med. 2007; 82: 361–9.
- 32Abuhusain H, Chotirmall SH, Hamid N, O'Neill SJ. Prepared for internship? Ir Med J. 2009; 102: 82–4.
- 33Smith CM, Perkins GD, Bullock I, Bion JF. Undergraduate training in the care of the acutely ill patient: a literature review. Intensive Care Med. 2007; 33: 901–7.
- 34Laack TA, Newman JS, Goyal DG, Torsher LC. A 1-week simulated internship course helps prepare medical students for transition to residency. Simul Healthc. 2010; 5: 127–32.
- 35Teo AR, Harleman E, O'Sullivan PS, Maa J. The key role of a transition course in preparing medical students for internship. Acad Med. 2011; 86: 860–5.
- 36Wayne DB, Butter J, Siddall VJ, et al. Mastery learning of advanced cardiac life support skills by internal medicine residents using simulation technology and deliberate practice. J Gen Intern Med. 2006; 21: 251–6.
- 37Barsuk JH, Shubhada NA, Cohen ER, et al. Mastery learning of temporary hemodialysis catheter insertion by nephrology fellows using simulation technology and deliberate practice. Am J Kidney Dis. 2009; 54: 70–6.
- 38Wayne DB, Barsuk JH, O'Leary KJ, et al. Mastery learning of thoracentesis skills by internal medicine residents using simulation technology and deliberate practice. J Hosp Med. 2008; 3: 48–54.
- 39Oermann MH, Kardong-Edgren S, Odom-Maryon T, et al. Deliberate practice of motor skills in nursing education: CPR as exemplar. Nurs Educ Perspect. 2011; 32: 311–5.
- 40Sawyer T, Sierocka-Castaneda A, Chan D, et al. Deliberate practice using simulation improves neonatal resuscitation performance. Simul Healthc. 2011; 6: 327–36.
- 41Stevens LM, Cooper JB, Raemer DB, et al. Educational program in crisis management for cardiac surgery teams including high realism simulation. J Thorac Cardiovasc Surg. 2012; 144: 17–24.
- 42Thomas ET, Williams AL, Reichman EF, et al. Team training in the neonatal resuscitation program for interns: teamwork and quality of resuscitations. Pediatrics. 2010; 125: 539–46.
- 43Steinemann S, Berg B, Skinner A, et al. In situ, multidisciplinary, simulation-based teamwork training improves early trauma care. J Surg Educ. 2011; 68: 472–7.
- 44Yee B, Naik VN, Joo HS, et al. Nontechnical skills in anesthesia crisis management with repeated exposure to simulation-based education. Anesthesiology. 2005; 103: 241–8.
- 45Issenberg SB, McGaghie WC, Petrusa ER, et al. Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Med Teach. 2005; 27: 10–28.
- 46McGaghie WC, Issenberg SB, Cohen ER, et al. Medical education featuring mastery learning with deliberate practice can lead to better health for individuals and populations. Acad Med. 2011; 86: e8–9.
- 47Giglioli S, Boet S, De Gaudio AR, et al. Self-directed deliberate practice with virtual fiberoptic intubation improves initial skills for anesthesia residents. Minerva Anestesiologica. 2012; 78: 1–6.
- 48Price J, Naik V, Bodhwani M, et al. A randomized evaluation of simulation training on performance of vascular anastomosis on a high-fidelity in vivo model: the role of deliberate practice. J Thorac Cardiovasc Surg. 2011; 142: 496–503.
- 49Kessler DO, Auerbach M, Pusic M, et al. A randomized trial of simulation-based deliberate practice for infant lumbar puncture skills. Simul Healthc. 2011; 6: 197–203.
- 50Zendejas B, Cook DA, Hernandez-Irizarry R, et al. Mastery learning simulation-based curriculum for laparoscopic TEP inguinal hernia repair. J Surg Ed. 2012; 69: 208–14.
- 51Rudolph JW, Simon R, Dufresne RL, et al. There's no such thing as “nonjudgmental” debriefing: a theory and method for debriefing with good judgment. Simul Healthc. 2006; 1: 49–55.
- 52Zigmont JJ, Kappus LJ, Sudikoff SN. The 3D model of debriefing: defusing, discovering, and deepening. Semin Perinatol. 2011; 35: 52–8.
- 53Kuiper RA, Heinrich C, Matthias A, et al. Debriefing with the OPT model of clinical reasoning during high fidelity patient simulation. Int J Nurs Educ Scholarsh. 2008; 5: Article17.
- 54Raemer D, Anderson M, Cheng A, et al. Research regarding debriefing as part of the learning process. Simul Healthc. 2011; 6(Suppl): S52–7.
- 55Wayne DB, Didwania A, Feinglass J, et al. Simulation-based education improves quality of care during cardiac arrest team responses at an academic teaching hospital: a case-control study. Chest. 2008; 133: 56–61.
- 56Cohen J, Cohen SA, Vora KC, et al. Multicenter, randomized, controlled trial of virtual-reality simulator training in acquisition of competency in colonoscopy. Gastrointest Endosc. 2006; 64: 361–8.
- 57Barsuk JH, Cohen ER, McGaghie WC, et al. Long-term retention of central venous catheter insertion skills after simulation-based mastery learning. Acad Med. 2010; 85(10 Suppl): S9–12.
- 58Crofts JF, Bartlett C, Ellis D, et al. Management of shoulder dystocia: skill retention 6 and 12 months after training. Obstet Gynecol. 2007; 110: 1069–74.
- 59Draycott TJ, Crofts JF, Ash JP, et al. Improving neonatal outcome through practical shoulder dystocia training. Obstet Gynecol. 2008; 112: 14–20.
- 60Phipps MG, Lindquist DG, McConaughey E, et al. Outcomes from a labor and delivery team training program with simulation component. Am J Obstet Gynecol. 2012; 206: 3–9.
- 61Zendejas B, Cook DA, Bingener J, et al. Simulation-based mastery learning improves patient outcomes in laparoscopic inguinal hernia repair: a randomized controlled trial. Ann Surg. 2011; 254: 502–9.
- 62Barsuk JH, McGaghie WC, Cohen ER, et al. Simulation-based mastery learning reduces complications during central venous catheter insertion in a medical intensive care unit. Crit Care Med. 2009; 37: 2697–701.
- 63Cohen ER, Feinglass J, Barsuk JH, et al. Cost savings from reduced catheter-related bloodstream infection after simulation-based education for residents in a medical intensive care unit. Simul Healthc. 2010; 5: 98–102.
- 64Matheson C. The educational value and effectiveness of lectures. Clin Teach. 2008; 5: 218–21.
10.1111/j.1743-498X.2008.00238.x Google Scholar
- 65Olson CA, Tooman TR. Didactic CME and practice change: don't throw that baby out quite yet. Adv Health Sci Educ Theory Pract. 2012; 17: 441–51.
- 66Carter MB, Wesley G, Larson GM. Lecture versus standardized patient interaction in the surgical clerkship: a randomized prospective cross-over study. Am J Surg. 2006; 191: 262–7.
- 67Carter MB, Wesley G, Larson GM. Didactic lecture versus instructional standardized patient interaction in the surgical clerkship. Am J Surg. 2005; 189: 243–8.
- 68Subramanian A, Timberlake M, Mittakanti H, Lara M, Brandt ML. Novel educational approach for medical students: improved retention rates using interactive medical software compared with traditional lecture-based format. J Surg Educ. 2012; 69: 253–6.
- 69Khan MN, Telmesani A, Alkhotani A, Elzouki A, Edrees B, Alsulimani MH. Comparison of jeopardy game format versus traditional lecture format as a teaching methodology in medical education. Saudi Med J. 2011; 32: 1172–6.
- 70McCoy CE, Menchine M, Anderson C, Kollen R, Langdorf MI, Lotfipour S. Prospective randomized crossover study of simulation vs. didactics for teaching medical students the assessment and management of critically ill patients. J Emerg Med. 2011; 40: 448–55.
- 71Haney M, Silvestri S, Van Dillen C, Ralls G, Cohen E, Papa L. A comparison of tele-education versus conventional lectures in wound care knowledge and skill acquisition. J Telemed Telecare. 2012; 18: 79–81.
- 72Bensalem-Owen M, Chau DF, Sardam SC, Fahy BG. Education research: evaluating the use of podcasting for residents during EEG instruction: a pilot study. Neurology. 2011; 77: e42–4.
- 73Wang CL, Schopp JG, Petscavage JM, Paladin AM, Richardson ML, Bush WH. Prospective randomized comparison of standard didactic lecture versus high-fidelity simulation for radiology resident contrast reaction management training. AJR Am J Roentgenol. 2011; 196: 1288–95.
- 74Behar S, Upperman JS, Ramirez M, Dorey F, Nager A. Training medical staff for pediatric disaster victims: a comparison of different teaching methods. Am J Disaster Med. 2008; 3: 189–99.
- 75Chenkin J, Lee S, Huynh T, Bandiera G. Procedures can be learned on the Web: a randomized study of ultrasound-guided vascular access training. Acad Emerg Med. 2008; 15: 949–54.
- 76Diachun LL, Dumbrell AC, Byrne K, Esbaugh J. … But does it stick? Evaluating the durability of improved knowledge following an undergraduate experiential geriatrics learning session. J Am Geriatr Soc. 2006; 54: 696–701.
- 77Hoag K, Lillie J, Hoppe R. Piloting case-based instruction in a didactic clinical immunology course. Clin Lab Sci. 2005; 18: 213–20.
- 78White M, Michaud G, Pachev G, Lirenman D, Kolenc A, FitzGerald JM. Randomized trial of problem-based versus didactic seminars for disseminating evidence-based guidelines on asthma management to primary care physicians. J Contin Educ Health Prof. 2004; 24: 237–43.
- 79Zitzmann MB. Comparing the learning outcomes of lecture and self-instruction methods in a senior clinical laboratory science course. Clin Lab Sci. 1996; 9: 198–201.
- 80Abraham GJ, Dhume VG, Diniz RS. Comparison of didactic lecture, self-reading and self-instruction as learning methods in medical students of western India. Med Educ. 1981; 15: 222–5.
- 81Agee N, Komenaka IK, Drachman D, Bouton ME, Caruso DM, Foster KN. The effectiveness of grand rounds lectures in a community-based teaching hospital. J Surg Educ. 2009; 66: 361–6.
- 82Shiroma PR, Massa AA, Alarcon RD. Using game format to teach psychopharmacology to medical students. Med Teach. 2011; 33: 156–60.
- 83Hessheimer HM, Rogo EJ, Howlett B. Use of questioning during lectures in a dental hygiene didactic course. J Dent Educ. 2011; 75: 1073–83.
- 84Rubio E, Bassignani MJ, White MA, Brant WE. Effect of an audience response system on resident learning and retention of lecture material. AJR Am J Roentgenol. 2008; 190: W319–22.
- 85Costa ML, van Rensburg L, Rushton N. Does teaching style matter? A randomized trial of group discussion versus lectures in orthopaedic undergraduate teaching. Med Educ. 2007; 41: 214–7.
- 86Bryner CL Jr. Learning as a function of lecture length. Fam Med. 1995; 27: 379–82.
- 87Markova T, Roth LM, Monsur J. Synchronous distance learning as an effective and feasible method for delivering residency didactics. Fam Med. 2005; 37: 570–5.
- 88Solomon DJ, Ferenchick GS, Laird-Fick HS, Kavanaugh K. A randomized trial comparing digital and live lecture formats. BMC Med Educ. 2004; 4: e27.
- 89Kelly PA, Haidet P, Schneider V, Searle N, Seidel CL, Richards BF. A comparison of in-class learner engagement across lecture, problem-based learning and team learning using the STROBE classroom observation tool. Teach Learn Med. 2004; 17: 112–8.
- 90Knowles MS. Andragogy, not pedagogy. Adult Leadership. 1968; 16: 350–352, 386.
- 91Neufeld VR, Barrows HS. The McMaster philosophy: an approach to medical education. J Med Educ. 1974; 49: 1040–50.
- 92Muller S (Chairman). Physicians for the twenty-first century: report of the project panel on the general professional education of the physician and college preparation for medicine. J Med Educ. 1984; 59(11 Pt 2): 1–208.
- 93Barrows HS. A taxonomy of problem-based learning methods. Med Educ. 1986; 20: 481–6.
- 94Berkson L. Problem-based learning: have the expectations been met? Acad Med. 1993; 68: S79–88.
- 95Vernon DT, Blake RL. Does problem-based learning work? A meta-analysis of evaluative research. Acad Med. 1993; 68: 550–63.
- 96Albanese MA, Mitchell S. Problem-based learning: a review of literature on its outcomes and implementation issues. Acad Med. 1993; 68: 52–81.
- 97Norman GR, Schmidt HG. Effectiveness of problem-based learning curricula: theory, practice and paper darts. Med Educ. 2000; 34: 721–8.
- 98Colliver JA. Effectiveness of problem-based learning curricula: research and theory. Acad Med. 2000; 75: 259–66.
- 99Distlehort LH, Dawson E, Robbs RS, Barrows HS. Problem-based learning outcomes: the glass half-full. Acad Med. 2005; 80: 294–9.
- 100Blumberg P, Michael JA. Development of self-directed learning behaviours in a partially teacher-directed problem-based learning curriculum. Teach Learn Med. 1992; 4: 3–8.
10.1080/10401339209539526 Google Scholar
- 101Haidet P, O'Malley KJ, Richards B. An initial experience with “team learning” in medical education. Acad Med. 2002; 77: 40–4.
- 102Touchet BK, Coon KA. A pilot use of team-based learning in psychiatry resident psychodynamic psychotherapy education. Acad Psych. 2005; 29: 293–6.
- 103Haidet P, Morgan RO, O'Malley K, Moran BJ, Richards BF. A controlled trial of active versus passive learning strategies in a large group setting. Adv Health Sciences Educ. 2004; 9: 15–27.
- 104Searle NS, Haidet P, Kelley A, Schneider VF, Seidel CL, Richards BF. Team learning in medical education: initial experience at ten institutions. Acad Med. 2003; 78: S55–8.
