Medical Student Milestones in Emergency Medicine

The goal of medical education is for trainees to be able to competently practice medicine independently by the end of their medical training.1 Accordingly, medical educators have sought instruments to assess clinical competence through the continuum from medical school, to residency, to unsupervised clinical practice.2, 3 This movement toward competency-based medical education has been championed by numerous stakeholders, including the Accreditation Council for Graduate Medical Education (ACGME), American Board of Medical Specialties, and CanMEDS.4, 5 The ACGME subdivided clinical competencies into six initial core competencies. Each competency has defined domains of behavior called subcompetencies, with specified intermediate steps toward competency termed milestones.6 This competency-based progression makes the fundamental skills explicit for learners and teachers, and it allows for the evaluation of learners against specific criteria.

In 2012 the ACGME mandated transition to the Next Accreditation System, which involves implementation of milestones specific to each specialty in an effort to improve competency-based assessment.7 Under this initiative, the ACGME led a group to define the milestones specific to emergency medicine (EM), with five levels of proficiency, beginning at Level 1 to reflect the competencies expected of a medical school graduate, through Level 5, which is expected of a clinician after years of clinical practice.8-10

Notably, while the Level 1 milestones are designed to correspond to the level of competency of graduating students entering residency, the graduate education model of milestones has not yet been adapted to undergraduate medical education. As a result, there is a need for competency assessment of medical students as they progress toward fulfilling the Level 1 milestones. Efforts are under way by the Association of American Medical Colleges (AAMC) to generate core entrustable professional activities (EPAs) for medical students entering residency.11 These EPAs are means to translate competencies into clinical practice. However, the knowledge and skills expected of medical students in fourth-year EM clerkships both crosscut other clerkships and contain unique EM-specific competencies. The objective of this study was to identify milestones for assessing the competence of fourth-year medical students on their EM rotations.

Methods

Results

The results of the expert panel regarding the importance of each proposed milestone that a student should be able to achieve by the end of the EM clerkship are presented in Table 1. After the first round, 12 milestones were revised based on participant comments on issues of clarity or redundancy. There were 12 milestones selected for removal due to low agreement (confirmed during the second round) or because they were consolidated into other milestones. One milestone (basic life support/cardiopulmonary resuscitation) was added based on comments. In the second round, the 12 milestones selected for removal had medium or high levels of agreement to drop them. The 24 milestones presented on the final list all had high levels of agreement; consensus was achieved by the expert panel.

The initial 39 proposed milestones spanned 16 of the 23 competencies outlined by the ACGME for EM residents. The final 24 milestones cover 13 of the 23 competencies (Data Supplement S1, available as supporting information in the online version of this paper). The competencies that were not included in the final list are also listed in Data Supplement S1.

Discussion

The 24 milestones identified in this study are aligned with the AAMC's CEPAER, as well as the ACGME EM milestones for residents to confirm content and relationship to other variable validity.1, 11 The EM medical student milestones cover the majority of both the EM Level 1 milestones as well as the CEPAER.

These milestones include some skills integral to EM practice, such as identification of “sick” patients, focused differential diagnosis, and recognizing emergency conditions. In addition, standard skills found in other clerkships, such as the ability to perform history and physical examination, professionalism, and communication, are included. It is important to remember we were attempting to determine competencies for all students in EM clerkships, not just those who will choose EM for residency. Therefore, some of the EM specific Level 1 milestones such as procedures and ultrasound were not included, as graduates entering other specialties may not be expected to be competent in these areas.

From the discussion involving the reactor panel, two key themes emerged. The first is that context of different institutions creates different expectations of their medical students. Examples include the proposed milestone “performs venipuncture or place IV” and use of the electronic medical record. Some institutions expect achievement of these milestones for their medical students, while others do not find it important for medical students to perform these tasks or lack the resources or have policy issues that prevent students from performing these tasks. The second theme from the reactor panel is that many panelists felt the list of 24 milestones is too long, would be difficult to assess in 1 month, and could be further consolidated. In response, we asked the reactor panel which milestone within a competency would be most appropriate for the assessment of medical students, and using the feedback of the reactor panel we pared down the list of 24 into 15 milestones to reduce redundancy within each ACGME competency (Data Supplement S2, available as supporting information in the online version of this paper). We present both the full list and the abbreviated list for consideration of EM medical student education leaders to begin the discussion of which competencies should be assessed. In the end, we believe that there should be a core set of competencies for all fourth-year medical students. However, programs may choose to add additional competencies based on their context and priorities. For example, at University of Michigan Medical School, students are assessed on delivery of bad news by a rigorously developed standardized patient program. Therefore, advanced communication would be determined to be a competency at this program.

Limitations

The inherent limitation with the use of a Delphi panel is the potential for bias. To address this we ensured that the expert panel was of reasonable size and that the experts included a variety of responsibilities to allow for diverse opinions and maintained 100% response rate. Nonetheless, we noted that while the process of providing the panelists the results of the entire panel to reflect on prior to responding for round 2 is intended to create consensus, some panelists were unmoved by the consensus and maintained close to their original scores. This was particularly noted on the context-related items such as IV placement. Further, some of the EM subcompetencies were not included in the medical student milestones. For graduating students entering EM, there will need to be curricular content and assessment to address the Level 1 milestones of these subcompetencies.

In addition, this is an initial determination of milestones requiring further validation and feasibility, possibly through the CDEM and medical student input. These milestones are intended for fourth-year students and may need adjustment for third-year students.25 With EM being offered at more than 50% of medical schools in the United States, the EM clerkship can play a key role in helping students reach ACGME Level 1 milestones and core entrustable professional activities.26

Conclusions

Our study found consensus support by experts for a list of 24 milestones for competency assessment of fourth-year medical students by the completion of their EM clerkships. The findings are useful for development of a valid method for assessing medical student performance as the student approaches residency. Notably, this allows each institution to tailor its curriculum toward the goal of having its students achieve these milestones by the end of their EM clerkships.

The authors acknowledge the following:

Expert Panelists: Douglas Ander, Daniel Cabrera, Wendy Coates, Nicole Deiorio, Christopher Doty, Cullen Hegarty, Kate Hogan, Joseph House, Sorabh Khandelwal, Amy Kontrick, Sarkis Kouyoumjian, David Lane, Luan Lawson-Johnson, David Manthey, Tom Morrissey, William Peterson, Camiron Pfennig, Susan Promes, Kevin Rodgers, Benjamin Sandefur, Sally Santen, Emily Senecal, Cemal Sozener, and David Wald.

Reactor Panelists: Douglas Ander, Arif Alper Cevik, Jamie Collings, Jeff Van Dermark, Sameer Desai, Gino Farina, Doug Franzen, Lisa Gehm, Kathy Hiller, Patrick Hinfey, Paul Ko, Joseph LaMantia, Aaron Leetch, Amy Leuthauser, Shawn Londob, Lucienne Lufty-Clayton, Todd Peterson, Thomas Regan, Colleen Roche, Dennis Ryan, Kaushai Shah, Scott Sherman, Matthew Tews, and Gregory J. Tudor. (Some joined the phone call late and may not have been captured in the roll call).