Subjective assessment of participants in education programs on clinical practice guidelines in the field of psychiatry
Funding information
This study was supported by the Japan Agency for Medical Research and Development (AMED) under grant number JP18dk0307060, JP21dk0307083 the AMED under grant number JP19dk0307083, the Japanese Society of Neuropsychopharmacology, and the Japanese Society of Mood Disorders. The funders had no role in the study design, data collection and analyses, decision to publish, or preparation of the manuscript.
Abstract
The Effectiveness of Guidelines for Dissemination and Education in psychiatric treatment (EGUIDE) project, which is a nationwide dissemination and implementation program for clinical practice guidelines (CPGs) in the field of psychiatry, is currently ongoing. In the current study, a subjective assessment of the participants in the EGUIDE programs was assessed using a questionnaire. Then, the relationships between the subjective assessment, the characteristics of the participants, and the clinical knowledge of the CPGs were evaluated. More than 90% of the participants gave a high rating for the components of content, recommendation, knowledge, skill, and adherence, but not for the component of confidence. A positive correlation was found between years of professional experience and the score of confidence. These results suggest that it may be necessary to apply the knowledge and skills of CPGs obtained in the education programs into practice to increase confidence in the proper use of psychiatric therapies based on CPGs.
1 INTRODUCTION
Clinical practice guidelines (CPGs) that give medical practitioners and patients reasonable treatment recommendations are important in “evidence-based medicine,” which is the standard manner of today's medical practices. The Effectiveness of Guideline for Dissemination and Education in psychiatric treatment (EGUIDE) project, which is a nationwide dissemination and implementation project for CPGs in the field of psychiatry, is currently ongoing. This project addresses "Treatment Guidelines II. Major Depressive Disorder" by the Japanese Society of Mood Disorders1 and "Guidelines for the Pharmacological Treatment of Schizophrenia" by the Japanese Society of Neuropsychopharmacology.2 The project is original in that it evaluates multiple indicators and provides feedback on the lecture and development of CPGs.3 As of June 2021, 243 medical institutions, including 44 universities, were participating in this project in Japan. The aim was that by promoting the EGUIDE project, psychiatrists will be educated to use the CPGs and that more appropriate treatments, based on CPGs, will be widely provided.
To date, it has been demonstrated that guideline-based treatment is not always widely used in Japan,4, 5 that participation in the EGUIDE programs improves understanding of the CPGs,6 that revisions of the training materials of EGUIDE programs improve participants' understanding of the CPGs,3 and that polypharmacy of antipsychotics significantly increases the simultaneous prescription of other psychotropics such as anticholinergics, anxiolytics, hypnotics, and mood-stabilizing agents in patients with schizophrenia.7 In the current study, the subjective assessment of the participants in the EGUIDE programs was assessed using a questionnaire. Then, the relationships between the subjective assessment, the characteristics of the participants, and the clinical knowledge of the CPGs were assessed.
2 METHOD
2.1 Participants
Psychiatrists were recruited from October 2016 to March 2018. Four sets of data, the subjective assessments of schizophrenia (SCZ) and major depressive disorder (MDD) programs and the clinical knowledge of the CPGs of SCZ and MDD, were available for a total of 344 participants.6 The mean age of the participants was 33.8 ± 6.9 years (mean ± SD), and the mean years of professional experience were 5.1 ± 6.1 years.
2.2 Subjective assessment of EGUIDE programs
The subjective assessment of each guideline program was evaluated using a questionnaire following the EGUIDE programs. It is a self-administered questionnaire that consists of the following six components on a scale of 1-5, with higher scores indicating a higher evaluation: (1) How would you rate the content of this course? The abbreviation for this question is "Content," with 5 = very satisfactory; 4 = moderate satisfaction; 3 = I cannot say I was either satisfied or dissatisfied; 2 = dissatisfied; and 1 = very dissatisfied; (2) How would you rate your recommendation of this course to a colleague or junior colleague? The abbreviation for this question is "Recommendation," with 5: strong recommendation; 4 = moderate recommendation; 3 = I cannot say that I would or would not recommend; 2 = no recommendation; and 1 = no recommendation at all; (3) How would you rate your clinical knowledge of treatment after attending this course? The abbreviation for this question is "Knowledge," with 5 = very increased; 4 = increased; 3 = I cannot say increased or confused; 2 = confused; 1 = very confused; (4) How would you rate your treatment skills after attending this course? The abbreviation for this question is "Skill," with 5 = very increased; 4 = increased; 3 = I cannot say increased or confused; 2 = confused; 1 = very confused; (5) How would you rate your confidence in your ability to perform proper treatment after attending this course? The abbreviation for this question is "Confidence," with 5 = great confidence; 4 = moderate confidence; 3 = I cannot say I am confident or unsure; 2 = no confidence; 1 = no confidence at all; and (6) How would you rate your future choice of treatment according to the guidelines after attending this course? The abbreviation for this is "Adherence," with 5 = will always choose; 4 = will mostly choose; 3 = I cannot say I would choose or not; 4 = no consideration of guidelines; 1 = no consideration of guidelines at all.
2.3 Assessment of clinical knowledge of CPGs
Working knowledge of the CPGs of SCZ and MDD was evaluated using a questionnaire both at baseline and after each of the programs. Each self-administered questionnaire consists of 37 questions, with a total score of 37 points (see Tables S1 and S2).6
2.4 Statistical analysis
The relationships between the demographic characteristics of the participants (age and professional experience), the total score of clinical knowledge following the programs, and each component score of subjective assessment (the aforementioned content, recommendation, knowledge, skill, confidence, and adherence) were analyzed using Spearman's rank correlation coefficient. All statistical analyses were performed using IBM SPSS 27.
3 RESULTS
In the subjective assessment, more than 90% of the participants gave a high rating of 4 or 5 for the five components of content (98.8% and 97.7%), recommendation (95.3% and 97.1%), knowledge (98.3% and 98.5%), skill (92.4% and 91.0%), and adherence (95.3% and 96.5%), respectively, in both the SCZ and MDD programs, but not for the component of confidence (42.2% and 44.8%, respectively, Figure 1). When we examined the relationships between age, professional experience, and the individual component scores of the subjective assessment, a positive correlation was found between years of professional experience and the score of confidence both in the SCZ and MDD programs (r = .249, P = 2.97 × 10−6 and r = .151, P =.005, respectively; Table 1). This correlation reached statistical significance when corrected for the multiple testing of the six components in each program (P < .0083). When the relationships between each component score of the subjective assessment and the total score of clinical knowledge of the CPGs were assessed following the programs, no significant association was found in either the SCZ or MDD programs (P > .05; Table S3).
| Content | Recommendations | Knowledge | Skills | Confidence | Compliance | |
|---|---|---|---|---|---|---|
| Schizophrenia | ||||||
| Age (y) | ||||||
| r | .040 | .107 | −.102 | −.115 | .126 | .058 |
| P value | .455 | .048 | .060 | .033 | .019 | .287 |
| Professional experience (y) | ||||||
| r | −.008 | .088 | −.142 | −.165 | .249 | .069 |
| P value | .882 | .104 | .009 | .002 | 2.97 × 10−6 | .202 |
| Major depressive disorder | ||||||
| Age (y) | ||||||
| r | .019 | .118 | −.073 | −.066 | .093 | −.007 |
| P value | .725 | .029 | .175 | .224 | .085 | .898 |
| Professional experience (y) | ||||||
| r | −.030 | .072 | −.181 | −.127 | .151 | .058 |
| P value | .574 | .181 | .001 | .019 | .005 | .287 |
Note
- Bold value indicates significant positive correlations are observed between "Confidence" and years of professional experience in either the schizophrenia or major depressive disorder program.
4 DISCUSSION
To our knowledge, this is the first study that assessed the subjective effect of educational program and the related factors in the field of psychiatric CPGs. The subjective assessment of participants in the EGUIDE programs was performed using a questionnaire. High rating scores of content and recommendation suggest that the CPG programs were highly regarded by the participants. High ratings of knowledge and skill suggest that most participants had a better understanding of CPGs. These results are consistent with the previous results of the objective improvement of the participants' clinical knowledge of the CPGs.6 Although knowledge of the guidelines may not be necessarily linked to treatment outcomes, a high rating score of adherence may lead to future proper treatment choices based on the CPGs.
A high rating for confidence was not achieved in either the SCZ or MDD programs. Most participants might be cautious rather than focusing on building their confidence. While the score of confidence was associated with years of professional experience, it was not associated with clinical knowledge of CPGs following the programs. On the contrary, an education program using case-based learning and supervision over time increased confidence as well as knowledge.8 Evidence-based medicine, which the CPGs are intended to support, relies on the expertise of medical practitioners.9, 10 In other words, clinical experience is a prerequisite for the optimal use of the probabilistic recommendations provided by the CPGs in practice. It may be that an application of the clinical knowledge of CPGs acquired in the program in daily practice will lead to the acquisition of confidence in the appropriate use of psychiatric therapies. A longitudinal study of changes in adherence to CPGs and confidence of the proper therapies based on CPGs is currently planned.
The limitations of this study are the number of participants and potential sampling bias, such as years of professional experience. Future researches with increased participants are needed.
5 CONCLUSION
The educational programs on CPGs contributed to a high subjective assessment of therapy knowledge and skills, as well as guideline-based treatment choices, but did not lead to an increased confidence in the appropriate use of therapies based on CPGs. To increase confidence, it may be necessary to apply the knowledge and skills obtained in the educational programs to real-world clinical practice.
CONFLICT OF INTEREST
The authors declare no conflict of interest.
AUTHOR CONTRIBUTIONS
KOg and SN were involved in data collection and data analysis and wrote the first draft of the manuscript. NHase, JM, KM, and MN were involved in the data analysis and contributed to the interpretation of the data and writing of the manuscript. MM, KOh, MT, TT, NHash, TO, HM, HH, KIc, TI, NY-F, AH, NS, KF, TNag, YT, RF, KN, TNak, MU, MF, HT, HY, HK, SO, KA, EK, MK, TK, TS, SY, JI, and HI contributed to the interpretation of the data and data collection. KIn and KW were involved in the study design and contributed to the interpretation of the data. RH supervised the entire project, collected the data, and was involved in the design, analysis, and interpretation of the data. All authors contributed to and approved the final article.
APPROVAL OF THE RESEARCH PROTOCOL BY AN INSTITUTION REVIEWER BOARD
This study was approved by the ethics committees of the National Center of Neurology and Psychiatry (A2017-105) and each participating university, hospital and clinic.
INFORMED CONSENT
All participants provided their written informed consent. Public availability of raw data was not planned in the research protocol approved by an Institution Reviewer Board. We did not obtain informed consent of the public availability.
REGISTRY AND THE REGISTRATION NO. OF THE STUDY/TRIAL
The protocol of this study was registered in the University Hospital Medical Information Network registry (UMIN000022645).
ANIMAL STUDIES STATEMENT
Not applicable.
Open Research
DATA AVAILABILITY STATEMENT
The data are not publicly available due to privacy and ethical restrictions (ie, we did not obtain informed consent on the public availability of raw data).
