2.1. The Dundee Ready Educational Environment Measure Questionnaire

DREEM has fifty items and a 5-factor structure: SPL (students’ perception of learning [12 items]), SPT (students’ perception of teachers [11 items]), SAS (students’ academic self-perception [8 items]), SPA (students’ perception of atmosphere [12 items]), and SSS (students’ social self-perception [7 items]).

A bipolar 5-point Likert scale with a neutral response is applied (0: strongly disagree; 1: disagree; 2: neither agree nor disagree; 3: agree; 4: strongly agree) to the 50 items. Among the fifty items, nine items (item 4, 8, 9, 17, 25, 35, 39, 48, and 50) are negative sentences; therefore, they should be reversely coded when scoring and analysis are performed. High scores indicate a positive evaluation regarding the education environment. The developers provided an interpretation guideline for each item or each factor. According to that guideline, items or factors with a score equal to or greater than 3.5 indicate a very positively evaluated area; items or factors with a score between 2 and 3 indicate a positively evaluated area but, however, require some efforts to enhance education environment, and items or factors with a score equal to or less than 2.0 indicate that have to demand attention to education environment [42]. The DREEM questionnaire enables researchers to quantitatively assess students’ perceptions of the educational environment. It also makes it feasible to compare results across different institutions, various curriculums, and different survey researches [17].

2.2. Cross-Cultural Adaptation Process

Wherever the cross-cultural adaptation guideline is presented, there is a slight difference in process; however, in general, a serial process of translation, back translation, review, and pretesting is performed [39–41, 43–45]. The World Health Organization (WHO) guidelines also recommend a serial process of forward translation, expert panel back translation, pretesting, cognitive interviewing, and fixing the final version [46]. In this study, we referred to the adaptation guidelines written by Beaton [39] as well as Sousa and Rojjanasrirat’s [40] six stages.

Before starting this study, the authors informed Dr. Sue Roff, the original developer of DREEM, about this cross-cultural adaptation and received her permission along with her kind and helpful advice via e-mail. Figure 1 depicts the overall process underlying the cross-cultural adaptation of DREEM.

During the first stage, two translators independently translated the English DREEM items into Korean. Their mother tongue was Korean, but they could speak English fluently. We provided this research’s aims and advance information to translator 1 (T1), while translator 2 (T2) was provided with no information regarding this study. During stage 2, we synthesized the two products obtained from T1 and T2. After we showed the synthesized version to T1 and T2, we received feedback from the two translators and revised the manuscript (early Korean version). During stage 3, two other translators independently translated the early Korean version produced in stage 2 into English. The two back-translators (BT1 and BT2) were not provided with any information on the original version of DREEM and the medical education system. During stage 4, we organized an expert committee where we developed the prefinal Korean version of DREEM after considering and analyzing all the products obtained from T1, T2, BT1, and BT2. A total of eight experts participated in the committee: one survey research methodologist, one healthcare professional, one language professional, one education professional, and four translators (T1, T2, BT1, and BT2). The first two members of the committee, and two translators (T1 and T2), were Korean medicine doctors who worked in the college of Korean medicine. Throughout stage 4, the committee members focused on semantic equivalence, idiomatic equivalence, experiential equivalence, and conceptual equivalence in order to maintain the equivalence between the original English version and the new Korean version [39, 43]. Dr. Sue Roff’s answers and advice were very helpful during this stage. We were able to develop the prefinal Korean version of DREEM in stage 4. During the next stage, we conducted an in-depth interview with two students to confirm the clarity of the items. They provided advice from a student’s point of view, and almost no changes were made in prefinal version. Consequently, the Korean version of DREEM was finalized. During the final stage, we performed a psychometric study using the Korean version of DREEM. This was followed by statistical analysis including descriptive study and confirmatory factor analysis.

2.3. Psychometric Testing and Statistical Analysis

After completing the cross-cultural adaptation, we performed a human subject research using the newly developed Korean version of DREEM. A survey was administered to second-year to sixth-year students in two universities. The online survey platform Survey Monkey (Survey Monkey, CA, USA) was used for collecting the responses. A total of 218 students and 233 students from two universities, respectively, voluntarily participated in the online survey. This human subject research was approved by the Institutional Review Board of Kyung Hee University Korean Medicine Hospital (KOMCIRB-161020-HR-059).

We performed descriptive statistical methods in order to summarize the basic characteristics and responses to the survey and confirmatory factor analysis to compare the latent variable structure of this population with the original structure and previous studies. For evaluating the goodness of fit of the original model using the acquired dataset, several model fit indices were calculated and discussed: goodness-of-fit index (GFI), adjusted goodness-of-fit index (AGFI), root mean square error of approximation (RMSEA), normed fit index (NFI), Tucker–Lewis index (TLI), and comparative fit index (CFI). With these model fit indices, we investigated the factor loading of each item, the construct reliability (CR) for the convergent validity, and the average variance extracted (AVE) for the discriminant validity. We used Microsoft Excel Office 365 (Microsoft, Redmond, WA, USA) for performing the descriptive statistics and R 3.6.1. (R Core Team, Vienna, Austria) and R packages “sem” for performing confirmatory factor analysis.

3.1. Cross-Cultural Adaptation of the DREEM Questionnaire

Table 1 presents the Korean version of DREEM, which was developed using the cross-cultural adaptation process.

3.2. Psychometric Test Results

3.2.2. Confirmatory Factor Analysis of the Psychometric Test

Figure 2 presents the result of the confirmatory factor analysis of the 5-factor model based on the original DREEM questionnaire with the acquired dataset. Factor loadings and construct reliabilities (CRs) for evaluating the convergent validity are also presented in the same figure. The indices for model fit are as follows: GFI = 0.75, AGFI = 0.73, NFI = 0.65, TLI = 0.73, CFI = 0.74, and RMSEA = 0.06. Table 4 shows the correlation coefficients and the average variance extracted (AVE) for evaluating the discriminant validity.

Table 4. Correlation coefficients, average variances extracted (AVEs), and construct reliabilities (CRs) of subscales for the convergent validity and the discriminant validity.

	SPL	SPT	SAS	SPA	SSS	AVE	CR
SPL	1					0.55	0.87
SPT	0.87 (0.76)	1				0.54	0.83
SAS	0.92 (0.85)	0.75 (0.56)	1			0.54	0.74
SPA	0.97 (0.94)	0.83 (0.69)	0.92 (0.85)	1		0.48	0.79
SSS	0.82 (0.67)	0.67 (0.45)	0.80 (0.64)	0.98 (0.96)	1	0.5	0.63

• The squares of correlation coefficients are expressed in parentheses. SPL: students’ perception of learning; SPT: students’ perception of teachers; SAS: students’ academic self-perception; SPA: students’ perception of atmosphere; SSS: students’ social self-perception; AVE: average variance extracted; CR: construct reliability.

4.1. The Cross-Cultural Adaptation Process

Cultural and linguistic diversity has caused problems with regard to translation, and many researchers have continued to study translation methods [47]. As a result, a general consensus has been formed for the cross-cultural adaptation process.

Numerous multinational studies have contributed toward DREEM’s development. It has been translated into various languages, used in various countries for many years, and has been identified as a nonculturally specific questionnaire [21, 22]. Some studies differed from consensus of cross-cultural adaptation process, such as multiple translators not participating, missing the back translation, or proceeding in a different order [26, 48–51]. Some did not elaborate on whether they followed the consensus process or not [15, 24, 37, 52–56].

This study encountered some difficulties because the expert committee found some expressions to be unclear during the process of creating the prefinal version. In response, the final version was produced after a comparison with other questions, agreements between experts, and communication with the original writer (S. Roff). In the cases of item #12 “timetable,” item #31 “empathy in my profession,” and item #33 “in class socially,” it was necessary to accurately define the meaning of the words. “Timetable” can be interpreted in various ways including a syllabus or a class schedule. Roff’s response was that it included all such meanings. The meaning of “my professional” in item #31 also included both doctor-doctor (the ability to form consensuses between doctors, the ability to discuss diagnoses, etc.) and patient-doctor relationships. In the case of item #33, “class” indicated all courses of study, including lectures and seminars. Internationally, the medical curriculum is not unified, and this can result in differences in terminology. The term “ward teaching” is translated into the Korean term “hoe-jin,” which refers to clinical practice conducted by professors, which involves seeing and examining patients in the ward. In Korea, ward teaching is usually conducted in the 6^th year of Korean medicine college. In the case of “constructive criticism” in item #32, students may find the term “constructive” vague. Based on the words and examples provided by Roff, a description that could supplement the word was added in the Korean version. In the case of item #34, we needed to grasp the meaning of individual “seminars/tutorials.” Roff suggested that the word was basically referred to a small number of people and that it was a concept that contrasted with the lectures. Therefore, “seminars/tutorials” was replaced by small group classes/small group activities in individual practice sessions.

4.2. Basic Statistics and the Evaluation of Construct Validity

The descriptive statistics of the results showed that many students were not satisfied with the education environments in Korean medicine colleges. The total and subscores of DREEM were almost equal to or less than half score, and 58% of items were identified as being problematic. This result is similar to that of a previous study in Korea [38]. Thus, education policy on Korean medicine requires further effort and investment.

Next, we examined whether the original 5-factor model of DREEM was suitable for the new dataset obtained from Korean medicine students. We conducted confirmatory factor analysis and calculated several parameters, including goodness-of-fit indices, convergent validity, and discriminant validity.

First, the goodness of fit can be evaluated when the original 5-factor model is applied to the new dataset. Generally, it has been known that RMSEA less than 0.05 is considered to be good, RMSEA between 0.05 and 0.08 is acceptable, RMSEA between 0.08 and 0.1 is marginal, and RMSEA over 0.1 is poor [57]. The RMSEA for our study was 0.06. We concluded that the original 5-factor model was acceptable for the new dataset obtained from the Korean medicine students. However, the other indices for model fit—GFI, AGFI, NFI, TLI, and CFI—were around 0.7. Among them, TLI and CFI, which were relatively sample size independent, were 0.73 and 0.74. In other countries, they also had results of about 0.7 [30, 32, 35], respectively, although it is suggested that these indices should be over 0.9 for a good model fit [58, 59]. Based on these results, we concluded that the goodness of fit of this model with our dataset was acceptable but not very good. Such less satisfactory results can imply different concept structures between countries or cultures.

Second, the convergent validity was examined using factor loadings and CRs. Ideally, they must be equal to or greater than 0.707 to demonstrate desirable convergent validity [60–62], but values over 0.5 are also generally accepted [63]. Among 50 items, only 2 items showed factor loading values over 0.707, and 28 items showed factor loading values over 0.5; however, the CRs of each factor (except for latent variable SSS) were all over 0.707. Based on these indices, we can consider the relatively low convergent validity of the model in this study. Low convergent validity indicates that the items cannot be explained by only the corresponding latent variable and that some other factors are influencing the information of the items. To consider good discriminant validity, the AVE of one factor should be larger than the shared variance estimates with any other factor including measurement errors [64, 65]. In this study, except for the only SPT-SSS pair, all squares of correlation coefficients were greater than the corresponding AVEs. Low convergent validity and low discriminant validity indicated that the original 5-factor model was not suitable for our dataset, which was obtained from Korean medicine students.

This result may have two possible explanations. First, as shown in the basic statistics, many respondents gave low scores across all the items. Such overall low scores may make it difficult to separate the latent variables, and this may induce low construct validity. Second, because of the differences in languages and cultures, the Korean students’ perceptions regarding medical education environments may have different structures from the perception of the other countries’ students. In order to clarify this issue, further studies with large amounts of samples should be conducted.

4.3. Limitations and Further Research

This study aimed to develop a Korean version of DREEM to promote more active use of DREEM in Korea and to reduce errors in the comparisons of Korean study results and those of other countries. Moreover, we aimed to obtain methodological justification through a strict process of cross-cultural adaptation, not simple translation by a few bilingual translators. Because of differences in terms of languages, cultures, medical systems, and education systems, the meanings of some items had to be modified despite the kind and considerable help of the original developer. We consider this to be an unavoidable aspect of localization that occurs in cross-cultural adaptation.

With regard to statistics, the convergence in each factor and the discrimination among factors were not clear. Therefore, our data were not considered to be suitable for the original 5-factor model. As has been suggested by some previous studies, five-factor structure of DREEM (original DREEM subscales) was not good model fit and new structure or abridged version was suggested [24–36]. Through further additional human subject researches in Korea, the structure model can be modified or respecified using consecutive confirmatory factor analysis or exploratory factor analysis.

Therefore, if researches using the Korean version of DREEM, which has undergone cross-cultural translation in this study, are conducted continuously, it could be possible to make policy decisions by analyzing what resource aspects should be put on to improve the quality of medical educational environments through comparison studies between educational institutions and countries. Moreover, it could be possible to objectively evaluate the effect of used resources and improved education systems if data are collected in time series for a specific population.