1.1 Background

Anger is a basic negative emotion that is closely connected to violence (Tonnaer et al., 2017) and generally accompanied by antagonistic thoughts directed towards a person or object viewed as the cause of an adverse event (Novaco, 1998). Manifestations of anger range from mild irritation to out-of-control rage (Spielberger et al., 1983). Anger can be suppressed or directed at others in the form of confrontation or aggressive behaviour (Lubke et al., 2015). Violent behaviour by patients can result in injury to medical staff or damage to hospital equipment. Importantly, poor communication is recognized as increasing the risk of violence (Duxbury & Whittington, 2005; Whittington & Wykes, 1996). Medical staff-related factors that put patients at risk of becoming angry to include poor communication and poor bedside manner (Chipidza et al., 2016). The knowledge, skills and attitudes of nurses are all relevant to effective communication and the formation of a good nurse–patient relationship (Happ et al., 2014). Importantly, good communication with patients can help nurses to detect a patient's anger at an early stage, identify possible causes of the anger and manage the patient's anger effectively (Chipidza et al., 2016; Flores, 2008; Kourkouta & Papathanasiou, 2014; Medved, 1990; Smitherman & Colleen, 1981; Thomas, 2003).

Since most components of communication techniques are learnable skills (Buckman, 2001), evaluating the communication skills of nurses and identifying deficiencies would help to target training to improve their ability to communicate with patients. A variety of instruments are available to assess the communication skills of nurses. For example, the Communication Skills Inventory developed by (Ersanli and Balci (1998) measures three dimensions of communication (psychology, emotion and behaviour) and consists of 45 questions answered using a 1–5 Likert scale. The Set Elicit Give Understand End (SEGUE) framework was developed by Makoul (2001) to evaluate the effectiveness of teaching and training in medical communication skills and has been used to assess communication ability in medical students (Sattler et al., 2017), nursing students (Son & Hee Kim, 2019) and qualified physicians (Bai et al., 2019). The Jefferson Empathy Scale (Hojat et al., 2002), which includes 20 items covering three dimensions (psychological cognition, emotional experience and behavioural assistance), has good internal consistency reliability and validity and has been translated into many different languages. The Amsterdam Attitude and Communication Scale (De Haes et al., 2001), which includes nine items, has a Cronbach's α coefficient of 0.70 and has been used to evaluate the communicative behaviour and attitudes of medical students (De Haes et al., 2005). However, there are no currently available measurement tools that specifically evaluate the ability of nurses to communicate with angry patients.

2.1 Aim

The aim of this study was to develop the Nurse's Communication Ability with Angry Patients Scale (NCAAPS) and evaluate its psychometric properties. It was anticipated that the future use of this new scale would help educators to identify specific areas of deficiency that could be targeted with training to improve the ability of nursing staff to communicate with angry patients.

2.2 Design

This study established the NCAAPS through a four-step process: (1) building an item pool and generating a scale; (2) testing the content validity, criterion-related validity and reliability of the scale; (3) exploratory factor analysis (EFA); and (4) confirmatory factor analysis (CFA).

The selection of candidate items for the NCAAPS was based on a literature review, previous data obtained by our research team, and the results of a focus group discussion. Seventeen candidate items were initially chosen following a comprehensive review of the literature relating to the ability of nurses to communicate with angry patients. An additional seven candidate items were included based on previous results obtained by our research team. The candidate items were reviewed by a focus group that included three nurse managers (with 18–33 years of experience working as registered nurses and more than 5 years of experience working as nurse managers), two nurses studying for a master's degree in nursing (grade 1 and grade 3) and two nurses studying for a doctoral degree in nursing (grade 1 and grade 4). The focus group held three discussions with each discussion lasting about 1 hr. The scale was improved by the addition, deletion or modification of items. Furthermore, the items were revised to ensure that their wording was precise and comprehensible and to ensure compliance with the current medical care environment.

2.3 Participants

A panel of 18 experts was assembled to further refine the item pool. The 18 experts included 6 charge nurses who were responsible for daily nursing management in the unit, a director of medicine who was responsible for daily medical management of the clinical department, 2 directors of nursing who were in charge of nursing management in the hospital, 1 member of staff responsible for dealing with hospital complaints, an Associate Dean of the hospital who was responsible for hospital management and 7 university lecturers working in the fields of interpersonal communication (3 lecturers), psychological care (2 lecturers) and nursing research (2 lecturers). Feedback from the expert panel was used to further revise the phrasing of each item in the item pool to make it more concise and easier to understand and to avoid redundancy. The final version of the NCAAPS consisted of 19 items with answers given on a 5-point Likert-type scale, where 1 means ‘strongly disagree’, 2 means ‘disagree’, 3 means ‘not sure’, 4 means ‘agree’ and 5 means ‘strongly agree’ (Table S1).

To enable further evaluation of the NCAAPS, a larger cohort of participants was recruited using three-stage hierarchical random cluster sampling from 2 August 2019 to 3 October 2019. In the first stage, three provinces were randomly selected from each of the three geographical regions (eastern China, central China and western China). In the second stage, 9 third-grade general hospitals were randomly selected from each of the chosen provinces. In the third stage, all emergency department nurses in each sampled hospital were selected. Study participants were enrolled using the following inclusion criteria: (a) registered nurse; (b) had been working in the emergency department of the sampled hospital for at least 3 months; (c) was working independently and communicated directly with patients and their family members; and (d) volunteered to participate in the study. The exclusion criteria were: (a) on vacation at the time that the NCAAPS was due to be administered; (b) were undergoing training at the emergency department; and (c) student nurses.

The following demographic characteristics for each participant were collected using specially designed forms: gender, age, length of service in the hospital, professional title, position, marital status, education level and communication skills training received after joining the hospital. In addition, the NCAAPS was administered to all the participants. All the NCAAPS questionnaires were filled in anonymously.

2.4 Data collection

The SEGUE framework and General Self-Efficacy Scale (GSES) were used as external criteria to assess the criterion validity of the NCAAPS. The SEGUE framework consists of two sections (doctor–patient communication content and communication skills) and five dimensions (preparation, information collection, information giving, understanding the patient and ending) (Makoul, 2001), and this scale has been used to evaluate communication skills in medical students (Sattler et al., 2017), nursing students (Son & Hee Kim, 2019) and doctors (Bai et al., 2019). The SEGUE framework has a Cronbach's α coefficient, CVI, test–retest reliability and KMO all exceeding 0.8 (Makoul, 2001; Xiong et al., 2019). In this study, 47 emergency department nurses with more than 1 year of professional experience were invited to complete the SEGUE framework and NCAAPS questionnaires, and 32 nurses (68.1%) agreed to participate. The nurses first filled out the NCAAPS questionnaire and then underwent a SEGUE framework evaluation carried out by two nursing students who were studying for a Master's degree and two trained, standardized patients. The GSES has a total of 10 items, that are answered using a 4-point Likert scale (completely disagree, slightly agree, mainly agree and completely agree). The Chinese version of the GSES has a Cronbach's α coefficient of 0.87, test–retest reliability of 0.83 and Spearman–Brown coefficient of 0.90 (Wang et al., 2001). The GSES was administered to all the study participants who completed the NCAAPS.

2.5 Data analysis

Structural validity was evaluated by EFA and CFA. The data were analysed with the principal component extraction method and varimax rotation using SPSS 18.0 (SPSS Inc., Chicago, IL, USA). The following criteria were used to determine which factors would be retained: (a) eigenvalues greater than 0.8; (b) the percentage of total variance explained >70%; and (c) a factor loading cut-off 0.40 (Loewenthal & Lewis, 2018).

CFA allows investigators to specify a hypothesized factor structure in advance and then test it, thereby determining how well the proposed model fits the data (Nunnally & Bernstein, 1994). AMOS 23.0 (IBM Corp., Armonk, NY, USA) was used to determine if the factor model emerging from EFA provided a good fit for the data. Two alternative models were tested against the hypothesized four-factor model that emerged from the EFA in this study: a first-order model and a correlated factors model that tested the idea that individual factors of the NCAAPS were related to one another. Support for the hierarchical model would suggest that all factors were not related to a higher-order factor. This model was analysed using the maximum-likelihood estimation method in AMOS with the covariance matrix generated in PRELIS. The following indices were used to determine the model fit: chi-squared test, relative chi-square (CMIN/DF), root mean square error of approximation (RMSEA), comparative fit index (CFI), normed fit index (NFI), non-normed fit index (TLI) and incremental fit index (IFI) (Kline, 1998).

2.6 Validity and reliability

The content validity of the scale was evaluated using the content validity index (CVI), which was obtained after evaluation of the scale by the panel of 18 experts (see above). Each expert was asked to assess the relevance of each item to the corresponding content dimension. A 4-level scoring method was used, where 1 = highly irrelevant, 2 = moderately irrelevant, 3 = moderately relevant and 4 = very relevant (Grant & Davis, 1997). Items with a score of 3 or 4 were considered to be relevant to the content being measured. The item-level CVI (I-CVI) was defined as the ratio of the number of experts who judged the item as relevant (i.e. score ≥ 3) to the total number of experts. The scale-level CVI (S-CVI) was calculated as the average CVI across items. Concerning the interpretation of the results, it is generally considered that when the number of experts is 6 or more, the content of the scale can reflect the measured content well when I-CVI is above 0.78 and S-CVI is above 0.80 (Davis, 1992; Li & Zheng, 2018; Polit et al., 2007).

Reliability describes the consistency and stability of the measurement tool and is most commonly assessed using Cronbach's α coefficient. Reliability was assessed following the administration of the NCAAPS to 30 emergency department nurses from the hospital. The internal consistency of the NCAAPS was evaluated using Cronbach's α coefficient (Kimberlin & Winterstein, 2008) and the odd-even split-half reliability (Gu et al., 2014). A Cronbach's α value >0.8 is generally taken to indicate good internal consistency (Kimberlin & Winterstein, 2008). In this study, reliability was considered to be acceptable for 0.7 ≤ Cronbach's α < 0.8, good for 0.8 ≤ Cronbach's α < 0.9 and excellent for Cronbach's α ≥ 0.9. Test–retest reliability is the most common method used to evaluate the stability of a scale, and a correlation coefficient >0.7 is generally taken to indicate a stable scale (Weir, 2005). To calculate the test–retest reliability of the scale, the 30 emergency ward nurses were asked to complete the NCAAPS within 2 weeks after the first administration.

2.7 Ethical considerations

The study was approved by the Ethics Committee [2019045]. The study was conducted with the consent and support of the managers of the nursing department of each included hospital. All participants provided informed written consent for inclusion in the study, and the NCAAPS questionnaires were completed anonymously.

3.1 Content validity

The S-CVI was 0.98 and the I-CVI ranged from 0.88–1.00, indicating that the NCAAPS had good content validity. The Spearman–Brown coefficient was 0.90, suggesting that the scale items had high homogeneity and that the scale had good internal consistency.

3.2 Characteristics of the study participants

A total of 501 questionnaires were distributed in this study, and 456 questionnaires were returned, equating to a response rate of 91.0%. The characteristics of the 456 study participants are shown in Table 1. The median age of the 456 respondents was 29 years (Q1, 26 years; Q3, 33 years), 63.1% of the participants were aged 26–35 years and 87.5% were female. The median duration of work experience was 6.5 years (Q1, 3 years; Q3, 10.8 years), and senior nurses accounted for 48.5% of the respondents. Approximately a fifth (21.9%) of the participants was the only child of their families (because of the one-child policy in the 1980s of China). The majority of participants were married (66.2%) and had children (56.1%). More than half the respondents (72.8%) were undergraduates, and 80.9% of the respondents had received communication skills training after joining the hospital. General Self-Efficacy Scale items score are showed in Table S2. The highest score for option 16 in anger perception is 0.75. The 11th option of the anger perception is the same as the first option of exploring the cause of anger with a score of 0.09 and is the lowest of the two. With a score of 0.76 in option 5 is the highest of the exploring the cause of anger. The first option of self-preparation received a score of 0.82, which is the highest of all the scoring options. Communication skills received the lowest score, 0.04, for ‘For unresolved patient anger, I can seek help from other medical team members in time’.

3.3 Construct validity

Bartlett's test of sphericity was significant (χ² = 6356.277, df = 171, p < .001), indicating that factor analysis was appropriate for the data. The Kaiser–Meyer–Olkin (KMO) statistic was 0.96 (i.e. >0.80), indicating that our data was suitable for factor analysis (Hutcheson & Sofroniou, 1999). EFA with varimax rotation yielded a 4-factor solution that explained 71.25% of the variance (Table 2). The scree plot identified two factors which accounted for only 62.6% of total variation in the data (Figure S1). However, we considered the cumulative percentage >70%, and finally four factors were retained. No items were loaded below 0.40, and no items were removed from the scale, hence the scale was formed from 19 items. The four factors were designated ‘anger perception’ (3 items), ‘exploring the cause of anger’ (6 items), ‘self-preparation’ (7 items) and ‘communication skills’ (3 items).

Following the identification of a 4-factor solution by EFA, CFA was performed to further test the structure of the NCAAPS. Goodness-of-fit indices were examined to determine the degree of fit between the data and a hypothesized first-order four-factor model assuming that the NCAAPS was composed of four separate correlated dimensions. The correlations between the four factors were statistically significant (p < .001). No further modification of the model structure was made because the majority of the fit indices indicated a reasonable fit to the data, with only small differences between the higher-order and first-order models. The goodness of fit indices was as follows: χ² = 654.254; df = 146; p < .001; CMIN/DF = 4.481; RMSEA = 0.087; CFI = 0.92; NFI = 0.90; TLI = 0.91; and IFI = 0.92 (Browne & Cudeck, 1993; Kline, 1998).

3.4 Criterion-related validity

Overall NCAAPS score was significantly correlated with both the SEGUE framework score (r = 0.62, p < .001) and GSES score (r = 0.55, p < .001). SEGUE framework score was also significantly correlated with the ‘anger perception’, ‘exploring the cause of anger’ and ‘communication skills’ dimensions of NCAAPS but not with ‘self-preparation’ dimension, whereas the GSES score was significantly correlated with all four dimensions (Table 3).

3.5 Reliability

Cronbach's α coefficient was 0.96 for the NCAAPS overall (19 items) and ranged from 0.81–0.92 for the four dimensions (Table 4). The corrected item-total correlations ranged from 0.60–0.81 (i.e. >0.40 for all items), indicating that the items were fairly homogeneous. The correlations between the four dimensions and the total scale are shown in Table 5. The test–retest reliability of the scale was 0.740 (p < .001), indicating that it was stable.

4.1 Limitations

This study has some limitations. First, although we recruited participants from 9 third-grade general hospitals from three provinces in China, it cannot be guaranteed that the sampled population is representative or that the findings are generalizable to other countries. Second, only nurses working in emergency departments were included, so it remains to be determined whether the NCAAPS would be suitable for nurses working in other hospital departments or settings. Third, the sample size was small for the huge nurse population in China, and the reliability coefficient for some factors was not satisfactory. Last but not least, the limitation is that items 4, 6, 7, 9, 10 and 12 had cross loadings (with two factors loading >0.4), and the overlap rate is too high.