Performed and missed nursing care in Swiss acute care hospitals: Conceptual considerations and psychometric evaluation of the German MISSCARE questionnaire

2.1 Conceptual definition of nursing care activities

2.2 Selection of the measuring instrument

Unfinished nursing care has been measured predominantly via three ‘parent instruments’, based on the mentioned approaches (Jones et al., 2015): (a) the Task Undone Survey (Aiken et al., 2001); (b) the BERNCA questionnaire (Schubert et al., 2007); and (c) the MISSCARE questionnaire (Kalisch & Williams, 2009). To our knowledge, so far performed nursing care in the context of nursing performance has been measured mainly implicitly on the assumption that activities that were not missed were performed (Dubois, D’Amour, Pomey, Girard, & Brault, 2013). Based on our conceptual definition, we concluded that the most useful of the three ‘parent instruments’ was the American MISSCARE Part A (hereafter referred to as MISSCARE). In addition, the conceptual background, development and spread of this instrument were reasons for the decision. It asks nurses, midwives and nursing assistants to rate on a 5-point response scale how frequently (‘always missed’ to ‘never missed’) they or their unit's staff missed any of 24 listed nursing activities. As repeated psychometric testing of it has indicated high acceptability, reliability and validity (Kalisch, 2016), we translated and culturally adapted it to German. Then, we conducted a pilot study using it and the previously used BERNCA-R to compare them and to have a decision base for subsequent data collection.

4.1 Design

This is a methodological study designed to further develop and evaluate the German MISSCARE and compare it with the BERNCA-R. For the pilot study, a quantitative crossover design was applied. Data collection phases lasted two weeks each, with a four-week washout period between. Participants were randomized into two groups: one completed the BERNCA-R in Phase 1 and the MISSCARE in Phase 2; and the other, vice versa. It was also possible to participate in only one phase.

4.2 Sample and setting

We conducted the pilot study in one tertiary and two Swiss university acute care hospitals, purposively chosen from the participating hospitals in the multicentre research programme. In two hospitals, we choose 21 units and included all nurses and midwifes working there. In the third hospital, 536 participants from all units were randomly invited. That led to a purposive sample of 1,030 nurses and midwives (515 randomly assigned per crossover group) from various departments. Nurse executives provided the names and email or postal addresses of the registered nurses and midwives. Requirements for inclusion were a Swiss nursing diploma in nursing or midwifery or equivalent foreign qualification, direct involvement in nursing care and informed consent.

4.3 Instruments

While the BERNCA-R was available and no adaptions were necessary, we conducted a 7-step best-practice process of translation and adaptation for the MISSCARE (Jones, Lee, Phillips, Zhang, & Jaceldo, 2001; Wild et al., 2005): (a) preparation; (b) forward translation; (c) reconciliation; (d) back translation; (e) harmonization; (f) expert review; and (g) this pilot study.

(a) The preparation included the literature review, the conceptual work and the resulting selection of the MISSCARE questionnaire. (b) After obtaining the first authors’ consent, two native German-speaking PhD-prepared nurse scientists with good knowledge of the English language translated the MISSCARE questionnaire from US English to German. (c) A master- and a PhD-prepared nurse scientist who also have good knowledge of the English language integrated the two translated versions. (d) Two native English speakers—one with a nursing background and one working as professional translator—back-translated the integrated version independently. (e) The master- and the PhD-prepared nurse scientist compared the two translations with the original version. No substantial discrepancies were found. (f) An expert panel of five master-prepared clinical nurse specialists, three experienced PhD-prepared nurse scientists and a statistician was established to evaluate face validity. The experts were provided with the original and the translated questionnaire, the conceptual background and the working definition, and gave written feedback on all contents of the questionnaire and on possible cultural adaptations. The master- and PhD-prepared nurse scientist again integrated the responses and developed a culturally adapted version.

We made four adaptations to the German MISSCARE: (a) we complemented the introduction with our working definition; (b) to reduce recall bias, we added a recall period of seven working days to the research question; (c) to focus explicitly on performed care, we adjusted the original 5-point response scale (‘nearly always performed’ value = 1; to ‘nearly always missed’ value = 5) and added a ‘activity not necessary’ (value = 1) option to prevent missing values or to force wrong answers. Since activities that are not necessary are not considered to have been missed, we valued it the same as ‘nearly always performed’ and; (d) to take cultural differences and conceptual inconsistencies into account, we defined registered nurses and midwives as the target group and reworded and adapted several items.

The 32-item BERNCA-R questionnaire (extended from the 20-item original BERNCA) was authored in German (Schubert et al., 2013). Using a 5-point response scale, it asks respondents to rate how often they were unable to carry out each of 32 listed nursing activities over the previous seven working days. The original BERNCA has been translated and adapted for various languages and cultures (American version: PIRNCA), and adapted for nursing home use (Jones, 2014; Zúñiga et al., 2016). Psychometric testing by the primary author has confirmed its validity, internally consistency and homogeneity (Schubert et al., 2007). The psychometric properties of the German BERNCA-R had not yet been reported.

4.4 Data collection

Invitations to participate, study details and login information for the online survey, were provided via either email or paper mail. Informed consent was obtained, participation was voluntary, and anonymity was guaranteed. In Phase 1, half of the participants were asked to complete the MISSCARE and the other half the BERNCA-R. For Phase 2, a second invitation was sent and the respective other questionnaire was provided. In addition, respondents’ socio-demographic and professional variables were recorded. A free-text item on both the MISSCARE and the BERNCA-R gathered qualitative responses asking: ‘Do you have any feedback on the comprehensibility of the text (question, items, response scale) or on the applicability of this questionnaire (procedure and effort to complete)?’

4.5 Data analysis

Descriptive statistics were applied using percentages and sum scores (MISSCARE possible range: 25–125, with higher scores indicating more missed activities; response categories ‘activity not necessary’ and ‘nearly always performed’ were combined; only fully completed questionnaires were included for analysis). We tested the two questionnaires’ psychometric properties via exploratory factor analysis (principal axis analysis, varimax rotation) and Rasch modelling. A Rasch analysis provides a detailed analysis of many aspects of a scale, including fit of items and persons, item bias, internal consistency, dimensionality and targeting (Hagquist, Bruce, & Gustavsson, 2009; Van Alphen, Halfens, Hasman, & Imbos, 1994). However, Rasch models assume locally independent items and lack of differential item functioning (DIF). In case of local dependency (LD) or DIF, extra terms to model these deviations from a Rasch model were included. This extension is known as the graphical loglinear Rasch model (GLLRM) (Kreiner & Christensen, 2007). Statistical analyses used SPSS Statistics, version 25 (IBM Corp., 2017), the R version 3.6.1 (R Core Team, 2018) and DIGRAM 3.66 (Kreiner & Nielsen, 2013).

To assess construct validity, on the same response scale used for the MISSCARE questionnaire, we asked clinical nurse specialists in one hospital for subjective overall assessments of performed and missed nursing care on their units. If this was impossible, they were asked to compare two units regarding the extent of missed nursing activities. A high degree of agreement between their unit-specific estimates and the corresponding mean sum score of the questionnaire would indicate good convergent validity. For discriminant validity, we compared the assumed differences between two units of a department with the mean sum score of the questionnaire (DeVellis, 2003).

The qualitative responses were analysed according to the framework analysis by Ritchie and Spencer (2002). In the first stage of familiarization, the data material was viewed. Second, the index categories, given by the question, were identified. Third, the data material was indexed using the NVivo 12 Pro software (QSR International Pty Ltd, 2018). Fourth, the contents of the individual categories per questionnaire were analysed and related or contradictory statements were explored. Finally, the findings were interpreted and summarized narratively.

4.6 Ethical considerations

The study followed the principals of good clinical practice and was approved by the governmental ethics committees (BASEC Nr. Req-2019–00392) (International Council for Harmonisation, 2016). Participation was voluntary, informed consent obtained and confidentiality assured.

5.1 The German MISSCARE

In Phase 1 and Phase 2, 169 (32.8%) and 121 (23.5%) respondents, respectively, completed the MISSCARE, resulting in 290 response sets (total response rate: 28.2%). Table 1 summarizes respondents’ socio-demographic and professional variables and response rate per hospital.

Of the 290 participants, 274 (94%) answered all items (total missing values: 96 (1.32%)). The mean sum score of the fully completed questionnaires was 41.8 (range: 25–82). Table 2 presents the distributions of all responses.

Free-text responses of forty-three participants were received for the MISSCARE. About half of these related to the comprehensibility of the questionnaire. Five of those described that ‘The questionnaire is clearly understandable’. Three participants stated that the formulations were long or complicated. Concerning the question, it was not clear to all participants that they were asked about the last seven working days: “Could not give details (…) because I have not worked for the last 7 days”. Concerning the items, seven participants reported that not all of them were relevant in their setting: “I work in a neonatological unit, so some questions could not be answered”. Concerning the response scale, there were two feedbacks pointing out that the distinction between the response options was not defined clearly enough and one person stated that the response option ‘always performed’ would have been useful. There were two answers regarding the questionnaire's applicability that suggested clarity and ease of use.

The exploratory factor analysis (EFA)’s sampling adequacy was marvellous (Kaiser–Meyer–Olkin (KMO): 0.91). The EFA scree plot indicated that a single-factor solution would explain 33.2% of the variance. While factor loadings are ideally above 0.5, three items loaded below 0.4 (Table 3). Internal consistency was relatively high (Cronbach's alpha: 0.917).

For the Rasch analysis, response categories 1 – 5 have been transformed first to 0–4, and then, the categories with the values 3 and 4 have been combined. As several item pairs showed LDs, a fit of items to the Rasch model was rejected. However, a GLLRM fit succeeded. The conditional likelihood ratio (CLR) test for homogeneity was not significant (p = .623), and inter-subsample item parameter estimates differed only randomly, indicating the absence of DIF. Furthermore, it showed significant LDs in eleven item pairs—mainly because they related to common topics. Figure 1 presents the item response theory (IRT) graph of the GLLRM. A targeting assessment indicated an individual mean of −1.23, where 0.05 would be optimal; that is, participants answered that the nursing activities have been performed for a disproportionately large number of items. As a result, the sum score distribution is right-skewed (Figure 2). The reliability of the measures is relatively high (0.9).

5.2 The BERNCA-R

In Phase 1 and Phase 2, 169 (32.8%) and 119 (23.1%) respondents, respectively, completed the BERNCA-R, resulting in 288 response sets (total response rate: 28.0%). Table 1 summarizes respondents’ socio-demographic and professional variables and response rate per hospital.

Of 288 response sets, 263 (91.3%) were complete (missing values: 76 (0.83%); range of missing values/item: 0–6).

Free-text responses of forty-nine participants were received for the BERNCA-R. About two-third of these related to the comprehensibility of the questionnaire. General answers on comprehensibility indicate that the questionnaire is understandable. Concerning the question, it was not clear to three participants that they were asked about the last seven working days: “(…) often work only one day in seven days”. Five others stated that they “(…) consider the period of 7 working days too short”. Concerning the items, eleven participants reported that not all of them were relevant in their setting: “Not all items are relevant to the ICU”. Concerning the response scale, there was no reply. Feedback on applicability was mixed: three respondents considered the questionnaire straightforward; three others had difficulty logging in and completing it.

The EFA’s sampling adequacy was marvellous (KMO = 0.93); the scree plot indicated that a single factor explained 36.3% of variance. However, five loadings were below 0.4, including two below 0.3, with residual correlations remaining high. The internal consistency was relatively high (Cronbach's alpha = 0.94). Two- to five-factor models did not yield a simple structure with clearly defined factors. In a further attempt, after we deleted six items that appeared to fit poorly conceptually, a factor analysis showed a one-factor solution with reasonably high loadings but still high residual correlations.

Neither a Rasch model nor a GLLRM achieved an adequate fit for the remaining items. In addition to LDs between many item pairs, there was DIF regarding age; that is, despite equal levels of perceived missed nursing care, different age groups responded differently to individual items. Recognizing that the BERNCA-R could not be Rasch-scaled, we discontinued the analysis.

5.3 Construct validity

We assessed construct validity by having eight clinical nurse specialists’ estimate missed nursing care for 13 units prior to data collection. Due to the described difficulties with the BERNCA-R, we continued the evaluation only with the MISSCARE. Four units with less than five responses were excluded. By presenting those estimations alongside the mean sum score per unit, Table 4 shows that the experts’ estimates and the mean sum scores per unit have the same tendency and that assumed differences between units are reflected in the figures.

6.1 Implications for nursing management and research

This methodological study is part of the continuous monitoring of nursing service context factors in a sample of Swiss acute care hospitals. Its findings will broaden the knowledge base concerning performed and missed nursing care and its measurability in the Swiss–German context. The German MISSCARE allows longitudinal monitoring of performed and missed nursing care—for which Swiss data are currently scant—and benchmarking of participating hospitals and units. Furthermore, relationships among nursing service context factors, performed and missed nursing care, and nurse-sensitive patient and nursing outcomes can be observed, explored and monitored. Our findings will help the participating hospitals’ nursing managers make timely evidence-based decisions regarding resource allocation, interventions, evaluation and overall patient care, for example staffing (Kontio et al., 2013). However, it is important to bear in mind that these results reflect only the frequency of performed and missed nursing care, not the quality or adequacy of performed activities.

Regarding implications for future research, our results regarding the German MISSCARE will guide structural adjustments of the instrument. Then, after adaption of the questionnaire, another data collection and psychometric analysis are indicated. Our working definition has proved to be valuable. It is however reasonable to refine and specify this theory-based definition with qualitative follow-up studies. Regarding the BERNCA-R, our results indicate that it should be revised.

6.2 Limitations

At least three notable limitations apply to this methodological study; first, as the studied concept and its operationalization are highly context-dependent, comparisons can only be made conditionally (Zeleníková et al., 2019); second, purposive sampling and voluntary participation may have led to selection bias; that is, only persons particularly interested in the studied phenomenon may have participated. In combination with the complex design, this could partly explain the low response rate and the results may not reflect the experience of non-participants (Vincelette, Thivierge-Southidara, & Rochefort, 2019); and third, we relied on self-reporting, which is prone to recall bias and social desirability.