Who's challenging who training for staff empathy towards adults with challenging behaviour: cluster randomised controlled trial
Abstract
Background
One in five adults with intellectual disabilities (ID) known to services display challenging behaviours (CBs), and these individuals are at risk for restrictive practices and poor care. Staff attitudes may contribute to the development and/or maintenance of CBs. We investigated the effectiveness of co-produced Who's Challenging Who? training delivered by people with ID to staff.
Method
This study involved a cluster randomised controlled trial (RCT) of Who's Challenging Who? training with follow-up at six and 20 weeks post-randomisation. Participants: two staff from each of 118 residential care settings for adults with ID at least one of whom displayed aggressive CB. Primary outcome: Self-reported Staff Empathy for people with Challenging Behaviour Questionnaire. Analysis: intention to treat of all randomised settings. ISCRTN registration: ISRCTN53763600.
Results
118 residential settings (including 236 staff) were randomised to either receive training (59 settings) or to receive training after a delay (59 settings). The primary analysis included data from 121 staff in 76 settings (51% of staff, 64% of settings). The adjusted mean difference on the transformed (cubed) Staff Empathy for people with Challenging Behaviour Questionnaire score at the primary end point was 1073.2 (95% CI: -938.1 to 3084.5, P = 0.296) in favour of the intervention group (effect size Cohen's d = .19).
Conclusions
This is the first large-scale RCT of a co-produced training course delivered by people with ID. Findings indicated a small positive (but statistically non-significant) effect on increased staff empathy at 20 weeks, and small to moderate effects for staff reported secondary outcomes in favour of the intervention group.
Introduction
Challenging behaviours (CBs) displayed by individuals with intellectual disability (ID) are actions that may place the individual at risk of harm or exclusion, or may place other people (e.g., carers) at risk of harm (Emerson & Einfeld 2011). Behaviours typically identified as challenging include anger and aggression, self-injurious behaviours (e.g., self-biting, hitting body parts against objects, scratching and gouging), destruction of property and inappropriate or risky sexual behaviour. CBs can have a direct impact on the quality of health and social care supports received by people with ID. For example, those who display CBs are more likely to be exposed to abusive practices including unnecessary psychoactive medication (Bowring et al. 2017). Carers of individuals with CBs may experience increased stress (Hastings 2002a; Hastings 2002b), and the cost of support services for people with CBs is high (Knapp et al. 2005). Population-based studies suggest that 18–19% of adults with ID known to services engage in CB that has significant impact on their lives (Jones et al. 2008; Bowring et al. 2016).
A range of personal and environmental risk factors have been associated with the emergence and ongoing maintenance of CBs, including genetic factors, other biological vulnerabilities, and severity of ID and communication impairments (Hastings et al. 2013). However, theoretically, many of the risk factors for CB are socially mediated and so highlight the inadvertent shaping of CBs through interactions with carers (Hanley et al. 2003; Beavers et al. 2013; Hastings et al. 2013). Support staff behaviours in particular are often targeted for change using Positive Behaviour Support (PBS) used in ID services (Gore et al. 2013) that focuses on reducing CBs whilst also building quality of life and key skills (e.g., communication).
Although strongly informed by person-centred values (Gore et al. 2013), a limitation of PBS is that it does not include explicit elements to either increase support staff motivation to engage in changing their own behaviour, or to engender attitude change. The latter is significant, since support staff beliefs and attitudes are a core part of theoretical models of why staff may behave in ways that increase the risk for the development and maintenance of CBs (Hastings & Remington 1994; Hastings 1997; Hastings et al. 2013). Staff understanding, for example, of the likely causes of CBs on a general or individual level may affect their behaviour towards individuals with ID. Individuals perceived as having ‘attention seeking’ CBs, for example, may be successfully ignored until CBs escalate and staff respond thus inadvertently reinforcing CB and increasing the future chances of such behaviour recurring when attention is low or not available.
Staff empathy (or understanding for the situation of people with CBs) is a crucial target for change in staff training interventions but has been neglected in ID research. First, people with ID and CBs themselves have clearly identified a lack of empathy for their situation from staff as a cause of friction and increased likelihood of CB (Griffith et al. 2013). Second, attribution theory (Weiner 1986) applied to the care context (e.g., Sharrock et al. 1990) suggests that CBs perceived as uncontrollable by the person are likely to lead to affective responses from staff related to sympathy and then more willingness to help. Third, empathy is also a key skill in human development (e.g., Eisenberg et al. 2010) such that increased ability to empathise with another's situation is associated with increased pro-social responses (e.g., helping) and less aggressive responses (potentially, less restrictive responses in a care environment). Thus, for pragmatic and theoretical reasons, increasing staff empathy for people with CBs should, in the long term, contribute to improved care.
The Who's Challenging Who? (WCW) training course for support staff was designed to fill the gap in the research literature for an empathy/attitude change intervention. WCW was developed to be specific to the ID services context, with content informed by systematic reviews and meta-syntheses of research on the experiences of people with ID and CBs and their carers (Griffith et al. 2013; Griffith & Hastings 2014). However, the theoretical and co-production basis of WCW was informed by research in mental health stigma. Specifically, applying Contact Theory (Allport 1954), and the results of meta-analytic research synthesis (Mehta et al. 2015), stigma reduction and attitude change interventions are more effective if they involve social contact with individuals from the stigmatised group. To be maximally effective, Contact Theory (Allport 1954) suggests that such contact should be intimate (in small group settings), involve a valued role for the stigmatised individuals, involve co-operative activity and be community-sanctioned (i.e., someone important to you communicates their support for the contact).
In earlier pilot research (Hutchinson et al. 2014), we trained two adults with ID and CB to co-deliver 10 small group training courses to 76 staff with a trainer without disability. A comparison of pre-training and post-training scores indicated positive changes were found in staff empathy towards people with ID and CBs, staff confidence in dealing with CBs and staff attitudes (Hutchinson et al. 2014). The pilot study also demonstrated the feasibility of staff recruitment, the co-production process with people with ID and delivery of the manualised training by two trainers per WCW course (one trainer for each course having ID).
In the current study, our aim was to evaluate the effectiveness of the WCW training course in a large-scale cluster randomised controlled trial (RCT) on staff empathy for people with ID.
Methods
Trial design and participants
The trial was registered on 08/12/2015 (ISRCTN53763600). The study protocol is published in full elsewhere (Randell et al. 2017). We used a cluster randomised design to evaluate the WCW intervention, with residential settings the unit of randomisation and all staff members recruited to the study within a setting allocated to the same arm (WCW training or waiting list control group). At each residential setting, two members of staff were recruited from amongst the staff team: one manager/lead staff member along with one other support staff member. In both arms of the trial, those staff taking part were not restricted from receiving training as usual (on ID social care and specifically, on CBs if provided) following their organisations' ongoing staff development policies. Data were not gathered on what training as usual may have been provided to staff and settings during the course of the study.
Residential settings were eligible for inclusion if they were based in a community setting, provided services via publicly funded contracts (e.g., local authorities, clinical commissioning groups), supported between one and 10 people with ID, employed staff who provided at least some 24-h support, provided care for at least one person with ID who displayed aggressive CB and could identify one manager/lead staff member and one other support staff member who could attend WCW training together. Staff were eligible for inclusion if they were either a manager (or lead staff member as defined by the service provider organisation) or a direct support worker whose roles were no more than 50% administrative/management. Staff who worked less than 70% of full-time equivalent were also ineligible.
Randomisation and masking
Settings were recruited in two phases, with phase 2 immediately following the 20-week data collection time point for phase 1. Randomisation occurred at one point in time for each phase, was carried out by a study-independent statistician from the Centre for Trials Research and used a dynamic balancing algorithm specifically designed for cluster randomised trials (Carter & Hood 2008). Allocations were stratified by residential setting size (i.e., the number of residents with ID in the setting), geographical region (West-Midlands or Manchester/North England) and phase of recruitment (i.e., phase 1 or 2) in 1:1 ratio. The trial statistician remained blind to allocation up until the point of data analysis. Settings, and staff members within them, could not be masked to the intervention but were recruited prior to randomisation.
Procedures
Residential setting managers were contacted by a Research Assistant for a structured telephone screening interview based on the inclusion and exclusion criteria. To establish whether at least one person engaged in aggressive CB at least weekly on average, items from the aggression/destruction scale of the Short Form of the Behavior Problems Inventory (Rojahn et al. 2012) were used as prompts during screening. Following the collection of baseline data, staff from settings allocated to receive WCW were required to attend a training session [up to six settings (12 staff) per session]. Staff from settings allocated to the control were offered WCW after the 20-week follow-up data collection time point (Fig. 1).
The WCW intervention evaluated in the pilot study (Hutchinson et al. 2014) was revised and extended for the current study. A meta-analysis of factors affecting outcomes in staff training in ID settings suggested that a combination of content training with coaching is more effective than classroom/workshop training alone (van Oorsouw et al. 2009). Thus, we developed an additional coaching element for post-training telephone mentoring support. Other data on training in ID services has highlighted the importance of supportive managers (Totsika et al. 2008); hence, our focus on training the setting manager or other senior staff member alongside other staff in each setting.
- Communication and how staff listening can prevent escalation of CBs
- How the living environment contributes to frustration and CBs
- The experience of being physically restrained
- What it is like to be on medication ‘for’ CB
- Experiences of feeling excluded because of CB
- Unhelpful attitudes and behaviour of support staff, and a discussion of positive qualities that contribute to good support/care
Each section of the training includes problems experienced in that domain from the perspective of people with ID and CB, and how people with ID suggest improvements that could be made to the support they receive in this area. Within each section of the training, there are videos of the trainers' experiences, experiential exercises with trainer feedback and the opportunity for the trainers with ID and CB to share more experiences with trainees.
At the end of the WCW training session, attendees write an Action Plan (based on one of the six areas in the training) of what they would do differently in their service on return. Action plans are shared with the group for feedback from the trainers and other attendees. Following WCW training, the trainer without ID contacted each residential setting manager/lead for a 30-min telephone coaching session based on the GROW model (Whitmore 1996) (Goal – what you want to do; Reality – where are you now; Options – what could you do; and Will – what will you do). The trainer followed a standard protocol for this coaching session, which involved finalising the Action Plan and discussing how this would be introduced to, adjusted, and then agreed with the staff team (e.g., as a part of a regular team meeting). After the Action Plan is agreed with the team, a further 30-min coaching session focuses on the implementation and monitoring of the Action Plan, especially dealing with obstacles. WCW training ends with this second coaching session.
In terms of contact theory (Allport 1954), WCW training was thus designed with all four of the key dimensions of contact in mind. First, the contact was intimate with a small group of staff attending the training session. Second, the contact involved a highly valued role for the stigmatised individual in that the person with ID was the lead trainer for the session. Third, the contact during the WCW session involve co-operative activity through multiple small and whole group practical exercises, discussion and planning. Finally, the contact was explicitly community-sanctioned through the selection and attendance at the WCW training of a manager or lead staff member from each residential setting.
Outcomes
Outcome measures were collected at baseline (i.e., prior to randomisation), 6 weeks and 20 weeks post-randomisation. Given the theoretical and potential practical significance of empathy and the prioritisation of empathy by people with ID themselves (Griffith et al. 2013), the primary outcome measure was staff self-reported empathy for people with challenging behaviour [Staff Empathy for people with Challenging Behaviour Questionnaire (SECBQ)] (Hutchinson et al. 2014), measured at 20 weeks post-randomisation. An existing general empathy scale was not used given that the focus of the intervention was specifically on increasing empathy in relation to people with ID and CBs. The SECBQ consists of five items rated on 6-point agreement scales: ‘I can relate to the everyday problems faced by people with intellectual disability/autism and challenging behaviour’; ‘I can easily see things from the point of view of people with intellectual disability/autism and challenging behaviour’; ‘I can imagine what it might be like to have intellectual disability/autism and challenging behaviour’; ‘I can understand why a person with intellectual disability/autism might present with behaviour (s) that can challenge’; ‘If I was faced with some of the life circumstances of people with intellectual disability/autism, I might resort to challenging behaviour’. Higher scores indicate higher levels of empathy. The internal consistency of the scale has been found to be good (Cronbach's alpha .72 in Hutchinson et al. 2014; .71 in the present study at baseline).
Secondary outcome measures included SECBQ at 6 weeks post-randomisation, challenging behaviour self-efficacy (Hastings & Brown 2002) (CBSE, 6 weeks and 20 weeks), similarities and empowerment attitude sub-scales from the Community Living Attitudes Scale (Henry et al. 1996) (CLAS_S and CLAS_E, 6 weeks and 20 weeks), emotional exhaustion, depersonalisation and personal accomplishment sub-scales of the Maslach Burnout Inventory (Maslach et al. 1996) (MBI_E, MBI_D, MBI_P, 20 weeks only), general positive contributions and positive work motivation sub-scales of the short version (Lunsky et al. 2014) of the Staff Positive Contributions Questionnaire (Hastings & Horne 2004) (SPCQ_GPC, SPCQ_PWM, 6 weeks and 20 weeks), recorded incidents of CBs and use of restrictive practices (20 weeks).
All WCW training attendees were asked to provide feedback about the course using a 10-question evaluation form focused on the following: participants' thoughts about the training content and delivery, whether they thought being trained by a person with ID was effective, and about the logistics of participating in the WCW training (Hutchinson et al. 2014). Questions were scored on a 4-point scale (1 = No; 2 = Yes, a little; 3 = Yes, mainly; 4 = Yes, definitely).
An audio recording of each WCW session was checked for fidelity to the WCW training manual by the Research Assistant (SF), and the first eight sessions were also rated by a second researcher who was not a part of the study. The fidelity rating tool had two sections. The first measured fidelity to the WCW manual and covered each of the six topics, as well as other session activities (e.g., introductions, review of learning outcomes, action plans). In total, there were 50 subcomponents of the training (reduced to 49 in Phase 2 to improve session timing) for full WCW delivery. Each subcomponent was rated on a scale of 0–2 (0 = missing/hardly delivered, 1 = partially delivered, 2 = fully/mostly delivered), with a maximum score of 100 for the fidelity to the manual for Phase 1 (Phase 2 maximum score = 98). The second section of the fidelity tool consisted of eight global ratings, including positive interactions between trainers and trainees, the active involvement of the trainer with ID, active participation of trainees in activities and good time management. The maximum score for the global ratings of the training was 16 for both phases, using the same 0–2 rating scale as the first section.
Statistical analysis
The sample size calculation was informed by the results from the Hutchinson et al. (2014) pilot study, where an effect size of .50 (standardised mean difference) for the staff empathy score (primary outcome) was observed. With a two-sided alpha of 0.05, and 90% power, the unadjusted sample size required was 172 (86 per arm) staff. With a cluster size of two staff per residential setting, and allowing an intra-class correlation coefficient (ICC) of 0.10, the variance inflation factor was 1.1. In addition, building in a 20% loss to follow-up meant an estimated 237 staff (thus, from 118 settings) were required.
All analyses were conducted using Stata v13.0. The primary analysis was conducted based on the intention-to-treat (ITT) principle. A two-level (staff nested within residential settings) analysis of covariance was fitted to provide a between-arm comparison of mean SECBQ scores at 20 week post-randomisation follow-up, adjusted for baseline SECBQ scores. The analysis also adjusted for residential setting size, phase and region (stratification factors at randomisation). Secondary outcomes were analysed similarly. Transformations were made to some outcome data to fulfil model assumptions (see full detail in note for Table 6). Recorded CB incident data were compared between arms using negative binomial regression models due to overdispersion, and hence a Poisson model did not provide an appropriate fit. Due to the presence of excess zeroes, analysis of the use of restrictive practices outcome involved fitting a zero-inflated negative binomial regression model. Differential intervention effects by length of time staff had worked in health/social care, length of time staff had worked in the specific setting and proportion of people with CB in the setting (all pre-specified in the Statistical Analysis Plan) were investigated by extending the primary analysis and fitting subgroup × trial arm interaction terms.
Sensitivity analyses comprised refitting outcomes as repeated measured within individuals, a full ITT analysis of the primary outcome [with missing data imputed using multiple imputation (50 imputations, imputation model included variables in the outcome model along with number of full-time staff, MBI_P at baseline and SPCQ_GPC at baseline)] and a complier average causal effect (CACE) analysis of the primary outcome (with intervention receipt defined as attending the training), and a two-stage least squares instrumental variables regression model fitted.
Role of the funding source
The funder of the study had no role in the study design, data collection, analysis, interpretation or preparation of this manuscript. D.G. and R.H. had full access to all study data, and RH had final responsibility for the decision to submit for publication.
Results
Participant characteristics
A total of 118 residential settings were randomised in total, with 59 settings randomised to each arm. Allocations were well distributed with respect to the balancing variables (phase, region and total number of residents living in the setting). There was an approximately equal number of settings randomised in each phase, the majority of settings were recruited from the West-Midlands (91/118, 77.1%), and the median number of residents living in a setting was 4.5 [interquartile range (IQR): two to seven residents]. Each residential setting identified the two members of staff required for this study, and hence 118 staff members were included in each trial arm (236 in total).
The primary outcome analysis included data from 121 staff in 76 settings (51% of staff and 64% of settings: 33 settings and 52 staff in the WCW arm; 43 settings and 69 staff in the control arm). Other 20-week outcomes were available for up to 117 staff from 71 settings (50% and 60% overall, respectively). Analyses of 6-week outcome data included up to 107 staff from 71 settings (45% and 60% overall, respectively). At both time points, a greater percentage of responses was obtained from Control than WCW participants (Fig. 1).
Overall, residential settings had a median number of nine full-time staff (IQR: 4 to 15 staff) and four part-time staff (two to six staff). The median number of residents displaying recent CB was 2 (IQR: one to four individuals). On average, 68% of residents in each setting had displayed recent CB (SD: 31.7%) (Table 1). Staff members were a mean age of 40 years (SD: 11.6 years). The majority were female (160/209, 77%) and white (184/206, 89%). Staff had worked in health or social care for a median of 11.0 years (IQR: 6.5 to 17.3 years), with people with ID for a median of 9.5 years (4.9 to 15.0 years), and in their current role for a median of 2.3 years (IQR: 0.8 to 6.5 years) (Tables 2 and 3). The majority of staff recruited worked full-time (185/208, 89%). Similar to other baseline variables, there was good balance between arms with respect to baseline measurement of outcome variables (Table 4).
| Variable | Control | WCW | ||||||
|---|---|---|---|---|---|---|---|---|
| n | Mean (SD) | Median (IQR) | Min to max | n | Mean (SD) | Median (IQR) | Min to max | |
| Total number of full-time care staff | 58 | 11.3 (8.41) | 9 (4 to 15) | 0 to 31 | 59 | 11.0 (10.00) | 8 (3 to 15) | 0 to 53 |
| Total number of part-time care staff | 58 | 5.2 (4.99) | 4 (2 to 7) | 0 to 25 | 59 | 4.0 (4.18) | 3 (0.5 to 5) | 0 to 17 |
| Total number of residents displaying aggressive behaviour* | 59 | 2.9 (2.03) | 2 (1 to 4) | 1 to 9 | 59 | 2.9 (2.13) | 2 (1 to 4) | 1 to 9 |
| Proportion of residents displaying recent challenging behaviour* | 59 | 0.7 (0.31) | 0.8 (0.3 to 1.0) | 0.1 to 1.0 | 59 | 0.7 (0.32) | 0.7 (0.4 to 1.0) | 0.1 to 1.0 |
| Total number of residents living in the setting (median, IQR) | 59 | 5 (3 to 6) | 59 | 4 (2 to 7) | ||||
- * For each resident, staff were asked whether they have displayed any of the listed behaviours from the Behaviour Problems Inventory Short Form Aggressive/Destructive Behaviour Scale (specifically, hitting others, kicking others, pushing others, biting others, grabbing and pulling others, scratching others, pinching others, verbally abusive with others, destroying things, bullying) at least once a month over the past 6 months.
| Variable | Control | WCW | ||||||
|---|---|---|---|---|---|---|---|---|
| n | Mean (SD) | Median (IQR) | Min to max | n | Mean (SD) | Median (IQR) | Min to max | |
| Age (years) | 99 | 39.8 (11.11) | 39.0 (31.0 to 49.0) | 20 to 70 | 105 | 39.5 (12.0) | 38.0 (29.0 to 47.0) | 19 to 68 |
| Length of experience in health/social care (years) | 101 | 12.8 (8.93) | 11.0 (6.4 to 17.3) | 0.3 to 39 | 100 | 12.6 (8.41) | 10.6 (6.5 to 17.5) | 0 to 36 |
| Length of experience working with people with intellectual disabilities (years) | 101 | 10.5 (8.18) | 9.2 (4.5 to 14.7) | 0.5 to 39 | 100 | 11.4 (8.05) | 10.0 (5.7 to 15.1) | 0 to 36 |
| Approximate length of time in current role (years) | 100 | 3.8 (3.82) | 2.3 (0.7 to 6.4) | 0 to 17 | 100 | 4.1 (4.13) | 2.4 (0.9 to 6.8) | 0 to 20 |
| Variable | Control | WCW | |||
|---|---|---|---|---|---|
| Frequency | % | Frequency | % | ||
| Male | 26 | 25.2 | 23 | 21.7 | |
| Female | 77 | 74.8 | 83 | 78.3 | |
| Job category | Co-ordinator role | 5 | 6.4 | 1 | 1.4 |
| Manager role | 32 | 41.0 | 39 | 53.4 | |
| Leader role | 11 | 14.1 | 8 | 11.0 | |
| Support worker role | 29 | 37.2 | 25 | 34.2 | |
| Assistant psychologist | 1 | 1.3 | 0 | 0.0 | |
| Full-time | 90 | 88.2 | 95 | 89.6 | |
| Part-time | 12 | 11.8 | 11 | 10.4 | |
| Ethnicity | White | 92 | 89.3 | 92 | 89.3 |
| Asian/Asian British | 2 | 1.9 | 1 | 1.0 | |
| Black, African/Caribbean, Black British | 8 | 7.8 | 8 | 7.8 | |
| Mixed/Multiple ethnic groups | 1 | 1.0 | 1 | 1.0 | |
| Other | 0 | 0.0 | 1 | 1.0 | |
| Highest education qualification | No formal educational qualifications | 2 | 2.0 | 6 | 5.9 |
| Standard Grades, O Grades, GCSEs, O Levels or equivalent | 35 | 35.7 | 25 | 24.8 | |
| Higher Grades, GCE A Levels, HNC, GNVQ or equivalent | 27 | 27.6 | 32 | 31.7 | |
| HND, other diploma or equivalent | 17 | 17.3 | 22 | 21.8 | |
| Polytechnic/University ordinary or honours degree | 15 | 15.3 | 14 | 13.9 | |
| Masters or Doctoral degree | 2 | 2.0 | 2 | 2.0 | |
| Formal health/social care qualification | 80 | 77.7 | 87 | 83.7 | |
| Health/social care qualification | GCSE (A-C) /L2 diploma | 11 | 16.2 | 13 | 19.1 |
| A-level/A2/AS/L3/L3-extended diploma | 26 | 38.2 | 34 | 50.0 | |
| Foundation degree/degree/HND/HNC | 15 | 22.1 | 9 | 13.2 | |
| Foundation degree/degree/HND/HNC | 10 | 14.7 | 8 | 11.8 | |
| Degree | 5 | 7.4 | 4 | 5.9 | |
| Master's degree | 1 | 1.5 | 0 | 0.0 | |
| Variable | Control | WCW | ||||||
|---|---|---|---|---|---|---|---|---|
| n | Mean (SD) | Median (IQR) | Min to max | n | Mean (SD) | Median (IQR) | Min to max | |
| Staff-reported measures | ||||||||
| SECBQ | 104 | 24.6 (3.89) | 25 (23 to 28) | 12 to 30 | 106 | 24.9 (3.38) | 25 (23 to 27) | 16 to 30 |
| CBSE | 103 | 27.4 (3.95) | 27 (25 to 30) | 20 to 35 | 104 | 26.9 (4.12) | 27 (24.5 to 30) | 16 to 35 |
| CLAS_S | 99 | 5.3 (0.52) | 5.4 (4.9 to 5.7) | 3 to 6 | 104 | 5.2 (0.54) | 5.3 (4.9 to 5.6) | 4 to 6 |
| CLAS_E | 97 | 4.5 (0.64) | 4.5 (4.2 to 5) | 2 to 6 | 103 | 4.4 (0.6) | 4.5 (4.1 to 4.8) | 3 to 6 |
| MBI_E | 102 | 13.5 (10.39) | 11.5 (6 to 17) | 0 to 46 | 101 | 13.9 (8.5) | 13 (8 to 20) | 0 to 47 |
| MBI_D | 101 | 2.3 (2.82) | 1 (0 to 3) | 0 to 13 | 104 | 2.6 (2.98) | 1 (0 to 5) | 0 to 14 |
| MBI_P | 102 | 39.9 (6.32) | 41 (36 to 45) | 20 to 48 | 103 | 40.8 (5.57) | 42 (37 to 45) | 19 to 48 |
| SPCQ_GPC | 100 | 20.7 (5.69) | 21.1 (18 to 24.5) | 5 to 30 | 106 | 22.9 (4.99) | 24 (20 to 26) | 5 to 30 |
| SPCQ_PWM | 104 | 15 (3.1) | 15 (13.5 to 18) | 3 to 18 | 105 | 15.6 (2.69) | 16 (14 to 18) | 6 to 18 |
| Setting-reported measures | ||||||||
| Total number of recorded incidents of challenging behaviour* | 48 | 41.3 (58.42) | 23.5 (5.0 to 45.0) | 0 to 270 | 53 | 62.6 (107.88) | 25.0 (10.0 to 6.0) | 0 to 557 |
| Total number of recorded incidents of aggressive behaviour* | 49 | 26.4 (44.61) | 9.0 (2.0 to 31.0) | 0 to 270 | 52 | 56.9 (136.24) | 17.0 (5.0 to 46.0) | 0 to 826 |
| Total number of incidents of the use of restrictive practices* | 49 | 4.1 (9.07) | 0.0 (0.0 to 3.0) | 0 to 47 | 49 | 13.1 (30.25) | 2.0 (0.0 to 12.0) | 0 to 177 |
- * In the last 4 months.
Twelve WCW training sessions were delivered in total in the intervention arm of the trial. The first eight WCW groups were rated independently by two raters for fidelity of implementation of the training. The ICC for the ratings of fidelity to the manual checklist was 0.97, indicating very high levels of inter-rater reliability [95% confidence interval (CI): 0.88 to 1.00]. For global fidelity ratings, the ICC was 0.98 (95% CI: 0.92 to 1.00). Group sizes of training sessions ranged from 4 to 10 staff members (median = 7 staff members). Intervention fidelity, assessed by both the fidelity to the manual checklist (overall median = 98 out of a maximum score 100, IQR = 97 to 99) and global rating of the session (overall median = 15 out of a maximum score of 16, IQR = 14 to 16), was very high across regions and phases (Table 5).
| Region | Phase | n | Fidelity to the manual (max = 100) | Global rating (max = 16) | ||||
|---|---|---|---|---|---|---|---|---|
| Median | Interquartile range | Min to max | Median | Interquartile range | Min to max | |||
| Midlands | 1 | 6 | 99 | 99 to 100 | 87 to 100 | 14 | 13 to 15 | 13 to 16 |
| 2 | 4 | 98 | 97 to 98 | 96 to 98 | 15 | 15 to 16 | 14 to 16 | |
| Total | 10 | 99 | 98 to 99 | 87 to 100 | 15 | 13 to 15 | 13 to 16 | |
| Manchester/North-West | 2 | 2 | 97 | 96 to 98 | 96 to 98 | 16 | 15 to 16 | 15 to 16 |
| Total | 2 | 97 | 96 to 98 | 96 to 98 | 16 | 15 to 16 | 15 to 16 | |
| Total | 1 | 6 | 99 | 99 to 100 | 87 to 100 | 14 | 13 to 15 | 13 to 16 |
| 2 | 6 | 98 | 96 to 98 | 96 to 98 | 15 | 15 to 16 | 14 to 16 | |
| Total | 12 | 98 | 97 to 99 | 87 to 100 | 15 | 14 to 16 | 13 to 16 | |
Main analysis
The staff and setting outcomes are summarised in Table 6. Staff allocated to Control had a mean score of 25.2 on the SECBQ at 20 weeks post-randomisation (SE: 0.38), and those allocated to WCW had a mean score of 25.9 (SE: 0.44). The adjusted mean difference on the cubed scale was 1073.2 (95% CI: −938.1 to 3084.5, P = 0.296). The setting-level intra-cluster correlation coefficient was 0.16. The standardised effect size (Cohen's d), based on the transformed model, was 0.19 – indicating a small effect in the direction of benefit of the WCW intervention. These findings were relatively insensitive to our pre-planned sensitivity analyses: our full ITT analysis, with missing responses included using multiple imputation, suggested a smaller effect (adjusted mean difference = 568.5, 95% CI: −1624.4 to 2761.5) and our CACE analysis, adjusting for departures from allocated intervention, suggested a larger effect (adjusted mean difference = 1106.1, 95% CI: −927.1 to 3139.3) (Supplementary Tables S1–S4).
| Outcome measure | Time point | Control | WCW | Adjusted mean difference* | 95% Confidence interval | P-value | ICC | Standardised effect size‡ | |||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| n | Mean | Standard error | n | Mean | Standard error | Lower limit | Upper limit | ||||||
| SECBQ† | 20 weeks | 69 | 25.2 | 0.38 | 52 | 25.9 | 0.44 | 1073.2 | −938.1 | 3084.5 | 0.296 | 0.16 | 0.19 |
| SECBQ† | 6 weeks | 56 | 24.5 | 0.44 | 51 | 25.5 | 0.47 | 1519.4 | −549.9 | 3588.7 | 0.150 | 0.20 | 0.28 |
| CBSE | 6 weeks | 55 | 27.8 | 0.36 | 51 | 28.9 | 0.43 | 1.1 | −0.05 | 2.2 | 0.061 | 0.00 | 0.41 |
| CBSE | 20 weeks | 66 | 27.9 | 0.35 | 48 | 28.9 | 0.42 | 1.0 | −0.1 | 2.0 | 0.079 | 0.00 | 0.35 |
| CLAS_S† | 6 weeks | 52 | 5.2 | 0.06 | 50 | 5.3 | 0.06 | 8.5 | −5.0 | 21.9 | 0.217 | 0.20 | 0.25 |
| CLAS_S† | 20 weeks | 64 | 5.3 | 0.06 | 47 | 5.3 | 0.07 | 0.4 | −1.3 | 2.2 | 0.640 | 0.00 | 0.09 |
| CLAS_E | 6 weeks | 52 | 4.5 | 0.07 | 49 | 4.7 | 0.07 | 0.2 | 0.0 | 0.4 | 0.044 | 0.00 | 0.40 |
| CLAS_E | 20 weeks | 62 | 4.5 | 0.06 | 47 | 4.7 | 0.07 | 0.2 | 0.0 | 0.4 | 0.037 | 0.00 | 0.42 |
| MBI_E† | 20 weeks | 67 | 14.0 | 0.86 | 46 | 12.1 | 1.05 | −0.3 | −0.7 | 0.1 | 0.120 | 0.00 | 0.30 |
| MBI_D† | 20 weeks | 66 | 2.3 | 0.33 | 46 | 1.5 | 0.39 | −0.3 | −0.5 | 0.0 | 0.053 | 0.00 | 0.37 |
| MBI_P† | 20 weeks | 67 | 38.7 | 0.65 | 46 | 40.8 | 0.78 | 0.2 | 0.0 | 0.3 | 0.048 | 0.03 | 0.38 |
| SPCQ_GPC† | 6 weeks | 53 | 22.7 | 0.61 | 50 | 23.3 | 0.63 | 13.4 | −57.4 | 84.2 | 0.710 | 0.21 | 0.07 |
| SPCQ_GPC | 20 weeks | 66 | 21.9 | 0.57 | 48 | 22.6 | 0.67 | 0.8 | −1.0 | 2.5 | 0.390 | 0.00 | 0.15 |
| SPCQ_PWM† | 6 weeks | 56 | 14.8 | 0.28 | 50 | 16.2 | 0.29 | 827.7 | 345.3 | 1310.2 | 0.001 | 0.00 | 0.66 |
| SPCQ_PWM† | 20 weeks | 68 | 15.3 | 0.27 | 49 | 16.0 | 0.32 | 380.0 | −150.6 | 910.5 | 0.160 | 0.00 | 0.26 |
| Outcome measure | Time point | n | Mean | Standard error | n | Mean | Standard error | Adjusted incidence rate ratio* | Lower limit | Upper limit | P-value | ||
| Challenging behaviour§ | 20 weeks | 30 | 40.9 | 11.32 | 27 | 59.5 | 21.55 | 1.0 | 0.5 | 2.1 | 0.947 | ||
| Aggressive behaviour§ | 20 weeks | 31 | 23.4 | 6.79 | 27 | 54.9 | 21.84 | 1.2 | 0.5 | 2.8 | 0.621 | ||
| Restrictive practices|| | 20 weeks | 32 | 2.6 | 1.14 | 25 | 4.6 | 2.72 | 0.7 | 0.3 | 1.7 | 0.405 | ||
- * Adjusted for variables balanced on at randomisation (phase, region, number of service users in the home) and corresponding baseline score.
- † Outcome measure and corresponding baseline score transformed to fulfil regression assumptions. Transformations were as follows: SECBQ at 20 weeks: cube; SECBQ at 6 weeks: cube; CLAS_S at 6 weeks: cube; CLAS_S at 20 weeks: square; MBI_E: square root; MBI_D: ln + 1; MBI_P: square root; SPCQ_GPC at 6 weeks: square; SPCQ_PWM at 6 and 20 weeks: cube.
- ‡ Glass's delta – Standardised effect size calculated as the difference in means (WCW - Control) divided by the standard deviation of the Control group.
- § Analysis based on a negative binomial regression model.
- || Analysis based on zero-inflated negative binomial regression model.
Staff-reported secondary outcomes (Table 6) followed a similar pattern to the primary outcome, with standardised effect sizes ranging from negligible (CLAS_S at 20 weeks) to a large effect in the direction of benefit of the WCW intervention (SPCQ_PWM at 6 weeks).
Analysing staff-reported outcomes as repeated measures over time demonstrated similar intervention effects to those in Table 6. There was some evidence to suggest that SECBQ scores increased between 6 weeks and 20 weeks. There was insufficient evidence to suggest that any of the outcomes were lower/worse at 20 weeks than they were at 6 weeks. There was also insufficient evidence to suggest any differential intervention effect over time (Supplementary Table S6).
Setting-level outcomes (i.e., incidents of CB and use of restrictive practices) were higher on average in settings allocated to the WCW arm than the Control arm. However, adjusted incidence rate ratios were low, and 95% CIs were wide, providing insufficient evidence to suggest a difference between arms (Table 6).
Subgroup analyses
Pre-specified subgroup analysis suggested a differential intervention effect on the SECBQ at 20 weeks according to the proportion of residents with CB within a setting. The trial arm main effect was 6678.0 (95% CI: 2495.4 to 10860.6, P = 0.002), indicating large differences between arms on the SECBQ at 20 weeks when the proportion of residents with CB was zero; the main effect for proportion of residents with CB in a setting was 3886.0 (95% CI: 204.3 to 7567.8, P = 0.039), indicating that as the proportion of residents with CB in Control arm settings increased, scores on the SECBQ at 20 weeks also increased. The interaction effect was −8412.0 (95% CI: –14045.8 to −2778.1, P = 0.003), indicating that the WCW intervention yielded higher SECBQ scores at 20 weeks than the Control arm when the proportion of residents with CB in a setting was low, with the effect diminishing as this proportion increased (Fig. 2). There was insufficient evidence to suggest any differential intervention effect according to length of time staff had worked in health/social care or length of time they had worked in the specific setting.
Training evaluation data
Evaluation data (Table 7) from trainees on the WCW courses indicate that the training was perceived to be enjoyable, and a good use of trainees' time. Trainees believed that being trained by a person with ID is an effective way to deliver training about CBs. Overall, the training materials and activities were perceived as appropriate, and participants had the opportunity to participate in the training. Participants largely believed that they would be able to apply the learning from the training to their job and did not feel uncomfortable at any point during the training. The training was delivered within an appropriate timeframe, and there was adequate time for breaks within the session.
| Mean (SD) | N | |
|---|---|---|
| 1. Did you enjoy the training? | 3.68 (0.52) | 85 |
| 2. Do you think the training being led by a person with an intellectual disability is an effective way to deliver training about challenging behaviour? | 3.87 (0.37) | 85 |
| 3. Were the training materials appropriate? | 3.65 (0.59) | 85 |
| 4. Were the training activities appropriate? | 3.64 (0.53) | 85 |
| 5. Did you have an opportunity to participate in the training session? | 3.86 (0.38) | 85 |
| 6. Did you feel uncomfortable or uneasy at any time during training? | 1.26 (0.69) | 85 |
| 7. Was the training delivered within an appropriate timeframe? | 3.71 (0.59) | 85 |
| 8. Was there adequate time for breaks? | 3.93 (0.26) | 85 |
| 9. Would you consider the training a good use of your time? | 3.64 (0.63) | 85 |
| 10. Will you be able to apply what you have learned to the job that you do? | 3.71 (0.49) | 84 |
Discussion
For the primary outcome (staff empathy or SECBQ score) at the primary end point (20 weeks post-randomisation), we found insufficient evidence to suggest that the WCW training was effective. Staff reported greater empathy for people with CB in the WCW arm of the trial, but this was a small effect (Cohen's d = .19), and the study was not powered to detect such small effect sizes. This conclusion was not affected by the results of planned sensitivity analyses (full ITT with multiple imputation, CACE analysis) (Supplementary Tables S1–S4). Staff in the WCW arm of the trial did report more positive outcomes on several secondary measures: staff had more positive attitudes towards people with ID and CBs in terms of their right to control their own lives (empowerment attitudes CLAS-E at both 6 and 20-week follow-ups), and staff reported increased personal accomplishment at work at 20 weeks (MBI_P) and increased positive work motivation at 6 weeks (SPCQ_PWM). All other staff reported outcomes were also in favour of the WCW arm of the trial but did not reach statistical significance at either 6 or 20 weeks. There was no evidence of a trial arm difference at 20-week follow-up for setting-level variables (recorded incidents of CBs and use of restrictive practices).
There was no evidence to suggest that WCW outcomes for staff improved or reduced over the follow-up period from 6 to 20 weeks post-randomisation. However, planned subgroup analyses revealed that the effect of WCW on the primary outcome (staff empathy) was greater in settings where the proportion of residents exhibiting CBs was smaller and gradually diminished as this proportion increased. It is not clear why the WCW training was more effective in settings where a smaller proportion of residents had CBs. Given the association between severity of ID and CB (e.g., Bowring et al. 2017), it might be that residents had less severe ID in the settings where a smaller proportion had CBs, were thus more similar to the trainers, leading to increased effectiveness of the training. Another possible explanation could be that staff experienced increased burnout in settings where a high proportion of residents had CBs rendering the training less effective, although additional analysis of the baseline data does not support this hypothesis about increased burnout in these settings (Flynn et al. in press). The interaction effect might also not be replicated in future research.
This was the first large-scale RCT of a co-produced staff training intervention, delivered by people with ID. An earlier systematic review (MacDonald & McGill 2013) of PBS staff training included no studies with a focus on changing staff empathy or staff attitudes, and no outcome measures designed to assess staff empathy. The recruitment target for staff and settings was achieved, the project was delivered to the planned timetable and the trainers with ID delivered the WCW training with a very high level of fidelity to the manual and in terms of the non-specific aspects of staff training in group settings. There was also some evidence of the effectiveness of the WCW training, on secondary staff attitude and work-related well-being and motivation outcomes. There were no adverse events reported during the course of the trial, and post-WCW training evaluation data from staff suggested that they enjoyed the training, thought it was of good quality, and that it was relevant to their work. Importantly, trainees were extremely positive in their belief that being trained by a person with ID is an effective way to deliver training about CBs.
Some weaknesses of the present study also need to be considered when interpreting the findings. First, positive effects were found only for staff reported outcomes, and staff were not blind to their allocation to trial arms. Second, retention in the study was lower than expected (e.g., 56% of staff retained to the 20 week follow-up point). Third, retention was also better in the control arm of the trial introducing bias with unknown impact. In terms of retention, future research studies of staff in ID service settings should consider retention strategies beyond those that could be employed within the present research (e.g., we used reminders, following up with managers, certificates of attendance at training for staff training portfolios). In particular, the use of incentives for staff and/or services may be worth exploring. It is also unclear why retention was better in the control arm of the study – potentially because staff knew they would then be offered access to WCW training and/or because they perceived themselves to still be a part of ongoing research (with staff in the WCW trial arm perhaps considering that their participation in the WCW training was the end of their involvement).
The effect size in favour of WCW was small in this study and smaller than found in the pre-post test pilot study. Reasons for a small effect may be many. First, full RCTs often lead to smaller effects than pilot studies (especially pilot studies without a control group). Second, retention of staff and settings in the study was a challenge and so complete data on outcomes could not be tested. Third, staff recruited were experienced in care settings and may either already be empathetic to the situation of people with ID and CBs or, conversely, they could be less likely to be affected by an attitude change intervention (having developed ingrained beliefs insensitive to change; cf. Hastings & Remington 1994). It is important to note, however, that the planned subgroup analyses showed no evidence of a differential effect of WCW by staff experience. Fourth, we defined the primary outcome as staff empathy specifically towards those with ID and CBs but there were larger effects on secondary outcomes including attitudes and work-related well-being. These outcomes may be equally or more valuable than changes in empathy. In addition, the measure of empathy was bespoke and there is only a small amount of information available about its psychometric properties. Other ways of measuring empathy could be explored. An additional perspective, based on the theoretical rationale provided in this paper, might be that there is value in seeing even small shifts in staff empathy that may then affect their interactions with people with ID and CBs in the longer term (which was not measured in this research). In addition, large changes are unlikely from a short intervention. Further research would be needed to examine all of these possibilities.
Given that WCW is short, low cost to deliver (we estimate approximately £360 for a group training of six staff, excluding trainee travel and staff replacement costs), has some positive outcomes, receives positive evaluations from staff, has no known adverse effects and is demonstrably led by people with ID, it has potential to contribute to the range of training staff are offered when working with individuals with CB. In future, it would be important to consider when in the work cycle WCW training might be more effective (e.g., for new staff vs. experienced staff perhaps set in their ways), the delivery context (i.e., with a mixed group of staff from different settings vs. training for a whole staff group who work in a particular setting) and also how to increase the effectiveness of WCW in settings where a larger proportion of residents have CBs. The research also demonstrates that it is possible to robustly test the effectiveness of co-produced interventions in the field of ID.
Source of Funding
National Institute for Health Research School for Social Care Research (CO88/T14-035/WURH-P64). This paper summarises independent research by the National Institute for Health Research School for Social Care Research. The views expressed in this presentation are those of the author (s) and not necessarily those of the NIHR SSCR, NHS, the National Institute for Health Research, or the Department of Health. The funder of the study had no role in the study design, data collection, analysis, interpretation or preparation of this manuscript. D.G. and R.H. had full access to all study data, and R.H. had final responsibility for the decision to submit for publication.
Ethical standards
The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008. Ethical approval was granted by the Social Care Research Ethics Committee on 12 May 2015 (reference 15/IEC08/0030).
R.H., R.M., D.G., Z.T. and P.H. designed and managed the study. E.R., S.F., R.K. and G.M. were responsible for data collection and trial management. D.G. carried out the data analysis. L.R., Z.T., P.H., A.M., P.B. and B.W. co-produced the final version of the WCW training course. All authors made a contribution to the interpretation of the study findings and/or writing and editing the paper.
R.H., R.M., D.G. and Z.T. received funding from the NIHR School for Social Care Research for the study. A.M., P.B., B.W. and P.H. received payments from the research grant to design and deliver the WCW training.
Acknowledgments
We would like to thank all the staff and intellectual disability services who took part in the study, and the members of the study steering committee and advisory group.
