Can the delivery of behavioural support be improved in the NHS England Low-Calorie Diet Programme? An observational study of behaviour change techniques
Abstract
Background
Previous research has illustrated a drift in the fidelity of behaviour change techniques (BCTs) during the design of the pilot NHS England Low-Calorie Diet (NHS-LCD) Programme. This study evaluated a subsequent domain of fidelity, intervention delivery. Two research questions were addressed: (1) To what extent were BCTs delivered with fidelity to providers programme plans? (2) What were the observed barriers and facilitators to delivery?
Methods
A mixed-methods sequential explanatory design was employed. Remote delivery of one-to-one and group-based programmes were observed. A BCT checklist was developed using the BCT Taxonomy v1; BCTs were coded as present, partially delivered, or absent during live sessions. Relational content analysis of field notes identified observed barriers and facilitators to fidelity.
Results
Observations of 122 sessions across eight samples and two service providers were completed. Delivery of the complete NHS-LCD was observed for five samples. Fidelity ranged from 33% to 70% across samples and was higher for group-based delivery models (64%) compared with one-to-one models (46%). Barriers and facilitators included alignment with the programme's target behaviours and outcomes, session content, time availability and management, group-based remote delivery, and deviation from the session plan.
Conclusions
Overall, BCTs were delivered with low-to-moderate fidelity. Findings indicate a dilution in fidelity during the delivery of the NHS-LCD and variation in the fidelity of programmes delivered across England. Staff training could provide opportunities to practice the delivery of BCTs. Programme-level changes such as structured activities supported by participant materials and with sufficient allocated time, might improve the delivery of BCTs targeting self-regulation.
Novelty statement
- Intervention fidelity (whether an intervention is implemented as intended) is associated with better intervention outcomes. Previous research has illustrated a drift in the fidelity of behaviour change techniques during the design of the NHS England Low-Calorie Diet pilot programme.
- Observations of remote one-to-one and group-based service delivery found an average of 55% of behaviour change techniques were delivered with fidelity.
- An under-delivery of behaviour change techniques during the NHS England Low-Calorie Diet Programme might impact participants' ability to sustain behavioural changes and improve their Type 2 Diabetes.
1 INTRODUCTION
Type 2 Diabetes Mellitus (T2DM) is one of the top 10 leading causes of death globally.1 In the UK, more than 4.3 million people are living with diabetes (90% of which have a T2DM diagnosis), costing the NHS 10% of the annual budget.2 Modifiable behaviours such as poor dietary quality, physical inactivity, smoking and excessive alcohol consumption increase both the risk of T2DM development and subsequent complications.3-7 Therefore, behaviour change is central to T2DM management.4
Interventions that target behavioural changes such as diet and physical activity are necessary to reduce health and economic burdens. Such interventions often include observable components designed to modify the cognitive and psychological processes underlying behaviour, known as behaviour change techniques (BCTs).8 Examples of BCTs include goal setting, action planning and problem-solving. However, as interventions are usually tested in tightly controlled trial settings, implementation of interventions at scale can lead to a loss in fidelity (whether a programme is implemented as intended). This can hinder the reproducibility and efficacy of interventions.9, 10
The National Institute of Health Behaviour Change Consortium model describes five domains of fidelity: programme design (the extent to which the intervention protocol reflects the evidence base); provider training (the extent to which deliverers are trained in an intervention's components); programme delivery (the extent to which intervention delivery adheres to the protocol); programme receipt (the extent to which intervention content is understood by participants); and programme enactment (the extent to which participants apply the intervention content in their daily lives).11, 12 For BCTs to exert their desired effects on health behaviours, they must be implemented as intended throughout each of these five phases.
An intervention to treat comorbid T2DM and obesity that includes the use of BCTs is the NHS England Low-Calorie Diet (NHS-LCD) (now known as Path to Remission).13 It is based on recent evidence demonstrating that low-calorie total diet replacement (TDR) approaches can be effective in achieving sustained T2DM remission (HbA1c <48 mmol/mol) in people with recently diagnosed T2DM (≤6 years).14, 15 The 52-week programme was initially piloted across 10 socio-demographically diverse areas in England. It includes a 12-week TDR phase, and subsequent stepped food reintroduction and weight maintenance support. BCTs are delivered via 20+ one-to-one or group-based health coaching sessions (delivered remotely during the COVID-19 pandemic), or digitally (web and app-based).
Previous studies by our research group evaluated three components of the pilot NHS-LCD providers' programme designs: the underpinning behavioural science theory,16 intended service parameters, and intended BCTs.12 On average, the four providers' designs had 79.5% fidelity to the BCTs stipulated in the NHS-LCD service specification,17 indicating some difficulty in translating the evidence base into programme designs.12 Although most BCTs were included within providers' programme designs, inclusion does not denote actual delivery. For example, the evaluation of another NHSE programme with a similar commissioning model, the NHS Diabetes Prevention Programme (NHS-DPP), found that although providers' session plans included 74% of the 19 BCTs in the NHSE service specification, the research team observed only seven of these BCTs in all eight observation sites, indicating difficulty in translating programme designs into programme delivery.18
Although quantitative evaluation of fidelity (such as that employed for the NHS-DPP), provides useful insight into a programme's implementation, it does not provide a clear understanding of why or how fidelity was influenced. This prevents making specific recommendations for programme commissioners and providers on how to optimise fidelity and programme outcomes. Understanding how fidelity can be improved is pertinent considering the increasing public and commercial availability of low-calorie diet programmes internationally. We are not aware of any studies that have evaluated the delivery of low-calorie diet programmes.
As part of an NIHR programme evaluation [NIHR132075], this study addressed two research questions: (1) To what extent are BCTs delivered with fidelity to providers programme plans in one-to-one and group-based delivery of the NHS-LCD? (2) What are the observed barriers and facilitators to BCT delivery?
The focus of this study was on the delivery of BCTs; fidelity to other intervention components will be reported separately.19
2 METHODS
Ethical approval was received from Leeds Beckett University [107887] and data collection occurred between January 2022 and February 2023.
2.1 Design, settings and participants
A mixed-methods sequential explanatory design was employed.20 Three providers were commissioned to deliver one-to-one or group-based behavioural support across six localities in England. However, due to limited engagement from one of the service providers, courses were sampled from two providers across five localities. Both providers delivered group-based courses, and one provider also delivered one-to-one courses. Due to COVID-19 all sessions were delivered remotely using videoconferencing software. NHSE stipulated that delivery staff must have sufficient and appropriate training and competencies.17 One provider recruited certified health coaches, whilst the other required a minimum of an undergraduate degree in Nutrition or a Health-related science.12 Both providers implemented session plans to guide delivery.
Courses of behavioural support were purposively sampled to ensure observation of a minimum of two courses per delivery model by each service provider. One-hundred and twenty-two live sessions were observed across eight samples. Two group-based courses were observed per provider and four one-to-one courses; however, only one complete one-to-one course was observed due to two participant withdrawals, whilst data collection for the fourth sample began mid-programme to ensure the remainder of the programme was observed. Table 1 outlines the coverage of session observations for each sample.
| Sample | Delivery model | Session numbers observed |
|---|---|---|
| 1 | Group | Full course |
| 2 | Group | Full course |
| 3 | Group | Full course |
| 4 | Group | Full course |
| 5 | One-to-one | Full course |
| 6 | One-to-one | 1–10 |
| 7 | One-to-one | 1–3 |
| 8 | One-to-one | 14–21 |
- Note: Provider and locality are not reported to protect anonymity; Sample numbers in Table 1 are different from Tables 2 and 3 to protect provider anonymity; Session 9 of samples 1 and 2 unobserved due to telephone delivery of a one-to-one progress review; a full course ranged from 20 to 21 sessions between providers.
2.2 Procedure
Informed consent was obtained from providers, delivery staff and programme participants. Two researchers observed the live sessions. Using a fidelity checklist, the delivery of planned BCTs was categorised by one researcher (TE, KD, PD, JM or CF) as delivered, partially delivered or not delivered. This checklist was based on the information gathered on BCTs in a previous study assessing providers' programme designs.12 All researchers completed the official Behaviour Change Technique Taxonomy (BCTT) v1 training.8, 21 BCTs were coded as delivered if they fulfilled the full BCTTv1 definition and they addressed the programme's target behaviour(s) or outcome(s). Partial delivery was coded if part of the BCTTv1 definition was met, and not delivered was coded if none of the BCTTv1 definition was met or the BCT addressed a behaviour or outcome outside of the programme's goals. BCT coding rules were previously reported.12 The checklist included space for field notes next to each BCT. The second researcher completed a checklist against other service parameters to be reported in a separate study.19 Following each session, the two researchers met to discuss their coding and field notes to achieve consensus. During early discussions within the research team, it became apparent that the BCT checklist field notes provided an opportunity to explore observed barriers and facilitators to BCT delivery, therefore research question two was devised a posteriori.
2.3 Analysis
Descriptive statistics were conducted using IBM SPSS 7 software to identify the fidelity of delivery for each included BCT and for all BCTs overall within each sample. To identify observed barriers and facilitators to fidelity, we used relational content analysis of field notes.22 To evaluate the co-occurrence of concepts, we used a subcategory of relational content analysis and proximity analysis.23 The aim was to link specific BCTs and their degree of fidelity to the barriers and facilitators that they frequently occur alongside.
Transcripts were imported into Nvivo 12 software and labelled with the following attributes: BCT label (BCTTv1), level of fidelity (delivered, partially delivered, not delivered), session number, programme phase (Total Diet Replacement, Food Reintroduction, Weight Maintenance), service provider, and delivery model (1:1, group). Two researchers (TE, KD) coded the data individually and in duplicate (see S2 for example field notes and codes), discussions were held to discuss interpretations and achieve consensus. Queries were run using Nvivo 12 software to explore and quantify the relationships between BCTs and codes. TE mapped out identified relationships and shared these with KD and later with the research team, to further discuss interpretations and relationships between constructs.
3 RESULTS
Overall, across samples, the fidelity of BCT delivery ranged from 33.3% to 69.9% (Table 2), with an overall mean of 55.1%. On average, fidelity was higher across group-based delivery models (64%) than one-to-one delivery models (46.2%), and fidelity was 18% greater for one of the providers compared to the other. We identified ‘partial delivery’ in an average of 11.8% (6.7%–17.8%) BCTs across all samples, with a mean of 33% (15.1%–60%) BCTs rated as ‘not delivered’. The designation of ‘partially delivered’ was similar across group-based (11.5%) and one-to-one (12.2%) delivery models.
| Sample | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 |
|---|---|---|---|---|---|---|---|---|
| Delivery model | Group | Group | Group | Group | One-to-one | One-to-one | One-to-one | One-to-one |
| Overall fidelity of BCT delivery (%) | ||||||||
| Yes | 67.4 | 69.9 | 50 | 68.8 | 60.4 | 53.2 | 33.3 | 37.8 |
| Partial | 17.5 | 8.6 | 11.2 | 8.7 | 12.1 | 12 | 6.7 | 17.8 |
| No | 15.1 | 21.5 | 38.8 | 22.5 | 27.5 | 34.7 | 60 | 44.4 |
The fidelity of individual BCTs varied considerably across the eight samples both between and within providers. For example, delivery of goal-setting (outcome), self-monitoring (behaviour and outcomes), graded tasks and problem-solving ranged from 0% to 100% (Table 3). Fidelity also varied by delivery model, for example, information about health consequences was delivered with higher fidelity in group-based samples (71.9%–93.1%) compared to one-to-one (0%–56.1%). The most consistently delivered BCT across all samples was instruction on how to perform the behaviour (66.7%–97.4%) and feedback on outcomes of behaviour (75%–100%), although it is important to note that feedback on outcomes was replaced by the provision of self-monitoring devices during COVID-19, which might explain this high fidelity.
| BCT (BCTTv1) | Provider A Group 1 | Provider A Group 2 | Provider B Group 1 | Provider B Group 2 | Provider B One-to-one 1 | Provider B One-to-one 2 | Provider B One-to-one 3 | Provider B One-to-one 4 | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Y (%) | P (%) | N (%) | Y (%) | P (%) | N (%) | Y (%) | P (%) | N (%) | Y (%) | P (%) | N (%) | Y (%) | P (%) | N (%) | Y (%) | P (%) | N (%) | Y (%) | P (%) | N (%) | Y (%) | P (%) | N (%) | |
| Goal setting (behaviour) (1.1) | 42.9 | 50 | 7.1 | 14.3 | 28.6 | 57.1 | 0 | 0 | 100 | 0 | 33.3 | 66.7 | 0 | 50 | 50 | 100 | 0 | 0 | ||||||
| Problem-solving (1.2) | 48.4 | 45.2 | 6.5 | 71 | 19.4 | 9.7 | 0 | 57.1 | 42.9 | 71.4 | 14.3 | 14.3 | 60 | 20 | 20 | 42.9 | 57.1 | 0 | 100 | 0 | 0 | 100 | 0 | 0 |
| Goal setting (outcome) (1.3) | 100 | 0 | 0 | 100 | 0 | 0 | 0 | 11.1 | 88.9 | 0 | 29.5 | 70.6 | 70 | 20 | 10 | 5.9 | 35.3 | 58.8 | 50 | 0 | 50 | 0 | 100 | 0 |
| Action planning (1.4) | 42.9 | 32.1 | 25 | 50 | 21.4 | 28.6 | 4 | 4 | 92 | 4.2 | 16.7 | 79.2 | 61.5 | 7.7 | 30.8 | 0 | 18.2 | 81.8 | 0 | 0 | 100 | 25 | 50 | 25 |
| Review behavioural goals (1.5) | 100 | 0 | 0 | 100 | 0 | 0 | 50 | 0 | 50 | |||||||||||||||
| Feedback on behaviour (2.2) | 55.6 | 44.4 | 0 | 88.9 | 0 | 11.1 | ||||||||||||||||||
| Self-monitoring of behaviour (2.3) | 48.1 | 14.8 | 37 | 66.7 | 11.1 | 22.2 | 100 | 0 | 0 | 100 | 0 | 0 | 0 | 100 | 0 | |||||||||
| Self-monitoring of outcome(s) of behaviour (2.4) | 66.7 | 0 | 33.3 | 100 | 0 | 0 | 0 | 0 | 100 | 100 | 0 | 0 | 0 | 0 | 100 | 0 | 0 | 100 | ||||||
| Feedback on outcomes of behaviour (2.7) | 80 | 15 | 5 | 75 | 20 | 5 | 94.7 | 0 | 5.3 | 100 | 0 | 0 | 100 | 0 | 0 | 94.1 | 0 | 5.9 | 100 | 0 | 0 | 100 | 0 | 0 |
| Social support (unspecified) (3.1) | 66.7 | 33.3 | 0 | 77.8 | 0 | 22.2 | 25 | 0 | 75 | 100 | 0 | 0 | 0 | 50 | 50 | 25 | 0 | 75 | 100 | 0 | 0 | |||
| Social support (practical (3.2) | 0 | 100 | 0 | 0 | 0 | 100 | 0 | 100 | 0 | 0 | 0 | 100 | 0 | 0 | 100 | |||||||||
| Instruction on how to perform the behaviour (4.1) | 97.4 | 2.6 | 0 | 92.1 | 5.3 | 2.6 | 87.5 | 0 | 12.5 | 94.4 | 0 | 5.6 | 78.6 | 0 | 21.4 | 91.4 | 0 | 8.6 | 100 | 0 | 0 | 66.7 | 0 | 33.3 |
| Information about antecedents (4.2) | 87.5 | 0 | 12.5 | 87.5 | 0 | 12.5 | 37.5 | 37.5 | 25 | 87.5 | 0 | 12.5 | 75 | 25 | 0 | 100 | 0 | 0 | 100 | 0 | 0 | |||
| Information about health consequences (5.1) | 77.2 | 5.3 | 17.5 | 71.9 | 0 | 28.1 | 87 | 4.3 | 8.7 | 93.1 | 6.9 | 0 | 28.6 | 7.1 | 64.3 | 56.1 | 3.5 | 40.4 | 0 | 20 | 80 | 16.7 | 0 | 83.3 |
| Information about social & environmental consequences (5.3) | 0 | 0 | 100 | 0 | 0 | 100 | 0 | 0 | 100 | 100 | 0 | 0 | 0 | 0 | 100 | 0 | 0 | 100% | ||||||
| Information about emotional consequences (5.6) | 100 | 0 | 0 | 100 | 0 | 0 | 33.3 | 0 | 66.7 | 100 | 0 | 0 | 100 | 0 | 0 | 0 | 0 | 100 | ||||||
| Demonstration of the behaviour (6.1) | 0 | 0 | 100 | 0 | 100 | 0 | ||||||||||||||||||
| Social comparison (6.2) | 20 | 30 | 50 | 40 | 20 | 40 | ||||||||||||||||||
| Prompts/cues (7.1) | 50 | 0 | 50 | 0 | 0 | 100 | ||||||||||||||||||
| Behavioural practice/rehearsal (8.1) | 66.7 | 33.3 | 0 | 66.7 | 0 | 33.3 | 0 | 0 | 100 | 0 | 0 | 100 | 0 | 0 | 100 | |||||||||
| Behaviour substitution (8.2) | 85.7 | 14.3 | 0 | 100 | 0 | 0 | 0 | 100 | 0 | 100 | 0 | 0 | 100 | 0 | 0 | 100 | 0 | 0 | ||||||
| Graded tasks (8.7) | 100 | 0 | 0 | 50 | 0 | 50 | 50 | 0 | 50 | 100 | 0 | 0 | 0 | 0 | 100 | 0 | 0 | 100 | ||||||
| Credible source (9.1) | 100 | 0 | 0 | 50 | 0 | 50 | ||||||||||||||||||
| Pros & cons (9.2) | 100 | 0 | 0 | 0 | 0 | 100 | ||||||||||||||||||
| Social reward (10.4) | 100 | 0 | 0 | 70 | 20 | 10 | 0 | 0 | 100 | 0 | 0 | 100 | 50 | 0 | 50 | 50 | 50 | 0 | ||||||
| Social incentive (10.5) | 50 | 0 | 50 | 50 | 0 | 50 | ||||||||||||||||||
| Self-incentive (10.7) | 33.3 | 33.3 | 33.3 | 33.3 | 33.3 | 33.3 | 0 | 0 | 100 | 0 | 66.7 | 33.3 | 0 | 0 | 100 | |||||||||
| Self-reward (10.9) | 100 | 0 | 0 | 25 | 0 | 75 | 0 | 100 | 0 | 0 | 0 | 100 | 100 | 0 | 0 | |||||||||
| Reduce negative emotions (11.2) | 66.7 | 33.3 | 0 | 66.7 | 0 | 33.3 | 33.3 | 33.3 | 33.3 | 100 | 0 | 0 | 100 | 0 | 0 | 50 | 50 | 0 | ||||||
| Restructuring the physical environment (12.1) | 75 | 25 | 0 | 75 | 0 | 25 | 50 | 0 | 50 | 50 | 50 | 0 | 100 | 0 | 0 | 50 | 0 | 50 | ||||||
| Restructuring the social environment (12.2) | 100 | 0 | 0 | 50 | 50 | 0 | ||||||||||||||||||
| Avoidance/reducing exposure to cues for the behaviour (12.3) | 100 | 0 | 0 | 100 | 0 | 0 | ||||||||||||||||||
| Distraction (12.4) | 50 | 50 | 0 | 0 | 0 | 100 | ||||||||||||||||||
| Adding objects to the environment (12.5) | 100 | 0 | 0 | 100 | 0 | 0 | ||||||||||||||||||
| Framing/reframing (13.2) | 11.1 | 33.3 | 55.6 | 33.3 | 11.1 | 55.6 | 100 | 0 | 0 | 100 | 0 | 0 | 100 | 0 | 0 | |||||||||
| Verbal persuasion about capability (15.1) | 50 | 50 | 0 | 50 | 50 | 0 | 100 | 0 | 0 | |||||||||||||||
| Mental rehearsal of successful performance (15.2) | 0 | 0 | 100 | 0 | 0 | 100 | 66.7 | 0 | 33.3 | 33.3 | 33.3 | 33.3 | 0 | 0 | 100 | 33.3 | 0 | 66.7 | 100 | 0% | 0 | |||
| Focus on past success (15.3) | 71.4 | 28.6 | 0 | 85.7 | 14.3 | 0 | 10 | 35 | 55 | 75 | 10 | 15 | 44.4 | 44.4 | 11.1 | 66.7 | 16.7 | 16.7 | 0 | 0 | 100 | 0 | 75 | 25 |
| Self-talk (15.4) | 0 | 50 | 50 | 100 | 0 | 0 | 0 | 100 | 0 | 0 | 0 | 100 | 100 | 0 | 0 | |||||||||
| Imaginary reward (16.2) | 0 | 0 | 100 | 0 | 0 | 100 | 0 | 0 | 100 | 0 | 0% | 100 | ||||||||||||
In contrast, the most consistently underdelivered BCTs were social support practical, demonstration of the behaviour, imaginary reward (0%), self-incentive (0%–33.3%), action planning and prompts/cues (0%–50%). Delivery staff were also a source of variability, for example, information about antecedents was delivered well by most coaches (75%–100%) with the exception of one group-based sample (37.5%). BCTs that were frequently partially delivered included problem-solving, reduce negative emotions and focus on past success.
3.1 Observed barriers and facilitators to BCT delivery
The relationships between BCTs and factors influencing their fidelity fell into the following five categories:
3.1.1 Alignment with the programme's target behaviours and outcomes
The lack of a clear link between the BCT and the programme's target behaviours or outcomes frequently coincided with the under-delivery of many BCTs. The BCTTv18 states that for a BCT to be present, it must be used to change the target behaviour or outcome. In the context of NHS-LCD, programme goals were divided into time-based phases: the first aimed to support TDR adherence to achieve weight loss and improved HbA1c; the latter phases (Food Reintroduction and Weight Maintenance) targeted healthy eating, physical activity and reduced sedentary behaviours to support weight maintenance. This was frequently observed as a barrier to the delivery of BCTs (e.g., problem-solving) during TDR, for example, by a coach focusing on physical activity (which is not recommended during TDR). In other instances, BCTs were presented without an explanation of how participants can use them to manage their TDR adherence. This was common for BCTs related to managing internal and external antecedents such as behaviour substitution, restructuring the environment, and using rewards and social support. It is important to note that a lack of focus was sometimes driven by the participant. For example, one participant suggested setting a goal to quit smoking. In these instances, the coaches often agreed with participants rather than correcting the deployment of the BCT. In contrast, the provider achieving 18% greater fidelity was observed as maintaining a clear focus on the programme's goals when discussing and delivering their content, for example by linking BCTs to TDR adherence.
3.1.2 Session content
Overall, including BCTs within activities was associated with increased BCT fidelity, especially if the BCT required active participation. For example, by presenting a problem-solving flipchart activity to the group and asking them to identify barriers and solutions. The use of supporting materials within activities, such as worksheets/books, was also observed as beneficial for fidelity (see S2 for example field notes). This was particularly noticeable for the delivery of SMART goal setting, which includes the BCTs goal setting and action planning, as observers reported higher fidelity when participants were instructed to turn to a SMART goal setting activity in their workbooks. Another self-regulatory BCT observed as having higher fidelity when included in an activity, was feedback on behaviour. An example of this is a step count activity where each week participants' steps were calculated, with the aim of reaching a destination on a map. This activity encouraged participants to self-monitor their physical activity and compare their progress. At the same time, this prompted the coach to discuss and provide feedback on physical activity levels.
Alternatively, another provider primarily delivered content through a PowerPoint presentation. This was associated with high fidelity of education and information giving BCTs, such as information about health consequences and instruction on how to perform the behaviour. Delivery of pre-prepared slides also potentially facilitated progress reviews and focus on past success. However, each of these were also well-delivered by the other provider, suggesting slides are supportive of, but not necessary, for the fidelity of these BCTs. In contrast, using slides was associated with ‘partial’ or an under-delivery of self-regulatory BCTs. For instance, problem-solving was often partially delivered as barriers were discussed without the participant engaging in solution generation. Despite coaches presenting a slide on SMART goal setting and showcasing a good theoretical understanding of SMART, a specific, relevant goal or subsequent action plan was rarely set. The higher fidelity of these BCTs suggests that activities supported by materials (e.g., a flipchart or worksheet/book) may facilitate delivery.
3.1.3 Time availability and management
Although activities that apply BCTs were related to increased fidelity, this was only the case when sufficient time was allocated to completion. For example, observers sometimes noted that progress on the step count progress activity (described above), was “briefly mentioned” by the coach, resulting in only partial delivery of feedback on behaviour and an under-delivery of social comparison (in a group context). Similarly, the inclusion of SMART goal-setting activities in participants' workbooks supported delivery only when sufficient time was dedicated to completing it; quick descriptions by the coach, or assigning completion as a ‘homework’ task, were linked to poorer fidelity. With only a couple of minutes left in their session, one provider tended to deliver goal-setting without action planning.
Poor time management was coded as a barrier across both providers but most frequently in group-based programmes. This was manifested in skipping content on information giving and placing activities towards the end of the session (e.g., SMART goal-setting and problem-solving). BCTs such as pros and cons, graded tasks, rewards and social support (practical) were often omitted. In contrast, substantial amounts of time were often given to information-giving BCTs, such as information on health consequences and instruction on how to perform the behaviour, impacting the time available for delivery of subsequent BCTs. This was coded most frequently for group-based programmes and importantly, was noted by observers as an over-allocation of time. For one provider, session time management was frequently observed as limiting the time available for goal-setting and action planning at the end of the session. Even when sufficient time was available, these BCTs were still not fully delivered. This suggests that the skill level of the coach in delivering these BCTs was also a contributing factor.
Running out of time was identified as a frequent barrier across both providers. For one-to-one programmes, this was often impacted by the participant, such as joining the session late, rather than due to poor coach time management. It was evident in the data that in one-to-one sessions, slides were more likely to be skipped and thus BCTs were coded as not delivered. Skipping slides most significantly impacted information-giving BCTs. This pattern did not occur in the same way in the group-based programmes. Furthermore, for the provider who predominately delivered BCTs through activities, running out of time resulted in activities towards the end of the session being skipped. Overall, running out of time most significantly impacted BCTs goal-setting, action planning and problem-solving, in addition to self-monitoring of behaviour and outcomes, information and instruction, and those BCTs that were less frequently present within programmes, such as graded tasks and mental rehearsal of successful performance.
3.1.4 Group-based remote delivery as a barrier and facilitator
Aspects of group-based delivery appeared to both facilitate and hinder BCT delivery, depending on circumstances. Activities involving group discussion and idea generation tended to facilitate the delivery of BCTs targeting self-regulation (e.g., action planning and problem-solving), management of internal and external antecedents of behaviour (e.g., reduce negative emotions and behaviour substitution), and self-belief (e.g., focus on past success). However, the group-based setting sometimes hindered individual engagement with a BCT. Furthermore, the level of group engagement also hindered BCT delivery; as the programme was delivered remotely via videoconferencing software due to COVID-19, many participants did not use their cameras, microphones or chat functions, resulting in the incompletion of the activity.
To facilitate group engagement, one of the providers utilised the “breakout rooms” function on the videoconferencing software. This function divided participants (and observers) into smaller discussion groups at random for a specified time before returning everyone to the “main room”. Breakout rooms were related to the successful delivery of activities focused on managing internal antecedents of behaviour, including information about antecedents, problem-solving, reduce negative emotions and social support. In contrast, breakout rooms were sometimes identified as a barrier to problem-solving. This variance was observed as being due to the level of engagement within each smaller group, for example, some participants would have “off-topic” discussions. Importantly, it was noted that activity completion was higher when the coach checked into each breakout room. Group discussions and idea generation when returning to the main room also appeared to resolve this barrier, although these discussions did not systematically take place.
Finally, remote delivery impacted the demonstration of the behaviour and behavioural practice/rehearsal in relation to physical activity. Providers replaced these activities with a video demonstration; however, coaches signposted participants to the video link as opposed to playing the video in-session.
3.1.5 Deviation from the session plan
In one-to-one sessions, deviation from the session plan appeared to be participant-led. For example, if a participant expressed challenges with dietary adherence due to life events such as bereavement or if they were already successfully using a BCT. In these instances, the coaches tailored the session to these needs, whilst weakening BCT fidelity. However, these were rare instances and deviation from the session plan was more frequently observed in group-based programmes. For one provider, coaches tended to miss a framing/reframing activity, where participants were instructed to view themselves as athletes aiming for a gold, silver or bronze medal. It was not apparent to observers the specific reason why this content was missed.
Furthermore, one coach delivering a group-based programme was less positive about setting outcome goals and told the group that they preferred action goals, without defining the meaning of this (see S2 for example field notes). This occurred on more than one occasion and was associated with an under-delivery of outcome goals. The coach further deviated from the session plan by delivering alternative content that was not mandated. In such instances, a significant amount of time was lost, leaving less time to deliver subsequent BCTs. Goal-setting (behaviour or outcome), action planning and problem-solving were primarily impacted, whilst focus on past success, pros and cons, instruction on how to perform the behaviour and graded tasks were also impacted due to insufficient time for their delivery.
4 DISCUSSION
Overall, the degree of fidelity in the delivery of BCTs across two providers commissioned to deliver face-to-face NHS-LCD programmes across England was 55% (range = 33%–70%). Thus, indicating a dilution in fidelity during the delivery phase of the NHS-LCD and variation in the fidelity of programmes being delivered across England. Whilst it would be unlikely that a programme be delivered with absolute fidelity, these results would be considered as low-to-moderate fidelity.9 Greater fidelity coincided with alignment with the programme's target behaviours and outcomes, use of structured activities and supporting participant materials, availability and management of time, group engagement, and coaches' preferences. Although fidelity was greater for group-based programmes, this should be interpreted in light of the impact of late attendance on the delivery of one-to-one sessions.
Although our evaluation of providers' programme designs found 79.5% of BCTs mandated in the NHSE service specification17 and clinical guidelines24, 25 to be included in session plans,12 45% of these BCTs were lost in programme delivery. A similar evaluation of the NHS-DPP also found weakened fidelity during the delivery phase as although providers planned to deliver 74% of BCTs in the NHS-DPP service specification, only 37% were delivered across all eight observation sites.18 Together this evidence demonstrates the difficulty of translating the design of nationally commissioned NHSE programmes into consistently delivered programmes.
Our previous study identified information-giving BCTs (e.g., instruction on how to perform the behaviour) as being the most frequently included BCTs across the two providers' session plans.12 The present study additionally reports these BCTs as being delivered with higher fidelity compared to self-regulatory BCTs (e.g., action planning). Similarly, an over-delivery of information about health consequences and an under-delivery of goal-setting and action planning was reported for the NHS-DPP.18, 26 This suggests that despite consensus on information and knowledge being insufficient for behaviour change, there is an emphasis on these BCTs in the design and delivery of NHS T2DM prevention and management programmes. In addition, perhaps a core belief about the role of education in determining behaviour within health promotion programmes.27 Furthermore, the increased delivery of information-giving BCTs may reflect their one-directional nature and thus ease of delivery, as opposed to the active participation and discussion required for capacity-building BCTs. BCTs of a higher difficulty may therefore require further training to ensure staff are skilled in their delivery. Likewise, observations of the NHS-DPP staff training found that although deliverers were informed about the BCTs included in programmes, they were not always shown how to deliver these BCTs or given opportunities to practice delivery.28 This may explain the similarities in the under-delivery of self-regulatory BCTs (which require a higher skill level) across these two NHSE-commissioned programmes. The dilution in fidelity may also reflect the limited health psychology expertise identified across the providers' programme development teams.12 Training and ongoing supervision should be provided by a Health Psychologist (the only psychologists who are specifically trained in BCTs) with an expectation that delivery staff meet a standard of proficiency in BCT delivery. NHSE required providers to evidence behavioural science expertise in their programme development teams during national NHS-LCD procurement, that took place following the pilots, which might improve the fidelity of future programmes.
The findings reported in this study highlight further programme-level features that might facilitate or hinder the delivery of self-regulatory BCTs. For instance, higher fidelity was observed when these BCTs featured within structured activities, were accompanied by participant materials such as worksheets, and had sufficient time for activity completion. Furthermore, we found evidence of coach autonomy influencing the under-delivery of outcome goals, whereby the coach expressed a disliking for a specific BCT, despite the NHSE service specification underlining their importance.17 The under-delivery of self-regulatory BCTs must be addressed, considering the evidence for their effectiveness in influencing behaviour and motivation in people with obesity,29 overcoming barriers to behaviour change,30 reducing HbA1c and BMI in people with T2DM,31,32 and their emphasis within the programme's underpinning clinical guidelines.24, 25 An upcoming meta-analysis also found the presence of action planning and problem-solving in low-calorie diet interventions to be independently associated with weight reduction.33 Furthermore, the Norfolk Diabetes Prevention Study found the greater the number of action plans set across the course of the programme, the greater the subsequent weight loss.34 This provides important information about BCT dose, suggesting that each failure to deliver action planning might be associated with poorer weight outcomes. Understanding of dose responses may also be included in subsequent staff training.
The influence of both coach-level (e.g., skills) and programme-level (e.g., content) factors on fidelity supports evidence for health professionals' implementation of complex interventions to be influenced by several factors.35 Normalisation process theory postulates that the gap between evidence and practice results from interactions between organisational contexts, individuals and groups tasked with implementation, and the associated procedures and materials.36 Behavioural science theories assert that for behaviour change to occur, individuals require intention, skills for action and environments that prompt behavioural changes.25,37 This applies to health professionals delivering programmes and should be considered in programme design and staff training. For instance, training could facilitate positive outcome expectancies and thus intentions to use the BCTs, by informing staff about their evidence base for behaviour change. Self-efficacy could be promoted by providing opportunities to practice and receive feedback on BCT delivery. And finally, environments could support delivery by including BCTs within structured activities and session materials (e.g., participant worksheets), and with appropriate time available for activity completion.
4.1 Strengths and limitations
To our knowledge, this is the first study to evaluate the delivery of low-calorie diet programmes, and the first evaluation of a health promotion programme that uses mixed-methods to link specific BCTs to observed barriers and facilitators to their delivery. Through this, we have been able to provide actionable recommendations for ongoing service improvement. Another study employing mixed-methods, qualitatively assessed factors influencing the fidelity of a back pain self-management through interviews with programme deliverers, finding that intrapersonal factors, such as knowledge, influenced fidelity intervention.38 Our observational methods were limited in their ability to identify intrapersonal factors. However, we were able to identify numerous programme-level factors, whilst also having the unique ability to link these factors to the delivery of specific BCTs. Our approach, therefore, highlights the role of both the coach and the provider organisation in influencing fidelity.
Another strength of this study is that the evaluation was conducted by a team of researchers independent of programme development. This minimises the impact of confirmation bias on the study results. However, a limitation of this study is that BCT coding was not conducted in duplicate. Nevertheless, the researcher was coding against a checklist of BCTs developed through our group's assessment of programme designs, where two researchers independently coded providers' session plans, resolved all discrepancies, and reported good inter-rater reliability.12
Participant withdrawal also impacted the completeness of session observation data. Although attempts were made to rectify this, due to the inevitabilities of recruiting replacements, sessions 11–13 within the one-to-one model were only observed in one sample. A further limitation is that as deliverers and participants were aware they were being observed, the observations may not have reflected the everyday realities of the health coaching and use of BCTs. To minimise the impact on session dynamics, researchers turned off their video cameras and microphones during sessions.
Finally, it is important to recognise the impact of COVID-19 and the subsequent remote delivery of session plans designed for face-to-face delivery. Barriers such as remote delivery and group engagement during video conferencing may not have occurred in the same way if programmes were designed for the purpose of remote delivery. As the national roll-out of the NHS-LCD will be transitioning to in-person face-to-face delivery, this may facilitate an improvement in the delivery of some BCTs.
4.2 Recommendations
- Providers should ensure staff training promotes positive attitudes towards included BCTs and provides opportunities to practice and receive feedback on the delivery of BCTs and time management skills.
- Providers should design session plans and content that prompt and support the delivery of BCTs requiring the active involvement of programme participants (such as self-regulatory BCTs). For example, by including BCTs within structured activities, supported by materials such as participant worksheets, and with sufficient time allocated for activity completion.
- Session content should be reduced for group-based programmes. This will optimise the value of the group-based context.
- Providers should consider ways to improve the fidelity of less frequent BCTs (i.e., through staff training or including them in structured activities).
- Setting expectations and explaining the importance of timely attendance to participants during session one might minimise the impact of late attendance on the time available for the coach to deliver session content.
- A Health Psychologist should be included within programme development teams to design behaviour change content, develop and deliver staff training in BCTs, and monitor delivery. Programme commissioners should require evidence of quality assurance methods from programme providers.
- Future research should evaluate the implementation of BCTs against clinical outcomes to elucidate the impact of fidelity on programme success.
5 CONCLUSION
The findings of this study indicate considerable dilution in fidelity during the delivery of the NHS-LCD and variation in the fidelity of programmes delivered across England. Considering intervention fidelity is known to impact outcomes,9 this highlights an important area for improvement in programme delivery to optimise intervention effects. To improve the delivery of BCTs, a combination of staff training and structural changes to session plans and content are needed to ensure staff have the self-efficacy and supportive environments needed to deliver BCTs effectively.
AUTHOR CONTRIBUTIONS
Tamla S. Evans led the design, data collection, analysis and write-up of this study. All authors made a significant contribution to the study and approved the final manuscript. Louisa J. Ells oversaw the study as Principal Investigator.
ACKNOWLEDGEMENTS
The Re:Mission study included a multidisciplinary team of academics from across the North of England. The authors would like to acknowledge all members of the team including: Dr Tamara Brown, Dr Susan Jones, Dr Pat Watson, Prof Maria Maynard, Karina Kinsella, Dr Simon Rowlands, Dr Tanefa Apekey, Dr Stuart Flint, Prof Janet Cade, Dr Samuel Fempong, Dr Adam Martin, Dr Maria Bryant, Dr Wendy Burton, Dr Chris Keyworth, Mick Martson and Dr Jennifer Logue. They would also like to acknowledge all members of ReMission Public and Patient Involvement group and the steering and oversight groups who were involved in the Re:Mission study.
FUNDING INFORMATION
This project [NIHR132075] is funded by the NIHR Health Service and Delivery Research Programme. The views expressed in this publication are those of the author(s) and not necessarily those of the MRC, NIHR or the Department of Health and Social Care.
CONFLICT OF INTEREST STATEMENT
All other authors confirm that they have no conflicts of interest to declare.
