A core outcome set for trials evaluating self-management interventions in people with severe mental illness and coexisting type 2 diabetes
Abstract
Background
People with severe mental illness (SMI), such as schizophrenia, have higher rates of type 2 diabetes and worse outcomes, compared to those without SMI and it is not known whether diabetes self-management interventions are effective for people who have both conditions. Research in this area has been impeded by a lack of consensus on which outcomes to prioritise in people with co-existing SMI and diabetes.
Aims
To develop a core outcome set (COS) for use in effectiveness trials of diabetes self-management interventions in adults with both type 2 diabetes and SMI.
Methods
The COS was developed in three stages: (i) identification of outcomes from systematic literature review of intervention studies, followed by multi-stakeholder and service user workshops; (ii) rating of outcomes in a two-round online Delphi survey; (iii) agreement of final ‘core’ outcomes through a stakeholder consensus workshop.
Results
Seven outcomes were selected: glucose control, blood pressure, body composition (body weight, BMI, body fat), health-related quality of life, diabetes self-management, diabetes-related distress and medication adherence.
Conclusions
This COS is recommended for future trials of effectiveness of diabetes self-management interventions for people with SMI and type 2 diabetes. Its use will ensure trials capture important outcomes and reduce heterogeneity so findings can be readily synthesised to inform practice and policy.
What's new?
What is already known
- People with severe mental illness are more likely to have type 2 diabetes. We need high-quality trials of self-management interventions in this population and a consensus on what outcomes should be prioritised.
What has this study found
- This study developed a core outcome set for self-management interventions for people with co-existing type 2 diabetes and severe mental illness.
What are the implications of the study
- This core outcome set will enable consistent reporting in future trials and help develop a robust evidence base for self-management interventions for those with co-existing type 2 diabetes and severe mental illness.
1 BACKGROUND
Severe mental illness (SMI; i.e. conditions such as schizophrenia and bipolar disorder)1 is associated with a higher risk of developing physical long-term conditions, such as type 2 diabetes.2 People with SMI are 2–3 times more likely to develop type 2 diabetes and experience poorer physical health outcomes compared with the general population.2 For example, people with SMI die on average 15–20 years earlier than those without SMI, mainly because of co-existing physical long-term conditions.3
Good outcomes for type 2 diabetes depend on effective self-management.4 The National Institute for Health and Care Excellence (NICE) recommends structured self-management education to support diabetes self-management,5 as they are effective at improving diabetes outcomes in the general population.6 However, the effectiveness of these programmes for people with SMI remains largely unknown.7
To establish the effectiveness of diabetes self-management programmes for people with SMI randomised controlled trials (RCTs) and systematic reviews of RCTs are needed.8 While there is limited research evaluating self-management interventions specifically for people with SMI and co-existing type 2 diabetes,7 previous research on generic self-management interventions has included those with both conditions.9 Systematic reviews have found variation in outcome selection, and incomplete reporting of outcomes, due in part to the lack of consensus about what outcomes are most important to those with type 2 diabetes and SMI, making it difficult to synthesise the results.10
Core outcome sets (COS) are an agreed standardised set of outcomes that should be used in all trials addressing a specific question.11 They provide a method of reducing outcome reporting bias in trials and ensure the clinical outcomes being reported are ones that are of greatest importance to all involved parties.12 They also reduce the heterogeneity of outcomes in trials, so findings can be more readily synthesised to inform policy and practice. Several COSs have already been developed for specific conditions and interventions, including for people with SMI.13 However, there is no COS for diabetes self-management interventions targeting people who have co-existing SMI and type 2 diabetes. People with SMI and co-existing physical long-term conditions experience unique issues and barriers to accessing healthcare services, and also have different priorities to those without SMI, due to the difficulties of managing co-existing symptoms and treatment burdens, therefore it is important to develop a specific COS for people who have co-existing SMI and type 2 diabetes, that accounts for these unique experiences and priorities.14, 15
The ‘Diabetes and Mental Illness—Improving Outcomes and Services (DIAMONDS)’ programme is a research programme that aims to develop and evaluate a tailored self-management intervention for people who have type 2 diabetes and SMI.16, 17 A necessary first step in this programme is to develop a COS for diabetes self-management interventions that are tailored specifically to people who have co-existing SMI. This will ensure the outcomes included in evaluation trials are of importance to those most affected, and the evidence can be synthesised to inform future policy.
2 AIMS AND OBJECTIVES
- Identify a list of potential outcomes from a systematic review of the literature (to identify those used in previous studies) and a multi-stakeholder workshop and service user panel meeting.10
- Rate identified outcomes for priority to include in the COS through a two-round Delphi survey with interested parties.
- Agree on which outcomes are included in the COS through a consensus workshop.
3 METHODS
We followed best practice methods as described in the COMET Initiative.18, 19 We reviewed methods employed in studies identifying health outcomes,20 to select appropriate consensus methods and ensure meaningful input from service users, carers and healthcare professionals.
We have published the study protocol21 and the study was registered on the COMET initiative website on 1 July 2016.22 Ethical approval was obtained from the Research Ethics Committee East Midlands—Leicester Central (reference: 16/EM/0149). Informed consent was obtained from all participants in the study.
Developing the COS included three steps:
3.1 Step 1—identifying outcomes
3.1.1 Review of existing evidence
We identified systematic reviews of trials and experimental studies involving people with SMI and physical long-term conditions. We used these to identify relevant studies with sample sizes over 20 participants,10 to ensure outcomes were practicable to measure in large trials.21 We extracted information about the outcomes used in these studies. We also incorporated outcomes from the AADE DSME Outcomes Continuum23 and from a systematic review of self-management outcomes which identified outcomes that are important to affected groups.24
3.1.2 Multi-stakeholder workshop and service user panel
We held a workshop attended by 19 stakeholders, including research team members and healthcare professionals who were identified through the research team's networks. Table 1 provides an overview of the attendee's professional background. Some participants had multiple professional roles.
| Professional role |
|---|
|
Attendees worked in three groups to identify relevant outcomes. Following this discussion, findings from the review were presented and the groups were asked to create a list of potential outcomes and remove any outcomes identified by the review that overlapped with each other.
This list was then presented to DIAMONDS Voice, our service user and carer panel consisting of between 6 and 8 members during the period of the study. The panel was invited to review the list and add relevant outcomes that had not already been identified, which was followed by telephone discussions with members who were not in attendance at the meeting. In total 4 service users and two carers reviewed the initial list through DIAMONDS Voice.
3.2 Step 2—rating outcomes
The aim of step 2 was to rate outcomes according to percentage agreement on the importance of each outcome, for inclusion in the COS through a two-round Delphi survey with all relevant stakeholders. Participants completed a survey which included the outcomes identified in step 1. The purpose of this survey was to reduce the list based on collated responses, considering different views between groups and allowing changes in opinion to enable the development of a group consensus.
3.2.1 Participants
- Adults with SMI and co-existing type 2 diabetes living in the community. We defined SMI as conditions which can present with psychosis, for example., schizophrenia and bipolar disorder.
- Carers of people who have SMI and co-existing type 2 diabetes. Carers were defined as anyone who provides unpaid support to the person, including family or friends.
- Health and social care staff from a range of professional backgrounds who provide care to people with coexisting SMI and type 2 diabetes.
- Health service managers and commissioners with responsibility for SM and/or type 2 diabetes
- Academic experts in relevant areas, such as diabetes, SMI, primary care and methodologists.
The DIAMONDS Voice Panel was also invited to take part in the Delphi survey.
3.2.2 Recruitment
Service users and carers were recruited through NHS community mental health teams using care coordinators. Other participants were recruited through NHS Trusts and third-sector organisations. Table 2 provides an overview of respondents.
| Stakeholder group | Number of respondents round 1 (n) | Number of respondents round 2 |
|---|---|---|
| Service user | 23 | 10 |
| Carer | 4 | 3 |
| Health and social care staff | ||
| Mental healthcare staff | 24 | 13 |
| Physical healthcare staff | 17 | 11 |
| Health service managers and commissioners | 7 | 2 |
| Academic experts | 9 | 9 |
| Total | 84 | 48 |
3.2.3 Data collection: Delphi survey round 1
The survey was administered online using Qualtrics, on paper or by telephone with a researcher. There was a four-week period for participants to complete the survey. While most healthcare staff and academics completed the survey online, most service users and carers used postal questionnaires and several service users accessed face-to-face or telephone support. The survey presentation was informed by DIAMONDS Voice, who piloted the survey prior to data collection.
During the first round, all potential outcomes that were agreed in step 1 were presented, and respondents were asked to rate the importance of each outcome on a scale from 1 to 9, independent of other outcomes, ranging from 1 being not important, to 9 being of critical importance. The survey also included free text boxes where participants could provide comments about their responses and suggest additional outcomes that were not included in the original survey. To reduce the risk of attrition we informed all participants that there would be two rounds of the survey before data collection.
3.2.4 Data analysis: Delphi survey round 1
Data were analysed by group (participants were grouped into academic experts, health services managers, mental health staff, physical health staff and service users and carers) and for all participants together. The number of respondents and distribution of scores were summarised and analysed. The proportion of participants scoring 1–3, 4–6 and 7–9 for each outcome was calculated.
Written responses by participants were analysed to assess how the survey should be modified for round 2. The main feedback was that the list was too long, and outcomes were often similar, making it difficult to differentiate their importance. Therefore, the research team merged certain outcomes which were able to be grouped into a single larger outcome.
3.2.5 Data collection: Delphi survey round 2
- Rating of outcomes across all participants.
- Rating of outcomes for their own group.
- Their individual scores from round one.
Participants were asked again to rate these outcomes, using the same 1–9 scale. This allowed participants to adjust their scores and enabled comparison between the rounds.
3.2.6 Data analysis: Delphi survey round 2
- Consensus ‘IN’, outcomes that should be included in the COS: >70% of respondents score the outcome as 7–9 (highest importance), and < 25% score the outcome as 1–3 (least importance).
- Consensus ‘out’, outcomes that should not be included in the COS: >70% of respondents score the outcome as 1–3 (least importance) and < 15% score the outcome as 7–9 (highest importance)
- No consensus, outcomes for which there is no consensus on their importance: Any other distribution of the scores.
We descriptively analysed the consistency of these categorisations within and across groups to explore the relevance of outcomes across all respondents, and to ensure the voices of minority parties were not overruled by the majority. However, no weighting was applied to the different stakeholder groups during the analysis.
3.3 Step 3—consensus workshop
We organised a Delphi consensus workshop that was attended by 10 participants. The purpose of the consensus workshop was to agree on a final set of outcomes. The attendees included a social epidemiologist, a clinical psychiatrist, two health services researchers, an academic diabetologist, a GP, a carer, a service user, a mental health professional and a lecturer in health psychology. Four additional clinical experts (1 diabetologist, 1 GP, 2 psychiatrists) provided virtual input in advance.
A summary of the results from rounds one and two of the survey was provided to participants in advance of the workshop. The results were also presented at the workshop, with an emphasis on the top 15 rated outcomes as there were no outcomes that met consensus for exclusion. Participants were guided by informing them of the following considerations: the purpose of the COS, the burden associated with the measurement of outcomes, the number of outcomes included in published COSs and whether the outcomes were measurable and feasible. The workshop participants were then asked to select two outcomes they considered the most important. This, along with the Delphi results, informed decisions within the group.
4 RESULTS
4.1 Step 1—identifying outcomes
We extracted 48 outcomes from relevant trials and key studies, and a further 28 outcomes were identified in a multi-stakeholder workshop and a service user panel (DIAMONDS Voice) meeting. The DIAMONDS Voice members added two outcomes to the list, specifically sleep problems and being listened to by healthcare professionals. This resulted in 76 outcomes which were included in round 1 of the survey in step 2. The final list of 76 outcomes is available in Data S1, these fell into the outcome domains outlined in Table 3.
| Domain |
|---|
| SMI symptoms and complications |
| Glucose control |
| Diabetes complications |
| Quality of life |
| Health state |
| Functioning (how well you can do things) |
| Self-management |
| Diabetes monitoring and treatment |
| Service utilisation |
| Patient experience |
4.2 Step 2—rating outcomes
4.2.1 Delphi survey round 1 results
In total, 84 people participated in round one of the surveys. The top-rated outcomes for round one (for which >90% of participants scored the outcome 7–9) were medication adherence, self-management of diabetes, glucose control, mental health-related quality of life (HRQoL), prescribed medications, attendance at annual diabetes check-ups, self-management of SMI and physical HRQoL. A summary of the proportion of respondents scoring 7–9 for these top-rated outcomes can be seen below in Table 4, and an overview of the full results can be found in the Data S1.
| Outcome | Percentage of all participants scoring 7–9 |
|---|---|
| Medication adherence | 99% |
| Self-management of diabetes | 96% |
| Glucose control (HbA1c) | 95% |
| Mental health-related quality of life | 94% |
| Prescribed medications | 93% |
| Annual diabetes care review | 92% |
| Self-management of SMI | 91% |
| Physical health-related quality of life | 90% |
4.2.2 Delphi round two survey results
The list of outcomes was reduced to 42 for round two of the survey, to minimise participant burden and ensure each outcome measured a distinct concept to better enable participants to assess each outcome.
A total of 48 participants from round one provided data for round two, giving a response rate of 57%. No formal analysis of attrition bias was conducted. Fifty-nine outcomes reached consensus to be considered for the COS and 17 outcomes did not reach consensus. Differences between scores on outcomes across different groups were marginal, with participant groups agreeing on what outcomes were important. No outcomes reached a consensus for exclusion; therefore, all outcomes were carried forward to the next step.
The top-rated outcomes in round two were self-management of diabetes, glucose control, blood pressure, diabetes complications, annual diabetes care review, depression, SMI symptoms, episodes of high and low blood glucose, diabetes distress, cholesterol levels, self-efficacy, health literacy, anxiety and satisfaction with care.
A summary of the proportion of respondents scoring 7–9 for these top-rated outcomes can be seen below in Table 5. A summary of the full results can be found in Data S2.
| Outcome | Percentage of all participants scoring 7–9 |
|---|---|
| Self-management of diabetes | 100% |
| Glucose control (HbA1c) | 98% |
| Blood pressure | 98% |
| Diabetes complications | 98% |
| Annual diabetes care review | 98% |
| Hypers (high blood glucose) | 96% |
| Depression | 94% |
| SMI symptoms | 94% |
| Hypos (low blood glucose) | 94% |
| Diabetes distress | 91% |
| Cholesterol levels | 91% |
| Self-efficacy | 90% |
| Health literacy | 90% |
| Anxiety | 90% |
| Satisfaction with care | 90% |
4.3 Step 3—agree outcomes
The final consensus workshop was facilitated by JT. A short report of the results from step 2 was provided to all workshop participants in advance of the meeting so they could see how different outcomes were rated across stakeholder groups. Service users/carer invitees were also offered a telephone call with a researcher to talk through the report in advance of the workshop.
The introduction to the workshop included reminding participants about the purpose of the core outcome set and how it would be used and summarised what would happen in the workshop and the overall aim. The workshop began with small group informal discussions about the results from step 2 with an emphasis on top-rated outcomes from both rounds of the Delphi survey. Groups were also provided with copies of the report they received in advance of the workshop. All outcomes from stage 2 of the Delphi survey were discussed, as no outcomes met the consensus for exclusion. The top 15 rated outcomes from the survey were identified.
In the whole group discussion that followed, the top 15 outcomes were discussed in turn. Specific discussions were also held about outcomes that were rated higher by service users than other stakeholder groups but did not make the top 15, round 1 outcomes that were rated highly but had been merged together to ensure these were not missed and other outcomes participants wanted to discuss.
A visual display of all the outcomes that were discussed was placed at the front of the room and updated as the workshop progressed. To ensure all outcomes could be covered in the time available, an informal voting technique was used, with the first question asked of the whole group being ‘should this outcome be included?’, with participants asked to raise their hands. All participants were offered an opportunity to express their views, starting with service user/carer participants to ensure discussions were led from this position. Participants were also reminded of any differences in the overall rating of these outcomes between stakeholder groups.
For outcomes that lacked consensus through discussion and informal voting, participants were asked to consider these outcomes in relation to the purpose of the COS and whether they were aligned with this. This helped participants consider factors other than importance, including the aim of diabetes self-management interventions and the feasibility of measurement. As these were more technical discussions, the facilitator ensured that any terminology used was explained so service user/carer participants could continue to contribute. As the focus of self-management interventions is to improve diabetes outcomes, a decision was made during these discussions to remove outcomes focused exclusively on mental health. Diabetes distress and HRQoL were selected to represent the emotional burden of managing diabetes alongside SMI.
- Blood pressure
- Body composition (body weight, body mass index (BMI) and body fat)
- Diabetes distress
- Diabetes self-management
- Glucose control (HbA1c)
- HRQoL
- Medication adherence
5 DISCUSSION
We have established for the first time a COS for diabetes self-management interventions for people who have type 2 diabetes and co-existing SMI. The seven outcomes included in the COS were selected through best practice methods described in the COMET Initiative.18 The outcomes relate to important clinical measures for type 2 diabetes, such as blood pressure, body composition and glucose control; measures of self-management itself, including diabetes self-management and medication taking; as well as subjective experiences associated with living with diabetes, including diabetes distress and HRQoL. While mental health outcomes were consistently prioritised, the focus of this COS is to evaluate self-management interventions for type 2 diabetes. Therefore, in Step 3 we decided to exclude measures related to symptoms of mental illness, but to include distress related to diabetes and overall HRQoL to ensure that mental health was retained in the COS. The need to focus on a narrow range of outcomes highlights the potential shortcomings of disease-specific interventions being too narrowly focused to address the broad range of issues experienced by people living with co-existing SMI.
Similar to other COS studies,13, 25 none of the originally identified outcomes reached consensus for exclusion, therefore we refined our list based on outcomes with the highest consensus for inclusion using a modified Delphi process with expert stakeholders involved in the final workshop. The outcomes included in our COS overlap with those identified in other COSs for effectiveness trials in diabetes, for example, blood glucose and body composition.25 Our set also features outcomes included in COSs for SMI-focused interventions, such as medication adherence and quality of life.13 While the outcomes included in other COSs for diabetes, including the COS developed for use in practice with people with type 1 and type 2 diabetes (ICHOM Diabetes),26 and the COS developed for trials evaluating interventions for type 2 diabetes (SCORE-IT),25 are relevant to the people who have co-existing SMI, they do not completely overlap. For example, we did not include outcomes related to complications of diabetes.25 While complications of diabetes were highly rated in the Delphi survey, these were excluded during discussions in the consensus workshop as we considered the implementation of the COS within a trial context. We needed to ensure that the number of outcomes was manageable and would not be burdensome to measure and that they would be relevant across all trials evaluating self-management of people with type 2 diabetes and co-existing SMI, including trials focused on people who are newly diagnosed with type 2 diabetes. However, glucose control, blood pressure and body composition represent key clinical indicators of future diabetes complications.27 In contrast to SCORE-IT,25 we also included diabetes distress. This reflects the need to account for the relationship between mental and physical health in interventions for type 2 diabetes in people who have co-existing SMI.
The next step is to identify measures that can be administered consistently across trials. Some outcomes have standardised clinical measures that are already in widespread use, such as HbA1c for measuring blood glucose control, or standardised blood pressure measures. Other clinical measures, such as body composition, are associated with more controversy in relation to their measurement. Body Mass Index (BMI) and body weight are not direct measures of body composition,28 while waist circumference is a particularly important measure in type 2 diabetes.29 Clinical guidelines, therefore, recommend that BMI and waist circumference are used together.5
Other outcomes included in our COS do not have the same widely recognised standardised clinical measures. There are many measures of HRQoL, some are generic while others are disease-specific.30 A recent narrative review has recommended the use of a combination of disease-specific and generic measures to capture an overall picture of HRQoL. However, it is key that the measures capture physical- as well as mental-HRQoL, to represent what is important to service users. Diabetes self-management measures can capture different self-care activities and other aspects of adherence to self-management regimen.31 Measures for medication taking are plentiful,32 and any measure would need to account for the medication regimens of both type 2 diabetes and SMI. Additionally, diabetes distress can be captured using several different instruments.33 Future research is needed to identify specific measures that can adequately capture the experiences of people living with co-existing SMI and diabetes.
The strengths of this study include the involvement of service users consistently throughout the consensus exercises, including during the final decisions about which outcomes to include. This was facilitated through strong networks in the research team and a flexible approach to collecting data, including providing support to complete surveys face-to-face and over the phone. The high level of consensus and prioritisation of outcomes reflect the relevance of items identified during the first stage. While we aimed to recruit a diverse range of participants, they may not be representative of all interested parties and our findings may not be transferable outside the UK. The provision of diabetes care for people with SMI varies across countries, and we were only able to include people from the UK who spoke English and had the capacity to engage in the process. The Delphi process was not effective in highlighting the most important outcomes for inclusion in the COS, and future studies may need to consider other methods to use that allow for this, as other COS studies have experienced similar problems.
6 CONCLUSION
This study is the first to develop a COS for evaluations of type 2 diabetes self-management interventions for people with co-existing SMI. This COS can be used in future trials to ensure a coordinated approach that will enable the synthesis of results, as well as ensure the outcomes being captured are those prioritised by service users, healthcare professionals and policymakers.
ACKNOWLEDGEMENTS
This paper reports work undertaken as part of a National Institute for Health and Care Research Programme development grant (Award ID: RP-DG-1214-10002).
CONFLICT OF INTEREST STATEMENT
DS is an expert adviser to the National Institute for Health and Care Excellence Centre for Guidelines; the views expressed are those of the authors and not those of National Institute for Health and Care Excellence. RIGH has received honoraria for speaker engagement, conference attendance and advisory boards from Abbott, AstraZeneca, Boehringer Ingelheim, European Association for the Study of Diabetes, Eli Lilly, Encore, Janssen, Menarini, Napp Pharmaceuticals Limited, Novo Nordisk, Omniamed, Roche and Sanofi.
Open Research
DATA AVAILABILITY STATEMENT
The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.
