Developing and exploring the validity of a patient reported experience measure for adult inpatient diabetes care

1 INTRODUCTION

Around 18% of people admitted as inpatients in England and Wales have diabetes.¹ Of those admitted with diabetes, only 8% have a reason for admission directly related to diabetes; most (92%) are admitted because of a health problem other than diabetes itself.² Data confirm that people with diabetes experience hospital-acquired harm,³ such as severe hypoglycaemia requiring rescue treatment and diabetes ketoacidosis.² Almost one-third of inpatients with diabetes experience a medication error during their hospital stay.²

To improve inpatient diabetes care, Diabetes UK recommends that every hospital has a multidisciplinary diabetes inpatient team, and a diabetes-educated inpatient workforce.⁴ National audit data are used to assess performance and to drive improvements in care. This was initially collected as part of the National Diabetes Inpatient Audit (NaDIA), which was an annual snapshot audit and included a questionnaire for people with diabetes asking about their experience of care. Inpatient data are now collected more continuously as part of the National Diabetes Inpatient Safety Audit (NDISA).

It is clear that robust evaluation of the quality of inpatient diabetes care should include feedback from people with diabetes on the inpatient care they have received. Patient Reported Experience Measures (PREMs) are designed to understand care experiences from the perspective of the patient including interactions with healthcare staff and the degree to which their needs are being met.^{5, 6} This approach is aligned with person-centred care crucial for the delivery of high-quality care^7-10 and increasingly linked with patient safety.^{11, 12}

PREM instruments measure patients' personal care experiences and focus on those aspects of care that matter to them.¹³ The ability of the existing NaDIA patient experience questionnaires to capture patient experience is limited; the existing survey used in the audit contains only 13 questions and does not allow for a thorough exploration of key aspects of person-centred care such as involvement in decisions, information and support, communication, confidence and trust in staff and care continuity.

‘Patient experience’ is sometimes equated to ‘patient satisfaction,’ however, these concepts are not interchangeable. An example of a measure of patient satisfaction in inpatient diabetes care is the Diabetes Treatment Satisfaction Questionnaire for Inpatients (DTSQ-IP).¹⁴ While patient satisfaction is concerned with patients' evaluation of the care provided relative to their expectations, patient experience—as measured by PREMs—focuses on specific interactions that patients have during their care.¹⁵ Measures of satisfaction are strongly influenced by patients' expectations while the measures of experience are concerned with value-free reports of care.¹⁶ Whilst satisfaction measures use evaluation-type questions (e.g. ‘How satisfied were you…’) experience measures use report-type questions (e.g. ‘Were you involved in decisions about your diabetes during your hospital stay?’). These measurement models require appropriate evaluation techniques with the former advocating psychometric methods and the latter proposing alternative methods.¹⁷ In the context of measuring healthcare quality, the measures of satisfaction were found fairly insensitive in detecting shortcomings in care provision¹⁶ while PREMs offer better insight into the aspects of healthcare that patients truly value, and produce data that are more actionable for quality improvement initiatives.^{18, 19}

The aim of this study was to develop and explore the validity of a new PREM for adult inpatient diabetes care.

2 METHODS

The Inpatient Diabetes Care (IDC) PREM was developed collaboratively by the research team including members of Oxford Brookes University, Picker Institute Europe, Oxford University Hospitals NHS Foundation Trust and the University of Oxford using an established survey-development methodology adapted to the needs of this project (Table 1).

3 RESULTS

4 DISCUSSION

We report on the development of the IDC PREM and investigation of its validity in capturing peoples' experiences of diabetes care throughout their inpatient stay from admission to discharge. The measure enables people with diabetes to provide feedback on how they felt about the management and self-management of their diabetes during their hospital stay, their care including medication and equipment, managing their diabetes with meals, the healthcare staff looking after them, communication about diabetes with healthcare staff and emotional needs in relation to diabetes. The PREM also provides an opportunity for the respondents to add free-text comments about diabetes care.

The PREM was developed in collaboration with people with diabetes and was informed by qualitative interviews, tested using cognitive interviews, and piloted. The tool performed well in the pilot with a low drop-out and missing data. Analysis of item frequencies and variances, and problem score calculations concluded that questions provided sufficient score differentiation.

A substantial number of eligible patients invited to complete the pilot survey did not participate (399 invitees out of 627 invited did not participate). There were a number of factors that affected participation. The feedback from the recruiting research nurses indicated that some survey recipients did not perceive themselves as eligible because their admission was not directly diabetes related or their type 2 diabetes was diet managed. Other survey recipients communicated that they did not want to complain therefore they preferred not to participate. Providing clearer information on eligibility (in writing and when talking to potential participants) and emphasising the value of all feedback independent of the type of diabetes or the positivity/negativity of the feedback, may help when developing recruitment strategies to maximise participation. Another factor that might have impacted participation rates was the timing of giving out the recruitment packs (which was dependent on research nurse availability).

Because of the participant identification strategy, the recruitment bias cannot be excluded; the use of a pre-defined sampling approach that stratifies by particular demographics to identify all eligible candidates may help to limit this bias. Some populations of inpatients with diabetes from ethnic minority groups were excluded. The exclusion of those who could not communicate in English to the level enabling participation means that the PREM has not been adequately tested with those who could not communicate in English about their diabetes care when inpatients. A translation helpline and/or having the survey available in other languages could improve accessibility in the future.

There was a substantial difference in uptake of the paper and online PREM; it is important to understand whether differences were likely due to recruitment methods or differences in those opting for one form over another.²⁵ There were differences in the ease of recruiting to the paper and online PREM with research nurses preferring to distribute research packs with paper PREM; paper PREM had also a higher response rate. There is not enough evidence to explain the preference for paper over online PREM. There is also not enough information to determine if the way of recruiting to the online PREM was efficient (the link www.brookes.ac.uk/prem was included in the written invitation or a clickable link was forwarded in an email by RNs to those participants who agreed to be contacted that way). The prevalent social context at the time of the study, of media discussion regarding the sharing of patients' health records with private companies,²⁶ might have impacted the willingness of participants to engage with online content for research in the NHS.²⁷

5 CONCLUSION

This new IDC PREM allows those staying in hospital as inpatients to share their experiences of diabetes care with the intention to improve existing services; this can be used as an audit tool, a service evaluation and improvement tool and a research tool.

The new IDC PREM functioned well in the pilot; item completion rates were good and there was no evidence of survey fatigue. The scoring system (problem scores) demonstrated a good capability of showing discrimination. Problem scores provide actionable feedback on specific aspects of care. We have not advised providing an aggregated score of patient experience as this would mask the granularity of the data that is obtained from looking at data on a question-by-question basis.

The tool may aid clinicians to better understand the patient's perspective and to suggest patient-focused goals. There is a need to improve the accessibility of the tool (e.g. for those who do not speak English or for patients with cognitive impairment) in patients with diabetes in acute hospitals.

FUNDING INFORMATION

The study received funding from the NIHR Oxford Biomedical Research Centre, Oxford, UK and the Oxfordshire Health Services Research Committee, Oxford, UK for providing funding for this study. The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health.

CONFLICT OF INTEREST STATEMENT

No conflicts of interest are declared by the authors.