1 INTRODUCTION

People living with mental illness experience an increased mortality rate of approximately 2.2 times that of the general population (Walker, McGee, & Druss, 2015). Those living with severe mental illness (SMI), such as schizophrenia, bipolar disorder, and major depressive disorder, have a reduced life expectancy of approximately 15 years (Hjorthøj, Stürup, McGrath, & Nordentoft, 2017), primarily due to preventable and premature cardiometabolic diseases (Correll et al., 2017). People with SMI have a 53% higher risk of having cardiovascular disease (CVD), are 78% more likely to develop CVD in the future, and are roughly 85% more likely to die from CVD in comparison with the general population (Correll et al., 2017). Further, people with SMI are more likely than the general population to meet criteria for metabolic syndrome (Vancampfort et al., 2015), a cluster of risk factors including abdominal obesity, dyslipidemia, hypertension, and insulin resistance (Zimmet et al., 2007), which is a significant contributing factor to the overall burden of CVD in this population. For example, among people with psychosis the incidence of metabolic syndrome has been documented as high as 61% which is approximately 2–3 times higher than the general population (Morgan et al., 2013).

Although there is only limited available data from Italian samples (Carrà et al., 2014; Clerici et al., 2014), internationally, the issues of physical health inequalities for people living with mental illness and the subsequent premature mortality have been identified as a public health crises (Thornicroft, 2011), with increasing calls for urgent action (Stewart, 2015; Suetani, Whiteford, & McGrath, 2015). A 2017 Forum on the need for a comprehensive approach to addressing the excess mortality in people with SMI in collaboration with the World Health Organization identified a multilevel intervention framework and the priorities for clinical practice, policy, and research agendas, including a focus across both individual factors (e.g., modifiable behavioral risk factors such as smoking and low levels of physical activity), health systems (including service delivery and human resources) and the social determinants of health (Liu et al., 2017). Additionally, in Australia and the United Kingdom a number of high-level policy-based documents have been developed including guidance from the Royal Australian and New Zealand College of Psychiatrists (The Royal Australian & New Zealand College of Psychiatrists, 2015) and a briefing report from Public Health England (Public Health England, 2018).

The aims of this study were to determine the psychometric properties of the M-BACK into Italian (M-BACK-IT) using established translation protocol and to determine the test–retest reliability of the instrument.

2 METHOD

The development of the English version of the M-BACK has been reported elsewhere and briefly involved a comprehensive literature review, and consultation and review by a multi-disciplinary panel of experts including clinicians, educators, and researchers (Watkins et al., 2017). Pilot testing was subsequently conducted with mental health nurses and structured feedback on the content, and usability of the tool was obtained. Feedback was collated, and consensus on the final version was then achieved through group discussion with the expert panel.

The translation process followed established protocol and involved a series of steps including (a) forward translation, (b) reconciliation, (c) back translation and review, (d) harmonization, and (e) cognitive debriefing and finalization.

The M-BACK questionnaire is intended to assess change over time and to detect change in the items following specific education or training targeting mental health professionals (Watkins et al., 2017). Internal consistency with Cronbach's alpha was tested and, as per the original validation study, test–retest for each item was determined by calculating intraclass correlation coefficients using a two-factor mixed effects model (Shrout & Fleiss, 1979). Likert-type items are typically considered as a continuous variable, and intraclass correlation coefficient (ICC) is the recommended method of assessing test–retest data for continuous data (Rankin & Stokes, 1998). Using ICC can determine the proportion of total variance that occurs between time points (Shrout & Fleiss, 1979).

Ninety-two mental health professionals from a private psychiatric hospital in northeast Italy were invited to complete the questionnaire. There were no other exclusion criteria. Participants were informed of the aims of the study that participation was completely voluntary, and there were no consequences for not participating. Participants completed a signed consent form. Thirty-eight professionals (41.3% of the invited staff) completed the questionnaire on the first occasion. Eight participants were unable to complete the questionnaire at the second time point, due to time constraints and clinical responsibilities. The majority of the sample who completed the questionnaire on the two occasions (seven days apart) were female (70%) and aged <50 years (80%).

Data were reviewed to ensure completeness. Cronbach's alpha and ICCs were calculated for the total score, as well as each of the four domains (Barriers, Attitudes, Confidence, and Knowledge). ICCs were interpreted based on Landis and Koch (1977), with ICCs of below 0.4 (poor to fair), 0.41–0.6 (moderate), 0.61–0.80 (excellent), and 0.81–1 (almost perfect) (Landis & Koch, 1977). The questionnaire was analyzed for individual items in addition to the four domain areas.

The study was approved by the University of Padua, Department of Biomedical Sciences Human Research Ethics Committee (HEC-DSB/01-19). Anonymity was ensured by using appropriate codes and storing demographic data separate from the completed questionnaires.

3 RESULTS

Thirty staff from a private psychiatric hospital in Northern Italy participated in this study. Participants had different background and roles, 40% were nurses, 30% psychologists, 17% exercise specialists, and 13% psychiatrists. In total, 70% of participants (n = 21) were female and the majority (80%) were aged <50 years. They completed the M-BACK-IT on two occasions, seven days apart.

The internal consistency of the questionnaire resulted in acceptable to good values with alphas ranging from 0.69 to 0.87 for the different domains (see Table 1 for the detailed values) and α = 0.71 for the total score.

ICC correlations for individual items ranged from 0.58 to 0.94 (see table 1).

4 DISCUSSION AND CONCLUSION

This paper describes the translation, and test–retest reliability of the M-BACK-IT, a novel tool designed to assess the attitudes, confidence, and knowledge of mental health practitioners in providing metabolic care to people experiencing mental illness and the perceived barriers in the delivery of appropriate interventions.

The M-BACK plays an important role in the bridging the established implementation gap between research and practice, for example by gaining an understanding of the barriers to the provision of metabolic screening and interventions, results of the M-BACK can inform the design and delivery of subsequent training and education. It is also an important tool by which the effectiveness of training and education for mental health professionals and other initiatives of mental health services to address the important area of metabolic health care can be evaluated. For example, using pre–post design, it also enables evaluation of the effectiveness of education regarding metabolic health care through pre- and post-training testing. Similarly, the M-BACK can be used to determine the effectiveness of staff-based interventions for mental health professionals (Fibbins, Ward, Watkins, Curtis, & Rosenbaum, 2018).

The Italian version of the questionnaire demonstrated acceptable to good internal consistency and good test–retest reliability, with an ICC of 0.87 for the total M-BACK-IT score and 0.86–0.91 for each of the four domains (Barriers, Attitudes, Confidence, and Knowledge).

Test–retest reliability analysis will rarely achieve perfect results (Wikman & Wärneryd, 1990). Within the domains, the greatest variance was found within the Attitude and Confidence domains and least variance within the knowledge domain. Item 12 in the Attitude domain “I am confident in using dietary interventions to prevent/treat metabolic syndrome in consumers” had the lowest ICC of any item throughout the questionnaire (0.58), although this was still classified as moderate according to the criteria of Landis and Koch (1977) (Landis & Koch, 1977). Possible explanations for why some items in a questionnaire may have greater variability using test–retest reliability can be due to a change of opinion or research undertaken by a participant in between test dates, perhaps even prompted from completing the questionnaire (Wikman & Wärneryd, 1990).

Given the acceptable outcomes of all results in the test–retest, it was determined that no changes to questions were required to ensure instrument validation. The current paper contributes to the recommendations of the recent Lancet Psychiatry Commission: Ensuring culturally appropriate scales are available in languages other than English is of key significance, given increasing international calls for a global approach to addressing the poor physical health of people with mental illness. Culture change and upskilling mental health clinicians regarding lifestyle interventions are critical, and therefore, having language-specific, validated tools to measure change and progress is imperative (Firth et al., 2019).

The M-BACK-IT questionnaire is not without limitations, and further research with a larger and more diverse sample of mental health professionals working across different settings is recommended. Determining whether responses vary as a consequence of professional background (i.e., investigating differences between psychiatrists, psychologists, and nursing staff) and specific characteristics of different mental health services (e.g., those treating inpatients or outpatients) would be of clinical relevance. This may also inform whether discipline-specific or setting-oriented versions of the instrument are required.

The M-BACK-IT tool seems to be a valid and reliable instrument to measure the attitudes, confidence, and knowledge of mental health staff in metabolic health and their perceptions of barriers to delivering such metabolic health care. There is a clearly identified need for initial and in-service training and education for mental health staff in regard to the metabolic health of mental health consumers.

CONFLICT OF INTEREST

The authors declare that they have no competing interests.