1 INTRODUCTION

Poststroke cognitive impairment (PSCI) is prevalent in over half of patients 6 months after stroke (Merriman et al., 2019), and cognitive assessment in stroke patients is essential. The standard process requires comprehensive assessment using well-validated cognitive tasks (Gorelick et al., 2011). Our preliminary work has validated the NINDS-CSN neuropsychological protocols (60, 30, and 5 minutes) in mild stroke patients (Chen, Wong, et al., 2015). However, there is no consensus on the definition of impairment in a particular domain, while PSCI is defined as deficits in at least one cognitive domain (Gorelick et al., 2011). For instance, the cutoff value would be set at 1, 1.5, or 2 standard deviations (SD) below the means of the healthy control, and it is inconsistent whether a single test or multiple tests should be considered when determining the impairment of a cognitive domain (Pendlebury, Mariz, Bull, Mehta, & Rothwell, 2013a). These would have affected the prevalence estimation of PSCI and hindered the comparison between various studies.

For time–cost optimization, cognitive screening tools have become the first choice in clinical practice, such as the Mini-Mental State Examination (MMSE) and the Montreal Cognitive Assessment Test (MoCA; Ghafar, Miptah, & O'Caoimh, 2019). Moreover, telephone scales like the NINDS-CSN 5-minute protocol and the six-item screener (SIS; Callahan, Unverzagt, Hui, Perkins, & Hendrie, 2002) are helpful for the large-scale epidemiological survey and long-term follow-up studies (Levine et al., 2018). However, the thresholds used in each study are inconsistent, varying from 21 to 26, probably due to different diagnostic criteria, culture difference, and distinct education level of the subjects (Stolwyk, O'Neill, McKay, & Wong, 2014).

This study was based on the employment of different neuropsychological definitions in stroke patients. The rate and classification of PSCI were compared; meanwhile, the diagnostic accuracy and the optimal threshold of the screening tools MoCA, MMSE, the NINDS-CSN 5-minute protocol, and SIS were investigated.

2 METHODS

3 RESULTS

A total of 498 patients were screened during the study period, and 409 were excluded (258 had geographic limits, 36 lost contact, 10 had severe limb impairment, 2 had aphasia, 5 had visual or hearing impairment, 10 had prestroke cognitive decline or in use of cognitive enhancers, 2 had Parkinson's disease, 40 had a hemorrhagic stroke, 34 underwent stenting or balloon angioplasty, and there were 12 deaths before assessment). Hence, this study included 89 stroke patients with an average age of 62.9 ± 8.6 years, an education level of 9.2 ± 4.2 years, and a male ratio of 65.2%. The severity of stroke at admission was assessed using the National Institutes of Health Stroke Scale (NIHSS), and the median score for NIHSS was 2 points (IQR, 0.8–5). All patients underwent a comprehensive neuropsychological assessment and the face-to-face cognitive screening tests. The test was performed at 7.3 months (IQR, 4.8–9.4) after stroke onset. Eighty patients underwent the telephone tests, and the median interval between the face-to-face test and telephone assessment was 2.8 months (IQR, 1.7–7.0).

Among the 80 stroke patients who completed both the face-to-face tests and the telephone tests, the rate of PSCI was found to be 37/80 (46.3%) from the most conservative definition (2 SD below the control mean), to 61/80 (76.3%) as with the least conservative definition (1 SD below the control mean). The classification also varied. In general, there was a higher rate of patients classified as single-domain impairment when more stringently defined. Nevertheless, those with amnestic single-domain PSCI were rare regardless of the different definitions, accounting for only 0%–7% of the total number. In addition, the PSCI rate was slightly lower when using multiple tests to define memory-domain impairment compared to using a single test, but the impact was not significant (Pearson's chi-Square: 0.100–0.517, p range: .47–.75) (Table 1). On the other hand, despite the varied definitions, the PSCI patients reported more depressive symptoms as assessed by the geriatric depression scale (p range: .02–.11) with medium effect sizes (Cohen's d: 0.36–0.53), but they showed no differences for neuropsychiatric symptoms as assessed using the neuropsychiatric inventory questionnaire (p range: .12–.93).

The ROC curve analyses showed an AUC value above 0.7 for each cognitive screening scale despite different neuropsychological definitions, indicating good validity (Figure 1). The face-to-face screening tests of MMSE and MoCA had broadly similar AUC values. The two telephone screening tests were comparable for the 2 SD cutoff definition (Figure 1c,f). In comparison between the telephone and face-to-face tests, the SIS scale was generally less accurate than the MoCA (p range: .002–.04), while SIS was alternative to MMSE at 1 SD cutoff (Figure 1a,d, p: .61 and .55 based on single test and multi-test, respectively). Besides, the NINDS-CSN 5-minute protocol was comparable to both face-to-face cognitive tests (p range: .12–.93) except for the most conservative definition at 2 SD cutoff and multi-test required for memory impairment (Figure 1f, p: .05 and .03 compared to MMSE and MoCA, respectively).

The sensitivities and specificities of each scale were listed according to different neuropsychological criteria (Table 2), and the cutoffs to detect PSCI were determined. In most cases, sensitivity and specificity were optimal with MMSE ≤ 27 points, MoCA ≤ 19 points, NINDS-CSN 5-minute protocol ≤23 points, and SIS ≤ 4 points. The MMSE scores were skewed to the high scores, while the MoCA scores were normally distributed (p = .627 by a Kolmogorov–Smirnov test). Correlation analysis suggested that the two had a high correlation (Spearman's r² = .723; p < .0001). The cognitive screening scales with the optimal cutoffs reached a good agreement with the comprehensive cognitive assessment (kappa: 0.318–0.630 for the telephone scales, 0.411–0.697 for the face-to-face scales). There was a greater consistency for the NINDS-CSN 5-minute protocol than the SIS scale, and a greater consistency for MoCA than MMSE as well. However, based on the education-stratified threshold, MMSE and MoCA were inconsistent with the comprehensive cognitive assessment (Kappa: 0.074–0.328).

4 DISCUSSION

Our results suggested that under different neuropsychological definitions, face-to-face MMSE and MoCA, and telephone scales of NINDS-5min protocol and SIS, were feasible and effective cognitive screening tools for stroke patients in China.

Previous studies have shown that the incidence rate of dementia and PSCI is affected by different diagnostic criteria (Pendlebury, Mariz, et al., 2013; Robertson et al., 2019). Our study indicated a lower rate of PSCI toward a more conservative definition, which was consistent with prior findings (Pendlebury, Mariz, et al., 2013). Besides, in line with clinical observations, stroke patients seldom suffered from amnestic single-domain PSCI (Camarda et al., 2018). Moreover, the rate of patients with PSCI in multiple domains increased as the definition was less conservative. Such patients were reported to progress to dementia at a faster speed (Kim et al., 2019); hence, the differences in definition may affect the identification of individuals along the disease trajectory (Edmonds et al., 2019).

The study found that each cognitive screening scale can effectively distinguish stroke patients with PSCI regardless of various neuropsychological definitions. As a face-to-face test, either MoCA or MMSE can be applied to stroke patients in cognitive screening, and the former was more accurate in tracking mild cognitive impairment, though they were of the same effectiveness to detect dementia (Pinto et al., 2019). For telephone screening, the NINDS-CSN 5-minute protocol was a more effective scale than SIS in stroke patients (Chen, Fan, et al., 2015). Furthermore, a similar accuracy of the NINDS-CSN 5-minute telephone scale to both MMSE and MoCA was found in all cases except for the most conservative definition, and the accuracy of the SIS telephone scale was comparable to that of MMSE for the less conservative definition. Since a less conservative definition corresponds to a more sensitive screening, the NINDS-CSN 5-minute telephone scale can be utilized as an alternative to MoCA and MMSE, and SIS test to MMSE.

Both MMSE and MoCA are commonly used cognitive screening scales in stroke patients (Rodrigues et al., 2019). This study showed a ceiling effect of the MMSE, which was also reported in previous observations (Trzepacz, Hochstetler, Wang, Walker, & Saykin, 2015). On the other hand, scores of MoCA were normally distributed, and the consistency with comprehensive cognitive assessment was better than MMSE under different definitions. Therefore, MoCA may be more suitable than MMSE in the cognitive screening of stroke patients (Ghafar et al., 2019). It is noteworthy that according to a previous review, the 19/20 cutoff for MoCA-BJ is at the lower bound of the optimal values ranging from 20 to 27 for cognitive normality in the chronic stroke phase (Chiti & Pantoni, 2014). Hence, there is a probability that PSCI would be false-negative with this cutoff. Meanwhile, a previous report also showed a majority of false-positive cases with the 21/22 cutoff in Chinese stroke patients (Wong et al., 2015). Therefore, appropriate precautions must be taken before using this statistically derived cutoff value to indicate a further comprehensive neuropsychological evaluation and to determine the potential clinical and functional importance.

In this study, more than half of the stroke patients were unable to participate in face-to-face cognitive tests because of geographical factors. Therefore, in studies of long-term cognitive outcomes, patients would potentially benefit from telephone screening (Moffatt, Hennessy, Marshman, & Manickam, 2019). A recent UK community study firstly explored the feasibility of the NINDS-CSN 5-minute scale for telephone assessment and demonstrated sufficient reliability for cognitive screening 1 year after transient ischemic attack or mild stroke (Pendlebury, Welch, et al., 2013). The SIS Mandarin Scale has been proved efficient for face-to-face screening of cognitive impairment in the elderly (Xue et al., 2018) and was also used as a telephone tool for cognitive follow-up in an American stroke study (Levine et al., 2018). Based on the preliminary validation of the NINDS-CSN 5-minute protocol and SIS as a telephone assessment tool in stroke populations, this study showed that under different definitions, the screening thresholds for PSCI were mostly ≤23 points and ≤4 points respectively, and the NINDS-CSN 5-minute protocol had a higher consistency than the SIS scale.

This study has the following limitations. First, this was a relatively small sample study with a majority of patients who underwent a mild stroke, and patients with right side hemiplegia, aphasia, and severe stroke were excluded. Second, we did not provide education-based or age-based cutoffs, and given that the average was comparatively well-educated, and in the early sixties, the generalization would be limited to those who are illiterate, younger, or older patients (Wu, Wang, Ren, & Xu, 2013). Third, the scales were assessed cross-sectionally, so their sensitivity to cognitive changes and cutoffs for cognitive decline remains worthy of further longitudinal investigations. Fourth, previous studies indicated progressive cognitive decline after stroke (Zheng, Yan, Zhong, Yang, & Xie, 2019), so the scores of telephone screeners, tested at 2.8 months after the face-to-face assessment, might be underestimated; however, this would possibly be neutralized by the practice effects associated with repeated testing items. Finally, cognitive screening tests were generalized rather than domain-specific; hence, assessment of multiple cognitive domains and clinical examinations were still needed for clinical diagnosis.

5 CONCLUSION

This study provides preliminary evidence for use of two cognitive screeners applied in face-to-face interview (MMSE ≤ 27 points and MoCA ≤ 19 points) and another two that can be administered over the phone (NINDS-CSN 5-minute protocol ≤ 23 points and SIS ≤ 4 points) for detecting PSCI in stroke patients. The telephone cognitive screens may be of particular relevance in research when seeking patient eligibility to participate in trials. Our findings may facilitate further studies in terms of choice for cognitive screening tools under different neuropsychological definitions.

CONFLICT OF INTEREST

The authors declare no conflict of interest.

AUTHORS' CONTRIBUTIONS

Minmin Ma designed the study and supervised the data collection. Xiangliang Chen initiated the study and drafted the manuscript. Yunfei Han was involved in data collection, and he critically revised the manuscript. Junshan Zhou preformed the statistical analyses. Xinfeng Liu supervised the data collection and analyses.