The KICA Screen: The psychometric properties of a shortened version of the KICA (Kimberley Indigenous Cognitive Assessment)
Abstract
Aim: To describe the development and psychometric properties of the KICA (Kimberley Indigenous Cognitive Assessment) Screen.
Methods: A short 10-item version of the KICA, the KICA screen was developed from original data of 363 Aboriginal people. The KICA Screen was subsequently independently validated in a non-random sample of 55 people living in Northern Queensland.
Results: In the original sample the KICA Screen showed an optimal cut-point score of 21/22 (out of a score of 25), and resulted in a sensitivity of 95.6% and specificity of 88.6% with AUC of 0.95 (95% CI 0.91–0.98). This cut point was subsequently tested on 55 people living in Northern Queensland, with a sensitivity of 82.4% and specificity of 88.5% for the diagnosis of dementia, and the area under the ROC (receiver operating characteristic) curve was 0.87 (95% CI 0.77–0.97).
Conclusions: The KICA Screen is a valid and acceptable screening tool among Indigenous Australians living in remote and rural areas of Australia.
Background
Dementia is a major cause of disability worldwide [1], but despite its high prevalence inadequate screening at the level of primary care remains a problem [2]. To encourage case finding, a tool needs to be quick, easy to administer, and culturally and educationally appropriate [3].
Aboriginal and Torres Strait Islander people have the worst health status of any population in Australia, with life expectancy approximately 17 years shorter than non-Indigenous people and burden of disease more than twice that of other Australians [4,5]. We have recently documented that dementia affects 12.4% of Indigenous Australians aged 45 years or older living in the Kimberley region [6], a prevalence estimate that is 5.2 times greater than for non-Indigenous Australians.
Indigenous Australians, including those from the Torres Strait, express a rich diversity of traditions, law and culture, with a large number of spoken languages [7], and often have limited Western style education. For these reasons, conventional screening for cognitive impairment may not be appropriate even when translated into traditional language [7,8].
The Kimberley Indigenous Cognitive Assessment (KICA) tool was developed by the authors to address a gap in the availability of a culturally appropriate assessment tool specifically for older Indigenous people living in remote and rural areas [9]. Initial results showed the effectiveness of the cognitive assessment section of the tool (KICA-Cog) to detect those with dementia [9] in populations of the Kimberley and Northern Territory [9,10]. The original KICA-Cog is freely available at the following website http://www.wacha.org.au/kica.html.
Despite its widespread use, a possible limitation of the KICA-Cog is the time required to complete the assessment, which hinders its acceptability in busy primary care settings. The aim of this study is to describe the psychometric properties of the KICA Screen and its subsequent validation in an independent population of older Aboriginal and Torres Strait Islanders living in Far North Queensland.
Methods
Development of KICA Screen
The KICA Screen (Appendix I) was derived from the original KICA-Cog items used to test a population sample of 363 people living in the Kimberley. Recruitment was based on semi-purposeful sampling. All residents over the age of 45 years living in six remote Aboriginal communities and a random sample of one third of eligible Indigenous people in one town were approached [6]. Their characteristics have been described in detail elsewhere [6]. Items that were best able to differentiate between the participants with and without cognitive impairment and dementia were determined by discriminant function analysis (DFA) from the full KICA-Cog. Sensitivity, specificity and cut points for the shortened version of the instrument were determined from the original validation sample.
Validation of KICA Screen in Queensland
The KICA Screen was then evaluated in a group of people from Northern Queensland over the age of 45 years. The Queensland study used convenience sampling methods, where a deliberate attempt was made to assess participants with varying degrees of cognitive impairment ranging from normal to dementia, using an approach similar to that of the original validation study in the Kimberley [9]. Participants were initially assessed with the KICA Screen and subsequently reviewed by a geriatrician who was unaware of the results of the KICA Screen. The Geriatrician Review involved a full medical, functional, psychosocial and cognitive assessment of the participant, as well as interview of carers and relevant family members in order to complete a Comprehensive Geriatric Assessment. The cognitive component of the review was based on the domains of the CIBIS/CIBIC Plus, that is, Orientation, Memory, Language/Speech, Praxis and Judgement/Problem Solving/Insight using the relevant prompts. As there is no alternative validated diagnostic interview schedule for this population, combination of formal and informal tools were used to assist clinical judgement in assessing performance across cognitive domains. This included use of the Folstein Mini-Mental State Examination (MMSE) when possible (although a diagnosis was not made on the basis of the overall score alone), delayed recall using shopping items, verbal fluency, description of safety tasks, clock face drawing, orientation, copying pictures, dressing praxis and others. This was combined with informant history and functional assessment to determine an overall assessment of cognition and care was taken to not overlap with items included in the KICA screen. This was supplemented by CT scan results when available.
This assessment involved determination of past and present history, medications, informant report, and behavioural, psychological and cognitive assessment using the domains of the Clinician Interview Based Impression of Change plus carer input (CIBIC-plus) [11].
Written records of this assessment were reviewed by two independent geriatricians or geriatric psychiatrists to reach a consensus diagnosis according to DSM-IV [12] and ICD-10 criteria for research [13]. The consensus diagnoses were used to classify participants as having no cognitive impairment, cognitive impairment not dementia (CIND) or dementia. In the case of dispute between a consensus diagnosis a third specialist (NL) was available to review the records, but this was not required. Verbal feedback was given to all individuals and family members. Following written consent, the results for each participant were provided to their health centre and doctor with recommendations as needed. At the completion of the study in each community, a report was provided to local councils and information about the study was presented at community meetings.
Statistical analysis
All data were analysed using the Statistical Package for Social Sciences (spss) version 15. Between-group comparisons were performed using Pearson's χ2 test or Student's t-test. DFA was used to determine the ability of individual items of the KICA Screen to differentiate between those participants with normal cognition and dementia. Sensitivity and specificity values and cut-off scores were determined by formulation of receiver operating characteristic (ROC) curves, and calculation of the area under the curve (AUC), for those with no cognitive impairment and definite dementia. The effects on sensitivity and specificity values were determined when the diagnosis of CIND was included in both normal and dementia groups. Multivariate logistic regression was then used to take into account the effect of potential confounding variables.
Ethics approval
Ethics approval was gained from Western Australian Aboriginal Health Information and Ethics Committee, the University of Western Australia Ethics Committee for Human Research, Ethics Committee for Cairns and Hinterland Health Service District and the District Managers for Cairns and Hinterland, Cape York and Torres Strait Health Service. Individual community council approval was obtained from each community and each participant or family member and carer provided informed consent.
Results
Initial development of the KICA Screen
Demographic features of the Kimberley cohort are described in Table 1. The mean age of those assessed was 60.6 (SD = 11.9) years, 291 (60%) received formal schooling, and 198 (55%) were women. Questions that assessed recall, registration and free recall were highly discriminatory items that correctly classified 96.7% of participants with or without dementia. The standardised discriminant function coefficient for registration was 0.51, for recall 0.27 and for free recall 0.59. The optimal cut-point score of 21/22 (case/non-case; out of a possible total score of 25) resulted in a sensitivity of 95.6% and a specificity of 88.6%. The AUC was 0.95 (95% CI 0.91–0.98). Sensitivity decreased to 87.8% when CIND was included in the dementia group, and specificity to 82.7% when CIND was included in the normal group, but the AUC remained unchanged. The difference in mean total KICA Screen for people with no education and some education was not statistically significant (P = 0.11), after adjusting for age and dementia diagnosis.
| Kimberley (n = 363) | North Queensland (n = 55) | |
|---|---|---|
| Age (years) | 60.6 (11.9) | 69.6 (45–95) |
| Mean s-KICA score | 21.4 (5.3) | 20.2 (4.7) |
| Sex (% women) | 198 (55%) | 35 (63.6%) |
| Received formal schooling | 219 (60%) | 49 (89%) |
| Informant living with participant | 206 (57%) | 24 (43.6%) |
| Normal | 289 (79.6%) | 26 (47%) |
| Dementia | 45 (12.3%) | 17 (31%) |
| CIND | 29 (7.9%) | 12 (22%) |
- CIND, cognitive impairment not dementia; KICA, Kimberley Indigenous Cognitive Assessment.
Validation of KICA Screen in Northern Queensland
The KICA Screen was subsequently tested in an independent sample of 55 people living in Northern Queensland. The mean age of these participants was 69.6 years, with a range of 45–95 years (Table 1). Most were women (63.6%) and had received some formal schooling (89%). Ninety-one per cent spoke English as their first language, and 7% spoke Creole. All participants were either Aboriginal (60%) or Torres Strait Islander (40%). Participants were from seven different communities: Yarrabah (18%), Mapoon (16%), Napranum (20%), Weipa (6%), Thursday Island (20%), Badu (16%) and Horn Island (4%). Carers were primarily the participants' children (34%), and female (74%), with 44% of informants living with the participant.
Twenty-six (47%) participants had no cognitive impairment, 17 (31%) had Dementia and 12 (22%) had CIND. The types of dementia were Alzheimer's disease [5], vascular dementia [3], dementia not otherwise specified [8] and dementia with multiple aetiology [1]. The mean score of the KICA Screen was 20.2 (SD = 4.7), and the internal consistency of the scale, as measured by Cronbach's alpha, was 0.76. The item that most successfully classified those with dementia versus those with no cognitive impairment was the free recall item, which correctly classified 88.4% of the group. The KICA Screen administered in this sample had a sensitivity of 82.4% and specificity of 88.5%, with a cut-point of 21/22 (case/non-case) differentiating those with dementia and no cognitive impairment. The area under the ROC curve was 0.87 (95% CI 0.77–0.97), and 82% of people with dementia were correctly classified with the KICA Screen. When CIND was included in the dementia group, sensitivity was reduced to 75.9%, and after CIND was included in the normal group specificity decreased to 71.1% (Figure 1).
ROC curves for the 16 item KICA-Cog, 10 item KICA Screen in Kimberley and Queensland samples, for dementia versus no cognitive impairment (CIND not included). CIND, cognitive impairment not dementia; KICA, Kimberley Indigenous Cognitive Assessment; ROC, receiver operating characteristic.
Discussion
This paper describes the development and basic psychometric properties of a short cognitive assessment tool for Aboriginal and Torres Strait Islander people over the age of 45 living in Australia. This is the first time that a cognitive screening tool has been validated in a sample including a Torres Islander subpopulation, suggesting its utility within the diversity of culture of this population.
Queensland participants were significantly older than the Kimberley participants and a larger proportion had access to formal education, a finding consistent with census data for education and language [14]. At a cut point of 21/22, the KICA Screen had slightly higher sensitivity in the Kimberley than the Queensland population. Some of the minor differences noted in sensitivity and specificity may relate to differences in selection of the cohort. The original KICA Screen was developed from a semi-purposeful sample recruited from various Kimberley communities, where all members of that community over the age of 45 years were recruited to take part. The KICA Screen tested in Queensland was on a much smaller sample that was specifically selected for this study and had a deliberate mix of normal and cognitively impaired participants. Previous studies on this tool have shown similar variations [10]. As described by Mulherin et al. [15], when study participants are not necessarily representative of patients seen in clinical practice, or the case mix differs between populations, a spectrum effect can apply usually affecting sensitivity. The predictive value of a screening tool will vary with the prevalence rate of a condition, such as dementia in the population tested [16]. Despite these limitations, the validity of the KICA Screen was comparable with the MMSE in a sample of Australian Aged Care Assessment Team clients (sensitivity 87.5% and specificity 85.3%) [16], and with the RUDAS in a multicultural cohort of community dwelling older people (sensitivity 84.3% and specificity 87.9%) [17].
The original KICA-Cog had a sensitivity of 90.6% and of 92.6% in the Kimberley [9], which may indicate that the KICA Screen is a slightly less sensitive tool. Those without dementia or cognitive impairment who screen positive KICA Screen may have transient diagnoses such as delirium or depression, or may have a stable but low level of cognitive functioning. This further highlights the need for medical review following a positive score on screening to prevent individuals being mistakenly diagnosed with dementia. To increase the specificity of the tool, the combination of the KICA Screen with an informant questionnaire would be useful, as seen in other tools developed for primary care, for example GP Cog [18].
In both populations studied the sensitivity of the KICA screen decreased when CIND was combined with those with dementia, whereas when CIND was combined with the normal groups specificity decreased. This pattern is seen with other cognitive screening tools [17], possibly because of the uncertain clinical significance of CIND.
Importantly, the KICA screen required less than 10 minutes to administer, similar to the administration time of the MMSE, whereas the KICA-Cog takes 25–30 minutes to complete. As a wide range of busy health professionals prefer brief screening tools, we recommend that the KICA Screen, at a cut point of 21/22, be considered for use in older Indigenous Australians. As with all screening tools, a positive response indicates that further clinical assessment is required to determine possible causes of cognitive impairment. Further validation studies on larger random community samples outside the original Kimberley cohort are required to better determine the test properties of the KICA Screen.
Acknowledgements
We thank the residents of the communities of Cape York and the Islands of the Torres Strait, and Jean Little, Judith Groube, Yarrabah Council staff, Yarrabah Primary Health Care Centre, Yarrabah hostel, Cape York Health Service District and the traditional owners and members of The Western Cape Communities, Mapoon, Napranum, Thursday and Badu Island communities, and members of the Steering Committee. Funding for the prevalence study was obtained through (Australian) National Health and Medical Research Council (NHMRC) grants 219194 and 353612. NHMRC had no role in the study design, study implementation, data interpretation or drafting of this paper. The Queensland project was funded by Queensland Health Strengthening Aged Care program.
Key Points
- •
The KICA Screen is a valid and acceptable brief screening tool for use with remote Indigenous Australians.
- •
The KICA Screen can be utilised within the diversity of culture and traditions of Indigenous Australians including Torres Strait Islanders.
- •
Further longitudinal evaluation of the KICA Screen tool is required.
Appendix
Appendix I
KICA Screen
Patient_________________ Date_______________ Assessor______________
I'd like to see if you can remember things. I'll ask you some questions.
Score
Orientation
1. Is this week pension/pay week? 0 1 ()
2. What time of year is it now? 0 1 () (May need to prompt, e.g. wet time? dry time? cold time? hot time?)
3. What is the name of this community/place? 0 1 ()
Verbal comprehension
4. First point to the sky and then point to the ground. 0 1 2 ()
Verbal fluency
5. Tell me the names of all the animals that people hunt.
(Time for one minute. Can prompt, e.g. ‘Keep going’, ‘Any more, e.g. in the water?’)
Total No._________
(0 animals) 0
(1–4 animals) 1
(5–8 animals) 2
(9 animals or more) 3 ()
Visual naming
6. I'll show you some pictures. You tell me what they are. Remember all these pictures for later on, because I'll ask you one more time.
Point to each picture and ask What's this?
(Show boomerang as example) Now remember them.
boy, emu, billy/fire, crocodile, bicycle 0 1 2 3 4 5 ()
□ Couldn't do due to poor vision. Name pictures for them to remember.
Frontal/executive function
7. Look at this. Now you copy it. (crosses and circles below) 0 1 ()
Free recall
8. You remember those pictures I showed you before? What were those pictures?
Tell me.
(Show boomerang picture as example) 0 1 2 3 4 5 ()
Cued recall
9. Which one did I show you before? (One of three pictures; use boomerang page as example)
If poor vision name the three options 0 1 2 3 4 5 ()
Praxis
10. Show me how to use this comb. 0 1 ()
Cut-off score ≤ 21/25 (Indicates possible Dementia; further screening required)
Total Score_______________
Refer to http://www.wacha.org.au/kica.html for full details
