Development and preliminary validation of an Observation List for detecting mental disorders and social Problems in the elderly in primary and home care (OLP)
Abstract
Objective
Even though the prevalence of mental disorders and social problems is high among elderly patients, it is difficult to detect these in a primary (home) care setting. Goal was the development and preliminary validation of a short observation list to detect six problem areas: anxiety, depression, cognition, suspicion, loneliness, and somatisation.
Methods
A draft list of indicators identified from a short review of the literature and the opinions of 22 experts was evaluated by general practitioners (GPs) and home care organisations for feasibility. It was then used by GPs and home care personnel to observe patients, who also completed validated tests for psychological disorders (General Health Questionnaire 12 item version (GHQ-12)), depression (Geriatric Depression Scale 15-item version (GDS-15)), anxiety and suspicion (Symptom Checklist-90 (SCL-90)), loneliness (University of California, Los Angeles (UCLA)), somatisation (Illness Attitude Scale (IAS)), and cognition (Mini-Mental State Examination (MMSE)).
Results
GPs and home care personnel observed 180 patients (mean age 78.4 years; 66% female) and evaluated the draft list during a regular visit. Cronbach's α was 0.87 for the draft list and ≥0.80 for the draft problem areas (loneliness and suspicion excepted). Principal component analysis identified six components (cognition, depression + loneliness, somatisation, anxiety + suspicion, depression (other signs), and an ambiguous component). Convergent validity was shown for the indicators list as a whole (using the GHQ-12), and the subscales of depression, anxiety, loneliness, cognition, and somatisation. Using pre-set agreed criteria, the list was reduced to 14 final indicators divided over five problem areas.
Conclusion
The Observation List for mental disorders and social Problems (OLP) proved to be preliminarily valid, reliable, and feasible for use in primary and home care settings. Copyright © John Wliey & Sons, Ltd.
Introduction
The prevalence of mental disorders in the community is high. WHO data for 14 countries show a 1-year prevalence of any psychiatric disorder of between 6% and 27% (Kessler et al., 2009). Mental disorders are even more prevalent in primary care (Jackson et al., 2007) (Linden et al., 2004; Kroenke et al., 2007), among the oldest old (Stek et al., 2007), and among elderly receiving home care (Bruce et al., 2002; Davitt and Gellis, 2011).
In the Dutch health care system, general practitioners (GPs) play a key role in the detection and treatment of mental disorders and social problems. In contrast with the increasing numbers of patients with mental disorders in later life, there are reports of decreasing access, diagnosis, and treatment of these patients (Ormel et al., 1991; Tiemens et al., 1998; Charney et al., 2003; Volkers et al., 2004; Verhaak et al., 2005). In about 50–80% of (elderly) patients depression is not recognised in primary care (Crawford et al., 1998; O'Connor et al., 2001b; Stek et al., 2007; May et al., 2014) or is not formally diagnosed, although GPs sometimes indicate that they are aware that patients have mental health problems (The Magpie Research group, 2005; Volkers et al., 2004). The under detection of mental disorders and social problems, such as depression, anxiety, or loneliness in the elderly, has several causes. GP-related causes include lack of knowledge and education, lack of time, and reluctance to talk about the topic (Boersma and Eefsting, 1996). Patient-related factors include presenting problems in a vague way (complaints of fatigue, appetite, and weight loss) (Ormel et al., 1991; Tiemens et al., 1998; Verhaak et al., 2006) (Bensing and Verhaak, 1994), or not mentioning these problems because of embarrassment or reluctance to recognise them as mental disorders (Docherty, 1997; O'Connor et al., 2001a). There are also indications that elderly patients present psychiatric disorders atypically (Tilburg and Beekman, 1999). Last, some mental problems, such as mood disorders, are detected more easily than others (such as anxiety or substance abuse; Verhaak et al., 2006), and moderately to very severe disorders are easier to detect than mild disorders (Ormel et al., 1991; O'Connor et al., 2001b). Although the prevalence of mental disorders and social problems in home-care populations is high, with a long duration of treatment, their under detection may be because of poorly trained and inexperienced personnel, a focus on somatic complaints with emphasis on certain signs and symptoms, and hesitancy to discuss these problems with clients (McDonald et al., 1999; Bruce et al., 2002; Brown et al., 2003). Most existing screening or diagnostic instruments for mental problems have disadvantages which limit their use in primary and home care, for example, the instruments have not been validated in these specific settings, they are time consuming and require the active participation of patients, or they are aimed at one type of pathology or require suspicion of possible pathology (Boersma and Eefsting, 1996; The Magpie Research group, 2005). In the area of dementia screening a successful short observation list is the OLD (Hopman-Rock et al., 2001; Wind et al., 2003). It was requested by the Netherlands Organisation for Health Research and Development (Zonmw) to develop and evaluate a similar instrument for early detection of mental disorders and social problems in the elderly. The aim of this study was therefore to develop and validate a short observation list that can be used to detect a broad range of mental disorders and social problems (namely, anxiety, depression, cognitive impairment, suspicion, loneliness, and somatisation) in elderly patients in a primary or home care setting.
Methods
Development and feasibility
The observation list was developed in several stages (see Flowchart: Table 1). As a first step, the literature was reviewed to identify signs and symptoms of the following mental disorders and social problems: anxiety, depression, cognitive impairment, suspicion, loneliness, and somatisation. Indicators were selected that could be observed during a home or practice visit and which were known to be manifest in elderly patients during early or mild stages of the conditions. These possible indicators were then evaluated by 22 experts (researchers and clinicians in the field of geriatrics, primary care, psychiatry, gerontology, psychology, and mental health). Their comments were used to refine the choice of indicators, resulting in a first draft of the indicator list with 36 possible signs and symptoms. The feasibility of this instrument in daily practice was then evaluated by eight GPs and two home care organisations.
| Stage | Selection criteria | |
|---|---|---|
|
Development and Feasibility |
Review of the literature | |
| ↓ | Signs and symptoms of mental disorders and social problems | |
| List of possible signs and symptoms (i = 178) | ||
| ↓ | Experts' opinion (n = 22) | |
| First draft list (i = 36) | ||
| ↓ | 8 GPs and 2 home care agencies' experience and opinion on feasibility | |
|
Preliminary Validation |
Second draft list (i = 24) | |
| ↓ | Reliability and validity. Reduction using pre-set criteria | |
| Final list (i = 14) |
Preliminary validation
In the preliminary validation study, a list with 24 indicators (remaining after the feasibility study) was used in daily practice by GPs and home care personnel. These indicators covered six different mental disorders and social problems, although they were not mentioned as such in the text that was used in the list. Observers indicated whether each sign or symptom was ‘present’, ‘doubtful present’, ‘not present’, or ‘could not be observed’. Observers were encouraged to use strategies to make signs ‘visible’, such as asking indirect questions or making small talk. Details about each visit were registered, namely, the reason for the consultation or home visit, its location (practice or at home), the length of the visit, and whether the patient was accompanied or alone. Results from this validation study were used to reduce the number of indicators to improve feasibility, validity, and reliability.
Procedure
Fifty GPs working in the city of The Hague were contacted, 12 of whom participated. These doctors had previously participated in research carried out by the Department of Public Health and Primary Care of the Leiden University Medical Center (LUMC). In the same period several home care agencies were invited to participate in the study. Nurses and carers from two agencies who wanted to join in the study participated in a group instruction meeting.
Inclusion criteria for patients were 65 years of age or older, no current psychiatric illness, and familiarity with the Dutch language. General practice patients were informed about the study by means of a poster in the waiting room, while home care patients (clients) were sent a letter about the study. Patients who met inclusion criteria and visited their GP or were visited by their home care agency during the study period were observed by their GP or nurse/carer, who completed the list afterwards. At the end of this visit, patients were given information about the study and were asked if they wanted to participate. If they agreed, an informed consent form was completed at a second visit, and they were given a questionnaire (including existing validated questionnaires) to complete. At this visit, the Mini-Mental State Examination (MMSE) (see next section) was administered by the GP or home care personnel. The LUMC Ethics Committee approved the study protocol.
Assessment
- The General Health Questionnaire 12 item version (GHQ-12) detects mental problems in general. A score of 3 or higher is indicative of mental disorder (Koeter and Ormel, 1991);
- The Geriatric Depression Scale 15-item version (GDS-15) is a self-rating scale designed to detect depression in a geriatric population. Respondents answer ‘yes’ or ‘no’ to 15 statements. A score of 6 or higher is indicative of depression (van Marwijk et al., 1995);
- The Symptom Checklist-90 (SCL-90-R) instrument evaluates a broad range of psychological problems and symptoms of psychopathology. The anxiety subscale evaluates 10 clinical symptoms of generalised anxiety, scored on a 5-point Likert scale running from ‘not at all’ to ‘very much’. Total scores range from 10 to 50, with a higher score indicating more symptoms. The Distrust and Interpersonal Sensitivity subscale evaluates 18 clinical symptoms of paranoia and suspicion and dissatisfaction with oneself (range 18–80). As dichotomous scores are not available, quartile scores were used (Arrindell and Ettema, 2003);
- The MMSE is a 30-item scale that assesses global cognitive function. A score of 23 or lower is indicative of cognitive impairment (Folstein et al., 1975);
- The University of California, Los Angeles (UCLA) Loneliness Scale detects loneliness. Twenty statements are scored on a 4-point Likert scale running from ‘never’ to ‘always’. Total scores range from 20 to 80, with a higher score indicating more loneliness. Because dichotomous scores are not available, quartile scores were used (Russel, 1996);
- The Illness Attitude Scale (IAS) assesses fears, attitudes, and beliefs associated with hypochondria and abnormal illness behaviour. For the indication of somatisation two subscales were used, namely, Health Anxiety (HA; cut-off score of 14 defines hypochondria) and Illness Behaviour (IB; quartile scores were used) (Speckens et al., 1996).
Statistical analysis
Patient characteristics observed by GPs or home care personnel are presented with means and standard deviations (SD) by group (primary care or home care) and for the entire population. Demographic and clinical characteristics were compared between primary care and home care patients using Student's t-tests for age and duration of visit and X2 tests for the other characteristics.
The psychometric properties of the draft observation list were assessed on factor structure, using exploratory principal component analysis (PCA) with Varimax Rotation, and for internal consistency, using Cronbach's α coefficient for the total list and six separate subscales (problem areas).
Convergent validity was assessed by comparing the number of observed indicators (‘present’ signs and symptoms and a combination of ‘present’ and ‘doubtful present’) and those identified with the corresponding gold standard instruments, namely, mental disorders detected with GHQ-21, depression detected with GDS-15, anxiety detected with the anxiety subscale of SCL-90, loneliness detected with the UCLA, somatisation detected with the subscales of the IAS, cognitive impairment detected with the MMSE, and suspicion detected with the subscale ‘Distrust and Interpersonal Sensitivity’ of the SCL-90. For each test a cut-off score was used (Student's t-test) if known, or otherwise normative data (analysis of variance (ANOVA) test) were used. Two-sided p-values lower than 0.05 were regarded as statistical significant.
Reduction
- more than 10% of the observations on a single indicator were scored as ‘could not be observed’;
- the value of Cronbach's α increased when the item was removed;
- there was no clear relationship (component loading < 0.40) with one of the components in the PCA;
- the entire subscale of indicators was removed if there was no significant association with the gold standard instrument (p < 0.05 on tests).
Statistical analysis was done using SPSS for Windows. We also performed receiver operating characteristic (ROC) analyses, the area under the curve (AUC), and 95% reliability intervals are reported.
Results
Development and feasibility
The experts indicated that, for use in daily practice, an observation list ideally should have preferable a maximum of 12 signs and symptoms, provide examples of behaviour, avoid vague somatic complaints such as headache, fatigue etc., and not make a distinction between primary or home care. It was questioned by the experts whether ‘suspicion’ could be viewed as a separate disorder or as a symptom of other disorders such as anxiety.
Based on feasibility in practise as evaluated by the GPs and home care organisations the initial list was reduced to 24 indicators (see Table 1).
Preliminary validation study
The second draft observation list was used by 12 GPs who observed 99 patients and by 13 home care workers (nurses and carers) who observed 81 patients. The background information of these patients is given in Table 2. Patients in home care were older, had a lower level of education, and lived alone more often than patients observed by their GP. Home care workers had more time to observe their patients and always visited them at home, whereas GPs did so in only 10% of the cases. Both types of professional observed about 75% of their patients when they were alone. Physical complaints were the most common reason for visiting the GP, while assistance with personal care was the most-mentioned reason for receiving home care. Two out of 24 indicators were categorised as non-observable (i.e. ‘loss of overview’ and ‘finding it hard to believe a doctor’).
| Primary care | Home care | All | |
|---|---|---|---|
| Age (yrs), mean ± SD | 77.4 ± 6.8 | 79.6 ± 6.7* | 78.4 ± 6.8 |
| Sex (%, female) | 64.2 | 68.8 | 66.3 |
| Education level (%) Low | 47.3 | 65.8** | 55.6 |
| Middle | 32.3 | 30.3** | 31.4 |
| high | 20.4 | 3.9** | 13.0 |
| Living situation (%, alone) | 57.4 | 83.1*** | 69.0 |
| Observations during regular consultation or visit | |||
| Number | 99 | 81 | 180 |
| Length (min) mean ± SD | 13.5 ± 4.5 | 32 ± 19.7 | 21.7 ± 16.4 |
| Length (min) range | 5–30 | 15–120 | 5–120 |
| Home visits (%) | 10 | 100 | |
| Patient/client alone (%) | 73.7 | 82.7 | 77.8% |
| Reason for visit (%) | |||
| Physical | 71.7 | ||
| Mental | 6.1 | ||
| Follow-up visit | 17.2 | ||
| Nursing | 26.3 | ||
| Care taking | 53.8 | ||
| Domestic help | 12.5 | ||
| Other | 5.1 | 7.5 | |
- * p < 0.05.
- ** p < 0.01.
- *** p < 0.001.
Reliability and validity
Cronbach's α coefficient for the total draft observation list was 0.87, and the values for four of the six predefined subscales (problem areas) were within the range of 0.78 to 0.88, with only the subscales ‘loneliness’ (0.72) and ‘suspicion’ (0.25) having lower values. A PCA was performed to identify domains covered by the 24 observation indicators. A seven-factor solution was retained according to the Kaiser criterion (Eigen values > 1), which accounted for 65.5% of the variance in the original matrix. To ease interpretation a second analysis was carried out with a forced six-factor solution, which explained 60.9% of the variance (see Table 3 for the individual component loadings). All but three component loadings of the observed 24 indicators included in the respective factors were between 0.57 and 0.82 (see Table 3); the exceptions were ‘loss of interest’ (0.47), ‘gloomy look’ (0.48), and ‘feels badly treated’ (0.47). The six factors could be characterised as cognition problems (factor 1), depression + loneliness (factor 2), somatisation (factor 3), anxiety + suspicion (factor 4), depression (indicators ‘change in eating or sleeping behaviour’ and ‘loss of interest’) (factor 5), and an ambiguous factor (‘depressed living situation’ and ‘building up a façade’) (factor 6).
| Factor | |||||||
|---|---|---|---|---|---|---|---|
| 1 | 2 | 3 | 4 | 5 | 6 | ||
| 1a | Is restless | 0.21 | 0.23 | 0.23 | 0.68 | 0.05 | −0.33 |
| 1b | Feels nervous/restless | 0.06 | 0.51 | 0.16 | 0.57 | 0.22 | −0.24 |
| 2 | Anxious in certain situations | 0.26 | 0.20 | −0.05 | 0.57 | −0.01 | 0.26 |
| 3 | Difficulty relaxing | 0.27 | 0.35 | 0.24 | 0.60 | 0.12 | −0.02 |
| 4 | Loss of interest | 0.10 | 0.36 | 0.14 | 0.20 | 0.47 | 0.19 |
| 5 | Limited ability to have fun | 0.16 | 0.76 | 0.04 | 0.02 | 0.11 | 0.04 |
| 6a | Gloomy mood | 0.15 | 0.82 | 0.19 | 0.16 | 0.01 | −0.08 |
| 6b | Looks gloomy | 0.06 | 0.48 | 0.27 | 0.44 | −0.19 | 0.27 |
| 7 | Change in eating or sleeping behaviour | 0.02 | 0.09 | 0.01 | 0.08 | 0.76 | 0.04 |
| 8 | Depressed living situation | −0.05 | 0.11 | 0.01 | 0.11 | 0.08 | 0.73 |
| 9 | Distrusts others | 0.13 | −0.09 | 0.16 | 0.65 | 0.26 | 0.29 |
| 10 | Feels badly treated | −0.25 | 0.23 | 0.02 | 0.47 | −0.35 | 0.05 |
| 11 | Is forgetfulness | 0.67 | 0.07 | −0.01 | 0.06 | 0.20 | 0.07 |
| 12 | Disorientation in time | 0.74 | −0.01 | 0.11 | 0.16 | −0.31 | 0.09 |
| 13 | Language problems | 0.79 | 0.04 | 0.19 | 0.18 | −0.09 | −0.05 |
| 14 | Building up a façade | 0.36 | 0.19 | −0.06 | −0.05 | 0.02 | 0.57 |
| 15 | Loss of overview | 0.70 | 0.21 | −0.10 | −0.02 | 0.13 | 0.04 |
| 16 | Dependency | 0.74 | 0.12 | 0.13 | 0.14 | 0.13 | 0.04 |
| 17 | Feels lonely | 0.07 | 0.66 | 0.21 | 0.14 | 0.03 | 0.17 |
| 18 | Social isolation | 0.04 | 0.59 | 0.06 | 0.24 | 0.06 | 0.23 |
| 19a | Preoccupied with physical complaints (physically) | 0.11 | 0.10 | 0.85 | 0.09 | 0.12 | −0.05 |
| 19b | Preoccupied with physical complaints (communication) | 0.04 | 0.22 | 0.80 | 0.16 | 0.04 | −0.07 |
| 20 | Worries a lot about own health | 0.12 | 0.11 | 0.78 | 0.12 | 0.15 | 0.07 |
| 21 | Finds it hard to believe a doctor | −0.06 | 0.11 | 0.71 | 0.04 | −0.34 | 0.02 |
- In bold component loadings > 0.45.
Separate analysis for primary care and home care showed some differences between them. The home care workers solution showed a relatively stronger first factor, while the GPs solution resulted in a more equally divided structure with 5 or 6 factors.
Convergent validity
Table 4 presents the results for the comparison between the draft observation list and the gold standard tests. Patients with mental problems (GHQ > 2) showed more signs than patients without these problems (Table 4), also when ‘doubtful’ observed signs were included. Similar significant results were found for patients with loneliness, depression, and cognitive impairment, who all had more signs than patients without these problems.
| Questionnaire | Number of observed indicators | Number observed indicators + doubtful |
ROC analysis Area under curve (95% confidence interval) for norm category |
||
|---|---|---|---|---|---|
| GHQ-12 | All indicators (i = 24) | ||||
| Normal (0–2) | 1.9 ± 2.8 | t = 3.1; df = 109: p = 0.003 | 4.2 ± 4.3 | t = 3.7; df = 109: p = 0.000 | |
| Psychiatric disorder (>2) | 3.8 ± 3.8 | 7.7 ± 5.1 | 0.70 (0.62–0.79) | ||
| GDS-15 | Depression indicators (i = 6) | ||||
| Non-depressed (0–5) | 0.3 ± 0.6 | t = 5.3; df = 139: p = 0.000 | 0.7 ± 1.2 | t = 5.2; df = 139: p = 0.000 | |
| Depressed (>5) | 1.2 ± 1.3 | 2.2 ± 1.8 | 0.70 (0.59–0.80) | ||
| MMSE | Cognition indicators (i = 6) | ||||
| Normal (>23) | 0.4 ± 0.8 | t = 6.6; df = 142: p = 0.000 | 0.8 ± 1.3 | t = 4.8; df = 142: p = 0.000 | |
| Cognitive impaired (0–23) | 2.1 ± 1.9 | 2.5 ± 1.9 | 0.71 (0.59–0.83) | ||
| IAS—Health anxiety | Somatisation indicators (i = 4) | ||||
| Normal (0–13) | 0.4 ± 0.8 | t = 1.4; df = 147: p = 0.160 | 0.9 ± 1.3 | t = 2.3; df = 147: p = 0.024 | |
| Hypochondriac (13) | 0.6 ± 1.0 | 1.5 ± 1.6 | 0.59 (0.49–0.69) | ||
| IAS—Sickness behaviour | Somatisation indicators (i = 4) | ||||
| First quartile (0–5) | 0.2 ± 0.7 | F = 1.605; p = 0.190 | 0.6 ± 1.1 | F = 2.5; p = 0.06 | |
| Second quartile (6–9) | 0.4 ± 0.8 | 1.0 ± 1.4 | |||
| Third quartile (10–12) | 0.6 ± 1.0 | 1.1 ± 1.4 | |||
| Fourth quartile (>13 highest problems) | 0.4 ± 0.8 | 1.5 ± 1.6 | 0.61 (0.51–0.71) | ||
| SCL-90 Anxiety | Anxiety indicators (i = 4) | ||||
| Below average | 0.3 ± 0.8 | F = 6.2; p = 0.056 | 0.6 ± 1.0 | F = 2.6; p = 0.001 | |
| Average | 0.5 ± 1.0 | 1.0 ± 1.2 | |||
| Above average | 0.6 ± 1.0 | 1.5 ± 1.2 | |||
| High | 1.0 ± 1.2 | 2.0 ± 1.6 | 0.54 (0.23–0.80) | ||
| SCL-90 Distrust and interpersonal sensitivity | Suspicion indicators (i = 2) | ||||
| Below average | 0.1 ± 0.3 | F = 0.5; p = 0.726 | 0.0 ± 0.1 | F = 0.4; p = 0.775 | |
| Average | 0.2 ± 0.4 | 0.1 ± 0.2 | |||
| Above average | 0.2 ± 0.5 | 0.0 ± 0.2 | |||
| High | 0.2 ± 0.5 | 0.1 ± 0.2 | 0.51 (0.40–0.62) | ||
| UCLA Loneliness Scale | Loneliness indicators (i = 2) | ||||
| First quartile (0–35) | 0.1 ± 0.3 | F = 2.8; p = 0.042 | 0.6 ± 0.7 | F = 3.6; p = 0.015 | |
| Second quartile (36–39) | 0.1 ± 0.2 | 0.3 ± 0.6 | |||
| Third quartile (40–45) | 0.1 ± 0.5 | 0.5 ± 0.7 | |||
| Fourth quartile (>46 highest loneliness) | 0.3 ± 0.4 | 0.8 ± 0.8 | 0.62 (0.52–0.73) | ||
Patients with hypochondria (as defined by the IAS-HA) had more signs considered ‘doubtful’ (significant difference found when included). There was no difference in observed or doubtful signs for the other subscale of the IAS, ‘Illness Behaviour Patients’. Patients with lower levels of anxiety on that last mentioned scale had borderline significantly fewer observed signs, but only if doubtful signs were included. The SCL-90 Anxiety subscale showed significant differences especially when ‘doubtful’ was included. The number of observed signs did not differ on the other SCL-90 subscale, ‘distrust and interpersonal sensitivity’ (indicator of ‘suspicion’).
Reduction
Using the pre-set criteria, the number of indicators was reduced (list of removed indicators results available on request), yielding a final list with 12 indicators (and two times subindicators) (see Table 5). For instance the subscale ‘suspicion’ was removed entirely because of the criterium ‘no significant associations with the gold standard’ as can be read from the results in Table 4. The reliability of the reduced final list was good (Cronbach's α = 0.84), the percentage of explained variance in the PCA was now almost 70% (results available on request), and four equal factors were identified, namely, depression + loneliness, hypochondria, cognition, and anxiety. Sensitivity final list was 0.72 and specificity 0.77. As can be seen in Table 6, more observed signs were positively associated with more mental problems (GHQ-12), depression (GDS-15), cognitive impairment (MMSE), hypochondria/somatisation (IAS-HA), anxiety (SCL-90), and loneliness (UCLA).
| Domain | Indicator | |
|---|---|---|
| Anxiety | 1 Seems agitated or tense | |
| a Restless behaviour (is restless) | ||
| b Restless feeling (seems to feel restless) | ||
| 2 Has difficulty relaxing | ||
| Depression | 3 Has limited ability to have fun/enjoy | |
| 4 Gloomy mood/outlook | ||
| Cognition | 5 Forgetfulness (complains of memory problems) | |
| 6 Disoriented in time | ||
| 7 Language problems | ||
| 8 Dependency of others | ||
| Loneliness | 9 Feels lonely | |
| 10 Social isolation, withdrawal | ||
| Somatisation | 11 Preoccupation with bodily complaints | |
| a Physical | ||
| b Communicative | ||
| 12 Worries a lot about own health | ||
- Preliminary norms (researchers are encouraged to do further research with this list with reference to our publication):
- Observed indicator = 1 point; when in doubt = 0.5 point (1 point in case of Loneliness or Somatisation); not present = 0 points; not observable = NA
- If sumscore ≥ 4 further diagnosis required (sensitivity 0.72, specificity 0.77)
- For depression domain score ≥ 1 further diagnosis required (sensitivity 0.68, specificity 0.81)
- For cognition domain score ≥ 2 further diagnosis required (sensitivity 0.72, specificity 0.83)
- For anxiety, loneliness and somatisation: because of low sensitivity and specificity no further action, these domains contribute to the total score.
- Possible questions to ease observation: ‘How old are you now?’ ‘What was the last time you was here’ ‘How are your grandchildren’ ‘how do you run your household’ etc.
| Questionnaire | Number of observed indicators | Number observed indicators + doubtful | ROC analysis area under curve (95% confidence interval) for norm category | ||
|---|---|---|---|---|---|
| GHQ-12 | All indicators (i = 14) | ||||
| Normal (0–2) | 1.0 ± 1.8 | t = 3.2; df = 168: p = 0.002 | 2.4 ± 2.8 | t = 4.5; df = 168: p = 0.000 | |
| Psychiatric disorder (>2) | 2.1 ± 2.4 | 4.7 ± 3.5 | 0.70(0.61–0.79) | ||
| GDS-15 | Depression indicators (i = 2) | ||||
| Non-depressed (0-5) | 0.2 ± 0.4 | t = 5.1; df = 153: p = 0.000 | 0.4 ± 0.7 | t = 4.5; df = 153: p = 0.000 | |
| Depressed (>5) | 0.7 ± 0.8 | 1.1 ± 0.9 | 0.73(0.62–0.84) | ||
| MMSE | Cognition indicators (i = 4) | ||||
| Normal (>23) | 0.2 ± 0.6 | t = 3.8; df = 151: p = 0.000 | 0.4 ± 0.8 | t = 3.2; df = 151: p = 0.002 | |
| Cognitive impaired (0-23) | 0.9 ± 1.1 | 1.0 ± 1.1 | 0.82(0.71–0.93) | ||
| IAS—Health anxiety | Somatisation indicators (i = 3) | ||||
| Normal (0–13) | 0.3 ± 0.8 | t = 1.7; df = 151: p = 0.091 | 0.8 ± 1.1 | t = 2.3; df = 153: p = 0.023 | |
| Hypochondriac (13) | 0.6 ± 1.0 | 1.3 ± 1.4 | 0.59(0.48–0.69) | ||
| SCL-90 Anxiety | Anxiety indicators (i = 3) | ||||
| Below average | 0.2 ± 0.6 | F = 2.0; p = 0.023 | 0.5 ± 0.9 | F = 5.7; p = 0.001 | |
| Average | 0.5 ± 0.9 | 0.9 ± 1.1 | |||
| Above average | 0.5 ± 0.8 | 1.2 ± 0.9 | |||
| High | 1.0 ± 1.2 | 1.6 ± 1.3 | 0.61(0.29–0.92) | ||
| UCLA Loneliness Scale | Loneliness indicators (i = 2) | ||||
| First quartile (0–35) | 0.1 ± 0.3 | F = 2.8; p = 0.042 | 0.6 ± 0.7 | F = 3.6; p = 0.015 | |
| Second quartile (36–39) | 0.1 ± 0.2 | 0.3 ± 0.6 | |||
| Third quartile (40–45) | 0.1 ± 0.5 | 0.5 ± 0.7 | |||
| Fourth quartile (>46) | 0.3 ± 0.6 | 0.8 ± 0.8 | 0.61(0.51–0.72) | ||
Discussion
The new Observation List for mental disorders and social Problems (OLP) in primary and home care proved to be reliable (Cronbach's α = 0.84), preliminary valid (significant associations with gold standards), and feasible in practise. It is possible to detect signs of depression, cognitive impairment, somatisation, anxiety, and loneliness. It is short and easy to use, taking only a few minutes to complete after the observation in a regular visit or conversation. The OLP could objectify the existing gut feeling of the observer and justify further screening.
The structure resulting from the final PCA analysis largely overlapped the pre-defined subscales, but some indicators were difficult to categorise. ‘Suspicion’ could not be detected with the chosen indicators and was related to anxiety (as already mentioned by the experts in the first phase). Indicators of somatisation were detected especially well by GPs, and results showed that these indicators were mainly associated with hypochondria. Depression and loneliness were recognised as separate problems, but factor analysis showed them to be associated, which is not uncommon (Cacioppo et al., 2007; Tiikainen and Heikkinen, 2005). Older adults consider both problems to be inherent to old age (Barg et al., 2006). Even so, passive observation on the part of the GP or carer will not detect loneliness—patients need to express feelings of loneliness in order for this problem to be detected. The OLP instruction provides suggestions to discuss this with the patient/client.
There was a clear positive association between the signs and most of the gold standards, even without inclusion of the so-called ‘doubtful’ signs. These signs were included in the analysis because doubts about the presence of signs also can arise during a consultation or home visit, as a result of time limitations or the situation at that moment. A re-evaluation of the same patient at a later occasion could clarify whether a certain sign is actually present. Also, because the observation list is intended to help observers get an overall picture of the patient rather than making a yes/no decision, doubtful signs may play a role in this process. Having a list of reliable and validated signs will help observers to evaluate patients in a more structured and objective way compared to biased, subjective, or prejudiced views (Koenig, 2007).
Limitations
It was sometimes difficult to find a reliable and valid gold standard for comparison, such as for suspicion. This certainly limited the measurement in that specific problem area. The UCLA Loneliness Scale proved to be less reliable in our sample of elderly patients and some gold standard instruments also lacked clear cut-off scores. In some primary care and home settings it was difficult to recruit patients because of refusal to participate, lack of elderly patients, or problems with the Dutch language. This led to less included subjects then foreseen and may have given also some bias in the results. As we did a preliminary validation only, we could not perform a test–retest reliability study or further elaborate on clinical utility and efficiency. We gave recommendations for cut-off points (see Table 5) and hope that future research could be done by others (the list is free to use). As an additional analysis we also gave the AUC parameters. For some domains these were rather low (<0.65). It was demanded by the sponsor of this research that as much as possible domains should be included; this restricted us from deleting too much indicators. However, all domains together contributed very well to the GHQ, and the total Cronbach's alpha was good.
Given the lack of use of instruments in current practice and problems with existing screening instruments in primary care (The Magpie Research group, 2004), new ways of evaluating patients are welcome. Our observation list is not intended as a screening or diagnostic instrument (as explained in the introduction these kinds of instruments have several restrictions), but serves to alert health care practitioners to the early presence of mental disorders and social problems, without yet specifically addressing these problems to the patient or client. A list of observable signs might overcome the problems caused by the low level of help-seeking behaviour for psychiatric disorders (Bland et al., 1997) and the poor personal insight of elderly individuals (O'Connor et al., 2001b). The increasing number of mental disorders in the population and the decreasing numbers of diagnosed patients (Jackson et al., 2007, Gonçalves et al., 2014) makes any improvement in detection welcome. Giving home care workers the possibility to systematically and objectively evaluate clients could improve their deliverance of care and prevent loss of quality of life. Owing to the growing population of elderly, a tendency to shorter hospital stays, technological advances, and other factors such as ageing policies, the number of elderly individuals receiving home care is likely to increase.
Both GPs and home care workers indicated that the instrument could be used in daily practice and that it allowed them to observe patients before focusing on specific problems. By referring to specific behaviours that they have observed, it might be easier for GPs or health care workers to bring up the possibility of mental disorders or social problems during the consultation or visit. After initial observation and possible detection of signs, standard screening or diagnostic protocols could then be used.
Conclusion
An observation list with 14 indicators in five problem areas was developed to support GPs and home care workers to detect mental disorders and social problems in their older patients and clients. The instrument is preliminary valid and reliable and can be used during a regular visit.
Conflict of interest
Key points
- Prevalence of mental disorders and social problems in older adults is high.
- Primary care and home care needed support to early detection.
- A short observation list has been developed and preliminary validated.
Acknowledgements
This study was funded by a grant from the Netherlands Organisation for Health Research and Development (ZonMw) and an unrestricted grant from Wyeth Pharmaceuticals. The authors would like to thank the advisors of the study, Marco Blom, Ernst Bohlmeijer, Jenny Gierveld, and Jeanne Beex.
Sponsored by Netherlands Organisation for Health and Development (Zonmw) project nr: 1360.0007, and an unrestricted grant from Wyeth Pharmaceuticals.
