Content validity of the Recap of atopic eczema (RECAP) instrument in Dutch, English and German to measure eczema control in young people with atopic eczema: a cognitive interview study
M.G. and A.R. contributed equally.
Abstract
Background
Recap of atopic eczema (RECAP) is a patient-reported outcome measure assessing eczema control. This instrument has been developed and validated in the UK. There are self-reported and proxy-reported versions in English, Dutch and German. However, it is unclear whether the self-reported version shows adequate content validity when completed by young people (8–16 years) in these languages.
Objectives
To assess the content validity (comprehensibility, relevance and comprehensiveness) of the English, German and Dutch versions of the self-reported RECAP in young people with atopic eczema and to identify the most appropriate age cutoff for self-completion.
Methods
We conducted 23 semistructured cognitive interviews with young people aged 8–16 years, using the ‘think-aloud’ method. In Germany and the Netherlands, participants were recruited in dermatology clinics and in the UK through social media and existing mailing lists. Interviews were audio recorded, transcribed verbatim and analysed in the three languages, using a problem-focused coding manual. Transcripts were coded by two independent reviewers in each country. Themes were translated into English and compared across the three countries.
Results
Significant age-related comprehensibility issues with the last three items of the questionnaire occurred with young people aged 8–11 years, causing difficulties completing RECAP without help. However, older children had only minor problems and were able to complete the questionnaire by themselves. The self-reported version of RECAP has sufficient content validity for self-completion in young people aged 12 years and above. However, the German version with some translational adaptations may be appropriate for children from the age of 8 years. There may be some situations where the proxy version is needed for older children too.
Conclusions
The self-reported version of RECAP is appropriate for use from the age of 12 years. The proxy version can be used in children younger than 12 years. Other measurement properties should be further investigated.
- Recap of atopic eczema (RECAP) is an instrument recommended by the Harmonising Outcome Measures for Eczema initiative for the core outcome domain of long-term control of atopic eczema.
- Content validity of RECAP for self-completion by adults and of the proxy version has been assessed.
- In this study, content validity (comprehensibility, relevance and comprehensiveness) of the self-reported version of RECAP among young people (aged 8–16 years) with atopic eczema across the UK, Germany and the Netherlands is assessed.
- Based on these findings, key recommendations on how to measure eczema control in young people with atopic eczema are formulated.
- The Dutch, English and German self-completion versions of RECAP are recommended for use in adolescents from the age of 12 years.
- The proxy version could be used in children younger than 12 years or where children are cognitively or physically incapable of reporting their experience of eczema control.
- Caregivers should be encouraged to complete RECAP together with their child where possible.
Atopic eczema (AE) is a common, inflammatory skin disease affecting both children and adults.1 A lifetime prevalence of 15–30% is reported for children and 2–10% for adults.2 The incidence of AE has increased over the past decades in industrial countries.3, 4 AE is a chronic relapsing condition, typically characterized by periods of flare and remission. In terms of AE, we talk about flares, which are periods of high disease activity. These ‘uncontrolled periods’ are associated with higher disease burden.5 In addition to lowering AE-related symptoms, treatment aims to reduce the intensity and quantity of flares. Assessing how well the eczema is controlled is therefore an important outcome when evaluating the efficacy of treatments.6
The patient’s perspective can be captured using patient-reported outcome measures (PROMs).7 For the assessment of the experience of eczema control, a new PROM called Recap of atopic eczema (RECAP) has been recently developed and validated in the UK.8 RECAP is recommended by the Harmonising Outcome Measures for Eczema (HOME) initiative as part of the core outcome set for AE.9 The questionnaire consists of seven questions with five response options each. Currently self-completion and proxy-reported versions are available and validated, using adults and parents of affected children.10-13 However, discrepancies between proxy- and self-reported PROMs in young people have been described in other clinical areas.14, 15 In addition, self-reporting could improve engagement and treatment adherence in young people as they become partners in their treatment.16, 17 Therefore, self-completion of RECAP is preferred. However, it is unclear whether the self-reported version shows adequate content validity when it is completed by young people with AE.
With this study, we aim to fill the content validation gap (comprehensibility, relevance and comprehensiveness) of the English, German and Dutch version of self-reported RECAP in young people with AE. Specific objectives of this study were (i) to assess the content validity (comprehensibility, relevance and comprehensiveness) of the English, German and Dutch versions of the self-completed RECAP instrument when completed by young people and (ii) to identify the most appropriate age cutoff for self-completion of the self-reported RECAP to ensure comprehensibility of the instrument.
Patients and methods
Participants and study design
We aimed to recruit at least five young people (age 8–16 years and affected by AE) per language (English, German, Dutch) for a qualitative study using semistructured cognitive interviews. Most children aged 8 years and over have the ability to read. In agreement with this, the self-completed version of the widely used EQ-5D-Y (EuroQol 5 Dimensions Youth) health status instrument is appropriate for those aged 8–15 years, while for children aged 4–7 years, the proxy version can be used.18 Written informed consent was given online by the parents or primary caregivers of the participating young people. Interviews were conducted by telephone or video call. One parent or caregiver was required to be present during the interview. He or she was just sitting next to the child and was instructed to be quiet. The background of the study was explained to the participants. Questions could be asked by the participants at any time during the interview. The interviews were audio recorded and transcribed verbatim.
Participants were asked to complete the questionnaire while reading out loud and saying what they were thinking about when trying to answer the questions. This ‘think-aloud method’ is a qualitative technique that provides insights into cognitive processes.19 An interview guide including probing techniques was used to structure the interview (Appendix S1; see Supporting Information). Aspects of comprehensibility, relevance and comprehensiveness (regarding the instructions, items, response options and recall period) to young people were assessed separately for each language.20 As the originally developed English version of the RECAP instrument is already finalized and in use, changes to the items included in the questionnaire were only considered if the problem was deemed to be significant and reoccurred across the interviews.20 The duration of the interviews was approximately 20–30 min. A voucher of €10 or £10 was sent as an inconvenience allowance to the participants.
In Germany and in the Netherlands, parents and primary caregivers of young people with AE were recruited in dermatology clinics. In the UK, participants were recruited through existing mailing lists with consent to contact and through social media. If a participant was recruited through social media, self-report of a doctor’s diagnosis of AE was used to confirm eligibility. Purposive sampling was used to ensure that young people of a range of different ages were recruited. All participants (except for one German girl) were native speakers.
Ethical approval to conduct this study in each country was obtained from the ethics committees of the participating institutions (Netherlands: MEC-2020–0417; Germany: 19-1521-101; UK: FMHS 18-1805).
Recap of atopic eczema questionnaire
RECAP is a seven-item questionnaire including items such as overall eczema control, itch frequency, itch intensity and impact on sleep, each with five response options. It is currently available in several different languages; however, it has not been validated in every language so far.21 The German adult and proxy versions of RECAP were obtained by translating the original RECAP using forward and backward translation for linguistic validation, with a subsequent cognitive debriefing to ensure content validity.12 The same was applied for the Dutch version. Due to the fact that in German and Dutch, children and young people are addressed differently from adults, unlike in English, an ‘informal’ version of RECAP in German and Dutch was created by replacing the formal pronoun with its informal equivalent. This replacement is not expected to alter the main content of the instrument in any way. This is only to make the instrument more suitable for the target population. One question of each language can be found in Appendix S2 (see Supporting Information).
Analysis
Transcripts were analysed using a problem-focused coding manual (Appendix S3; see Supporting Information). ATLAS.ti (https://atlasti.com), NVivo (QSR International, Doncaster, Australia) and/or Excel sheets were used to code the transcripts and summarize the results. After the transcripts were coded by two independent reviewers in each country, the data were analysed by six researchers (A.B., L.H., G.K., M.G., A.R., J.A.F.O.) experienced in qualitative research. The analysis of the data was conducted in the same language in which the interview took place. Themes were translated and compared across the three countries. The comments of the young people on the individual items of RECAP were evaluated, and based on these findings the items were assessed in relation to comprehensibility, comprehensiveness and relevance. If an issue with an item occurred, the reviewers classified it as either a minor or a major problem. When young people stated having problems with understanding specific words but were able to complete the question by themselves it was rated as a minor problem as this issue was rather negligible. Issues with the items were only rated as a major problem if explicit comments about rewording were made and/or if the young people had difficulty answering the question on their own. Additionally, the reviewers rated an issue as major when they wanted to discuss a question considering this item with the research team. In general, all results were discussed within the research team.
Results
Demographics
In total we recruited 23 young people from the three countries. We conducted seven, seven and nine interviews in the UK, the Netherlands and Germany, respectively (Table 1). Overall, the mean age of the young people was 10·7 years (SD 2·7) with a range of 8–16 years. Ten of the 23 participants were female (43%).
| UK (n = 7) | Netherlands (n = 7) | Germany (n = 9) | Total (n = 23) | |
|---|---|---|---|---|
| Age (years) | ||||
| 8–11, n | 5 | 3 | 7 | 15 |
| 12–16, n | 2 | 4 | 2 | 8 |
| Range | 8–15 | 8–16 | 8–14 | 8–16 |
| Sex | ||||
| Female | 2 | 3 | 5 | 10 |
| Male | 5 | 4 | 4 | 13 |
| Ethnicity | White 5, Asian 2 | White 3, mixed 2, black 1, Arab 1 | Not reported (one girl non-native) | Not available |
Relevance
All items on the RECAP questionnaire were considered to be relevant by the participants. In the UK, the response options were difficult for three young people because there were either too few options to choose from or they had problems deciding what to answer. For item 5 (‘Over the last week, how much has your eczema been getting in the way of day to day activities?’), item 6 (‘Over the last week, on how many days has your eczema affected how you have been feeling?’) and item 7 (‘Over the last week, how acceptable has your eczema been to you?’) there were minor problems stated by three young people, because they considered the items as overlapping or not related to eczema. In the Netherlands, only one child stated that item 7 was not considered relevant, because this skin disease was not acceptable to anyone. In Germany, no problems regarding relevance were observed. As all of these stated problems were minor and only occurred with a few young people, the reviewers reached a consensus to not recommend the removal or changing of these items.
Comprehensiveness
Regarding comprehensiveness only one minor problem occurred. In the UK, one child suggested including an additional question about ‘skin picking’, a disorder characterized by repetitive and compulsive scratching or picking at the skin, to which dermatological conditions such as AE may contribute.22 In the Netherlands and Germany, no mentionable problems emerged for the comprehensiveness of RECAP. As only one child wanted to add a question, the research team agreed that no further changes should be recommended.
Recall period
A recall period of 1 week was considered to be appropriate by all participants. Furthermore, there were no issues during the think-aloud process regarding the recall period. This means that the young people were able to accurately recall 1 week when answering the question.
Comprehensibility
In the UK, the interviews did not identify any issues that appeared to warrant recommending a change to the original scale; however, the study did identify issues around comprehensibility that appeared to be age related. Minor and major problems for the young people occurred with items 6 and 7 (Table 2). As item 6 was also a relevant problem in the German interviews and was rather problematic for younger children, the team decided that this issue appeared to be age related. Also, item 7 was decided by the reviewers to be an age-related problem for younger children.
| Type of problem | Age (sex) | Examples | |
|---|---|---|---|
| Instructions | Minor | 8 years (female) |
Mother: Did you understand this bit where it says the questions below provide a snapshot of your eczema? Do you understand that bit? Participant: No (female, 8 years) |
| Item 6 | Minor | 11 years (male), 13 years (male) |
Interviewer: So, what is your answer? Participant: I’m not sure (male, 13 years) |
| Item 7 | Major | 8 years (female), 11 years (male) |
Interviewer: Do you know what it means? Participant: No (female, 8 years) |
The results of the interviews in the Netherlands are depicted in Table 3. The title, item 3 (‘Over the last week, on how many days has your skin been intensely itchy because of your eczema?’) and item 5 were rated by the reviewers as minor problems. However, these problems could be neglected, because only a few young people had minor problems with understanding those, while item 7 and the response options were very difficult for the young people to comprehend. As already discussed for the UK, item 7 was deemed to be an age-related problem and therefore should not be altered. The response options were only problematic for item 7 because the participants did not understand the word ‘acceptabel’ (acceptable). As these problems only occurred for this specific item it was decided to not alter the response options.
| Type of problem | Age (sex) | Examples | |
|---|---|---|---|
| Title | Minor | 8 years (male), 15 years (female) | Participant: What is ‘atopic’? (male, 8 years) |
| Item 3 | Minor | 12 years (female) | Participant had difficulty estimating symptom severity (female, 12 years) |
| Item 5 | Minor | 9 years (male) | Participant: What are ‘bezigheden’ (day to day activities)? (male, 9 years) |
| Item 7 | Minor | 8 years (male), 8 years (male) | Participant thinks ‘acceptabel’ (acceptable) is a difficult word (male, 8 years) |
| Item 7 | Major | 9 years (male) | Participant does not know the meaning of ‘acceptabel’ (acceptable) (male, 9 years) |
| Response options | Major | 8 years (male), 9 years (male) | Participant does not know meaning of ‘acceptabel’ (acceptable) (male, 8 years) |
For Germany, the results of the interviews are depicted in Table 4. Some minor problems occurred with item 1 (‘Over the last week, how has your eczema been?’), item 3 and item 7. As these problems were only stated by a few young people, these issues are rather negligible. The young people had major issues understanding the title of the questionnaire, item 4 (‘Over the last week, how much has your sleep been disturbed because of your eczema?’), item 5 and item 6. Regarding the title, the sex-specific term ‘Patient/innen’ (male and female patients) was difficult to understand for the young people. For this reason, the questionnaire was renamed as ‘Fragebogen für Kinder und Jugendliche mit Neurodermitis’ (‘RECAP for children and adolescents with atopic eczema’). This alteration does not change the meaning, but it is more comprehensible for the young people. As the participants did not understand the translation of the word ‘disturbed’ (item 4) this word was altered into ‘gestört’, which is a more easily understandable translation for ‘disturbed’. Regarding item 5, the translation of ‘getting in the way of’ was slightly simplified. The same goes for item 6, as the word ‘affected’ was changed to a more comprehensible expression in German. We have paid great attention to making these adaptations conceptually equivalent to the original version. All these changes were discussed within the German research team with the help of a primary school teacher and paediatric linguist (D.G.). Therefore, these changes should now be comprehensible for the majority of young people from the age of 8 years, and the applied adaptions should not affect the meaning of the items.
| Type of problem | Age (sex) | Examples | |
|---|---|---|---|
| Title | Minor | 9 years (female), 9 years (female), 10 years (male) | Participant stalled while reading ‘Patient/innen’ (patients) and needed explanation from parent (female, 9 years) |
| Major | 8 years (male), 10 years (male) | Participant: I don’t know what ‘Patient/innen’ means (male, 10 years) | |
| Item 1 | Minor | 10 years (male), 10 years (male) |
Interviewer: Do you know the word ‘beurteilen’ (judge)? Participant: Not so well (male, 10 years) |
| Item 3 | Minor | 9 years (female), 9 years (female), 9 years (female) | Interviewer had to explain to the participant the difference between item 2 and item 3 (female, 9 years) |
| Item 4 | Minor | 9 years (female) | Participant had problems understanding the word ‘beeinträchtigt’ (disturbed) (female, 9 years) |
| Major | 8 years (female) |
Interviewer: What do you not understand? Participant: ‘Beeinträchtigt’ (disturbed) (female, 8 years) |
|
| Item 5 | Minor | 9 years (female), 12 years (male) | Participant struggled with the term ‘alltägliche Aktivitäten’ (day to day activities) but actually understood it very well (male, 12 years) |
| Major | 8 years (female), 8 years (male), 9 years (female), 10 years (male) |
Interviewer: Do you know what ‘alltägliche Aktivitäten’ (day to day activities) means? Participant: No (male, 8 years) |
|
| Item 6 | Major | 8 years (female), 9 years (female), 10 years (male) | Participant: I don’t understand the word ‘beeinflusst’ (affected) (female, 8 years) |
| Item 7 | Minor | 14 years (female) | Participant struggled with the word ‘klarkommen’ (in ‘how acceptable has your eczema been to you?’) (female, 14 years) |
In summary, only minor changes were made to the questionnaire. Some major problems (Table 5) were identified for young people between the age of 8 and 11 years and therefore we recommend that the RECAP proxy questionnaire is used for children under 12 years. The translational changes that were made in Germany were related only to language-specific issues and did not change the meaning of the questions in any way. All changes were only made in order to enhance the comprehensibility of the PROM. Due to these changes, the German child version of RECAP may be used in young people from the age of 8 years.
| Age range | Number of participants | Examples | |
|---|---|---|---|
| Title | 8–10 years | 2 | Participant: I don’t know what ‘Patient/innen’ means (male, 10 years) |
| Item 4 | 8 years | 1 |
Interviewer: What do you not understand? Participant: ‘Beeinträchtigt’ (disturbed) (female, 8 years) |
| Item 5 | 8–10 years | 4 |
Interviewer: Do you know what ‘alltägliche Aktivitäten’ (day to day activities) means? Participant: No (male, 8 years) |
| Item 6 | 8–10 years | 3 | Participant: I don’t understand the word ‘beeinflusst’ (affected) (female, 8 years) |
| Item 7 | 8–11 years | 3 | Participant does not know the meaning of ‘acceptabel’ (acceptable) (male, 9 years) |
| Response options | 8–9 years | 2 | Participant does not know meaning of ‘acceptabel’ (acceptable) (male, 8 years) |
Discussion
In this study, we assessed the content validity (comprehensibility, relevance and comprehensiveness) of the self-reported version of RECAP among young people with AE across the UK, Germany and the Netherlands. No comprehensibility issues were reported in adolescents above the age of 12 years. These children only had minor problems with the questionnaire and were able to fully complete it by themselves. Children younger than 12 years old reported problems with several items of RECAP and were thus unable to complete the questionnaires by themselves. In addition, all items and response options were considered relevant. Finally, children and adolescents did not report problems with comprehensiveness.
In our study, children below the age of 12 years reported difficulty understanding several terms that led to an inability to complete RECAP without help. These terms included the ‘day to day activities’ (item 5), ‘affected’ (item 6) and ‘acceptable’ (item 7). Interestingly, when explaining the terms ‘day to day activities’ (item 5) or ‘affected’ (item 6), children could understand these items and were able to provide an answer. This suggests a problem with the vocabulary of the children and not with the construct of these items. Adding an example would help children to understand these items. However, adding examples to the questionnaire leads to a restriction of the construct that each item is trying to capture and is therefore not preferable. As these items are designed to leave room for individual interpretation, adding examples could restrict the patients in doing so. Furthermore, we did not want to introduce issues of cross-cultural validity by including inappropriate examples.
A more pragmatic approach would be to encourage children and their caregivers to complete RECAP together. This provides children the opportunity to report their perspective of eczema control, without restricting the construct that is measured. Difficulties with the term ‘acceptable’ (item 7) could be more complex. Although none of the children possessed understanding of the term ‘acceptable’, explanation of the meaning of this item did not result in the ability to complete this item in all children younger than 12 years old. ‘Acceptability’ could be a more complex concept that requires greater abstraction ability, which is not yet present in young children.23 However, only limited struggles with this item were reported in the German version of RECAP, which uses a specific term ‘klarkommen’ (get along, cope). This would suggest a problem with linguistic comprehension instead of a problem in abstraction ability. Creating a new child version of RECAP could be an option. However, for uniformity purposes a single version of RECAP that captures exactly the same construct in all age groups should be pursued.
With the increasing number of potential treatment options for young people with AE, it becomes more important to assess effectiveness in ways important to young people.24 In addition to measuring patient-reported symptoms and quality of life, patients and professionals recently agreed that long-term control should be a core outcome for all AE trials.9 The added value of capturing young people’s own reported outcomes is known in paediatrics and is underlined by the US Food and Drug Administration.17, 25 In our study, we found that adolescents had no problems completing RECAP, while most younger children struggled with completing it by themselves. Self-completion of RECAP by adolescents and capable children provides clinicians and researchers with better information on perceived control over AE. For children with AE, this means that their care providers can better inform them how their peers perceived the effectiveness of treatment options, which can help with the shared decision process. In addition, self-completion promotes patient engagement and could therefore lead to greater treatment adherence.16, 26
A strength of this study is its multinational, multilingual approach to the content validity of RECAP among young people. Additionally, in accordance with the COSMIN criteria for good content validity studies, we included at least seven participants per language, and a topic guide was used during the cognitive interviews, making our findings more robust.20 A limitation of our study is the lack of information on AE severity and the educational and cognitive levels of the included participants, which may influence both relevance and comprehensiveness. However, several approaches were used to recruit patients from both dermatology clinics and the community, which should have ensured inclusion of people with a range of eczema severities. However, as the study population was recruited differently for the UK than for the Netherlands and Germany, this might have also influenced the results (e.g. a better understanding through patient education at the dermatological departments). This study only assessed the content validity of the languages German, English and Dutch, and it is possible that further studies are required in other languages.
With the increasing number of trials in children and the movement of clinicians to capture patient-reported effectiveness of treatment in clinical settings,27, 28 it is important to use validated and reliable outcome measures. RECAP, alongside another instrument called the Atopic Dermatitis Control Tool, is recommended by the HOME initiative as a core outcome measurement instrument for long-term control in AE.9, 29 Based on our findings, RECAP could be recommended as an outcome measure for long-term control in young people. In general, the self-reported version of RECAP is likely to be appropriate for children aged 12 years and older. Additionally, the German version will probably be understood by children of lower ages (8 years and older) due to the linguistic changes. Nevertheless, in all three languages, there might be some situations where the proxy version is needed for older children as well. Furthermore, as children below the age of 12 years reported several comprehensibility issues with RECAP, the proxy version should be used in children younger than 12 years or when children are cognitively or physically incapable of reporting their experience of eczema control. If there are any doubts from the parents' side that their child is not capable of self-completing the questionnaire, the proxy version should be used instead. This should be decided individually with the involvement of the parents. While using the proxy version of RECAP, we would encourage caregivers to complete RECAP together with their child for optimal assessment of perceived eczema control.
Further research is necessary to investigate the validity, responsiveness, reliability and interpretability of RECAP among different populations and age groups. Uptake of the HOME initiative core outcome set is needed to enable trial data to be compared and combined in meta-analyses. For successful implementation of the HOME initiative core outcome set, it is important that future clinical trials include HOME instruments such as RECAP. Trials involving children and young people now have guidance available on which version of RECAP to use.
In conclusion, RECAP is an outcome measure to capture ‘eczema control’ that can be used among all age groups. It can be used by proxy in children younger than 12 years and self-reported by adolescents and adults.
Acknowledgments
We thank Max van der Klooster for his help with transcribing the Dutch interviews. Additionally, we thank Madelon Bronner, Rosalie Baardman and Margreet Romeijn for their contribution to the translation process of the Dutch RECAP and interviews. We also thank all the participants in the think-aloud interviews. Open Access funding enabled and organized by Projekt DEAL.
Funding sources
None.
Conflicts of interest
C.A. has received institutional funding and consultancy fees from Dr Wolff GmbH and consultancy fees from Sanofi, LEO Pharma and Rheacell. C.A., K.S.T. and L.H. were involved in the original development of RECAP, and all three are members of the executive committee of the HOME initiative. L.H. has acted as a consultant for the University of Oxford on an educational grant funded by Pfizer, unrelated to the submitted work.
Ethics statement
The ethics committees of all participating institutions approved this project (Netherlands: MEC-2020-0417; Germany: 19-1521-101; UK: FMHS 18-1805).
Open Research
Data availability
The data are available upon request from the corresponding author.
