myTREEHOUSE Self-Concept Assessment: preliminary psychometric analysis of a new self-concept assessment for children with cerebral palsy
Abstract
Aim
To evaluate the preliminary validity and reliability of the myTREEHOUSE Self-Concept Assessment for children with cerebral palsy (CP) aged 8 to 12 years.
Method
The myTREEHOUSE Self-Concept Assessment includes 26 items divided into eight domains, assessed across three Performance Perspectives (Personal, Social, and Perceived) and an additional Importance Rating. Face and content validity was assessed by semi-structured interviews with seven expert professionals regarding the assessment construct, content, and clinical utility. Reliability was assessed with 50 children aged 8 to 12 years with CP (29 males, 21 females; mean age 10y 2mo; Gross Motor Function Classification System [GMFCS] level I=35, II=8, III=5, IV=1; mean Wechsler Intelligence Scale for Children - Fourth Edition [WISC-IV]=104), whose data was used to calculate internal consistency of the scale, and a subset of 35 children (20 males, 15 females; mean age 10y 5mo; GMFCS level I=26, II=4, III=4, IV=1; mean WISC-IV=103) who participated in test–retest reliability within 14 to 28 days.
Results
Face and content validity was supported by positive expert feedback, with only minor adjustments suggested to clarify the wording of some items. After these amendments, strong internal consistency (Cronbach's α 0.84–0.91) and moderate to good test–retest reliability (intraclass correlation coefficient 0.64–0.75) was found for each component.
Interpretation
The myTREEHOUSE Self-Concept Assessment is a valid and reliable assessment of self-concept for children with CP aged 8 to 12 years.
What this paper adds
- myTREEHOUSE is a population-specific assessment which offers a unique evaluation of self-concept.
- myTREEHOUSE is valid and reliable for children with cerebral palsy.
This article is commented on by Causgrove Dunn on page 573 of this issue.
Abbreviation
-
- WISC-IV
-
- Wechsler Intelligence Scale for Children - Fourth Edition
According to a number of systematic reviews and meta-analyses, children with a physical disability or chronic illness have lower self-concept than children with typical development; including children with cerebral palsy (CP),1 developmental coordination disorder,2 asthma, diabetes, epilepsy, and juvenile arthritis.3 However, concerns have been raised over the meaning of these findings, as all have used self-concept assessments designed for children with typical development. Our systematic review4 and others5, 6 have indicated that self-concept assessments designed for children with typical development are unsuitable for children with disabilities. In addition, these assessments may present children with constructs that do not reflect their differing life experiences, or omit other important constructs. In support of this, our Delphi consensus survey7 showed that while clinicians/researchers, parents, and children with CP agreed that some items from commonly used self-concept assessments for children with typical development were appropriate for children with CP, they also proposed several additional items critical for reflecting self-concept of children with CP. Measures specific to CP are internationally supported for accurate assessment of quality of life,8 gross motor,9 and upper limb performance.10 We therefore propose this approach as a mechanism to improve self-concept assessment for children with CP.
In response to this challenge, we designed a new CP-specific self-concept assessment, called the ‘myTREEHOUSE Self-Concept Assessment’ (myTREEHOUSE). myTREEHOUSE is based on the developmental theories of ‘self’ that propose that self-concept development is dependent on the interaction between socialization experiences and cognitive processes.11, 12 In the paediatric context, Harter11 proposed that cognitive development impacts two areas of self-development. First, ‘differentiation’ involves the ability to compare one's performance between the real and ideal self. Second, ‘integration’ involves the ability to conceptualize one's overall self-worth based on various performance areas. (It is important to differentiate self-concept from self-esteem. Although still debated, self-esteem is commonly defined as the evaluation of self.13) In contrast, self-concept is the (evolving) product of this evaluation. Applying the developmental theory of ‘self’, we developed myTREEHOUSE to provide a child-reported evaluation of self-concept across three Performance Perspectives: (1) Personal – children's evaluation of their performance against their own personal standards; (2) Social – children's evaluation of their performance against the performance of their peers; and (3) Perceived – children's perception of how significant others like parents or teachers might evaluate their performance.
myTREEHOUSE then assesses each Performance Perspective across eight domains, including five domains from assessments for children with typical development13 that have been employed for children with CP,4 and three additional CP-specific domains derived from our Delphi consensus survey.7 The domains identified from existing assessments that were supported by Delphi survey participants are Physical Appearance, Learning Abilities, Social Skills, Emotional Regulation, and General Self. The new CP-specific domains are Physical Abilities, Personal Agency, and Ability to Participate.
In addition to evaluating performance, myTREEHOUSE also asks children to provide an Importance Rating – where children rate the importance they place on each item. Items viewed as more important are rated higher.11 Using the Importance Rating, a Personal Concern Score is derived, which highlights the difference between each child's Personal Performance Score and their Importance Rating, and flags key items or domains that are adversely impacting the child's self-concept. Self-report on constructs of self-concept is considered effective from the age of 8,11 which is the lower limit we recommend for using myTREEHOUSE.
This paper provides a preliminary psychometric analysis of the myTREEHOUSE Self-Concept Assessment for children with CP aged 8 to 12 years. Objectives were to evaluate its (1) validity, including face and content validity, and (2) reliability, including internal consistency and test–retest reliability.
Method
Ethical approval was obtained from the National Health & Medical Research Council registered Human Research Ethics Committees of the Cerebral Palsy League (EC00417) and the Australian Catholic University (EC00205).
Administration of the myTREEHOUSE Self-Concept Assessment
myTREEHOUSE includes 26 items divided into eight domains. Each item is linked to three statements or questions that enable the child to rate their performance across the three Performance Perspectives, for example: (1) Personal – ‘I am good at learning things’; (2) Social – ‘I can learn things as well as other children’; and (3) Perceived – ‘Would other people think you are good at learning things?’ (Table 1). An additional question for each item seeks the child's Importance Rating for the item: ‘Is it important to you that you are good at learning things?’
Domains | Items | Question/statement format | Order of questions present |
---|---|---|---|
Physical Abilities | Gross motor ability | I am good at moving around | 2 |
Fine motor ability | I can do things well with my hands | 10 | |
Oral motor ability | I can speak clearly | 18 | |
Physical Appearance | Appearance of my whole body | I have a good looking body | 5 |
Appearance of my facial features | I have a nice looking face | 13 | |
Appearance when I move | I look good when I move around | 21 | |
Learning Abilities | Learning ability | I am good at learning things | 4 |
Memory ability | I am good at remembering things | 12 | |
Problem solving ability | I am good at working things out | 20 | |
Social Skills | Interacting with others | I get along with people | 1 |
Making new friends | I can make new friends easily | 9 | |
Keeping friends | I can keep friends | 17 | |
Communicating with others | I am good at telling people what I want | 25 | |
Emotional Regulation | Behaviour management | I am well behaved | 6 |
Emotional reactivity | I do not get too upset about things | 14 | |
Emotional regulation | I know how to make myself feel better when I am upset | 22 | |
Personal Agency | Trying my best | I always try my best | 3 |
Setting my own goals | I can achieve what I want | 11 | |
Making my own decisions | I am able to make my own decision | 19 | |
Ability to Participate | Participation in outdoor activities | I am good at doing things outside | 8 |
Participation in indoor activities | I am good at doing things inside | 16 | |
Participation in self-care activities | I can look after myself well | 24 | |
Participation in school activities | I do well at school work | 26 | |
General Self | Desire to change myself | There is not many things I would change about myself | 7 |
Being a good person | I am a good person | 15 | |
Liking myself | I am happy with myself | 23 |
Two administration methods are available: a Questionnaire Version suitable for older children, and a Game Version suitable for children with lower reading skills. For both methods, presented items are identical. The Game Version is structured around a game board depicting a picture of a treehouse that ‘belongs’ to the child (i.e. myTREEHOUSE). The treehouse is reached by a universally accessible ramp with four levels leading up from the ground. Each level of the ramp represents a successively more positive rating. The upper two ramps are accompanied by pictures of ‘thumbs up’ to help children orient their responses with high agreement. The lower two ramps have pictures of ‘thumbs down’ to orient towards low agreement. Items are presented verbally, accompanied by a 3cm2 picture card. The child rates each question by placing the picture card on the ramp of choice.
Each item is scored using a 4-point rating scale. The response options for statements in the Personal and Social Performance Perspectives are as follows: ‘a lot like me’ (4 points); ‘a bit like me’ (3 points); ‘not really like me’ (2 points); and ‘not like me at all’ (1 point). The response options for questions in the Perceived Performance Perspective and the Importance Rating are as follows: ‘really yes’ (4 points); ‘sort of yes’ (3 points); ‘sort of no’ (2 points); and ‘really no’ (1 point).
myTREEHOUSE produces two types of scores: the Performance Perspective Scores and a Personal Concern Score. The Performance Perspective Scores are a summation of scores within each of the Performance Perspectives. Higher scores indicate higher self-concept from that perspective, with a possible score range of 26 to 104 for each perspective. The Personal Concern Score is calculated by first subtracting the Personal Performance Perspective Score from the Importance Rating at the item level. Negative scores are assigned a value of zero to avoid cancelling the effect of other positive scores. The subtracted scores are then summed to obtain the Personal Concern Score; with a possible score range of 0 to 78. Higher scores indicate greater concerns; for example, an item with a high Importance Rating (‘really yes’=4 points) but low Personal Performance Perspective Score (‘not like me at all’=1 point) will obtain a high Personal Concern Score (3 points).
Validity Evaluation phase
This study phase aimed to test the face and content validity of myTREEHOUSE. These evaluations included the relevance of the items to assess self-concept and representativeness of the items for children with CP.
Participants
Purposive sampling was used to recruit an expert reference panel for validity checking who were (1) psychologists or other allied health professionals working for Australian CP service organizations or universities, (2) with training or experience in measuring self-concept, (3) with children with CP. In addition, a senior speech pathologist, experienced with children with CP, was recruited to evaluate the language presented in the draft assessment.
Procedure
Semi-structured interviews were conducted by the first author (SKC) with each expert, either face-to-face, via Skype, or teleconference. During the 60-minute interview, participants were guided through a set of PowerPoint slides that described the assessment and prompted evaluation of (1) assessment constructs, (2) item content, (3) language and phrasing of items, and (4) clinical utility of the assessment. At the end of the interview, a summary of responses was checked with the participant. Each interview was voice recorded and transcribed by the first author (SKC) to enable later analysis. Responses were collated and content analysis performed to identify common themes. Changes recommended by the majority of experts resulted in a change to the assessment. In addition, changes suggested by a minority were considered on a case-by-case basis by the authors. Finally, the speech pathologist reviewed the appropriateness of the language level with respect to the expected level and range of language comprehension in the target population.
Results of the Validity Evaluation phase
Six psychologists and one paediatric physiotherapist participated. A good spread of experience and recent clinical training was obtained. Three participants had over 10 years experience, one reported between 5 years and 10 years experience, and three had less than 5 years experience. Participants responded positively to the overall proposed assessment and strongly supported the dual administration method. Some suggestions were made to improve sentence structure and presentation of individual items to increase ease of understanding. Feedback from the speech pathologist was used to increase readability. Recommendations for new items and changes to existing items are reported in Appendix S1 (online supporting information). After implementation of these changes, a final assessment was produced (Table 1).
Reliability Evaluation phase
This study phase aimed to assess the internal consistency and test–retest reliability of myTREEHOUSE.
Participants
Participants were recruited from a state-wide community rehabilitation service in Queensland, Australia. Letters were sent to all children who (1) had a diagnosis of CP, and (2) were aged between 8 years and 12 years. Children with parental consent to participate, were then screened for inclusion criteria of (3) cognitive function, determined as a Verbal Comprehension Index of 70 or above on the Wechsler Intelligence Scale for Children - Fourth Edition (WISC-IV),14 and (4) functional communication skills, determined as levels I to III on the Functional Communication Classification System (FCCS).15
Measures
In addition to myTREEHOUSE, the following measures were included. Parents completed a questionnaire about their child's demographic characteristics and CP classifications, including their Gross Motor Function Classification System - Expanded & Revised (GMFCS - E&R) level,16 Manual Ability Classification System (MACS) level,17 and FCCS level.15
The WISC-IV14 assesses intelligence for children aged 6 years to 16 years 11 months using five composite scores ranging from 40 to 160 that represent intellectual functioning. In this study, screening was performed with the Verbal Comprehension Index, following the recommendations of Yin Foo et al.,18 to eliminate the negative impact of poor fine motor performance for children with CP. The Verbal Comprehension Index has a reliability coefficient of 0.94 and stability coefficient of 0.89.14
Procedure
An information sheet and an expression of interest reply slip was mailed to eligible families, and a follow-up phone call was provided 2 weeks later. Before participation, written consent was gained from caregivers and assent was gained from children (either verbally or in writing).
At first administration, caregivers completed the demographic questionnaire while their child completed myTREEHOUSE and WISC-IV assessments with the first author (SKC). For myTREEHOUSE, children with sufficient reading ability were provided with the option of completing the Questionnaire Version or playing the Game Version; otherwise the Game Version was presented. The retest for myTREEHOUSE was conducted 14 to 28 days after first administration (mean 18d, SD 3.94d). As the Game Version was novel, participants who used this version were asked to provide their opinion about the suitability of this administration method.
Statistical analysis
Data were analysed using SPSS (version 22; IBM Corp., Armonk, NY, USA). Internal consistency was calculated using Cronbach's α. A moderate Cronbach's α of between 0.70 and 0.90 was taken to indicate strong internal consistency.19 Test–retest reliability was calculated using intraclass correlation coefficients (ICC). Each ICC (3,1) was employed according to the following reliability indicators: 0.90 and higher as excellent, 0.75 to 0.90 as good, and below 0.75 as poor to moderate reliability.19 Pearson's rank correlation coefficients were used to examine intercomponent correlations. Significance levels were set at p<0.05.
Results of the Reliability Evaluation phase
Participants and administration methods
Contact was made with 471 families, and 58 families agreed to participate; however, on further assessment, eight children did not fulfil the inclusion criteria for communication and/or cognitive functioning. Of the 50 remaining participants (mean age 10y 2mo, mean WISC-IV 104), 35 participants (mean age 10y 5mo, mean WISC-IV 103) agreed to complete the retest measure. There were no significant differences in age, sex, GMFCS - E&R level, MACS, FCCS, and WISC-IV between the total sample and the retest sample. Participant characteristics are presented in Table SI (online supporting information). Of the total sample, 37 children used the Game Version, of whom 24 participated in the retest measure.
Reliability
All Performance Perspectives showed strong internal consistency: Personal (α=0.87), Social (α=0.91), and Perceived (α=0.89) (Table 2). Strong correlations were found between Performance Perspectives (r=0.78–0.85) and as predicted, the Personal Concern Score was negatively correlated with all Performance Perspective Scores (r=−0.51 to −0.76) (Table 3).
Components | Mean (SD) n=50 | Sample score range (min–max) n=50 | Cronbach's αn=50 | ICC (95% CI) n=35 |
---|---|---|---|---|
Overall scale | a | a | 0.96 | a |
Personal Performance Perspective | 84.72 (11.83) | 46–98 | 0.87 | 0.71 (0.50–0.84) |
Social Performance Perspective | 80.30 (14.10) | 39–104 | 0.91 | 0.75 (0.56–0.86) |
Perceived Performance Perspective | 83.82 (12.12) | 45–104 | 0.89 | 0.70 (0.49–0.84) |
Importance Rating | 86.64 (10.81) | 49–103 | 0.84 | 0.64 (0.39–0.80) |
- a This assessment does not yield total scores. Only component scores are available.
Personal Performance Perspective | Social Performance Perspective | Perceived Performance Perspective | Personal Concern Score | |
---|---|---|---|---|
Personal Performance Perspective | 1 | |||
Social Performance Perspective |
r=0.85 (p=0.001) 95% CI 0.74 to 0.91 |
1 | ||
Perceived Performance Perspective |
r=0.78 (p=0.001) 95% CI 0.64 to 0.87 |
r=0.82 (p=0.001) 95% CI 0.70 to 0.89 |
1 | |
Personal Concern Score |
r=−0.76 (p=0.001) 95% CI −0.86 to −0.61 |
r=−0.54 (p=0.001) 95% CI −0.71 to −0.30 |
r=−0.51 (p=0.001) 95% CI −0.69 to −0.27 |
1 |
All Performance Perspectives showed moderate to good test–retest reliability: Personal (ICC=0.71), Social (ICC=0.75), and Perceived (ICC=0.70) (Table 2). The Importance Rating showed lower reliability (ICC=0.64) (Table 2).
The wide range of scores on all Performance Perspectives demonstrates that myTREEHOUSE has the potential for discriminant validity (Table 2).
Participant feedback on the Game Version
Nineteen Game Version participants provided feedback on this administration method. Children provided favourable comments, which included: ‘[the board game is] more fun than homework’, and ‘it was easy and kind of fun’. They found the treehouse and the ramps helpful when making their evaluations. Children liked the picture cards, but thought they could be slightly enlarged.
Discussion
This is the first study reporting on the development of the myTREEHOUSE Self-Concept Assessment for children with CP aged 8 to 12 years, which measures self-concept using three Performance Perspectives and an Importance Rating over eight domains. Before this study, self-concept assessments developed for children with typical development were used for children with CP; however, those assessments did not fully capture the self-concept of children with CP. myTREEHOUSE was developed in response to the need for population specific self-concept assessments for children with physical disability,6 like CP,7 and the lack of well-validated assessments for this population.4, 5
myTREEHOUSE showed strong internal consistency across Performance Perspectives, with values comparable to existing self-concept measures validated for children with typical development20-22 or for children with CP, such as the Self-Description Questionnaire-I (Cronbach's α=0.76–0.94).23 Values were also comparable with those reported for other CP-specific measures of psychological constructs, such as the CP Quality of Life Questionnaire for Children (Cronbach's α=0.80–0.90).8
myTREEHOUSE showed moderate to good test–retest reliability. This finding is comparable with other CP-specific measures of psychological constructs, such as the CP Quality of Life Questionnaire for Children (ICC=0.76–0.89).8 Test–retest reliability is a recommended inclusion in psychometric testing13 but has not been consistently reported in evaluations of other self-concept assessments.4, 5 The current findings indicate that the myTREEHOUSE self-concept constructs are fairly stable and can be reliably assessed over time.
Impairments caused by CP may limit a child's access to their environment, resulting in a different life experience compared with children with typical development. myTREEHOUSE was constructed to include several CP-specific areas identified as important contributors7 under the domain names of Physical Abilities, Personal Agency, and Ability to Participate. Their inclusion provides a more comprehensive evaluation of self-concept for children with CP.
myTREEHOUSE has the unique ability to provide clinicians with a measure of self-concept from three Performance Perspectives (i.e. Personal, Social, and Perceived). Personal and Social Performance Perspectives are common features in most self-concept assessments for children with typical development13 but they are not evaluated as individual components, making it difficult to ascertain their relative contribution. Measuring these components separately, myTREEHOUSE allows clinicians to assess the relative weight of these perspectives to assist with determining appropriate intervention programmes to target the core difficulties for children with low self-concept.
Moreover, myTREEHOUSE has introduced a new evaluative perspective, the Perceived Performance Perspective, which evaluates a child's perspective of how others might view their performance. To date, this perspective has not been evaluated as an independent construct in self-concept instruments. Unlike their typically developing peers, children with CP are consistently exposed to evaluation by clinicians and caregivers, often being commented on in their presence, about their proficiency related to various areas of impairment. This reality for children with CP makes the Perceived Performance Perspective more important for their sense of self. By preadolescence, children have mastered the ‘role-taking’ skill (i.e. being able to step into another's shoes),24 which enables them to take in the judgement of others and incorporate this knowledge into their evaluation of self-concept.11, 12 Thus, the myTREEHOUSE Perceived Performance Perspective can provide clinicians with an insight into the best approach for discussing treatment progress to protect or promote each child's self-concept.
Furthermore, myTREEHOUSE also incorporates a Personal Concern Score that provides clinicians with the ability to identify domains of particular concern to the child. Harter22 introduced a similar scoring format in the Self-Perception Profile for Children and stressed its significance in interpreting self-concept,11 but this type of rating has not been included in other self-concept assessments used with children with CP.4 The myTREEHOUSE Personal Concern Score allows clinicians to target specific domains that are negatively impacting a child's overall self-concept.
Lastly, myTREEHOUSE includes two administration methods to facilitate the participation of children with a wide range of abilities: a Questionnaire Version and a pictorial Game Version. Stone and Lemanek25 emphasized the importance of designing self-report assessments for children to suit their capacity (e.g. attention span, reading and writing skills) and maintaining their interest using pictorial cues. Our Game Version, which uses not only pictorial cues but is presented in a game format, received favourable responses from both allied health professionals in the expert reference group and children with CP who participated in this study. Children indicated that they enjoyed the interactive nature of the game with the ramps making rating scales easy to comprehend.
This study introduces preliminary psychometrics for myTREEHOUSE. Sample size in this study was modest because of strict inclusion criteria for functional communication and cognitive ability, which resulted in a reduction in eligible participants. However, these criteria are important to ensure that participants have the language and cognitive ability to perceive the self and to engage in self-concept evaluation.5 Further investigation with a larger sample would allow for evaluation of construct validity using factor analysis to confirm the domains.
Conclusion
The new myTREEHOUSE Self-Concept Assessment is a unique, population-specific assessment that is valid and reliable for assessing self-concept of children with CP. It provides comprehensive evaluation across eight domains and three different Performance Perspectives. Domains include items from instruments for children with typical development, as well as new CP-specific items. Finally, myTREEHOUSE is one of the first self-concept instruments to provide a Personal Concern Score which can be used to identify domains of particular concern for each child.
Acknowledgements
We would like to thank the Cerebral Palsy League (Brisbane, Australia) for their in-kind support and acknowledge the faculty grant awarded by the Australian Catholic University in support of this study. We would also like to express our appreciation to all of the professionals, parents/caregivers, and children who participated. The authors have stated that they had no interests which might be perceived as posing a conflict or bias.