Efficacy, model of delivery, intensity and targets of pragmatic interventions for children with developmental language disorder: A systematic review

INTRODUCTION

In addition to acquiring words and grammar during the early stages of language development children must acquire pragmatic rules that is, the knowledge necessary for the appropriate, effective, rule-governed employment of speech in interpersonal situations (Ninio & Snow, 1996: 4). Pragmatic skills imply the use of language in different social situations and is traditionally studied in the fields of developmental psychology and linguistics. In psychology research focus has been on what Tomasello (2008: 7) called ‘the cooperative infrastructure of human communication’, that is, the cognitive abilities and social skills that lie beyond the use of language (Bates, 1976; Stephens & Matthews, 2014). Linguistic studies, on the other hand, consider pragmatic skills within the context of Austin's (1962) speech acts that relate to the speaker's intention behind their behaviour (Cameron-Faulkner, 2014) as well as a scalar (quantity) implicature approach put forward by Grice (1975) (for more information on theoretical differences, see Falkum, 2019).

The development of pragmatics and the successful application of pragmatic skills are based on the interaction of two crucial skills: (1) adequate social skills, supported by social cognition; and (2) language skills through which the message is realised, by applying various pragmatic devices (Adams et al., 2005). According to Prutting and Kirchner (1987), pragmatic devices involve three aspects that are mastered synchronously: (1) verbal aspects that integrate elements such as variety of speech acts, topic selection, topic introduction, topic maintenance, turn-taking initiation, turn-taking response, revision, pause time-overlap, cohesion, etc.; (2) paralinguistic aspects such as intelligibility, vocal intensity and quality, prosody, fluency; and (3) non-verbal aspects such as physical proximity and contacts, body posture, foot/leg and hand/arm movements, gestures, facial expression and eye gaze.

This systematic review focuses on examining the use of the verbal and paralinguistic aspects in pragmatic interventions involving children with developmental language disorder (DLD).¹

METHOD

Selection of papers and reliability of search procedures

Selection of the papers was conducted on four consecutive steps using specialist software supporting systematic reviews (EPPI—Reviewer 4) and Excel:

• Stage 1: The initial search aimed to find papers that focused on different language domains (phonology, vocabulary, morphosyntax and pragmatics) and which would serve the WG1 subgroup within COST Action IS1406 for their further analysis given their specific interest. The papers were screened by researchers in the COST Action based on title and abstract analysing the previously prescribed criteria such as target group and intervention targets. A total of 20% of the papers were double screened by two independent reviewers (see Frizelle et al., 2021, for details).
• Stage 2: To identify papers specifically relevant to the domain of pragmatics two independent reviewers (K.J.L. and J.K.K.) screened all papers allocated to the domain of pragmatics on title and abstract. Due to the updated searchers this was carried out twice and yielded 1031 papers.
• Stage 3: The first stage of full text screening (n = 259) was completed against the inclusion/exclusion criteria by K.J.L. and J.K.K. Disagreements were resolved through discussion.
• Stage 4: Full text screening was completed by K.J.L., J.K.K. and E.B.S. on the papers that emerged from stage 3 and only those that focused specifically on oral pragmatic outcomes. Finally, 11 papers satisfied all initially prescribed criteria and were included in the analysis (Figure 1).

Risk of bias

K.J.L., J.K.K. and E.B.S. appraised the quality of each paper using the Cochrane Risk of bias tool for RCTs (Higgins et al., 2011). In total five different types of biases were evaluated: (1) selection bias that covers two aspects: random sequence generation and allocation concealment; (2) performance bias that stands for blinding of participants and personnel; (3) detection bias which evaluate blinding of outcome assessment; (4) attrition bias that control incomplete outcome data; and (5) reporting bias which deals with selective reporting. In addition, we analysed the faithful and correct implementation of the key components during the intervention (fidelity measurements). Bias risk was rated as high, low, or unclear using the Cochrane risk-of-bias tool for RCTs (Higgins et al., 2011). Two reviewers rated each article independently and disagreements were resolved by consensus. The critical appraisal for each paper is shown in Figure 2.

RESULTS

A total of 11 studies examined language pragmatic as an outcome variable and were included in our analyses. An overview of the designs, samples, outcome variables and results are presented in Table 1. Only one of the included studies was carried out in a non-English-speaking country (Italy), with the remaining studies being conducted in the respective countries; Australia (three studies), UK (two studies), Canada (one study) and the United States (four studies). Seven of the studies were randomized controlled studies and the remaining studies were pre-post-test intervention-control group studies.

Regarding the efficacy of the interventions, five studies (Abdul Aziz et al., 2016; Allen & Marshall, 2011; Joffe et al., 2019; Kelley et al., 2015; van Kleeck et al., 2006) showed significant effects in favour of the DLD group for the outcomes used to measure children's pragmatic language, with effect sizes ranging from medium 0.04 to large 1.50. The large effect size of d = 1.50 was produced using a 10-item narrative checklist of explicit narrative knowledge by Joffe et al. (2019). Allen and Marshall (2011) measured two pragmatic language outcomes; verbal initiations and verbal responses; however, significant effects were only shown for verbal initiations. The areas of pragmatic language, for which the respective interventions proved to be effective, varied considerably from the child's ability to engage in speech regulation (social speech, private speech, and inaudible speech) to the ability to recall a narrative (ERRNI) and to respond adequately to inferential questions. The interventions that did not prove to be effective measured pragmatic language using the Narrative Language Ability Index (NLAI and Renfrew Bus Story), the CCC2 checklist and communicative acts (measured by number of verbal productions) and answers (measured as number of inadequate and adequate answers) as outcome measures. To summarize, the results gained across the included studies regarding the effects of the interventions did not show a clear picture of which aspects of pragmatics that certainly lead to improvement. Furthermore, several of the intervention studies held small sample sizes.

Regarding control of biases as the internal validity in the RCT studies, four studies did not report about performance bias and three studies indicated a risk of detection bias. Two papers showed high risk of selection bias whereby Lavelli et al. (2019) showed high bias for: random sequence generation and allocation concealment. There was an unclear risk of bias for two papers on selection allocation concealment and for one paper in detection bias and attrition bias. The high risk of bias and unclear bias in some studies reduced the reliability of the interpretation of the results.

We analysed the specific content of the included interventions to better understand what the main targets of the interventions were. In four papers, conversation was the target of the intervention, while narrative skills were targeted in the remaining seven studies. The conversation skills were based on verbal initiations, verbal responses, turn-taking skill, question answering, etc. These skills have been confirmed in several previously published studies (Katsos et al., 2011; Befi-Lopes & Rocha, 2004) as vulnerable areas that may be weak in children with DLD. In most interventions the SLP encouraged and observed the conversation between the child and his/her primary caretaker (usually the mother) or used shared book-reading as the context of the intervention activity. Apart from Van Kleeck et al. (2006), which consisted of scripted book reading, all other studies were carried out within the context of a shared book-reading activity (Joffe et al., 2019; Kelley et al. 2015; Kramer et al., 2009; Lavelli et al., 2019; Wake et al., 2013, 2015). In Van Kleeck et al. (2006), children participated in individual 15-min shared book-reading sessions twice per week with graduate and undergraduate research assistants from programmes in communication sciences and disorders. In some interventions, for example, that by Lavelli et al. (2019), parental responsiveness was combined with dialogic reading, that is, parents were first explicitly taught conversation strategies after which they applied them in the reading sessions with their child. Interventions that focused on narrative skills mostly measured various elements on the microstructure level (five papers: Fey et al., 2010; Kelley et al., 2015; Wake et al., 2013, 2015; Wilcox et al., 2011) and quite rarely elements on the macrostructure level, for example, Joffe et al. (2019) and Kramer et al. (2009).

Since one of the criteria for including studies was that the aim of the study was to change an aspect of language pragmatics, different aspects of dose form were sampled like private speech by Abdul Aziz et al. (2016) or verbal initiations and verbal responses by Allen and Marshall (2011). Interestingly, in Abdul Aziz et al. (2016), paralinguistic aspects (inaudible speech) that accompanied verbal aspects were observed and encouraged. It was operationally defined as any whispering, unintelligible muttering or silent lip movements that cannot be heard to be coded into one of the measured verbal categories. Representation of this type of pragmatic device in the included studies was however extremely low.

We analysed the model of delivery, unit of allocation, level of service delivery and setting of delivery in all 11 studies (Table 2).

Interventions can either be direct, meaning that the SLP or a similar specialist bears full responsibility for delivering the training or indirect, meaning that a parent or other professionals, supervised by an SLP, in some respects help deliver the training (Boyle, 2009). In our included studies, four interventions were delivered directly and seven indirectly. It is important to emphasize that the type of model delivery in these studies had different goals. For example, in Joffe et al. (2019), based on an indirect model, the aim was to investigate the efficacy of teaching assistants in delivering narrative and vocabulary intervention to mainstream secondary school-age students with language disorders. In studies based on a direct model of delivery, such as Fey et al. (2010), Kramer et al. (2009), Van Kleeck et al. (2006) and Wilcox et al. (2011), the aim was to examine the efficacy of a particular type of intervention in improving pragmatic skills. Direct intervention was always conducted by a specialist (SLP), while indirect intervention was either delivered by parents as in Allen and Marshall (2011), by a teaching assistant as in Joffe et al. (2019) or by some other trained professionals as in the intervention reported by Kelley et al. (2015) and Wake et al. (2013). In several of the interventions conducted indirectly children showed improvement, however, as emphasized by Allen and Marshall (2011), indirect invention relies heavily on parental motivation for intervention to succeed.

In six studies, the preferred unit of allocation was one to one, which is a common mode of SLP intervention. It was not only interventions based on direct mode that were organized on a one-to-one unit, but also some interventions that were based on indirect mode were conducted on an individual basis. That was the case in the studies by Allen and Marshall (2011), Lavelli et al. (2019) and Wake et al. (2013, 2015). Nearly half of all studies had a small group as their main unit of allocation (Abdul Aziz et al., 2016; Fey et al., 2010; Joffe et al., 2019; Kelley et al., 2015), whereas in Wilcox et al. (2011) a large group was the unit of allocation.

To appreciate the role of intensity for the effects of the individual studies, we evaluated three quantitative aspects of dosage: one component of dose (the session length), dose frequency and total intervention duration (Table 2).

All these aspects were highly variable among the studies. The total number of intervention sessions ranged from four (e.g., Allen & Marshall, 2011; Kramer et al., 2009) to 32 (Lavelli et al., 2019). In two studies (Kelley et al., 2015; Wilcox et al., 2011) information about total number of intervention sessions was not stated, and it was not possible to calculate. Session length ranged from 15 min (Allen & Marshall, 2011; Van Kleeck et al., 2006) to 150 min (Wilcox et al., 2011). In Allen and Marshall (2011), parents were asked to play/carry out activities for 15 min, while in Wilcox et al. (2011) the primary aim was to examine the efficacy of a new preschool oral language and early literacy curriculum package called Teaching Early Literacy and Language (TELL). Since TELL was part of the curriculum, it was offered four times weekly with 150 min of instruction scheduled every day. In four studies (Fey et al., 2010; Kelley et al., 2015; Kramer et al., 2009; Lavelli et al., 2019) no information was reported about session length. Except for Lavelli et al. (2019), dose frequency and total intervention duration were the most consistently reported data about dosage across the analysed studies. Dose frequency ranged from once to four times per week. Wilcox et al. (2011) was the only study not to report information about total intervention duration. In the other studies, duration intensity ranged from 1 to 18 weeks. The shortest duration was reported by Kramer et al. (2009). This study was based on dynamic assessment intervention that is defined by a test–teach–retest format. All three children included in the study participated in two mediation sessions; the first mediation session took place on an average of 3 days after the test phase, and the second mediation session took place the day after the first session except in one case where the session took place 2 days later. In all cases, both mediation sessions were conducted within 1 week. The longest duration within the included studies was for the intervention conducted by Wake et al. (2013, 2015) and consisted of an 18-session programme (three 6-week blocks of weekly sessions, starting every 3 months) delivered by the authors on a one-on-one basis.

DISCUSSION

The choice of effective service delivery is a major concern for practitioners, service commissioners and policymakers. Narrative synthesis of the results of studies within systematic reviews are widely regarded as one of the most valued and reliable ways of synthesizing research evidence (Marshall et al., 2011). To our knowledge, our systematic review is the first review to exclusively investigate interventions that target pragmatic language-focused interventions in children with DLD including various dose forms. The purpose of this study was to systematically evaluate peer-reviewed articles from the last 15 years concerning the effect and content of pragmatic based speech–language interventions for children with DLD. There are still mis-conceptual believes that children with language disorders show exclusively difficulties with the lexicon and grammar, whereas pragmatic difficulties primarily are seen in children with autism spectrum disorder. However, several studies have pointed to the fact that difficulties within the pragmatic domain also are the hallmark of children with DLD (Befi-Lopes & Rocha, 2004; Olsen et al., 2010; Katsos et al., 2011; Mäkinen et al., 2014). These results align with recent studies showing that children with DLD are delayed in their understanding of ToM (Nilsson & Jensen de López, 2016), which may be mediated by their delayed language.

Narrative synthesis of the 11 papers that met the inclusion criteria for this study showed a high degree of diversity in reporting about components relevant for confirming intervention effectiveness. That was especially the case with targets, intervention outcomes and intensity of the intervention. Moreover, the included studies were not uniform in terms of risk-of-bias reporting and this might affect the outcomes and conclusions of the studies (Higgins et al., 2011). In our study, performance and detection bias were the most frequent aspects with risks of biases that might influence the results. Despite this diversity among the included studies, some important conclusions can be drawn from the results of this systematic review.

First, when dealing with pragmatic language, researchers are mostly oriented towards conversation and narrative skills. A similar pattern appears for interventions based on single case designs (Jensen de López et al., under review). In relation to pragmatic devices, researchers are mostly focused on the verbal pragmatic aspects, and to a lesser extent on those that are paralinguistic. It means that verbal pragmatic elements such as speech acts, topic selection, turn-taking, etc. are more often the focus of SLP research than paralinguistic elements like prosody and fluency, or metalinguistic skills like explicit awareness of pragmatic rules and when these have been violated.

What seemed a bit surprising in our results is that even for pragmatic interventions; the dominant unit of allocation is one-to-one which is a common mode of intervening in speech, language, and communication disorders. There are advantages of individual interventions such as the full attention of the SLP to the one client and his/her needs or better management and control of multiple goals and targets during the intervention, and less opportunities for distraction of the child. Group intervention was also present in some of the included studies but to a lesser extent. An important advantage of group intervention in addition to establishing specific speech and language behaviours is that it is directed towards the development of appropriate interpersonal and social skills (Lieberman & Michael, 1986) and is embedded in an ecological valid setting. These kinds of skills are important building blocks for the development of pragmatic language abilities and intrinsic aspects of pragmatic language (Adams et al., 2005). Yet few interventions seemed to have capitalised on these specific skills.

The included studies also differed in terms of reporting the intensity of the interventions. Dose frequency and total intervention duration are the most consistently reported data about intensity. In two recently published systematic reviews the authors dealt with papers in which dosage was statistically analysed (Frizelle et al., 2021; Segura-Pujol & Briones-Rojas, 2021). In Frizelle et al. (2021), the results showed that frequent interventions (two to three times per week) that target language goals for short periods or less frequent interventions (once per week or fortnight) targeting language goals for longer yield the best outcomes in relation to composite language measures. The preliminary conclusions of Segura-Pujol and Briones-Rojas (2021) suggested that greater stimuli variability provided better results in treating morphology, whereas greater exposure to stimuli does not necessarily translate into improved performance for vocabulary tasks. Similar conclusions cannot be drawn from data obtained in our pragmatic intervention review since we were not restricted to papers in which dosage was experimentally manipulated as our goal was to analyse the range of intensities in the interventions. In fact, the intervention with the longest intensity (Wake et al., 2013, 2015) did not show statistically significant changes.

Furthermore, there are differences in the effectiveness of the interventions among the included studies and some studies did not report effectiveness. Therefore, the results of the efficacy of the interventions included in our review did not show a clear picture of which interventions were most effective for improving pragmatic language. In addition, if we add the fact that the analysed studies had different research objectives—in some of them the aim was to test the intervention effectiveness, while in other studies intervention was the means for achieving a different goal — this further complicates the drawing of reliable conclusions.

There seems to be a clear need for further research that can identify pragmatic interventions that are effective for children with DLD and across languages other than English. It is also important that future intervention studies include outcome measures with high levels of ecological validity to certify that there has been a transfer of the intervention to the everyday activities of the child. Finally, it is equally of importance to establish what type of content and context, but also which level on intensity is most effective for treating pragmatic language disorders in children with DLD. As mentioned earlier there are a vast amount of pragmatic intervention studies for children with autism disorder, however due to the diversity of the two groups we cannot expect these existing interventions to be the most effective options for children with DLD.

CONCLUSIONS

Several systematic reviews focusing on pragmatic interventions are available today, but none of them include children with DLDs. Therefore, the motivation of this research was to fill the gap in the literature and to give directions for clinical work.

Despite a high diversity in the 11 included studies regarding intervention targets, intensity and outcomes, several relevant conclusions, especially for clinicians, can be drawn. The evidence suggested that interventions for pragmatic language are mostly focused on encouragement of conversation and narrative skills, very often through parent–child interaction or shared book-reading activities. Further, they are feasible for all models of delivery (individual, small and large group). Pragmatic interventions can also be carried out indirectly if the motivation of non-specialists to participate in the intervention is ensured and if their training for the implementation of therapy is under the continuous supervision of a specialist.

CONFLICTS OF INTEREST

The authors have no known conflict of interest to disclose.