2.1. Eligibility Criteria

Given that RCTs are considered the gold standard for evaluating the effectiveness of interventions [20], only RCTs were included in this review. The population targeted adults aged ≥18 years old with or without medical, psychological, and behavioral problems. The intervention used a conversational (chatbot) interface to deliver any form of psychotherapy with self-guided or therapist support incorporating AI technology. Response generation contained rule-based or other AI technologies. Input and output modalities involved written, spoken, visual, or emoji. The presentation could use either an embodied or nonembodied chatbot. The comparator included treatment as usual, waitlist, placebo control, or any type of intervention. The psychological outcomes included depressive, anxiety, and stress symptoms at postintervention and follow-up assessments. No restrictions were imposed on the population and publication date. This review included published and unpublished trials in the English language [21]. The details of the eligibility criteria can be found in Table S2.

2.2. Search Strategy

A scoping search for existing systematic reviews with similar aims was conducted in the Cochrane Database of Systematic Reviews, Joanna Briggs Institute, and the PROSPERO database to prevent any duplication. An iterative process was used to develop the search terms. An initial keyword search comprising “artificial intelligence” AND “psychological outcomes” was used to conduct a simple search. After the inclusion of potential articles, the search terms and keywords were revised in consultation with a university librarian. The eventual search terms comprising both keywords and index terms for the respective databases can be found in Table S3.

Following the development of the search terms, a three-step search [22] was conducted from inception to February 9, 2023. First, a search was conducted in nine English electronic databases (PubMed, Embase, CINAHL, Cochrane Library, Scopus, IEEE Xplore, Web of Science, PsycINFO, and ProQuest Dissertations and Theses) to locate relevant articles. Second, a search for unpublished trials was conducted in three clinical trial registries (ANZCTR, ISRCTN, and CenterWatch), and an email was sent to all corresponding authors to obtain information about their trials. Finally, a hand search of the reference lists of the selected studies and gray literature was conducted to maximize the search. We contacted the authors via email for additional data when the information included in their publication was insufficient.

2.3. Study Selection

EndNote X20 was used to manage the retrieved citations from the search. Duplicates were removed using automated and manual functions. Two authors (W.W. and S.H.W.) screened all the articles by title and abstract, with reference to the eligibility criteria. When disagreements occurred, a third author (L.Y.) was consulted. Inter-rater reliability was measured using Cohen’s kappa (κ), l, with −1 suggesting an absence of agreement, and 1 indicating perfect agreement [23]. Values greater than 0.75 were considered excellent agreement, whereas values between 0.40 and 0.75 were considered good agreement [24].

2.4. Data Management and Extraction

The data extraction form was designed with reference to the Cochrane Handbook [22]. Two reviewers (W.W. and S.H.W.) extracted all the data independently. The data elements extracted included trial characteristics (number of studies, author, publication year, country, recruitment setting, design, nature of participants, mean age, gender distribution, AI-based psychotherapeutic intervention, name, comparator, sample size, psychological outcomes, measures, attrition rate, intention-to-treat analysis [ITT], missing data management [MDM], protocol, trial registration, and grant support), description of the intervention (intervention content, type of AI chatbot, psychological principle, duration of the intervention, follow-up assessment, frequency of use, and mean amount of time engagement in minutes), and psychological outcomes (depressive, anxiety, and stress symptoms) at postintervention and follow-up assessments (mean, standard deviation, and total numbers).

2.5. Risk of Bias Version 2 (RoB 2.0)

The Cochrane risk of RoB 2.0 [25] was used to appraise the methodological quality of all included studies. Risk of bias was performed via an Excel tool to implement RoB 2.0 by two independent reviewers (W.W. and D.A.). The risk of bias was evaluated against the following five domains of bias: (1) randomization process, (2) deviations from intended intervention, (3) missing outcome data, (4) measurement of the outcome, and (5) selection of the reported result [25]. Two reviewers responded to signaling questions in each domain to select the options of “yes,” “probably yes,” “probably no,” “no,” or “no information.” The RoB 2.0 algorithmic tool rates the risk of bias as “low,” “high,” or “some concerns” [25].

2.6. Certainty of Evidence

The grading of recommendations, assessment, development, and evaluation (GRADE) criteria was used to assess the overall certainty of evidence [26]. To determine the certainty of evidence, two reviewers (D.A. and L.Y.) independently evaluated the studies based on the following domains: risk of bias, inconsistency, indirectness, imprecision, and effect. The ratings were classified as very low, low, moderate, or high, and the decision was determined based on justifications [26]. Publication bias was determined using the Egger regression test [27] and funnel plot of precision using standardized mean difference [28]. Publication bias was ascertained using a p-value of less than 0.05 from the Egger test and asymmetrical funnel plot [29].

2.7. Data Synthesis

We used the meta [30] and metaphor [31] packages of R software to conduct the meta-analysis, subgroup analysis, and meta-regression analysis. Prediction interval (PI) was used based on t-distribution (t) to predict a range of true effects for future trials with similar settings [32]. A 95% PI was used to estimate the 95% probability that the next trial will be contained within this range. A statistically significant effect is expected for a future trial if all values of the 95% PI are on the same side of the null of 0, whereas an insignificant effect is expected if all values are on both sides of the null of 0 [32]. Hedges’ g was used to communicate the effect size because of its precision for studies with small sample sizes [33, 34]. Random-effects model was used to assume that the observed estimates of treatment effect can vary across studies [35]. Restricted maximum likelihood method was used as the estimator for random-effect meta-analysis to provide unbiased estimates [36]. Hartung–Knapp adjustment for random-effects models was selected to prevent counterintuitive effects [37]. A 95% confidence interval was used to communicate the precision of the summary estimate and derive the p-value [22].

Heterogeneity was assessed using Cochran Q test and I² values [22]. A p-value of <0.01 indicated heterogeneity. The extent of heterogeneity was quantified using I² values [22]. A Cochran Q test p-value of <0.01 and I² > 50% indicated heterogeneity [22]. Additional subgroup and meta-regression analyses were conducted to explore the reasons for heterogeneity [22].

Subgroup analyses were conducted based on the predetermined groups based on the nature of participants (depression ± others, stress/distress ± others, other condition, or healthy), age groups (18–30, 31–40, 41–50, or >50), type of AI-based chatbot (Deprexis vs. others), different comparator (passive vs. active), type of psychotherapy (cognitive behavioral therapy [CBT] vs. others), type of platforms (Internet vs. others), response generation (rule-based vs. NLP), and embodiment (yes vs. no) use of ITT/MDM (yes vs no), and protocol publication/trial registration (yes vs. no). Significant subgroup differences were determined based on the Q statistic with a subgroup effect of p < 0.1 [38]. Meta-regression analyses were conducted to examine the effects of potential covariates (publication year, duration of intervention, sample size, attrition rate, and portion of males) on the psychological symptoms. The relationships were expressed using coefficient β, which represents the change in the value of depressive symptoms relative to the unit change in the covariates [39]. A p-value of <0.05 was used to conclude the association between the covariate and outcomes based on effect size [22].

3.1. Trial Characteristics

The characteristics of the 30 trials evaluating the effect of AI-based psychotherapeutic interventions involving 6100 participants can be found in Table 1. The trials were published from 2009 [12] to 2022 [70]. They were conducted in Argentina (n = 1) [71], China (n = 2) [70, 72], Germany (n = 10) [11, 12, 26, 39, 40, 42, 48, 50–52], Italy (n = 2) [7, 44], Korea (n = 2) [16, 47], Romania (n = 1) [73], the United Kingdom (n = 3) [8, 41, 74], the United States (n = 5) [2, 9, 23, 45, 46], and three countries (n = 1) [43]. The participants were recruited from the community (n = 21) (1–3,5,6,8,9,11–16,19,20–23,25,27,28), clinical setting (n = 6) [11, 16, 26, 39, 43, 47], and a mixture of both (n = 3) [48, 50, 52]. Twenty-five trials adopted a two-arm RCT, three trials [40, 49, 72] adopted a three-arm RCT, one trial [7] adopted a four-arm RCT, and one trial [23] adopted a crossover design. The sample sizes of the trials ranged from 21 [44] to 1013 [48]. Half of them reported follow-up outcomes after postassessment, which ranged from 2 weeks [74] to 12 months [48].

3.2. Description of AI-Based Psychotherapeutic Interventions

Fourteen types of AI-based chatbot were found, including Deprexis (n = 11) [11, 12, 14, 26, 39, 40, 42, 48, 50–52], Manage Your Life Online (n = 3) [8, 41, 74], Woebot (n = 3) [2, 39, 73], Therapy Empowerment Opportunity (n = 2) [7, 44], and others. The content of the interventions is described in Table 2. Table S6 provides a summary of 14 different types of AI-based chatbots. The psychological principle was largely grounded in CBT (n = 25), and the area of use was mainly for the treatment of various conditions (n = 26). The main functions included counseling (n = 18), therapy (n = 18), and monitoring (n = 20) via the Internet (n = 18). Most interventions (n = 27) were self-guided and four of them [7, 40, 44, 46] were supported by a therapist. Two trials [40, 46] used therapists for counseling and therapy, while others used a self-help version of an AI-based chatbot. Berger et al. [40, 42] adopted one intervention group with a low-intensity therapist-guided self-help version of Deprexis, and Fitzsimmons-Craft et al. [46, 50] relied on human authoring of conversations via a chatbot (Tessa). Two trials [7, 44] used the mobile personal health care agent (m-PHA) to communicate with patients, but the therapist supervised the m-PHA interactions with the patients. The therapists provided support regarding the events mentioned during the therapy sessions, as well as reviewing the notes and recollections [7, 44]. The duration of intervention ranged from one time [41] to 16 weeks [58]. Most trials did not report the frequency and time of usage. Response generation contained rule-based (n = 16), NLP (n = 14), and other AI technologies. Input and output modalities involved written (n = 30), spoken (n = 3) [43, 70, 72], visual (n = 1) [72], and emojis (n = 1) [71]. Seven embodied chatbots [9, 16, 23, 43, 47, 49, 70] were observed.

3.3. Individual Quality Assessment

A total of 30 RCTs were evaluated against the RoB 2.0 criteria (Figure S1). Twenty-three trials used ITT analysis, and seven trials used perprotocol analysis. Low risk of bias rating across five domains was found in the majority (79.1%) of trials with ITT analysis but less than half (48.6%) of trials with perprotocol analysis. Nine trials (30%) did not provide information on allocation. Seventeen trials (56.7%) rated some concerns for deviations from the intended intervention because the participants and personals were aware of the assigned intervention. Table 1 displays that 13 trials (43.3%) did not publish a protocol or did not register in clinical trial registries, so the selection of the reported results had some concerns. The attrition rate ranged from 0% [9] to 59.7% [71]. The majority (80%) of trials received grants from various sources, including 16 nonindustry-sponsored grants, seven industry-sponsored grants, and one trial [45] that included both industry-sponsored and nonindustry-sponsored grants. However, six trials did not report or had no grant support. Even though seven trials mentioned industry-sponsored grants, two trials [23, 42] declared that they were not involved in data analysis.

3.4. Depressive Symptoms

A total of 29 arms of 26 RCTs [2, 7, 9, 11, 12, 14, 16, 23, 26, 39–48, 50–52, 70, 72–74] among 4349 individuals, eight arms of six RCTs [7, 23, 41, 44, 72, 74] among 418 individuals, and eight arms of seven RCTs [11, 12, 26, 40, 46, 48, 50] among 2268 participants evaluated the effects of AI-based psychotherapeutic interventions at the postintervention assessment and 2 weeks to 3 months and 6–12 months of follow-up assessments (Figure 2). The meta-analyses showed that AI-based psychotherapeutic interventions significantly reduced depressive symptoms at the postintervention assessment (t = −4.40, p = 0.001) with medium effect size (g = −0.54, 95% CI: −0.79 to −0.29) and 6–12 months of follow-up assessment (t = −3.14, p < 0.016) with small effect size (g = −0.23, 95% CI: −0.40 to −0.06) compared with the comparators. No differences were observed between the intervention and comparator at 2 weeks to 3 months of follow-up assessment (t = −0.08, p = 0.936).

The 95% PIs were −1.77 to 0.69, −1.27 to 1.24, and −0.64 to 0.19 for three-time points. Given that the 95% PI contained values on both sides of the null of 0, suggesting that the intervention will predict an insignificant reduction of depressive symptoms in future similar studies. Heterogeneity was substantial (I² = 70%–85%) for postintervention assessment and 2 weeks to 3 months of follow-up assessment and moderate (I² = 42%) for 6–12 months of follow-up assessment. Given the presence of substantial heterogeneity at the postintervention assessment, subgroup analyses, and meta-regression analyses were conducted to explore the reasons for heterogeneity.

We conducted subgroup analyses as shown in Table 3 and Figures S2–S26. Significant differences (p < 0.1) were found between subgroups based on participants’ nature, age group, embodiment, ITT/MDM, and protocol/registration on reduction of depressive symptoms at three-time points. Trials that were conducted among the participants with depression or depression combined with other health issues had a larger effect on reducing depressive symptoms at postintervention (g = −0.81, 95% CI: −1.16. to −0.45) and follow-up 2 weeks to 3 months (g = −0.64, 95% CI: −2.22. to 0.95) compared with their counterparts. The interventions used embodied chatbot (g = −0.57, 95% CI: −1.17 to 0.03) among those aged 31–40 (g = −0.57, 95% CI: −1.17 to 0.03) using ITT or MDM (g = −0.35, 95% CI: −1.11 to 0.40) had a greater effect on reducing depressive symptoms at follow-up 2 weeks to 3 months than their counterparts. We observed trials with a protocol or registration (g = −0.32, 95% CI: −0.65 to 0.02) have a greater effect on the reduction of depressive symptoms at follow-up 6 months to 12 months than those without a protocol or registration. Hence, between-trial heterogeneity could be partially explained by participant characteristics.

A series of random-effects meta-regression analyses were conducted to evaluate the effect of the various covariates on the effect size of depressive symptoms (Table 4). The univariate meta-regression analyses concluded that publication year (β = 0.017, p = 0.617), duration of intervention based on the number of days (β = −0.004, p = 0.270), sample size (β = 0.001, p = 0.383), attrition rate (β = −0.003, p = 0.774), and the portion of males (β = −0.011, p = 0.134) had no effects on depressive symptoms. Thus, the between-trial heterogeneity could not be explained by these covariates.

3.5. Anxiety Symptoms

A total of 19 arms of 17 trials [2, 7, 16, 23, 40–42, 44–47, 49, 50, 70, 71, 73, 74] involving 2922 participants at the postintervention assessment, six arms of five trials [7, 23, 41, 44, 74] including 220 participants at 2 weeks to 3 months of follow-up assessment, and four trials [11, 26, 46, 50] of 1080 participants at 6–12 months of follow-up assessment were found. Meta-analyses showed no differences between the intervention and comparator at the postintervention assessment (t = −1.95, p = 0.067) and 2 weeks to 3 months (t = 2.08, p = 0.093) and 6–12 months (t = −2.82, p = 0.067) of follow-up assessment, as shown in Figure 3.

The 95% PIs were −1.19 to 0.71, −0.11 to 0.65, and −0.99 to 0.51 for three-time points. Hence, the intervention will predict an insignificant reduction in anxiety symptoms compared with comparators in future similar studies. Heterogeneity was substantial (I² = 78%) at the postintervention assessment, insignificant (I² = 0%) at 2 weeks to 3 months of follow-up assessment, and moderate (I² = 45%) at 6–12 months of follow-up assessment. To explore the sources of heterogeneity, subgroup analyses and meta-regression analyses were performed.

We conducted a series of subgroup analyses for three-time points (Table 3 and Figures S27–S49). Significant differences (p < 0.1) were found between subgroups based on the nature of participants, age groups, the type of AI chatbot they used, and how they used ITT/MDM to improve anxiety symptoms at follow-up 2 weeks to 3 months and 6 months. Subgroup analyses showed that AI-based psychotherapeutic interventions using Deprexis with people aged 41–50 in Europe who had depression or depression along with other health problems had a bigger effect (g = −0.37, 95% CI: −0.70. to −0.05) on lowering anxiety symptoms at follow-up 6 months later than the other groups. We found a smaller effect size in the trials using ITT or MDM (g = 0.05, 95% CI: −1.67 to 1.76) on decreasing depressive symptoms at follow-up 2 weeks to 3 months when compared to its counterpart.

The univariate meta-regression analyses suggested that the publication year (β = −0.001, p = 0.993), duration of intervention based on number of days (β < −0.007, p = 0.056), sample size (β < −0.001, p = 0.671), attrition rate (β = −0.007, p = 0.339), and portion of males (β = −0.006, p = 0.407) had no effects on anxiety symptoms (Table 4). Therefore, the cause of high heterogeneity could not be explained by these covariates.

3.6. Stress Symptoms

A total of eight arms of seven RCTs [7, 23, 27, 41, 44, 47, 74] among 421 participants at the postintervention assessment and six arms of five RCTs [7, 23, 41, 44, 74] involving 312 participants at 2 weeks to 3 months of follow-up assessment were pooled to evaluate the effect of intervention on stress symptoms (Figure 4). Meta-analyses did not yield any significant differences (t = −1.18 to 2.04, p = 0.098–0.277) between the intervention and comparator at the postintervention assessment and 2 weeks to 3 months of follow-up assessment.

A series of subgroup analyses were performed for two-time points (Table 3 and Figures S50–S67). Significant differences (p < 0.1) were revealed between subgroups based on the nature of participants, their different comparators, the type of psychotherapy they received, the platforms they used, and how the embodied chatbot presented itself to decrease stress symptoms at postintervention and follow-up assessment. Trials that were conducted among participants with conditions other than stress or distress had a larger effect on reducing stress symptoms at postintervention (g = −0.32, 95% CI: −4.67 to 4.02) and follow-up (g = −0.20, 95% CI: −0.78 to 0.39) compared with their counterparts. The interventions that adopted CBT (g = −0.33, 95% CI: −0.79 to 0.12) using non-Internet platforms (g = −0.33, 95% CI: −0.79 to 0.12) and passive control (g = −0.40, 95% CI: −1.07 to 0.27) had a greater effect on decreasing stress symptoms at postintervention than their counterparts. Trials conducted in non-Europe (g = −0.20, 95% CI: −0.78. to 0.39) when the interventions used embodied chatbot (g = −0.20, 95% CI: −0.78. to 0.39) had a greater effect on improving stress symptoms at follow-up assessment when compared to their counterparts.

3.7. Depressive, Anxiety, and Stress Symptoms

Three RCTs [8, 41, 74] were found to examine the effect of intervention on the total scores of depressive, anxiety, and stress symptoms using the 21-item Depressive, Anxiety, and Stress Scale [75] in 295 participants at the postintervention assessment and 2 weeks to 3 months of follow-up assessment (Figure 5). The meta-analyses did not reveal any differences between the two groups (t = 1.34–1.46, p = 0.281–0.311).

3.8. Overall Evidence

The GRADE criteria were used to evaluate 10 outcomes of this review (Tables S7), and the certainty of evidence ranged from very low to moderate. Inconsistency, indirectness, and imprecision were downgraded due to the presence of high heterogeneity, various populations and interventions, a small sample, and a wide confidence interval. Given the more than 10 trials for depressive and anxiety symptoms at postintervention, funnel plots and Egger’s test were performed. No evidence of publication bias was found because of symmetrical funnel plots and the Egger tests (p = 0.091–0.983; Figures S68 and S69).

4.1. Summary of Findings

Through 13,546 records from the 12 databases, three clinical trial registries, and other methods by using three-step comprehensive searching, we found 30 RCTs among 6100 samples across nine countries. Our review showed that AI-based psychotherapeutic interventions significantly reduced depressive symptoms at postintervention assessment with a medium effect size and 6–12 months of follow-up assessment with a small effect size compared with comparators. No significant effect of AI-based psychotherapeutic interventions was found on anxiety, stress, or the total scores of depressive, anxiety, and stress symptoms at postintervention or different periods of follow-up assessments. A series of subgroup analyses revealed significant differences in the reduction of psychological symptoms at various points based on participants’ nature, age group, type of AI chatbot, type of psychotherapy, type of platform, embodiment, different comparator, ITT/MDM, and protocol/registration. The random-effects univariate meta-regression did not detect a significant covariate on depressive and anxiety symptoms at postintervention. The majority (79.1%) of trials with ITT analysis and less than half (48.6%) of trials with perprotocol analysis rated a low risk of bias across five domains using the RoB 2.0 criteria. No publication bias was detected for depressive and anxiety symptoms at postintervention. The certainty of evidence ranged from very low to moderate for 10 psychological outcomes according to the GRADE criteria.

4.2. Depressive Symptoms

In line with a piece of previous meta-analytic evidence [18], we found that depressive symptoms significantly reduced following AI-based psychotherapeutic interventions at postintervention. Our result also indicated a significant effect at 6–12 months of follow-up assessment. Thus, AI-based psychotherapeutic interventions reduce immediate and long-term effects. AI chatbots can be designed to deliver various psychotherapies using AI technology according to different psychological principles, such as CBT [40], method of levels therapy [44], or problem-solving therapy [66]. Users may engage in the intervention in text-based or voice-activated conversations [70], and such interactions can offer psychological, relational, and emotional support [76]. Chatbots can also provide initial counseling, guide users to use a self-help library, and lead users to correct services [74]. Chatbots use AI algorithms to interpret user dialogues and conduct useful interactions. They may have a low attrition rate due to increased engagement and motivation [17]. Therefore, AI-based psychotherapeutic interventions can ameliorate depressive symptoms. Given that only seven trials included 6–12 months of follow-up assessment, a conclusion of the long-term effect of intervention cannot be made.

In our review, the majority of the interventions used rule-based response generation and less than half used NLP. Rule-based response generation consists of simple dialogue components based on rules, following a predefined decision tree and communicating in a scripted manner [74]. Conversely, generative-based response generation is more complex and relies on ML to construct its dialogues; AI uses this method to generate possible answers and enhance conversational proficiency [11]. With the increasing integration of AI technology into psychotherapy [77], future interventions can consider using advanced generative deep learning techniques that may allow AI chatbots to interact with users in an empathetic, coherent, and personalized manner [7, 74].

Seven interventions used embodied conversational agents in our review. Our subgroup analysis showed a greater effect size for embodied agents compared with nonembodied agents. An embodied conversational agent is a computer-based dialogue system with a virtual embodiment (full body or face-only) that typically interacts with users using multimodal communication cues of speech, text, animated facial expressions, or gestures [73]. Evidence showed that embodied conversational agents can build trust and rapport and can create a sense of warmth, leading to companionship and long-term usage [13, 78]. Future interventions can consider adopting embodied agents. Only one intervention [57] used emojis (images depicting facial expressions) to share and track the participants’ moods over time. Considering emotions can be used to express, imitate, and appraise the varying degrees of emotions [79]; more research is needed to evaluate its effectiveness.

Notably, the intervention failed to demonstrate superior effects at 2 weeks to 3 months of follow-up assessment in eight trials. Most comparators (62.5%) were active control groups, such as using another conversational chatbot [44, 51, 53], stress management training and CBT [47, 48], and e-books on depression [53]. This finding aligns with the results of a previous mixed-method review [16] demonstrating similar patterns. Our review revealed comparable effects between AI-based psychotherapeutic interventions and active comparators. Furthermore, a few of the participants (25%) had depressive problems at 2 weeks to 3 months of follow-up assessment. The plausible interpretation of the findings suggested that AI-based psychotherapeutic intervention may not alleviate depression symptoms in persons who are not depressed. However, we could not conclude an absolute treatment efficacy on the reduction of depressive symptoms at 2 weeks to 3 months of follow-up assessment.

Our subgroup analyses revealed that intervention had a greater effect size among participants with depression or depression combined with other health issues aged 31–40 than other age groups. One reason could be that younger adults had greater knowledge of AI [72] and more engagement in activities [80] than older adults. Therefore, young adults were more likely to adhere to interventions than older adults. Consistent with a previous review 18], the intervention significantly improved depressive symptoms in participants with depression or depression combined with other health issues. This finding suggests that interventions were more effective for treatment in depressive participants compared to other health conditions. Hence, the intervention was more beneficial for the young depressive group. Our subgroup results showed a significant subgroup difference based on the nature of participants, age groups, embodiment, ITT/MDM, and protocol/registration at follow-up 2 weeks to 12 months, but the subgroup analysis only used 1–4 trials. Hence, the results should be interpreted with caution. Hence, more investigations are recommended for future trials to confirm the findings.

4.3. Other Psychological Outcomes

Contrary to our expectations, the meta-analyses revealed that AI-based psychotherapeutic interventions did not improve anxiety symptoms, stress symptoms, and a combination of depressive, anxiety, and stress symptoms at postintervention and follow-up assessments compared with comparators. These findings are inconsistent with a previous review [17]. One possible reason may be attributed to the fact that most comparators are active control for these outcomes. Another possibility is the differences between depressive, anxiety, and stress symptoms [81]. Stress symptoms are a sense of feeling overwhelmed that measures chronic nonspecific arousal, tension, agitation, and irritability; anxiety symptoms are a sense of fear or dread that focuses on autonomic arousal, physical symptoms of anxiety, and the subjective experience of anxious affect; and depression symptoms are a sense of unhappiness or sadness, such as dysphoria, hopelessness, low self-esteem, anhedonia, and loss of interest [82, 83]. These discrepancies of feeling with specific cognitive processes and coping strategies may explain the different results [83]. At this stage, we can only speculate about the reason for this occurrence. Hence, conclusions cannot be drawn, and further studies are required.

According to our subgroup analyses, we found significant differences between subgroups based on participant’s nature, age groups, type of AI chatbot, psychotherapy, platform, comparators used, and how they used ITT/MDM to improve anxiety and stress symptoms at postintervention and follow-up assessments. However, these subgroup comparisons used only 1–6 trials in each group, and we also found an uneven number of trials in the subgroups. It is therefore important to evaluate the data cautiously[38]. Therefore, we advise further research to validate the results in subsequent studies.

4.4. Strengths and Limitations

The current systematic review has several strengths. This review was the first to examine the short- and long-term effects of AI-based psychotherapeutic interventions on psychological outcomes. A comprehensive search strategy, including 12 databases and three clinical trial registries, was used to identify 30 RCTs to reduce publication bias. The random-effect meta-analysis applied the restricted maximum likelihood method [36] with Hartung–Knapp adjustment [37]. The 95% PI for the meta-analyses was reported to predict true effects in future settings [32], and the certainty of evidence on each outcome was assessed.

Notwithstanding the strengths, this review had several limitations. First, the psychological outcomes were self-reported, which may cause social desirability bias. Second, the number of trials included in some meta-analyses was limited, especially for follow-up assessment; thus, statistical power was reduced. Third, the uneven number of trials in the subgroups may have failed to estimate valid results [38]. Fourth, included interventions were designed from a wide variety of psychological principles, and six meta-analyses revealed substantial heterogeneities that restricted the accuracy of pooled estimates. Fifth, the certainty of the evidence for the six outcomes was either very low or low, which may eliminate the confidence in implementing AI-based psychotherapeutic interventions. Sixth, some trials did not provide a regimen of intervention that limited the feature comparison. Lastly, the majority (n = 18) of the trials were from European countries, which might restrict the generalization of the findings.

4.5. Clinical Implications and Future Research

In this review, we found that depressive symptoms had small to medium effects at postintervention and follow-up assessments at 6–12 months. Given that the intervention was under variable comparator conditions, the active control groups may mask effectiveness [84]. Hence, small to medium effects can be considered either clinically important differences or the minimum clinically important differences [71, 85]. The participants could have experienced meaningful treatment benefits from AI-based psychotherapeutic interventions. However, the certainty of evidence quality of six outcomes was very low or low; thus, AI-based psychotherapeutic interventions can be supported as a supplementary intervention. Given the shortage of mental health workers globally, such intervention can be considered adjunctive to the usual treatments during the therapeutic process. Interventions can be incorporated into comprehensive web applications to facilitate access to psychotherapy amid physical distancing requirements, particularly during the ongoing COVID-19 pandemic. However, designing an interface adaptable to diverse user profiles presents certain challenges. Technical challenges are encountered in interpreting emotions in dialogues and improving features of chatbots in a human-like manner. Privacy and security of interventions are other important issues to pay attention to during the development of the intervention.

Despite the COVID-19 pandemic driving the use of AI technology, AI chatbots may have the risk of being used inappropriately. Healthcare research teams should collaborate closely and regularly with computing scientists to modify and upgrade human–computer interactions. The subgroup results suggest that intervention can target depressive populations aged 31–40 years. Sustainable heterogeneities exist in some meta-analyses, suggesting that the interventions varied across regimes of interventions, settings, and populations. Future interventions should consider using standardized regimes among specific populations in the same setting to draw a conclusive result. In addition, future research should include more detailed content and regimen according to the Template for Intervention Description and Replication Guide [86]. Given the low or very low certainty for the six outcomes, well-designed RCTs are necessary to minimize selection, performance, and reporting biases by reporting allocation concealment, blinding participants, use of ITT or MDM, and registering/publishing trial protocols. Future RCTs should recruit large samples in non-European countries to improve the generalizability of the findings.