1 INTRODUCTION

Major depressive disorder (MDD) is a major health challenge and often manifests itself in a recurring or chronic condition.¹ In the absence of biomarkers,² guiding diagnosis and treatment has traditionally relied on subjective self-report measures such as questionnaires and interviews by mental health professionals.³ As these are administered at sporadic points in time, they are (a) prone to retrospective bias,⁴ and (b) even full episodes might be missed.⁵ Patterns such as moment-to-moment fluctuations in symptoms which might be central regarding potential triggers and prodromal warning signs will remain undetected.⁵

With the features of near real-time, continuous, active and passive data collection, the use of ambulatory assessment (AA) is advantageous as it reduces retrospective recall bias and increases ecological validity.^{5, 6} AA involves the assessment of momentary self-reported symptoms and behaviors assisted by personal digital devices while patients perform their normal daily activities in their natural environment.⁷ One idea to reduce reliance on self-reports is to derive objective parameters from speech. Two different streams of parameters have been used in the past: acoustic and linguistic features. Multiple studies revealed differences in the acoustic dimension of speech (e.g. pitch, jitter) between depressive and non-depressive states⁸ or between depressed and healthy individuals (for reviews see⁹ and¹⁰). Leveraging the development of natural language tools such as the Linguistic Inquiry and Word Count (LIWC¹¹), linguistic style or the choice of words in association with MDD is also investigated. Pennebaker and colleagues¹² state that individuals' everyday word use, interpreted as a behavioral manifestation of thoughts and emotions, can reveal affective and cognitive processes characteristic of mood disorders.

As the potential linguistic feature space is extensive, researchers either pursue a brute-force approach (e.g. References [13, 14]) or inform their feature selection by theoretical considerations of MDD (e.g.^{15, 16}). With regard to the latter, in many studies the use of (1) positive and negative emotion words, (2) first person singular pronouns, and (3) past tense words were analyzed. Traditional theories of depression such as Beck's Cognitive Model of Depression¹⁷ or heightened self-focus theories¹⁸ suggest such word use. However, empirical results are often mixed which might be due to the variety of methodological approaches and samples. Improved depressive states have been found to be associated with: (i) heightened use of positive emotion words and little use of negative emotion words (e.g.^14-16), but also greater use of both positive and negative emotion words¹⁹; (ii) little use of first-person singular pronouns (review by References [14, 20, 21]), but also opposite effects²² and null-effects¹⁶; (iii) little use of past tense (e.g. References [23, 24]), but also the contrary.¹⁹

In the light of future automatic everyday monitoring systems, which shall monitor the change of trajectories of patient diseases and prevent upcoming episodes, two approaches are of particular relevance: (1) longitudinal studies exploring within-person effects and (2) data collection in the field instead of controlled therapy sessions. According to current findings, only two studies meet these criteria.^{13, 22} In the first study, 120 linguistic and acoustic features extracted from mental health patient recordings were analyzed.¹³ Positive and negative emotion words and overall speech features were associated with global clinical assessments and depression subscores. However, only 15% of the sample were MDD patients and machine learning results refer to the full set of speech features. In another study, speech samples from MDD patients were aggregated to pre- and post-treatment assessments.²² Decreases in negative emotion words, no difference in positive emotion words and increases in first person pronouns for post- versus pre-treatment assessment were identified.

To inform the development of an automatic monitoring tool, robust studies in the natural environment of patients during and between state of the art assessed depressive episodes are needed. Those that exist exploit the potential of AA only weakly, for instance by aggregating multiple assessment points. Additionally, while there is a growing body of work on interindividual differences, studies on longitudinal intraindividual differences during states of different depression severities are lacking. Understanding the fluctuations that occur may help to get a clearer clinical picture and provide warning of impending episodes.

With the present study, we aim to overcome the above-mentioned limitations. Based on the available evidence, the linguistic style of speech of clinically diagnosed MDD patients in association with their reported depressive state has not yet been compared longitudinally during treatment with high temporal resolution. We analyzed a dataset of speech samples and concomitant self-reported momentary affective states collected longitudinally via AA from MDD patients undergoing sleep deprivation therapy (SDT) in an exploratory way. SDT is a chronotherapeutic intervention that can rapidly improve depression severity.²⁵ For our linguistic study, this is advantageous, as maximum variance in depressive symptomatology is gained within a short period of time, enabling to observe associated modifications in linguistic style within days. Finally, instead of averaging data from multiple sampling points into composite measures, we stay on the most granular level of our data collection (multiple data points per day per person) expecting to capture the dynamic ebb and flow of affective states.

To limit alpha error inflation and informed by the existing literature, we decided to include four LIWC word categories that closely resemble psychopathological phenomena. Accordingly, we formulated four hypotheses regarding associations between LIWC categories and concomitantly reported affective momentary states: With lower reported depression severity, we expected (1) more words in the LIWC category positive emotion words, (2) less words in the LIWC category negative emotion words, (3) less words in the LIWC category first person pronouns and (4) less words in the LIWC category past tense. In addition, we analyzed associations of these categories with more broadly defined reported momentary affective states (i.e., positive and negative affective state, valence, energetic arousal, calmness). We hypothesized that the associations of LIWC categories with reported negative affective state are in the same direction as those for reported depression severity and in the opposite direction for the remaining reported affective states.

2 MATERIALS AND METHODS

Detailed information on the sample, the study procedure and assessment tools can be found in Reference [8] and Appendix S1. Data were collected from 30 inpatients at the Central Institute of Mental Health in Mannheim, Germany, who experienced an acute depressive episode (ICD-10) at admittance to the hospital. Patients were informed about the study procedures, gave informed consent before being included in the study, and could withdraw at any time. Patients underwent SDT as part of their treatment, which involved staying awake for 36 h. AA collection covered the period before, during, and after SDT (up to 26 days in total). In the first week of the study, 3 e-diary prompts per day (morning, afternoon, and evening) were send via smartphones, which was reduced to morning and evening prompts after the first week to minimize patient burden. E-diary prompts included a request to respond to items about the current affective state on (a) the short version of the Center for Epidemiologic Studies Depression Scale (Allgemeine Depressionsskala in German (ADS-K)²⁶), (b) 15 positive and negative affective state items from an item pool,²⁷ and (c) a short version of the Multidimensional Mood Questionnaire (MDMQ²⁸), and to record a selfie video to report current feelings (“Please describe in 10–20 s how you currently feel.”).

From originally 899 recorded selfie videos we had to exclude 155 files, mostly for technical reasons (see Appendix S1). We obtained transcripts of the audio tracks using an automatic speech recognition system²⁹ and corrected them manually. The patients' transcripts were analyzed using the language processing tool LIWC,¹¹ a well-established, transparent and reliable computer text analysis program that automatically classifies words into categories stored in a predefined dictionary.³⁰ The output is a percentage of words allocated to the linguistic categories in relation to the total number of words in a text sample. The German LIWC 2015 version used in this study contains 18,711 words and word stems in more than 80 word categories.³⁰ We focus on four LIWC categories: positive emotion words (posemo), negative emotion words (negemo), first person singular pronouns (I), and past tense (focuspast).

To analyze the within-subject association of LIWC categories and reported momentary affective states, we applied multilevel modeling³¹ with person centered time-variant level-1 predictors (LIWC categories) and added the predictors time and time² in hours (centered at 2 p.m.) as covariates. We calculated separate models for each LIWC category: positive emotion words, negative emotion words, first person singular, past tense, and each affective state as outcome (reported depression severity (ADS-K), positive affective state, negative affective state, valence, energetic arousal, calmness), resulting in 24 models. As the number of speech samples was limited, we included all reported momentary affect ratings and speech recordings available irrespective of the time of assessment. We set the initial α level to 5%, applied Bonferroni corrections construct-wise (αadj = 0.008), and performed all analyses in R (version 4.3.1 [16 June, 2023]). See⁸ for details of statistical parameters and used R packages.

3 RESULTS

In the final analysis, we included 744 pairs of sampled speech and affective states, an average of 34 ± 20.16 pairs per person (range = 5–87). Descriptive statistics on word categories and reported affective states, ICCs, and reliability indices (McDonald Omega and Spearman-Brown coefficients) are presented in Table 1. The upper part depicts the four LIWC categories analyzed, starting with the total word count of all speech samples with 28,128 words. Hereafter follow the four LIWC categories with the highest percentage of words found for first person singular pronouns (10.81%), followed by positive emotion words (6.22%), negative emotion words (4.64%), and past focus (3.72%). The amount of between-person variance represented by the ICCs (0.14–0.32) indicates that most of the variance is within-subject, strongly arguing for the chosen assessment and analysis approach. The lower part of Table 1 refers to the reported affective momentary states. Mean reported depression severity (ADS-K) is presented for all assessment points (1.2 ± 0.3) and for the baseline assessment only (1.5 ± 0.3), both to be classified as medium high on a scale from 0 to 3. Negative (2.4 ± 0.5) and positive (2.1 ± 0.4) affect items were medium high on average (scale 1–5), as well as MDMQ scores (valence: 44.7 ± 10.8; energetic arousal: 41.6 ± 10.6; calmness: 43.4 ± 11.45; scale 1–100). The ICCs (0.26–0.60) encourage us to have sufficient within-person variance to run multilevel regression analysis. The reliability indices were good to excellent as evaluated according to McDonald Omega and Spearman-Brown coefficients.

In Figure 1, we illustrate Pearson correlation coefficients with person-mean-centered scores. The left graph displays correlations of the six affective state variables, the right graph displays correlations of the four LIWC categories. The color scheme encodes the direction and the strength of the correlation; blue indicates a positive correlation with darker blue indicating stronger correlations; red reflects the same pattern but for negative correlations. On the y-axis and the diagonal, the names of the respective variable pairs are depicted.

Regarding correlational analyses, two main patterns emerged (Figure 1): The e-Diary items on affective states showed a high coherence while the LIWC categories were weakly correlated. In detail, we found strong positive correlations (r > 0.5) between reported depression severity and negative affective state; strong negative correlations between reported depression severity and positive affective state, and all MDMQ subscales. The same was found for negative affective state except for a moderately strong negative correlation with energetic arousal. Correlations between positive affective state and other momentary states were opposite to the pattern found for the reported depression severity (all strong). Valence correlated strongly positively with energetic arousal and calmness. Calmness and energetic arousal correlated moderately in a positive way.

Regarding LIWC features, positive and negative emotion words showed the highest (negative) correlation, which can be classified as moderate. Past focus words correlated weakly with negative emotion words and first person singular pronouns. The other pairings were not meaningfully linked to each other.

Table 2a represents the fixed effects of the separate multilevel models with positive and negative emotion words, first person singular, and past focus as statistical predictors. Momentary reported depression, positive and negative affective state are defined as outcomes. Table 2b has the same structure, but reports outcomes related to MDMQ dimensions valence, energetic arousal and calmness. All models also included the centered time and time² variables as statistical predictors. As findings are comparable, we report the simpler models here and expanded models in the Appendix S1.

4 DISCUSSION

This is the first study to investigate momentary affective states and concomitant speech samples collected longitudinally via AA from patients diagnosed with MDD during SDT inducing rapid shifts in symptomatology. We found a coherent pattern for the LIWC categories positive and negative emotion words in association with concurrently reported affective states. Specifically, we found that using more positive emotion words and fewer negative ones is linked to lower reported depression severity and negative affective states. Additionally, it corresponds to higher levels of positive affective states, (positive) valence, energetic arousal, and calmness. These results suggest that changes in linguistic style extracted from ambulatorily assessed everyday speech samples may be indicative of mood changes.

As hypothesized, we found a positive association between MDD severity and the frequency of negative emotion words and a negative association between MDD severity and positive emotion words. This replicates previous work^{14, 15} and is in line with Beck's depression model,¹⁷ suggesting that patients suffering from MDD have a severely negative attitude. Interestingly, our results generalized over the more broadly defined affective states assessed (positive and negative affective states, valence, calmness, energetic arousal). This connects to discussions on speech features being considered disorder-specific or transdiagnostic.^{13, 16}

With respect to our hypothesis on first person singular pronoun use, our results were more limited. An increased use was observed in association with negative affective states and a trend for depressive symptoms, but not with any of the other variables. In a previous review, a small positive correlation was found indicating higher self-focus during depressive states.²⁰ One possible explanation for the lack of a significant relationship between reported depression and first person singular personal pronouns lies in the nature of the task. Unlike studies that have collected language content during therapy sessions,^{15, 16} or free spontaneous speech,¹⁹ patients in this study talked about their current mood in a selfie video, which might have been beneficial as more mood related content has been reported. Furthermore, earlier studies concluded that linguistic indicators of self-referencing seem to rather reflect the negative affective component of depressive symptoms. Thus, they might be rather broader indicators of negative affectivity than specifically indicating depressive states.²¹

Our fourth hypothesis, which was based on prior studies showing that MDD patients refer more frequently to the past in their narratives compared to healthy participants,^{23, 24} was not supported by our data. However, in a more recent study, more past focus words were associated with less depression severity.¹⁹ Surprisingly and not in line with any of these previous findings, in our study the amount of past words was not associated with MDD. It is likely that our task instructions resulted in a rather low variance of past word frequencies between time points and different affective states in contrast to earlier language samples that often instructed a life review feature.²³

Our results suggest that short speech samples collected in everyday life may serve as a behavioral marker of changes in depression severity and may to some extent compensate for the lack of objective biomarkers in future studies. The use of emotion words in particular (positive and negative) may provide clinically meaningful information that could contribute to the detection of impending episodes. Taking the advantage of the widespread availability of smartphones and smartwatches that people wear every day, speech-based monitoring of MDD is a promising approach. It should be noted that we consider such an everyday speech tool to be a supportive monitoring system, coexisting with clinical sessions which are essential. However, due to its continuous and unobtrusive applicability, speech-based monitoring could offer the crucial advantage of bridging the time between clinical sessions. Patients who are approaching their personal relapse threshold could be identified earlier, for examp;e, if their linguistic style shows the use of more negative emotion words.

While promising in theory, the development of a speech-based monitoring system faces many challenges. We have to identify a speech sampling strategy that is (a) most informative, (b) comes with the smallest patient burden, (c) preserves the privacy of patients and bystanders, and (d) impacts smartphone battery life minimally. More specifically, it is unclear whether active sensing (i.e. asking patients to actively record a speech sample or call a system) is necessary or whether passive sensing is sufficient. Especially in the case of passive sensing, the need for privacy-preserving tools requires technological development, for example, to filter out bystander's speech or to create transcripts in real-time.

This pilot study had several limitations. First, the sample size was limited and future studies are needed to investigate replicability of results as well as generalizability beyond an inpatient sample. Still, the dataset at hand is unique as it is based on a true within-subject design with a relatively high number of data points per patient. A meaningful amount of variability in reported depression severity over a short period of time is stimulated by the study design; this is crucial from a theoretical perspective, as meaningful variance is necessary in both parameters to uncover existing associations. Second, SDT might have additional effects, such as fatigue, beyond the antidepressant one. This has to be explored in future work. Third, as we focused our analysis on four LIWC categories, it remains unclear whether other word categories are also sensitive to changing depressive states. However, as a vast variety of possible linguistic features is assess- and extractable in theory, feature selection should be made in a considered manner in order to limit alpha error inflation and to increase replicability.³² We decided to focus on features resembling psychopathological phenomena in MDD as closely as possible and for which both a theoretical foundation and previous empirical evidence are available. Fourth, LIWC is a word count based automated language analysis and does not consider the context in which the categorized words are used or negations. However, recent findings show comparability between LIWC-based and hand-labeled (and thus “corrected”) categories.²² LIWC categories can be compared across studies and have thus proven to be reliable and valid.²¹ Fifth, although we used automated transcription, its manual correction was time-intense. For the future, especially if linguistic analysis is used in the context of AA, it is crucial to have reliable speech-to-text tools, as manual correction won't be feasible. Finally, our speech task instruction might have biased word use. Whereas this might have increased number of negative and positive words, it might have limited words related to the past. While previous LIWC studies mostly collected longer speech samples or narratives, we requested brief samples to reduce patient burden. Still identifying significant associations between linguistic style and reported depression severity supports the reliability of the measurement and is promising for AA speech sample collection, where minimizing effort per assessment is crucial for feasibility.

To conclude, our study provides evidence for associations between fluctuations in the use of positive and negative emotion words and momentary affective states. These changes happened within a relative short time, not lagging behind and as such are a real marker. We want to particularly emphasize that the sample consisted of clinically diagnosed patients with an acute depressive episode. The intervention study design involving SDT ensured a maximum of within subject dynamics of affective states within just a few days. The use of these words as a marker is promising for the development of future technology predicting upcoming episodes on an individual level. And this research adds important observations with respect to the aim of developing an automated depression monitoring system.

AUTHOR CONTRIBUTIONS

Marcella Rietschel, Jerome C. Foo, Josef Frank, Stephanie H. Witt, Lea Sirignano, Maria Gilles, Fabian Streit, Ulrich W. Ebner-Priemer planned the investigation and developed the sampling scheme. Maria Gilles, Lea Sirignano were responsible for data collection. Lisa-Marie Hartnagel preprocessed the data, carried out statistical analysis, interpreted results and drafted the manuscript. Matthias F. Limberger, Ulrich W. Ebner-Priemer and Lea Sirignano contributed to analysis, interpretation and the drafting of the manuscript. Andrea B. Horn contributed to Linguistic Inquiry and Word Count analyses. All authors revised and edited the manuscript critically and had final approval of the version to be published.

FUNDING INFORMATION

This work was funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation)—GRK2739/1—Project No. 447089431—Research Training Group: KD²School—Designing Adaptive Systems for Economic Decisions; ERA-NET NEURON “EMBED-impact of Early life MetaBolic and psychosocial strEss on susceptibility to mental Disorders; from converging epigenetic signatures to novel targets for therapeutic intervention” [01EW1904]; supported in part by the DFG Project No. 402170461 [TRR 265 “Losing and Regaining Control over Drug intake”] and the Federal Ministry of Research and Education (BMBF) [RELATER 01EF1803B: Improving communication in psychiatric care of refugees using mobile technology].

CONFLICT OF INTEREST STATEMENT

Ulrich W. Ebner-Priemer reports consultancy for Boehringer-Ingelheim and speaker honorarium from Angelini Pharma, which both had no influence on the content of this article. All other authors declare no conflict of interest.

ETHICS STATEMENT

The Ethics Committee II of the Medical Faculty Mannheim, University of Heidelberg, approved the study protocol.