Continuous outcome measurement in modern data-informed psychotherapies
Continuous outcome measurement in psychotherapies has become a central research topic only in the last two decades1. Here we provide a short introduction to the relevant concepts and discuss the opportunities and challenges of their implementation in clinical practice.
Most continuous outcome measurement systems comprise short self-report questionnaires which assess patient progress on a session-by-session basis. Feeding this psychometric information back to therapists enables them to evaluate whether their current approach is successful or adaptations are necessary. In order to help therapists judge whether a particular patient is improving or at risk for ultimate treatment failure, many routine outcome monitoring (ROM) systems include feedback and empirically-based decision rules.
Decision rules are generated based on datasets from clinical practice settings1. Based on such large archival datasets, expected recovery curves can be estimated and used to build thresholds indicating which scores are reflective of an increased risk for treatment failure. Having identified a patient as at risk, some ROM/feedback systems provide therapists with additional clinical support tools2. These support tools have incorporated process measures designed to assess specific change factors within and outside treatment that impact outcome.
Originally, these tools comprised two elements to help therapists adapt treatments specifically for patients at risk for treatment failure: a) an additional assessment of potential problem areas (e.g., suicidal ideation, motivation) to elucidate the patient’s individual risk profile, and b) a decision tree directing therapists to specific interventions depending on the identified risk profile. New developments have built on these ideas and included multimedia instruction materials and machine learning prediction models in order to help therapists provide the specific interventions that are most promising for a particular patient3.
Over 40 randomized clinical trials (RCTs) and several meta-analyses provide a compelling evidence base for ROM and feedback. Feedback-informed treatments have been shown to result in improved outcomes, reduced dropout, and higher efficiency than standard evidence-based treatments2, 4. The most recent and comprehensive meta-analysis reported a significant effect size advantage of d=0.15 for progress feedback compared to treatment as usual4. This effect was slightly higher for the subgroup of patients showing an initial treatment non-response (d=0.17).
When evaluating the size of these effects, it is important to keep two issues in mind. First, these effects come on top of the effects of effective evidence-based treatments. Second, feedback is a minimal low-cost technological intervention that does not put much of a burden on either patients or therapists. Accordingly, the largest RCT to date (N=2,233) demonstrated the cost-effectiveness of adding ROM and feedback to evidence-based psychotherapies within the UK Improving Access to Psychological Therapies (IAPT) system. While feedback was associated with a non-significant increase of costs per case (£15.17), it helped a significant amount of 8.01% more patients to be reliably improved at the end of the treatment5. A further enhancement of feedback effects has been repeatedly documented for additional clinical support tools2, 4.
However, not all therapists show improved outcomes when using psychometric feedback. The main reason for this seems to be the different extent to which therapists make use of the information provided by feedback systems. This usage determines the extent to which feedback is advantageous3, 4.
As a result of the above research, the use of progress feedback and empirical-based decision rules is now considered an important clinical competence and a significant component of training. As such, current challenges and research questions in this field mainly deal with the implementation of feedback systems in order to further increase uptake by mental health professionals.
The recent debate about the development and implementation of “personalized” or “precision” mental health care has also influenced research on measurement-based and data-informed psychotherapies6, 7. This development includes data-informed recommendations and decision rules for treatment selection derived from statistical and/or machine learning algorithms7. These approaches aim to predict the optimal treatment package, module or strategy given a patient’s characteristics.
Data-informed treatment selection and routine outcome monitoring have recently been combined in comprehensive decision support systems, which form the basis of modern data-informed therapies (DITs). Such DITs include (intensive) assessments before and during treatment, allowing the immediate application of empirical findings to clinical practice and enabling clinicians to develop individualized diagnoses, case conceptualizations, and treatment options, especially for patients at risk for treatment failure. The strength of such data-rich research has also been shown in other areas of public health, such as new treatment options for Parkinson's disease or patient-tailored tumor therapies.
An example of such a system is the Trier Treatment Navigator (TTN), which empirically supports clinical decisions that need to be made at the beginning of psychotherapy as well as during ongoing treatment. At the beginning of treatment, an algorithm is used to generate patient-specific treatment strategy and dropout risk predictions. Having decided on an initial treatment plan, the TTN further supports therapists with ongoing personalized feedback on their patients’ progress over the course of treatment. In order to enable therapists to evaluate these changes, a dynamic threshold indicates whether these changes are as expected or whether they indicate an increased risk for treatment failure. If the patient’s scores exceed the threshold, the TTN alerts the therapist and provides additional information on potential risk areas that might be impeding improvement. On the basis of this risk assessment, the therapist is supported with multimedia learning tools suggesting alternative clinical interventions (e.g., via video or text material).
A recently published study evaluated both components of this comprehensive navigation system in a sample of 538 patients8. Each patient-therapist dyad was randomized to either the therapist having access to the TTN (intervention group; N=335) or not (treatment as usual; N=203). Analyses revealed that patients who received their prospectively predicted optimal strategy had greater early improvements (d=0.3). The analyses regarding the personalized feedback during treatment showed that therapist variables significantly predicted or moderated the effects of the system. For example, therapists’ symptom awareness and attitude towards and confidence using feedback had an impact on treatment outcome8.
Thus, the technical implementation of DITs does not seem sufficient. Rather, quality standards for implementation as well as the scientific training of therapists are necessary, and these factors require further study. Furthermore, new methodological and technological advancements might further improve DITs (e.g., more intensive measures several times a day or digital phenotyping of stress markers).
In summary, DITs have the potential to broaden our understanding of clinical concepts and improve clinical practice. The integration of modern technologies in continuous outcome monitoring has become more sophisticated and builds a bridge to precision mental health care9. We think it is time to abandon the seemingly perpetual cycle of developing and testing new treatment packages for the average patient, which are seldom more effective than available treatment options. Instead, we encourage progress monitoring in daily practice and an increased focus on patients at risk for treatment failure.
