Research Article
Evaluation of Treatment Integrity: The Treatment Specificity Index
Florian Grikscheit,
Florian Weck, Martin Hautzinger, Thomas Heidenreich, Maria Weigel, Visar Rudari, Christine Schimming, Ulrich Stangier,
Corresponding Author
Florian Grikscheit
University of Frankfurt
Please address correspondence to: Florian Grikscheit, Goethe University Frankfurt, Institute for Psychology, Department of Clinical Psychology and Psychotherapy, Varrentrappstr. 40–42, D-60486 Frankfurt, Germany. E-mail: [email protected]Search for more papers by this authorFlorian Grikscheit,
Florian Weck, Martin Hautzinger, Thomas Heidenreich, Maria Weigel, Visar Rudari, Christine Schimming, Ulrich Stangier,
Corresponding Author
Florian Grikscheit
University of Frankfurt
Please address correspondence to: Florian Grikscheit, Goethe University Frankfurt, Institute for Psychology, Department of Clinical Psychology and Psychotherapy, Varrentrappstr. 40–42, D-60486 Frankfurt, Germany. E-mail: [email protected]Search for more papers by this authorAbstract
Objective
To differentiate between treatments in a randomized controlled trial (RCT), therapies need to display not only high purity but also high specificity. The aim of this study was to demonstrate the benefits of calculating a treatment specificity index for the evaluation of treatment differentiation.
Method
Based on an RCT of relapse prevention in depression, comparing a cognitive with a psychoeducational treatment, the specificity and the purity index were calculated.
Results
As indicated by the specificity index, both conditions differed in their levels of implemented specific and common interventions. A significant relationship was found between symptom change before a therapy session and treatment specificity.
Conclusions
The specificity index is an appropriate method for enhancing the internal validity of RCTs in evaluating treatment integrity.
References
- Barber, J. P. (2009). Toward a working through of some core conflicts in psychotherapy research. Psychotherapy Research, 19, 1–12. doi:10.1080/10503300802609680
- Beck, A. T., Ward, C. H., Mendelson, M., Mock, J., & Erbaugh, J. (1961). An inventory for measuring depression. Archives of General Psychiatry, 4, 561–571. doi:10.1001/archpsyc.1961.01710120031004
- Bhar, S. S., & Beck, A. T. (2009). Treatment integrity of studies that compare short-term psychodynamic psychotherapy with cognitive-behavior therapy. Clinical Psychology: Science and Practice, 16, 370–378. doi:10.1111/j.1468-2850.2009.01176.x
- Bockting, C. L. H., Schene, A. H., Spinhoven, P., Koeter, M. W. J., Wouters, L. F., Huyser, J., & Kamphuis, J. H. (2005). Preventing relapse/recurrence in recurrent depression with cognitive therapy: a randomized controlled trial. Journal of Consulting and Clinical Psychology, 73, 647–657. doi:10.1037/0022-006X.73.4.647
- Boswell, J. F., Gallagher, M. W., Sauer-Zavala, S. E., Bullis, J., Gorman, J. M., Shear, M. K., Woods, S., & Barlow, D. H. (2013). Patient characteristics and variability in adherence and competence in cognitive-behavioral therapy for panic disorder. Journal of Consulting and Clinical Psychology, 81, 443–454. doi:10.1037/a0031437
- Crits-Christoph, P., Gibbons, M. B. C., & Mukherkee, D. (2013). Psychotherapy process-outcome research. Bergin and Garfield's handbook of psychotherapy and behavior change. ( 6th ed.) ( S.298-340.) New York: Wiley & Sons.
- Dennhag, I., Gibbons, M. B. C., Barber, J. P., Gallop, R., & Crits-Christoph, P. (2012). How many treatment sessions and patients are needed to create a stable score of adherence and competence in treatment of cocaine dependence? Psychotherapy Research, 22, 475–488. doi:10.1080/10503307.2012.674790
- Fava, G. A., Rafanelli, C., Grandi, S., Conti, S., & Belluardo, P. (1998). Prevention of recurrent depression with cognitive behavioral therapy: preliminary findings. Archives of General Psychiatry, 55, 816–820. doi:10.1001/archpsyc.55.9.816
- Frank, E., Kupfer, D. J., Wagner, E. F., McEachran, A. B., & Cornes, C. (1991). Efficacy of interpersonal psychotherapy as a maintenance treatment of recurrent depression: contributing factors. Archives of General Psychiatry, 48, 1053–1059. doi:10.1001/archpsyc.1991.01810360017002
- Hautzinger, S., Schlösser, R., & Barocka, A. (2006). Manualisierte Psychoedukation (MAPE): Anleitung für Therapeuten und Materialien für Patienten [Manualized psychoeducation (MAPE): introduction for the therapist and material for the patient]. Unpublished manuscript, University of Tübingen and University of Jena, Germany.
- Jarrett, R. B., Kraft, D., Doyle, J., Foster, B. M., Eaves, G. G., & Silver, P. C. (2001). Preventing recurrent depression using cognitive therapy with and without a continuation phase: a randomized clinical trial. Archives of General Psychiatry, 58, 381–388. doi:10.1001/archpsyc.58.4.381
- Leichsenring, F., Salzer, S., Hilsenroth, M. J., Leibing, E., Leweke, F., & Rabung, S. (2011). Treatment Integrity: An unresolved issue in psychotherapy research. Current Psychiatry Reviews, 7, 313–321. doi:10.2174/157340011797928259
10.2174/157340011797928259 Google Scholar
- Loeb, K. L., Wilson, G. T., Labouvie, E., Pratt, E. M., Hayaki, J., Walsh, B. T. … Fairburn, C. G. (2005). Therapeutic alliance and treatment adherence in two interventions for bulimia nervosa: a study of process and outcome. Journal of Consulting and Clinical Psychology, 73, 1097–1107. doi:10.1037/0022-006X.73.6.1097
- Luborsky, L., McLellan, A. T., Woody, G. E., O'Brien, C. P., & Auerbach, A. (1985). Therapist success and its determinants. Archives of General Psychiatry, 42, 602–611. doi:10.1001/archpsyc.1985.01790290084010
- Luborsky, L., Singer, B., & Luborsky, L. (1975). Comparative studies of psychotherapies: Is it true that “everyone has von and all must have prizes”? Archives of General Psychiatry, 32, 995–1008. doi:10.1001/archpsyc.1975.01760260059004
- Portney, L. G., & Watkins, M. P. (2009). Foundations of clinical research: applications to practice ( 3rd ed.). London: Pearson.
- Reisner, A. D. (2005). The common factors, empirically validated treatments, and recovery models of therapeutic change. The Psychological Record, 55, 377–399.
- Risch, A. K., Stangier, U., Heidenreich, T., & Hautzinger, M. (2012). Kognitive Erhaltungstherapie bei rezidivierender Depression: Rückfälle verhindern, psychische Gesundheit erhalten. [Cognitive maintenance therapy for recurrent depression: preventing relapse, promoting mental health]. Berlin , Heidelberg, New York: Springer.
10.1007/978-3-642-04889-0 Google Scholar
- Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlation: uses in assessing rater reliability. Psychological Bulletin, 86, 420–428. doi:10.1037/0033-2909.86.2.420
- Southam-Gerow, M. A., & McLeod, B. D. (2013). Advances in applying treatment integrity research for dissemination and implementation science: introduction to special issue. Clinical Psychology: Science and Practice, 20, 1–13. doi:10.1111/cpsp.12019
- Stangier, U., Hilling, C., Heidenreich, T., Risch, A. K., Barocka, A., Schlösser, R., …Hautzinger, M. (2013). Maintenance cognitive-behavioral therapy and manualized psychoeducation in the treatment of recurrent depression: a multicenter prospective randomized controlled trial. The American Journal of Psychiatry, 170, 624–632. doi:10.1176/appi.ajp.2013.12060734
- Strunk, D. R., Cooper, A. A., Ryan, E. T., DeRubeis, R. J., & Hollon, S. D. (2012) The process of change in cognitive therapy for depression when combined with antidepressant medication: Predictors of early intersession symptom gains. Journal of Consulting and Clinical Psychology, 80, 730–738. doi:10.1016/j.brat.2010.03.011
- Teasdale, J. D., Segal, Z. V., Williams, J. M., Ridgeway, V. A., Soulsby, J. M., & Lau, M. A. (2000). Prevention of relapse/recurrence in major depression by mindfulness-based cognitive therapy. Journal of Consulting and Clinical Psychology, 68, 615–623. doi:10.1037/0022-006X.68.4.615
- Waltz, J., Addis, M. E., Koerner, K., & Jacobson, N. S. (1993). Testing the integrity of a psychotherapy protocol: assessment of adherence and competence. Journal of Consulting and Clinical Psychology, 61, 620–630. doi:10.1037/0022-006X.61.4.620
- Webb, C. A., DeRubeis, R. J., & Barber, J. P. (2010). Therapist adherence/competence and treatment outcome: A meta-analytic review. Journal of Consulting and Clinical Psychology, 78, 200–211. doi:10.1037/a0018912
- Webb, C. A., DeRubeis, R. J., Dimidjian, S., Hollon, S. D., Amsterdam, J. D., & Shelton, R. C. (2012). Predictors of patient cognitive therapy skills and symptom change in two randomized clinical trials: the role of therapist adherence and the therapeutic alliance. Journal of Consulting and Clinical Psychology, 80, 373–381. doi:10.1037/a0027663
- Weck, F., Hilling, C., Schermelleh-Engel, K., Rudari, V., & Stangier, U. (2011). Reliability of adherence and competence assessment in cognitive behavioral therapy. The Journal of Nervous and Mental Disease, 199, 276–279. doi:10.1097/NMD.0b013e3182124617
- Weck, F., Rudari, V., Hilling, C., Hautzinger, M., Heidenreich, T., & Stangier, U. (2013). Relapses in recurrent depression one year after maintenance cognitive-behavioral therapy: the role of therapist adherence, competence, and the therapeutic alliance. Psychiatry Research, 210, 140–145. doi:10.1016/j.psychres.2013.05.036
- Weck, F., Weigel, M., Hautzinger, M., Barocka, A., Schlösser, R. G. M., & Stangier, U. (2012). Relapses in recurrent depression 1 year after psychoeducational treatment: the role of therapist adherence and competence, and the therapeutic alliance. Psychiatry Research, 195, 51–55. doi:10.1016/j.psychres.2011.07.025
- Weck, F., Weigel, M., Richtberg, S., & Stangier, U. (2011). Reliability of adherence and competence assessment in psychoeducational treatment: influence of clinical experience. Journal of Nervous and Mental Disease, 199, 983–986. doi:10.1097/NMD.0b013e3182392da1
- Zobel, I., Karim, A., Kech, S., Berger, M., & Schramm, E. (2008). Wie adhärent wird Clinical Management in randomisierten, kontrollierten Studien durchgeführt? [The Adherence of Clinical Management in a Randomized Controlled Trial]. Psychotherapie, Psychosomatik, medizinische Psychologie, 58, 395–402. doi:10.1055/s-2008-1067361
