Purpose: The aim of this article is to compare treatment efficacy of cognitive-behavior therapy (CBT) and short-term dynamic psychotherapy (STDP) in high-quality studies.

Methods: A systematic review was conducted using Medline and PsycINFO. Effect sizes were calculated.

Results: Eleven publications (seven individual studies) respected the inclusion criteria. The results reveal that both CBT and STDP are efficacious treatments. In anxiety disorders, CBT shows a small to moderate advantage over STDP in primary symptoms from pretreatment to post treatment and a negligible to moderate advantage from pretreatment to follow-up. Results are mixed in personality disorders. In depressive disorders, CBT shows a negligible to small advantage from pretreatment to post treatment in primary symptoms, but STDP shows a small advantage from pretreatment to follow-up. In bulimia nervosa, CBT shows a small advantage in primary symptoms from pretreatment to post treatment. Differences are also observed in secondary symptoms for each disorder. Post hoc analyses revealed that STDP shows a negligible to small advantage over CBT from post treatment to follow-up.

Conclusion: Differences in treatment efficacy seem to vary according to disorders, type of treatment outcomes (primary or secondary), and measurement time (post treatment or follow-up), but results have to be interpreted cautiously.

Keywords: Cognitive therapy; Psychotherapy, psychodynamic; Review; Treatment outcome

CBT: Cognitive-Behavior Therapy; IP: Interpersonal Psychotherapy; LTDP: Long-Term Dynamic Psychotherapy; STDP: Short-Term Dynamic Psychotherapy

Psychoanalysis gave rise to a variety of models across the years, including short-term dynamic psychotherapy (STDP). STDP differs from long-term dynamic psychotherapy (LTDP) in that it puts a greater emphasis on developing a therapeutic alliance and positive transference, focuses on specific themes, conflicts, or client’s feelings in the here and now and towards the therapist (transference), and is more goal-oriented. In addition, the therapist plays a more active role and the number of sessions is shorter in STDP than in LTDP [1]. Meta-analyses and literature reviews have investigated the efficacy of STDP in comparison to no treatment and other forms of psychotherapies [2-8]. The results of these studies are summarized in Supplementary Table 1. With few exceptions, the results suggest that STDP is more efficacious than no treatment at post treatment and as efficacious as other psychotherapies at post treatment and follow-ups.

Cognitive-behavior therapy (CBT) is another short-term model that proved to be efficacious with many axes I and axis II disorders [9]. Based on results of a meta-analysis comparing CBT to other treatment modalities (including psychodynamic therapy), Tolin has suggested that CBT was the treatment of choice for many if not most disorders [10]. A reanalysis replicated the results for disorder-specific symptoms, but additional analyses revealed that CBT did not show superiority over other models for non-disorder specific symptoms or for anxiety disorders [11]. In Tolin’s study [10], psychodynamic therapy included both LTDP and STDP, precluding any conclusions about comparisons between CBT and STDP. Direct comparisons between CBT and STDP were investigated in only a few meta-analyses, and results were mixed. For example, one meta-analysis concluded that CBT was more efficacious than STDP in a variety of disorders including depression [8], whereas another concluded that STDP was as efficacious as CBT in depression [6]. The former meta-analysis was criticized for including studies with poor methodological designs [1], whereas the latter focused on a single disorder and included studies with designs that were not optimal (for example, treatments in one study were not manualized; treatment integrity was not always assessed by external objective raters). The quality of study designs may be an important parameter in explaining differences between meta-analysis results. Studies with poorer designs are more likely to inflate treatment efficacy [12].

However, what are the criteria for high-quality designs in studies comparing treatment efficacy of CBT and STDP? First, studies should be randomized controlled trials comparing treatment efficacy of CBT and STDP. Randomized controlled trials provide the best design for investigating treatment efficacy, and trials comparing CBT and STDP allow direct comparisons of treatment effect size because the data are extracted from the same experimental comparisons [13]. Second, treatment should be delivered according to treatment manuals or treatment manual-like guides. There is evidence that studies that do not use treatment manuals are more sensitive to therapist effects [14]. Third, treatment adherence should be assessed with direct observation (audio or video recordings) by external objective raters. The mere presence of treatment manuals does not ensure that treatments are delivered as intended or that therapists do what they report doing. Treatments that are not delivered appropriately may result in poor outcomes and misleading conclusions [15]. Fourth, therapists should be experienced clinicians (either clinical psychologists or psychiatrists) trained in the modality they are delivering in the study. It has been suggested that psychodynamic therapy is more complex to deliver than CBT and therefore studies using trainees rather than experienced therapists may be biased in favor of CBT [16]. Fifth, outcome measures should be validated. Sixth, there should a sufficient number of participants per treatment group. Hsu suggested that a sample size of at least 20 participants per treatment group decreased the likelihood of finding baseline group differences after randomization [17]. Finally, the number of sessions should be sufficient to enable improvement on treatment outcomes. There is however no clear-cut number of sessions allowing to differentiate between ultra-brief therapy, short-term therapy, and long-term therapy. In the treatment of depression, it has been argued that a minimum of 13 sessions of STDP is necessary to achieve prominent change [6]. An artificial cut-off for Axis I disorders may be 30 sessions. In the treatment of personality disorders (Axis II), treatments providing 40 sessions or lower could be considered short-term treatments.

Regarding systematic review and meta-analyses, there may be methodological issues in combining results of studies conducted in children and adults, treatments delivered in individual and group formats, or treatments delivered or not in combination with another treatment (group counseling, pharmacotherapy, and placebo pills). Intervention techniques used with children may be different than those employed with adults. Participants in treatment groups may socialize and interact with each other, which could contribute to treatment outcome [18], and CBT may be more suitable to group format than STDP. Studies providing CBT and STDP in combination with another treatment prevent sound conclusions about their individual efficacy (unless the design included a treatment group without combination). To our knowledge, no systematic reviews have compared treatment efficacy of CBT and STDP in high-quality studies among adult participants, provided in a face-to-face (as opposed to remote therapy) and individual format, and on a variety of disorders.

The aim of this systematic review is to compare the treatment efficacy of CBT and STDP in high-quality studies. More specifically, the article sought to answer the following question: Is CBT more efficacious than STDP? Studies providing a direct comparison of CBT and STDP delivered to adults in individual and face-to face settings (as opposed to remote therapy), and without treatments being combined with another modality will be considered in this systematic review. The results of this review could be important for therapists, insurance companies, policymakers, and clients.

Search of the literature

A computerized search of the literature was conducted in Medline and PsycINFO databases on studies published up to December 21, 2014, using a combination of keyword terms covering the concepts of CBT, STDP, and randomized controlled trials. (The complete keyword combination is listed in Appendix) Additional references were identified through examination of reference lists of articles that were assessed for eligibility and from past readings of the first author.

Inclusion and exclusion criteria

Titles and abstracts of the retrieved articles were screened to select studies that met the following inclusion criteria:

1. Studies providing a direct comparison of CBT and STDP. The categorization used by original authors served as guiding principle of classification and was complemented, if necessary, by requiring a clear reference to specific themes and technique interventions, as well as relevance of technique interventions to target population for each treatment modality. For CBT, themes included automatic thoughts, cognitive distortions, or learning theory; technique interventions included cognitive restructuring, exposure and response prevention, role play, or relaxation. For STDP, themes included the unconscious, drives, transference, or internal conflicts; intervention techniques included interpretation, confrontation, or clarification. Examples of non-representative intervention techniques are a focus on relaxation training in social phobia for CBT or a proscription to address conflicts for STDP. Interpersonal psychotherapy (IP) was not considered to be a form of STDP in this systematic review in accordance with founders of IP and because IP showed a strong adherence to ideal CBT prototype [19]. However, psychodynamic interpersonal therapy [20] may be considered to be a form of STDP and was included in this systematic review.
2. Participants aged 18 years or older (or mean age greater than 18 years old if age range was not provided).
3. Therapies provided in individual format, face-to face (excluding remote therapy such as videoconferencing).
4. Studies published in English or French.
5. Participants randomly assigned to CBT or STDP.
6. Use of treatment manuals or treatment manual-like guides.
7. Assessment of treatment adherence using direct observation (audio or video recordings) and external independent raters.
8. Use of clinical psychologists, psychiatrists, or advanced trainees in psychology or psychiatry, and who were trained in the treatment modality they delivered in the trial. Due to the small number of studies using experienced therapists, it was decided to include in the systematic review studies with advanced trainees in psychology or psychiatry.
9. ix. Use of validated measures within the disorder being studied.
10. A minimum of 20 participants per treatment group.
11. Treatment length ranging from 13 to 30 sessions (or mean number of sessions ranging from 13 to 30 if treatment length was not provided) in the case of Axis I disorders and up to 40 sessions in the case of personality disorders. Studies delivering CBT and STDP in combination with other treatments were excluded when no data on the individual treatments were available.

Outcome measures

Treatment efficacy was investigated in two types of treatment outcomes: primary symptoms and secondary symptoms. Outcomes for primary symptoms were measures of target symptoms, that is, symptoms specific to the patient population being studied. Outcomes for secondary symptoms were measures of symptoms other than the primary symptoms. Measures such as therapy satisfaction or alliance were not included in the review. Change in outcome measures were investigated at post treatment and at follow-up. When more than one follow-up was available, the longest follow-up was selected to investigate longer-term change, as long the criterion of a minimum of 20 participants per treatment group was still respected.

Data synthesis

Our approach for synthesizing data included three steps. In the first step, within-group effect sizes were calculated for both CBT and STDP to investigate the magnitude of change in outcome measures. Cohen d statistic was used as the estimate for effect size, with values of 0.20-0.49 representing small effect size, values of 0.50-0.79 representing moderate effect size, and values of 0.80 and above representing large effect size [21]. For each outcome, d was calculated by subtracting the post treatment (or follow-up) mean from the pretreatment mean, and dividing the difference by the pooled pretreatment standard deviation of the two treatment groups. When a study included more than one outcome, a mean effect size was calculated (one for primary symptoms, one for secondary symptoms) per measurement time (post treatment and follow-up) to assess the overall outcome of the study following a procedure described by Rosenthal [22]. Odds ratio were converted into d following procedures described by Chinn [23]. When the data did not permit a direct computation of d, a value for d was estimated from other statistics (e.g., F) following procedures described by Rosenthal [24]. Additional information regarding calculation of effect size is available from first author upon request. When necessary, signs were reversed in order that positive effect sizes reflect improvement. It should be noted that this first step cannot be considered a systematic review of the treatment efficacy of CBT and STDP (a systematic review on CBT and another on STDP would be needed to address these issues). Nevertheless, it seemed relevant to provide an estimate of the magnitude of change associated with CBT and STDP in outcome measures in the studies included in this systematic review.

In the second step, a count approach was used to record the results of the original CBT-STDP comparisons (whether Chi Square, analyses of variances, etc.) for each primary and secondary outcomes.

In the third step, between-group effect sizes were calculated to compare the efficacy of CBT and STDP on each outcome measures at post treatment and follow-up using two procedures. In the first procedure, a change score was estimated. That is, between-group effect sizes were estimated by calculating the difference between the pretreatment and the post treatment (or follow-up) means of the CBT group minus the difference between the pretreatment and the post treatment (or follow-up) means of the STDP group. The difference was then divided by the pooled standard deviation of the two groups at pretreatment. In the second procedure, CBT and STDP scores were directly compared at post treatment and follow-up rather than calculating change scores [25]. That is, the difference between the post treatment (or follow-up) means of CBT and STDP was calculated and this difference was then divided by the pooled standard deviation of the two groups at post treatment (or follow-up). For both procedures, effect sizes were calculated for primary and secondary symptoms, at post treatment and follow-up. When comparing the two treatment groups, effect sizes were calculated so that a positive value reflects better outcome in CBT than in STPD, and that a negative value reflects worse outcome in CBT than in STDP. Excel 2010 (Microsoft; Redmond, WA) was used for computing effect sizes and to plot results. The Exploratory Software for Confidence Intervals (© Cumming, 2012) was used to compute the confidence intervals of Cohen’s d. This software allows a maximum of 100 participants per treatment group. Pursuant to recommendations made by Borenstein et al. [26], a meta-analysis will be conducted provided that studies were comparable, which in our case was defined as having comparable study samples (in terms of disorders and participant characteristics) and treatment modalities within a theoretical orientation. The first author proceeded to data extraction and effect size calculation.

Inclusion of studies

A flow chart of article selection is presented in Supplementary Figure 1. The literature search identified 871 articles after duplicates were removed. After screening for title and abstract, 117 full-text articles were assessed for eligibility, of which 38 were excluded because CBT and STDP were not compared, one because the modality was group therapy, 16 because CBT and STDP were delivered in conjunction with another modality, nine because CBT was compared to IP, and eight for unforeseen reasons (literature review, description of protocol, bogus study, and case studies). Of the remaining 45 articles, 27 were independent studies and 18 were companion papers (either follow-up studies or studies on other outcome measures that were or were not relevant for our purposes (such as treatment alliance)) of these 27 independent studies.

Of the 27 independent studies, 20 were rejected because they did not meet our study quality criteria (5 out of 20 did not meet the random assignment criterion, 12 out of 18 the treatment manual criterion, 14 out of 19 the integrity check criterion, 8 out of 19 the therapist experience criterion, 11 out of 20 the number of participants criterion, and 9 out of 20 the number of sessions criterion; the number of studies does not always adds up to 20 because in some instances it was unclear whether criteria were met), leaving a total of seven independent studies for the systematic review. The companion papers of these seven independent studies were screened to assess whether the criterion for number of participants (the only criteria that might have changed at follow-up) was met. Four of these companion papers met the inclusion criteria. Therefore, the final sample of articles included in this systematic review was eleven publications, representing seven independent studies, three companion follow-up studies, and one companion paper presenting additional primary and secondary outcomes. The seven independent studies will be labeled according to authors and year of the first article presenting study data.

Study characteristics

Studies that were included in this review are detailed in Table 1. The disorders being studied were generalized anxiety disorder, social anxiety disorder, cluster C personality disorders, avoidant personality disorder, depression (two studies), and bulimia nervosa. Treatment modalities varied for both CBT and STDP. One of the studies focused on older adults, whereas another included only female participants. The number of participants per treatment group mainly ranged between 25 and 30 at post treatment and between 23 and 26 at follow-up. The total number of participants was 767 participants at post treatment and 696 participants at follow-up. The number of treatment sessions ranged between 16 and 30 for Axis I disorders (up to 40 for Axis II), and lengths of follow-up were 6 month (2 studies), one year (1 study), and two years (3 studies). The diversity in disorders and in models of CBT and of STDP suggested that it was not indicated to conduct a meta-analysis. Therefore, effect sizes of individual studies will be presented.

First Step: Magnitude of change following CBT and STDP

In a first step, the magnitude of change associated with CBT and STDP was investigated (Supplementary Table 2). In anxiety disorders, with studies by Leichsenring [27-30], CBT is associated with a large decrease in primary symptoms (ds ranging from 1.06 to 1.97; mean = 1.56) and with a moderate to large decrease in secondary symptoms (ds ranging from 0.52 to 1.12; mean = 0.85) from pretreatment to post treatment and from pretreatment to follow-up. SDPT is associated with a moderate to large decrease in primary symptoms (ds ranging from 0.65 to 1.34; mean = 1.11), and from a small to moderate decrease in secondary symptoms (ds ranging from 0.26 to 0.75; mean = 0.55).

In personality disorders, with studies by Svartberg [31] and Emmelkamp [32], CBT is associated with a small to large decrease in primary symptoms (ds ranging from 0.47 to 1.15; mean = 0.84) and with a moderate to large decrease in secondary symptoms (ds ranging from 0.75 to 1.17; mean = 1.00). STDP is associated with a moderate to large decrease in primary symptoms (ds ranging from 0.64 to 1.09; mean = 0.82) and with a small to large decrease in secondary symptoms (ds ranging from 0.39 to 1.00; mean = 0.67).

In depressive disorder, with studies by Shapiro et al. [33,34] and Thompson et al. [35,36], CBT is associated with a large decrease in primary symptoms (ds ranging from 1.84 to 2.27; mean = 2.10) and in secondary symptoms (ds ranging from 0.89 to 1.65; mean = 1.27). STDP is associated with a large decrease in primary symptoms (ds ranging from 1.57 to 2.63; mean = 2.07), and with a moderate to large decrease in secondary symptoms (ds ranging from 0.73 to 1.85; mean = 1.46).

Finally, in bulimia nervosa, with a study by Garner et al. [37], CBT is associated with a large decrease in primary symptoms (d = 1.00) and a moderate decrease in secondary symptoms (d = 0.72), while STDP is associated with a moderate decrease in primary symptoms (d = 0.65) and a small decrease in secondary symptoms (d = 0.43) from pretreatment to post treatment. No data were available from pretreatment to follow-up.

Second Step: Comparisons between CBT and STDP using a count approach

In the second step, we counted the results of the original CBT-STDP comparisons for each primary and secondary outcomes provided in the studies included in this review.

In anxiety disorders, with studies by Leichsenring [27-30], CBT shows an advantage over STDP in primary symptoms in five out of the nine comparisons from pretreatment to post treatment and in two of the eight comparisons from pretreament to follow-up. In secondary symptoms, the outcome is more favorable in CBT than in STDP in one out of the five comparisons from pretreatment to post treatment, and no significant differences are observed on any of the three comparisons from pretreatment to follow-up.

In personality disorders, with studies by Svartberg [31] and Emmelkamp [32], CBT shows an advantage over STDP in primary symptoms in three out of the four comparisons from pretreatment to posttreatment, and in two of the five comparisons from pretreament to follow-up. In secondary symptoms, the outcome is more favorable in CBT than in STDP in one out of the four comparisons from pretreatment to posttreatment and in two out of the four comparisons from pretreatment to follow-up.

In depressive disorder, with studies by Shapiro et al. [33,34] and Thompson et al. [35,36], no significant differences are observed in primary symptoms on any of the eight comparisons from pretreatment to posttreatment and on any of the four comparisons from pretreatment to follow-up. Similarly, no significant differences are observed in secondary symptoms on any of the fourteen comparisons from pretreatment to posttreatment and on any of the two comparisons from pretreatment to follow-up.

Finally, in bulimia nervosa, with a study by Garner et al. [37], CBT shows an advantage over STDP in primary symptoms in five out of the seventeen comparisons and in secondary symptoms in three out of the five comparisons from pretreatment to posttreatment. In none of the above comparisons do STDP show an advantage over CBT.

Third step: comparisons between CBT and STDP using an Effect size approach

The first step suggested that CBT and STDP models reviewed in this article are all efficacious treatments. The second step suggested that significant differences between CBT and STDP always favored the former, but in most of the comparisons (about 75%), no significant differences were observed. It seems difficult to conclude whether or not the two treatments differ based on such numbers. Aggregating multiple comparisons into a single effect size per study can allows drawing meaningful conclusions based on the effect size. In the third step, two procedures are used two investigate differences between CBT and STDP: effect sizes based on change scores and effect sizes based on endpoint scores.

Change scores: The results of the first procedure, providing a change estimate from pretreatment to posttreatment and from pretreatment to follow-up, are illustrated in Supplementary Figure 2 (Panels A to D).

In anxiety disorders, with studies by Leichsenring [27-30], the effect sizes suggest that the decrease in primary symptoms is greater in CBT than in STDP from pretreatment to posttreatment, and the magnitude of the difference between the two treatment modalities is moderate in the case of generalized anxiety disorder and small in the case of social phobia (ds = 0.64 and 0.42, respectively; Supplementary Figure 2, Panel A). From pretreatment to follow-up, the advantage of CBT is maintained in generalized anxiety disorder, but it decreases to become negligible in social phobia (ds = 0.72 and 0.05, respectively; Supplementary Figure 2, Panel C). Regarding secondary symptoms, CBT shows a small advantage over STDP in generalized anxiety disorder and in social phobia from pretreatment to posttreatment (ds = 0.42 and 0.26, respectively; Supplementary Figure 2, Panel B). From pretreatment to follow-up, the difference is maintained in generalized anxiety disorder, whereas it decreases to become negligible in social phobia (ds = 0.33 and 0.16, respectively; Supplementary Figure 2, Panel D).

In personality disorders, with studies by Svartberg [31] and Emmelkamp [32], one study shows a small benefit of STDP over CBT in primary symptoms in Cluster C personality disorders, whereas the other shows a moderate benefit of CBT over STDP in avoidant personality disorder from pretreatment to posttreatment (ds= -0.26 and 0.61, respectively; Supplementary Figure 2, Panel A). From pretreatment to follow-up, the advantage of STDP over CBT decreases to become negligible in Cluster C personality disorders, whereas the advantage of CBT over STDP decreases to become small in avoidant personality disorder(ds = -0.18 and 0.25, respectively; Supplementary Figure 2, Panel C). Regarding secondary symptoms, the two studies show an advantage of CBT over STDP in secondary symptoms from pretreatment to posttreatment, but the magnitude of the difference between the two treatment modalities is negligible in Cluster C personality disorders and moderate in avoidant personality disorder (ds = 0.12 and 0.72, respectively; Supplementary Figure 2, Panel B). These results are maintained from pretreatment to follow-up (ds = 0.09 and 0.74, respectively; Supplementary Figure 2, Panel D).

In depressive disorder, with studies by Shapiro et al. [33,34] and Thompson et al. [35,36], CBT shows a negligible to small advantage over STDP in primary symptoms from pretreatment to posttreatment (ds = 0.16 and 0.22, respectively; Supplementary Figure 2, Panel A). However, the effects are reversed from pretreatment to follow-up, with differences of a small magnitude favoring STDP over CBT (ds = -0.36 and -0.25, respectively; Supplementary Figure 2, Panel C). Regarding secondary symptoms, the differences between the two treatments are negligible from pretreatment to posttreatment, although in opposite directions (ds = -0.19 and 0.16, respectively; Supplementary Figure 2, Panel B). One study provided data in secondary symptoms from pretreatment to follow-up, and the results suggest a moderate advantage favoring STDP over CBT (d = -0.54; Supplementary Figure 2, Panel D).

In bulimia nervosa, with a study by Garner et al. [37], CBT shows a small advantage over STDP in primary symptoms (d=0.37; Supplementary Figure 2, Panel A) and in secondary symptoms (d=0.29; Supplementary Figure 2, Panel B) from pretreatment to posttreatment. No data were available at follow-up.

An examination of the confidence intervals reveals that the possibility of negligible differences between CBT and STDP could not be ruled out in any of the primary and secondary symptom comparisons. In most of these comparisons, the possibility that CBT is more efficacious than STDP could not be ruled out. Often, the possibility that STDP is more efficacious than CBT could also not be excluded.

Endpoint scores: The results of the second procedure, comparing CBT and STDP endpoint scores at posttreatment and at follow-up, provide, with few exceptions (mostly on the range of the magnitude), results that are similar to those of the first procedure and thus are not reported here. These results are illustrated in Supplementary Figure 3.

Post Hoc investigations: change scores between posttreatment and follow-up

The results of the third step suggest a different pattern of associations at the two measurement points for the different disorders, with differences in effect sizes being in general lower or effects reversed in favor of STDP at follow-up in comparison to posttreatment. In order to investigate the discrepancy between posttreatment and follow-up results, an effect size comparing change scores of CBT and STDP between posttreatment and follow-up was calculated for primary and for secondary symptoms. The results are illustrated in Supplementary Figure 4.

In anxiety disorders, with studies by Leichsenring [27-30], the results suggest that the decrease in primary symptoms from posttreatment to follow-up is greater in STDP than in CBT in social phobia, with difference in magnitude being in the small range (d = -0.30; Supplementary Figure 4, Panel A). Other differences fell into the negligible range.

In personality disorders, with studies by Svartberg [31] and Emmelkamp [32], differences between CBT and STDP in change from posttreatment to follow-up were meaningless.

In depressive disorder, with studies by Shapiro et al. [33,34] and Thompson et al. [35,36], STDP shows a small advantage in primary symptoms (ds = -0.40 and -0.27; Supplementary Figure 4, Panel A) and a negligible advantage in secondary symptoms (d = -0.17; Supplementary Figure 4, Panel B) over CBT from posttreatment to follow-up. No follow-up data were available for bulimia nervosa in the study by Garner et al. [37].

The aim of this systematic review was to compare the treatment efficacy of CBT and STDP in high-quality studies. The results revealed that only a small number of study designs met our high-quality criteria. CBT and STDP models reviewed in the current article were all efficacious in reducing primary and secondary symptoms. An examination of the studies’ original comparisons revealed that significant differences between the two treatment modalities were always in favor of CBT, but in most of the comparisons (about 75%), no significant differences were detected. Based on this count approach, it is difficult to conclude whether or not the two treatments differ. An effect size approach was then undertaken, aggregating multiple comparisons into a single effect size per study to allow drawing meaningful conclusions based on the effect size. In anxiety disorders and bulimia nervosa, this effect size approach revealed that CBT showed an advantage over STDP in primary and secondary symptoms. In personality disorders, results were mixed in primary symptoms, but CBT showed an advantage over STDP in secondary symptoms. In depressive disorder, the outcome was more favorable in primary symptoms for CBT than for STDP from pretreatment to posttreatment, but the results were reversed from pretreatment to follow-up, with STDP showing and advantage over CBT. Results were mixed in secondary symptoms from pretreatment to posttreatment in depressive disorder, but the benefit was greater for STDP than for CBT from pretreatment to follow-up. The magnitude of the differences ranged from negligible to moderate across disorders, and differences in effect sizes were in general lower or effects reversed in favor of STDP at follow-up in comparison to posttreatment. An examination of the confidence intervals revealed that the possibility of negligible differences between CBT and STDP as well as the possibility of an advantage of one modality over the other could not be excluded in many of these comparisons. Discrepancies between the count approach and the effect size approach may be explained by differences in the computation underlying the different statistical strategies (e.g., effect size and analyses of variance) and by the aggregation of significant and non-significant results in the effect size approach.

In the effect size approach, the findings of an advantage of CBT over STDP for anxiety and depressive disorders in primary symptoms from pretreatment to posttreatment replicate those of a meta-analysis that concluded that CBT was more efficacious than psychodynamic therapy at posttreatment [10] (although in this case psychodynamic therapy combined LTDP and STDP). However, the results of the current systematic review suggest that STDP shows an advantage over CBT in depressive disorder from pretreatment to follow-up, contrasting with results of other meta-analyses that concluded to a disadvantage of STDP [8] or that the two treatment modalities were not different at follow-up [6]. Additional research is thus needed to clarify the comparison of long-term treatment efficacy of CBT and STDP in depression. The results of this systematic review for personality disorders are concordant with those of a meta-analysis which concluded that CBT and psychodynamic therapy (pooling LTDP and STDP) were both efficacious with personality disorders [13].

An interesting result is that the decrease in symptoms seems to some extent greater in STDP than in CBT from posttreatment to follow-up (Supplementary Figure 4). This result is supported by at least three meta-analyses who reported data suggesting that the decrease in symptoms from posttreatment to follow-up may be greater in STDP than in other psychotherapies [3,5,7]. It would therefore be interesting to examine variables and processes associated with the difference in patterns of change between posttreatment and follow-up. Other high-quality studies are needed to investigate whether this effect is replicated and, in longer follow-up, whether this effect stabilizes, decreases, or increases.

Numerous forms of STDP have been developed and can generally be differentiated on an expressive/supportive continuum [38] or on an emotion-focused/interpretive continuum [5]. There is some evidence that STDP focusing on affects are more efficacious than the other forms of STDP [39, 40]. Indeed, in our results, the affect phobia model [41], which focuses on affects, seems to perform better in comparison to CBT than the other STDP models in primary symptoms at posttreatment [31]. Randomized controlled trials comparing CBT and expressive or emotion-focused STDP are warranted.

Studies included in this review were all randomized controlled trials and were thus concerned with internal validity and efficacy at the expense of external validity and effectiveness. There is no consensus as to whether results of randomized controlled trials can be generalized to “real-life” private practice [42] given that in private practice clients are more heterogeneous (e.g., presence of co morbidity) and therapists are more free in their use of techniques. There is however some evidence that developing an individualized case formulation and treatment plan specially tailored to the needs and difficulties of a particular client is not more effective than following treatment protocols developed in a “one size fits all” spirit [43,44].

An important limitation of this systematic review is that a small number of studies respected the high quality design criteria. A single author proceeded to study selection and extraction of data. Within studies, multiple measures of primary symptoms (and secondary symptoms) were pooled, and therefore the significant group differences reported in the original study analyses may have been lost in the number of measures. Unfortunately, it was not possible to limit the systematic review to studies using experienced therapists. More importantly, the data were deemed unsuitable for conducting a valid meta-analysis given the diversity in disorder populations and in treatment modalities within theoretical approaches. Therefore, quantitative results were reported qualitatively, and moderator analyses (investigated for example researcher’s allegiance or duration of follow-up), test for file drawer effect, and sensitivity analyses could not be conducted.

Despite these limitations, the current systematic review adds to the literature in several ways. First, it goes further than the original studies. These studies were based on the null hypothesis statistical testing and results and discussions focused on whether or not differences were detected rather than on the magnitude of the difference (negligible, small, moderate, or large). Our effect size approach focuses on the magnitude of the difference, and provides estimates of the precision of the magnitude (the confidence intervals) [45]. Second, it clarifies treatment differences when multiple primary and secondary measures were used within a study, as it were the case for each individual studies included in this systematic review. For example, in bulimia nervosa, CBT showed an advantage over STDP in primary symptoms in five out of the seventeen original comparisons. It seems difficult to conclude whether or not treatments differ based on those numbers. Our effect size approach, aggregating multiple comparisons into a single effect size, allows drawing meaningful conclusions based on the effect size. Third, it re-analyses the results of the original studies and provides different conclusions. None of the studies’ original comparisons revealed an advantage of STDP over CBT; it would therefore be difficult to argue that STDP is associated with a more favorable outcome than CBT based on the original results. Our effect size approach, however, suggests that STDP may show negligible to moderate advantage over CBT in some instances. Fourth, it highlights differences between treatment modality according to disorders, types of treatment outcomes (primary or secondary), and measurement time (posttreatment or follow-up). Finally, it suggests a different pattern of symptom change from posttreatment to follow-up.

We conclude that very few studies have compared treatment efficacy of CBT and STDP using high quality-designs. Differences in treatment efficacy seem to vary according to disorders, type of treatment outcomes, and measurement time, but results have to be interpreted cautiously given the confidence intervals and the small number of studies included in the systematic review. Nevertheless, returning to Paul’s [46] seminal view on outcome research, the question of the delivery of the right treatment by the right therapist to the right client for the right disorder, and under the right circumstances still seems relevant.