Evidence Based on Inadequate Research

/in /door

Abstract van een artikel waarvan beide auteurs (Prof. Volker Tschuschke en Peter Schultess) aanwezig zullen zijn op de SARC Conference (Science and Research Committee) van de EAP (Wenen 16 en 17 februari 2016)

Fundamental Reflections on Psychotherapy – Research and Initial Results of the Naturalistic – Psychotherapy Study on Outpatient Treatment in Switzerland (PAP-S)

Philippe Vrancken zal participeren aan deze conferentie en zal via dit kanaal verslag uitbrengen.

Volker Tschuschke, Aureliano Crameri, Margit Koemeda, Peter Schultess, Agnes Von Wyl & Rainer Weber

Abstract

The paper deals with the current controversy regarding the subject of appropriate and necessary research strategies in psychotherapy, and it takes a clear stand in favor of a naturalistic, process-outcome-oriented research approach (practice-based evidence). Based on the design and the initial results of the PAP-S study, conducted by the Swiss Charter for Psychotherapy, it will be pointed out how useful empirical research can be carried out in psychotherapy.

…the powerful National Institute of Health (NIMH) in the USA, adopted the same principle of methodological research as can be found in pharmaceutical research – the randomised controlled trial (RCT). This move implied that studies had to include manual-guided psychological or psychiatric treatment as well as samples of mental syndromes as defined in the DSM. (Duncan & Miller, 2006).

Critics of the RCT paradigm – which is called the “gold standard” of psychotherapy research by its apologists – see a veritable array of points of criticism, which is why the RCT approach is a false path for psychotherapy as a scientific discipline.

1 Research Bias: It has been proven that there is a highly significant positive correlation (>80) between the preferred therapy model of researchers and the therapy model tested as successful in the studies (Luborsky et al., 1999). RCT research is based up to 90% on behavioural, respectively cognitive-behavioral, therapy methods; it takes place almost exclusively in centers and institutes for behavioral research.

2 Patients rejected by EST study researchers: Up to two-thirds of all patients (who appear in the therapist’s office with anxiety disorders and depression, for example) are excluded from RCT studies because of the rigid diagnostic criteria for inclusion. That is, they are not “pure” enough for the studies in diagnostic terms. The higher the number of rejected patients, the more successful the therapies prove to be (Messer, 2004). Thus, EST studies (all characterised by the RCT paradigm) involve extremely selective patient groups. There is no room for co-morbid patients; thus, the “normal” cases of multiple problems from among the patients who appear in therapists’ offices are all factored out.

3 EST Studies scarcely have lasting effects: EST – respectively RCT – studies examine almost exclusively patients with certain anxiety disorders (phobias, generalised anxiety disorder) and patients with depression. The average patient in these studies retains mild, although clinically significant, symptoms following the treatment applied in an EST study (Westen & Morrisson, 2001). Only about 40% of all patients benefit; in fact, only 37% of the depressive patients do. If the drop-outs are included, then the rate of success is reduced to just 27%. Over 50% of the patients seek therapy anew within two years following EST therapy (Messer, 2004).

4 Impact of Comparative Studies: Many so-called bona fide therapy methods (Wampold, 2001) cannot show that they have participated in RCT studies [or do not want to]. Nonetheless, they have an extensive, solid canon of basic hypotheses that have been in use for a long time, and that possess a long tradition of empirical research, but they reject the RCT paradigm (for example psychodynamic, person-centered, and family therapy). These are not considered for direct comparison with behavioral or cognitive behavioral therapies, precisely because they are used to treat mixed disorders (for example maladjustment, major depression combined with anxiety disorder). Yet, if the aforementioned therapy forms are compared with behavioral / cognitive-behavioral therapy forms, then analogous effects appear in 41 of the 49 EST studies (Messer, 2004) listed by the task force (Chambless et al., 1996).

5 Randomization: It is not that simple to randomly allocate patients to various therapy forms, or to non-therapeutic activities (randomization). In any event, it is possibly unethical. A therapist cannot decline to give direct help to a person who is suffering, and no ethics commission today would tolerate that. Moreover, a great many people have predispositions: they often enough prefer – for whatever reason – specific treatment forms. Besides, scientific-methodological guidelines that are now available offer numerous tests and procedures for determining indication for, and prognosis of, treatment. The results of these tests would bar therapists from leading patients to treatments from which they would most likely not benefit (for example: Eckert, 2009)

6 Control Group Studies: Based on their RCT paradigm, EST studies want to suggest that all other possible variables that have an impact – except for one independent one, namely the applied therapy – are “controlled”, that is, kept equal under both conditions (intervention and control group), so that no variable other than the independent one can be held responsible for the psychotherapeutic transformation of the patient. Other unknown variables that possibly facilitate the patient’s transformation are excluded as “control” variables. The absurdity of this assumption is evident alone for the fact that, out of 168 living hours per week, a person spends only 1 or 2 in therapy. That means the patient is not under supervision for 166 hours of the week, and that, during this time, he or she is exposed to hundreds of situations and stimuli; and that he or she experiences positive or negative events of a more or less dramatic kind – all of which can influence him or her. And each patient in a study has his or her individual life-universe, incomparable with that of any other person. So, one wonders what is actually being “controlled” in such studies, since nothing in the experience of therapy patients in a study can be “controlled”. Laboratory mice or rats in studies can perhaps be controlled in that they are given only certain foods or medicine, and their general living conditions in their cages remain constant between experiments. Human therapy patients would only then be participants in real “control” studies if they were locked up in a low-stimulus cell and allowed to leave for their two therapy sessions.

7 Manual-guided Therapies: Duncan and Miller (2006) criticise the manual-guided treatment approach as limiting and not allowing an adequate view of the therapy process. Furthermore, they quote numerous studies that have failed to bring forth statistical proof that manual-guided approaches are superior – and all this with a variance clearing of significantly less than 10% (in cases of therapy success) using the applied therapy technique. There is apparently even empirical evidence that manual-guided therapy can be harmful. It is of significance that psychotherapy researchers representative of behavioral therapy disregard the words of one of the grand old men of behavioral therapy, Aaron T. Beck, who said: “Being able to conduct cognitive therapy using a manual is just as unlikely as it is to perform surgery using a manual.” (Beck, 2000). The therapeutic bond, which is far more important, and the individual needs of the particular patient (of far greater importance in psychotherapy) are ignored; instead, a technical operation is conducted. However, it is precisely this that is not psychotherapy.

8 Analysis of variance and EST – respectively RCT – studies: There is no dissent among psychotherapy researchers on the matter that the actual technique of therapeutic treatment explains outcome variance only minimally. The highest assumed rate of therapeutic variance is 15% (Assay & Lambert, 1999) and the lowest is 1% (Wampold, 2001). This corresponds to the “dodo bird verdict” in therapy research (“All have won and all must have prizes”). This equates to there being no difference in effectiveness among therapy methods as long as professionally experienced psychotherapists and “real” patients are considered in studies. Wampold et al. (1997) examined 277 research studies done between 1977 and 1995 and found no therapy method to be superior. The Human Affairs International Real – World-Study (Brown et al., 1999) included more than 2000 psychotherapists and 20,000 patients, respectively clients, and found that the therapy outcomes of the 13 different therapy concepts involved (including medication and family-therapeutic approaches) did not vary. The dodo bird verdict – known in the German-speaking countries as the ‘equivalency research paradox of psychotherapy forms’, according to which no psychotherapy concept is generally superior to any other treatment concept – is one of the best verified results of psychotherapy research (Lambert & Ogles, 2004). Since a mere 1-15% variance pertains to the therapy method, but 7% to the therapeutic alliance (which is not examined in EST studies), 40% even to the patient variables (therapy viability, treatment motivation, specificity of the disorder, severity of suffering, etc.) and altogether about 87% to the extra-therapeutic variables (including patient variables) (Wampold, 2001), then we are amazed to ascertain that EST – respectively RCT – studies revolve around the variance of 1-15% (therapy technique), yet completely ignore all other aspects.

9 Conclusion: Not only do EST/RCT studies reveal nothing about psychotherapy, they are, on the contrary, risky, because they are completely misleading. It is precisely EST/RCT studies which render a completely distorted image of professional, serious psychotherapy, because they describe psychotherapeutic reality in a very biased manner:

  • in most studies, bias on the part of the researcher leads subconsciously or consciously to manipulated results there are no representative patients who seek therapy;
  • on the contrary, those chosen are just not representative (no external validity)
  • the complex disorders of the “typical” patients who show up in psychotherapeutic offices (co-morbidities) are and have been excluded from RCT research (no external validity)
  • control studies (control of influence-variables) are not possible (thus no internal validity)
  • randomization is in effect not possible. If implemented, it is unethical at the least
  • placebo control is not possible in psychotherapy (double blind studies all the more so)
  • EST/RCT studies are not superior to bona fide studies
  • manual-guided psychotherapeutic treatment cannot make sense and be effective because it makes free, spontaneous intervention impossible and because it takes “standardized” patients as its basis and assesses the therapist’s relationship to the manual as more important than the therapist-patient relationship; and because it looks upon psychotherapy as if it were a surgical technique, a pill that could be administered at the prescribed moment
  • EST/RCT studies cover only those techniques which make up the 1-15% variance of therapeutic outcomes; consequently, EST/RCT studies theoretically give evidence in a dwindling domain of therapeutic effects and suppress 85-99% of possible influences
  • interim conclusion: thus, RCT (EST) studies have no validity whatsoever Thus the following must be put on record: RCT/EST studies are per definitionem invalid, in psychotherapy, since they cannot yield valid results – due to the deficits described above. If the 1-15% variance in therapy method is the major discovery of empirical, evidence-based psychotherapy research, then we know nothing of the effectiveness of psychotherapy. We especially lack knowledge about the real ingredients of therapeutic treatment, because we do not conduct process research in order to detect the mechanisms of change. The medical model is just not suitable for psychotherapy research. RCT studies in psychotherapy research lack sufficient complexity (Henningson & Rudolf, 2000). They do not illustrate the process of change and reveal nothing about the effectiveness of psychotherapy, because the verum (the elements of change) results, in effect, from manifold variables during therapy, variables that have to be controlled, which means, specifically: extensive process research must be conducted. Psychotherapy should not pander to the other medical disciplines by surrounding itself with aura of science (an inauthentic one, at that). Psychotherapy has a more complex object of study than pharmaceutical research (for instance); psychotherapy research must be commensurate to its complexity. It cannot settle for the inadequate research paradigm used in the medical field.

Prof. Dr. V. Tschuschke, Department of Medical Psychology, University Hospital of Cologne, D-50924 Koln, Germany