Guest guest Posted July 2, 1998 Report Share Posted July 2, 1998 Hi All This is a copy of a letter I am writing to The Addiction Newsletter and possibly other professional addiction forums on Project MATCH, an investigation into methods of alcoholism treatment. If youre not familiar with MATCH, you might like to visit Stanton Peele's Addiction Website where he discusses it. If any body spots any factual errors, please let me know - I think I have something important to say - and I dont want it blown by a goof. Pete ---------------------- Cool Briton PERSONALITY-DISORDERS LIST: http://rdz.acor.org/athenaeum/lists.phtml?personality-disorders _____________________ " Every great scientist is part B.F. Skinner and part P.T. Barnum " - Bart Simpson's science teacher Always happy to oblige: Sir, As an undergraduate in Psychology I am at best at the very beginning of a career in the addiction field. Having admitted my lowly academic status, what I am about to say may be dismissed by some as sophomoric - to which I reply perhaps it is - that a sophomore has the learning to make all the points I will make here, and hence it is all the more disturbing that they need to be made. Much of what I am about to say in respect of the project has been said before - what I believe is most important is not its manifest failings, but how those failings came to exist and the response of the project's supporters when these failings are indicated. The first terrible failing is the lack of a control group. As a result, MATCH gives us no information whatsoever on whether *any* of the treatments studies are effective over Natural History, let alone Placebo Treatment. This is a devastating failure. How could this happen? The sad and astonishing fact is that it is now *routine* to perform alcohol studies without control groups, despite the enormous historical evidence of the importance of subjecting any form of treatment to clinical trial. If asked what the most important medical breakthrough of history is, it is not I believe antibiotics or any actual material treatment at all - but the perfection of the randomized clinical trial, by which all other discoveries are either made or confirmed. With it effective treatments are verified and worthless ones discarded. The randomized trial is the very keystone of scientific Medicine and Psychology - yet this precious tool is routinely disregarded in alcohol studies. why is this? the usual explanation is that such is the seriousness of alcoholism that it is unethical not to offer all subjects treatment - and my understanding this is the argument given in the case of MATCH. This argument is so manifestly fallacious that is amazing (to me, anyway) anyone would dare offer it, and it is difficult to know where to start in refuting it. Firstly, it is, as far as I know, an argument peculiar to alcohol science. Clinical trials for antidepressants routinely test against placebo. this is despite the manifest discomfort of the patient, and the occasional *fatal* nature of depression through suicide, and that there are many effective antidepressant drugs already licensed. in fact, clinical trials of new antidepressants often test against other antidepressants as well as placebo as a measure of effectiveness. Secondly, if the normal practice of equal numbers in each condition were followed, 75% of subjects would have been allocated a treatment condition. Given the large sample size, it would be perfectly feasible to run only 10% of subjects in the control condition, so for 90% of subjects the use of a control would have had no effect whatsoever. Given that subjects were being *granted* an offer of treatment, i see no ethical problem in a situation where there was a 10% chance an individual would be put in the control condition. then, one needs to remember what the control condition usually is in studies of psychotherapy - a waiting list, whereby the subject is offered therapy after the trial has been finished. in other words, the only inconvenience the subjects in the control condition would suffer is having to wait until the end of the study for their treatment! the argument " It's unethical not to offer them something " is completely irrelevant. of course, this means that they are apparently left to continue to suffer during the length of the study, but this in itself presumes that the treatments offered are effective above placebo - and until controlled studies are done, this assertion cannot be made - there is still a desperate need for adequate controlled studies in alcoholism/addiction treatment which MATCH ought to have helped provide. Looking in the larger perspective, estimates of the number of sufferers from alcohol problems are put in millions in the US alone. When this suffering community is remembered, the small number of participants placed in a control condition of MATCH would be but the tiniest dram in a great still of alcoholism. By failing to provide a controlled study, and hence a measure of the absolute effectiveness of the treatment modalities, the interests of all these millions have been compromised dreadfully - a splendid opportunity to test whether effective treatment is currently being offered has been completely squandered. this i believe to be a monstrous disservice, and leads me to what is in fact is the true nature of the ethics in this situation - that such is the seriousness of alcoholism/addiction that it is unethical *not* to perform controlled studies which are almost the only means by which meaningful conclusions can be drawn. The cynic might suggest that the team were concerned that a controlled study might be unethical in that if it showed no above-placebo effectiveness for therapy it might bring down an industry worth millions of dollars. If the failure to provide a control were not enough, what is even more disturbing is the claims that are being made. Apparently, some professionals are claiming that MATCH demonstrates that Alcoholics Anonymous is effective! As previously noted, the lack of control means that in fact nothing is shown to be effective over placebo. Even more remarkable still, is that AA was *not even studied* by MATCH. What *was* studied was Twelve Step Facilitation - but this is by no means the same thing. if in fact there had been a control group, success for TSF would certainly have had implications for AA, but even then it would be a gross misrepresentation of what was actually found to merely state " AA is shown effective " . Another disturbing claim I have seen is that MATCH shows that TSF is more effective than the other conditions. I saw on promotional material for a treatment centre a newspaper report that the study showed that twelve-step treatment was " the best " . In fact, there was no statistically significant difference between the three conditions in overall performance. Given the large sample size and hence strong statistical power of the study, if any difference actually did exist it was very likely to have been found. Certainly, if any difference does exist, its effect size must be very small to escape detection, and of no practical significance. Newspapers are notorious for errors, particularly in matters relating to science, but if journalists were *always* presented with the statement " MATCH found no significant difference in effectiveness between all three treatments " it is difficult to see how this misrepresentation came about. it is hard not to conclude that the results were falsely reported to the media - and this misreporting has already been used by a treatment centre to support its approach. In fact, given that it employed only a third the number of sessions, in at least cost-effective terms, arguably MATCH shows MI is the most effective. The equality of effectiveness is itself something with a significance that has been missed. It has been noted that all treatments apparently succeeded at levels way above that which is actually seen in clinical practice. the explanation usually offered for this is that MATCH's treatment was of a particularly high quality, above that typically found in the field. What is more rarely discussed is the fact that there were several known prognosticators of recovery that would suggest greater success irrespective of treatment quality. for example, those with co-existing drug problems were excluded, as were those with insecure housing. It must also be remembered that the Motivational Interviewing condition provided just *four* sessions, and on average, participants in that condition attended only three. Could superior quality of MI *really* explain the greater success rate observed? Also, in the other two conditions, participants were given *12* sessions. In order to believe treatment quality is what is responsible, then MI needs to have improved over 4 sessions (with an average attendance of 3) to produce exactly the same greater efficacy as the improvement over 12 in the other two. is that likely? Also, consider the nature of the treatments themselves. Critics of TSF suggest that it includes components that appear completely contradictory to the central tenets of Cognitive Behavioural Therapy - or at least are very hard to square with them. These are primarily: Belief one is powerless over alcohol(Step 1), Surrender to God (Step 3) and Moral Inventory (Steps 4, 8, and 10). These appear to contradict the principles of CBT which emphasise belief in Personal Self-Efficacy, Internal Locus of Control, and avoiding negative self assesments. The TSF/AA advocate will immediately protest 'This shows you dont understand what the Steps are really about' but whether *I* understand what the Steps are " really about " doesnt matter - what matters is how the " alcoholic in the street " understands them, and how they square up with well-established research on cognitive therapeutic principles. with the possible exception of Moral Inventory, I would suggest to the TSF novice at least there appears to be immediate, profound contradictions between the objectives of CBT and TSF. Hence, it is hard to imagine how *both* these approaches could be effective - certainly *equally* effective. The more the quality of treatment improved, the *greater* one would expect the disparity of the more effective over the less effective to be - yet what one sees in MATCH is apparently very high quality treatment producing the same effectiveness. (The success rate in MATCH is not so high that a ceiling effect can be offered as explanation). Another extraneous variable is the influence of Alcoholics Anonymous. TSF is intended as an introduction to AA, and many TSF participants attended it. Also some attended from other groups. If AA is indeed effective, then would expect it to have an incremental effect over and above TSF. This in itself has an implication, since it suggests that TSF *alone* would perform more poorly than the other conditions - but that does not matter much as TSF encourages AA participation (but it means that more client time is required, and gives them more opportunity for non-compliance). Of course, some participants attended AA from the other conditions too - but not as often. Also, TSF is *intended* as an introduction to AA - the TSF clients will hear the same thing in AA they hear in their sessions. The clients in the other sessions hear something completely different - and in the case of those in CBT, something seemingly contradictory. are they as likely to benefit from AA attendance as much as those in the TSF condition? isnt it more likely that AA is likely to *undermine* their therapy, if it has any effect at all? If the TSF clients had attended a treatment-neutral support group (such as SOS) or a CBT-friendly support group (such as SMART) then they might have performed even better. since no attempt was made to test this, we cannot know. Given all these factors, it is in my view highly unlikely that improved treatment quality is the explanation for the universally high success rate observed. a far more parsimonious explanation is that simply the known prognosticators of success are responsible. since clients were randomly assigned across all groups, these are the *only* variables that are likely to affect each condition equally. with all the others, there are good reasons to think they would have different strengths of effect in the different conditions, and hence make it unlikely that the even success rate would be seen if they were operating to any great extent. A rather odd objection to what I have written above I have heard is: " But MATCH wasnt intended to show the absolute or even relative effectiveness of the treatments. it was supposed to show the effect of matching clients to treatments. " I think I have shown it *ought* to tried to show absolute effectiveness above - and in fact, although that may not have been its intention, it *does* show relative effectiveness - even if that relative effectiveness is widely misreported. in that respect, we are lucky that there were three conditions, because in a sense any two act as a kind of placebo for the third. the equal effectiveness (despite the extranneous variables I have referred to) is in my opinion indicative that possibly *none* of the treatments would have shown performance over placebo - especially when the diversity of their approaches is remembered. Now finally I can move into an area where a failure might refer to an outcome rather the Project itself. MATCH failed to show significant effects of client matching except for two that are explicable in terms of the familywise error of performing that number of comparisons. Again, considering the sample size and hence statistical power of the study, this suggests that there probably isnt anything " out there " . However, these negative outcomes serve to illustrate another monumental *design* failure of the study, which again is amazing because to me it is so obvious. In studies in Social Science, one of the commonest confounding variables is self -election. It is a very easy matter to show an association between therapy attendance and recovery in a wide variety of settings. clearly, someone motivated to get well is likely to attend more sessions, but someone motivated to get well is also likely to get well anyway. this is before one even considers any placebo effect of the therapy itself. The problem is self-selection - the person chooses to attend. one cannot require clients to attend at gunpoint, but when it comes to assessing outcomes, the effect of self-selection must always be considered, especially in the absence of a control. Probably over 90% of the conviction of the effectiveness of AA in its supporters, and 95% of their offered evidence, is based on what might be vulgarly called the " Bums on Seats " principle, that there are over a million AA attendees wordlwide at any one time, and many tens of thousands of people who attribute their recovery to AA. The flaw in this argument is the self-selection principle - in all but a handful of places, there are few alternatives, and even if there were, someone who attends *any* group or treatment is self-selecting to attend it, and hence is likely better motivated than one who doesnt, and better outcomes than Natural History (if they indeed occur) explicable by higher motivation rather than attendance. as a result any number of testimonials is meaningless - and the need for controlled studies. Even if a therapy is 100% effective, it is useless if the client refuses to undertake it. Hence, *how willing* a client is to undertake a particular therapy is of considerable importance, and how remarkable and unfortunate it is that MATCH made no attempt to examine it. Of all the various client variables that were examined by MATCH, apparently one that no-one thought to examine was: " If we advise the clients on the three treatments available, which will they tend to prefer? " amazingly, apparently the clients werent even *asked* this question, let alone any investigation done on what was the effect of granting their wish. I hope I will be forgiven for not providing a citation to support my suggestion that clients are more likely to attend and comply with a therapy that they themselves have chosen in preference to alternatives than to one which they have been randomly assigned - which means, if the therapies are effective, they are more likely to get better simply on the grounds of improved attendance if nothing else. of course, in practice they are likely to choose one most in accordance with their existing beliefs and personality, and hence both compliance and behavioural change are more likely to occur from that perspective as well. MATCH could have been vastly improved by only randomly assigning half the clients, and the other half allowed to choose their treatment condition. If this had been done, it would be a surprise ( to me at least) if the self-selectors did not outperform the randomized clients at least within any single treatment modality, and possibly an effect might be observed across modalities as well. once again, a precious source of information squandered. Even if the clients were still all randomized, then there could have been considerable benefit from asking the question - and correlating the answer with attendance and outcome. Even if the treatments are all equally effective, then the one to be preferred is the one where the client is most likely to turn up. Without even considering the actual nature of the treatments, we can see why people might prefer one over the others. Some people might prefer MI because it only " required " four sessions - some people might think the others gave them three times as much (this issue of course, is vastly more significant when clients are paying for treatment sessions, especially if not reimbursed). When one considers the nature of the treatements, reasons why people might express preferences become even greater. I do not intend to examine the religious/spiritual debate over the 12 Steps here. Again, what I think, or what is " true " does not matter, what matters is what the " Alcoholic in the Street " thinks, and how s/he reacts to them in consequence. Different US courts have ruled that forced attendance at AA is unconstitutional on the basis that AA is a religious program. whether those rulings are " correct " does not matter either. far more significant is that these appeals appeared before the courts in the first place, and that someone can feel strongly enough on the issue to prefer to sit in jail rather than attend AA. It's hard to imagine someone registering that much objection to MI or CBT sessions. for each of the handful of such " refuseniks " there are likely to be many who grudgingly comply but who would far more willingly participate in other treatments, possibly to greater effect. Of non-coerced clients, there could be very many people who would prefer non-TSF treatments on ideological grounds, possibly to the point of not undergoing treatment if only TSF were available - but at present over 90% of alcoholism treatment in the US is TSF-based. TSF/AA " refuseniks " need not all be atheists and agnostics. its perfectly possible that many religious people would object. firstly, there are those who do not belong to monotheistic religions, such as those who are Hindus, Buddhists, Pagans, or have " New Age " beliefs. To those who would say " but anything can be your Higher Power " - that may not be good enough for these people - the Steps say quite clearly " God as we understood Him " - which in particular might be aversive to women. Secondly, TSF/AA could be aversive to those with very *srong* monotheistic beliefs - they might consider involvement in a *second* spiritual program reflects a disloyalty or lack of faith in their Church. Branching into subjectivism, I will say that I have met Christians who have felt guilty and pressured by their Church not to attend twelve step programs. However, an entirely *secular* therapy might be acceptable both to them and their Church - particularly one that was not open-ended. A feature of the last 30 years in the West has been not only increased secularism and feminism but also growth in Eastern religions, New Age beliefs, and resurgent fundamentalist monotheism in both Christian and Islamic forms - the adherents of all of which might object to the 12 step program. Hence, there is every reason to think that both MI and CBT might be more frequently chosen as preferred treatment to TSF - and it is important to know if this is true simply in terms of providing treatment that clients will accept. Finally, I wish to comment on the assesment of outcomes in MATCH. Outcome has been measured on factors like the reduction in days drinking. If it is considered acceptable to measure success in terms of the *reduction* of drinking days, rather than in the number of complete abstainers, it becomes reasonable to ask why there was no evaluation of a Controlled Drinking therapy in the study. It is paradoxical to justify abstinence programs in terms of drinking reduction, when that is not their intended outcome. Again I am likely to hear " But MATCH wasnt about that. It was only intended to evaluate abstinence programs. " I return again to my original point - that the principal significance of MATCH is what it didnt do rather than what it did. Why did such a heavily-funded study not examine a kind of treatment for which there is already supportive evidence as a comparison with the others? clearly, comparing a controlled drinking treatment with abstinence treatments may be a little tricky in evaluation, but if merely reduced drinking is accepted as evidence for the effectiveness of an abstinence program, there is no reason why it cannot be done. It is my belief that Project MATCH is a monument to what is terribly wrong with the predominant nature of the alcoholism/addiction science and treatment. Indeed, the treatment industry could be said to itself be suffering from a crippling disease in which denial of sickness is a primary symptom - it is addicted to a Disease Model of addiction, and an ineffective anstinence/12-step treatment paradigm, despite mounting adverse consequences, namely a complete failure to justify these approaches empirically. Like the worst of addicts, it desperately attempts to cover up the effects of its addiction - but there comes a point where no amount of covering up can conceal the truth, and that point is fast arriving. Again, I am left with the impression that MATCH was intended not to test if the treatment of alcoholism applecart were safely balanced but instead to thrust chocks under its wheels to ensure it could not be upset. My comments might appear precocious from a mere undergraduate, but then it was the little boy who said the Emperor had no clothes. Pete ---------------------- Cool Briton PERSONALITY-DISORDERS LIST: http://rdz.acor.org/athenaeum/lists.phtml?personality-disorders _____________________ " Every great scientist is part B.F. Skinner and part P.T. Barnum " - Bart Simpson's science teacher Quote Link to comment Share on other sites More sharing options...
Guest guest Posted July 4, 1998 Report Share Posted July 4, 1998 At 07:36 03/07/98 +0100, you wrote: >Below is a copy of a letter concerning Project MATCH, an >evaluation of three kinds of alcoholism therapy - Twelve >Step Facilitation (TSF), Motivational Interviewing (MI), >and COgnitive Behavioural Therapy (CBT), which I have sent to The >Addiction Newsletter and may well send to other forums. Pete- that was (is) brilliant and I think you should send it to as many forums as possible- including the usenet newsgroup arf12s. It could do with some quality input... I just want to pick up on one bit of it, where I think it explains some of my own recovery history; >> >>In studies in Social Science, one of the commonest >>confounding variables is self-selection. It is a very easy >>matter to show an association between therapy attendance >>and recovery in a wide variety of settings. clearly, >>someone motivated to get well is likely to attend more >>sessions, but someone motivated to get well is also likely >>to get well anyway. this is before one even considers any >>placebo effect of the therapy itself. The problem is >>self-selection - the person chooses to attend. one cannot >>require clients to attend at gunpoint, but when it comes to >>assessing outcomes, the effect of self-selection must >>always be considered, especially in the absence of a >>control. Probably over 90% of the conviction of the >>effectiveness of AA in its supporters, and 95% of their >>offered evidence, is based on what might be vulgarly called >>the " Bums on Seats " argument, that there are over a >>million AA attendees worldwide at any one time, and many >>tens of thousands of people who attribute their recovery to >>AA. The flaw in this argument is the self-selection >>problem - in all but a handful of places, there are few >>alternatives, and even if there were, someone who attends >>*any* group or treatment is self-selecting to attend it, >>and hence is likely better motivated than one who doesn't, >>and better outcomes than Natural History (if they indeed >>occur) are explicable by higher motivation rather than by >>attendance. as a result any number of testimonials for >>AA or other method are >>meaningless - and the need for controlled studies. This is the explanation for why NA " worked " for me. It didn't. I was committed to my own recovery and I went looking for help and all I found was Narcotics Anonymous, so I used that. So *I* worked for me! I have long had a tendency to think I needed external validation and until I was able (some time later) to confront that, it actually made considerable sense for me to use a group setting and to have a social structure of similarly (abstinence) motivated people to share my journey with, especially at the beginning. Thanks - I can now take back all of my " gratitude " and direct it where it belongs- to myself. What this means is that I think I now have te answer to something that has puzzled me for a long time- how would I have done if there had been the choice of programs and information that I now have available. I think I would have done very well, and I think I could have avoided getting sucked into the religion and anti-thinking and submissive philosophy and got my own life back considerably earlier. Joe Berenbaum Slogan removed for routine maintenance ---- Read this list on the Web at http://www.FindMail.com/list/12-step-free/ To unsubscribe, email to 12-step-free-unsubscribe@... To subscribe, email to 12-step-free-subscribe@... -- Start a FREE E-Mail List at http://makelist.com ! Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You are posting as a guest. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.