Book Review

Snake Oil Science: The Truth about Complementary and Alternative Medicine

by R. Barker Bausell
(Oxford 2007)

Packaging Placebos

Snake Oil Science is the first scientific evaluation of complementary and alternative medicine (CAM), according to the author. Until recently there hasn't been much scientific evidence to evaluate. Sure, we have millions of anecdotes based on personal experience and they're convincing and persuasive to millions of people, but they aren't scientific evidence. It's much harder to control for bias when dealing with personal experience; yet, as Bausell documents, medical researchers can be biased, self-deceived, and driven by wishful thinking in their efforts to prove some favored treatment works. If nothing else, Bausell's book should leave the reader with an appreciation of what a good scientific study in medicine should look like and a feeling of dismay at the number of low quality studies that get published in peer reviewed journals and then find their way into the mass media, embellished and shining, promising more than they can ever hope to deliver.

What qualifies Bausell to do an evaluation of scientific methods and then, by extension, to do a scientific evaluation of that Promised Land called CAM? He's an expert in biostatistics and he served for five years as the director of research at the University of Maryland's NIH-funded Complementary Medicine Program (now called the Center for Integrative Medicine). He knows how to tell a well designed and implemented medical study from a faulty or incompetent one, and he knows how to evaluate the statistical data that is the backbone of such studies.

It might seem obvious to most readers of The Skeptic's Dictionary, but apparently many members of the CAM research community and the general public, as well as many members of the press, don't understand the importance of double-blind, randomized clinical trials that employ placebo groups and have low attrition rates. (Some studies present a false sense of success only because they don't mention that, for example, 60% of those getting acupuncture to cure their heroin addiction, dropped out of the study before it was completed. Guess who dropped out and who stayed in the study, and guess what that can make the data look like if the attrition rate isn't mentioned.) Many people don't appreciate the importance of having a large number of participants in a clinical trial (at least 25 should be in each group, according to Bausell). Many also think there's no difference between peer reviewed journals. They rank a study in the Journal of Scientific Exploration, the Journal of Noetic Sciences, or the Journal of the Australasian College of Nutritional and Environmental Medicine as high as they would an article in Lancet, the Journal of the American Medical Association or the New England Journal of Medicine. According to Bausell,  many CAM practitioners or researchers, as well as many of their clients, think biased personal experience trumps an unbiased scientific study. Bausell makes it clear, however, that bias is a problem for all medical researchers and isn't restricted to just CAM researchers.

The good news for the millions of people who spend billions of dollars on CAM every year is that most of the CAM therapies work. This is an obvious fact, of course. These treatments would have died with the first shamans had there not been a large base of satisfied customers. The bad news is that their effectiveness is weak, temporary, based on subjective rather than objective outcomes, and the evidence that they work for the reasons their advocates claim is either non-existent or very weak. They work but they don't work any better than a placebo. In fact, Bausell argues, that's what CAM delivers with each dose of hope it packages—placebos and a few other artifacts unrelated to the hocus pocus or metaphysical beliefs in which these therapies are wrapped.

Bausell doesn't deny that millions of people find relief from their symptoms after receiving some type of CAM treatment. How could he? This fact is as well known as the fact that millions of people find relief from their symptoms after receiving medical treatment from a practitioner of scientific medicine. The scientific evidence strongly indicates that most of the relief from CAM and a good deal of the relief from scientific medicine, is coming from the placebo effect. Bausell focuses on CAM but we should not forget that much of what he says will apply equally well to scientific medical research as well.

The most common types of CAM therapies are acupuncture, chelation therapy for disorders other than the removal of ions of heavy metals such as lead, chiropractic, healing prayer, herbal remedies, homeopathy, hypnosis, massage therapy, and spiritual/energy healing (e.g., reiki, therapeutic touch,  intuitive healing). Also popular are therapies involving breathing and relaxation techniques, biofeedback, guided imagery, meditation, naturopathy, osteopathy, and vitamin therapy.

What qualifies a treatment as CAM? Bausell notes that CAM therapies share several characteristics. They all aim to treat some medical disorder, but the same treatment might be considered scientific or CAM, depending on what it's being used for. Chelation to treat autism is CAM; chelation to treat lead poisoning is scientific medicine. Transcendental meditation as a lifestyle choice is not CAM. Meditating to treat some medical disorder is CAM. CAM treatments "usually (but by no means always) ... employ physiologic mechanisms of action currently unknown to biological science; are holistic in nature; are noninvasive, natural, and nontoxic; are applicable to an extremely wide range of medical conditions, and are individualized for each patient" (p. 19). Many CAM providers, but not all, "tend to place less value on scientific evidence [than practitioners of scientific medicine], often arguing that standard scientific methods are inadequate to evaluate the effectiveness of the treatments they offer."

However one defines CAM, says Bausell, these treatments "are practiced in the absence of both scientific evidence proving their effectiveness and a plausible biological explanation for why they should be effective, and their practice continues unabated even after (1) there is scientific evidence that they are ineffective and (2) their biological basis is discredited"  (p. 21). In some books, these facts alone qualify CAM as pseudoscience, though this inflammatory word is not used in Bausell's book.

Bausell reminds us that there are some science-based medical practices that are out-of-date or have proved to be based on errors, but when such practices have been shown to be ineffective or their biological bases have been discredited, the practices are discontinued. (For example, the notion that peptic ulcers are caused by stress, which led to various treatments no longer used, has been replaced by the notion that these ulcers are caused by Helicobacter pylori (H. pylori), a type of bacteria, and these ulcers are now treated with drugs that kill the bacteria.) This self-corrective feature of science is what some consider its main difference from pseudoscience. CAM therapists, writes Bausell, "simply do not value and most, in my experience, do not understand the scientific process" (p. 22). Fortunately for the CAM healers, most of their clients don't understand the scientific process, either.

Since the mid-1950s, when H. K. Beecher published a study that claimed that more than one-third of patients participating in clinical trials benefit without receiving any active treatment, medical researchers who understand what this means have designed their experiments to account for the placebo effect. New drugs or therapies might work, but if they don't work any better than a placebo then that's what they are: a placebo. This fact seems very difficult for many people to grasp. Acupuncture, for example, is currently enjoying a surge of popularity in the U.S., even though the preponderance of the evidence supports beyond a reasonable doubt that acupuncture works exactly the same way a placebo works. (I'll return to this point later in a brief discussion of the scientific experiments described by Bausell that demonstrate this fact.) Many CAM researchers seem oblivious to the requirement to rule out the placebo effect and not only don't use placebo groups, they don't use controls at all. Bausell explains the lack of placebo controls in CAM as partially due to the difficulty of devising adequate placebos. For example, it is easy to devise an adequate control for homeopathic remedies (since one vial of water looks and tastes just like any other vial of water), but it would be very difficult to devise fake spinal manipulation.

The placebo effect is just one of many confounding factors that make it difficult to assess properly the effectiveness of a treatment, i.e., to weed out the effectiveness due to the treatment itself from the effectiveness due to other factors. The researcher must also consider "artifacts such as the natural history of a disease (that is, the tendency for people to get better or worse during the course of an illness irrespective of any treatment at all), the fact that people behave differently when they are participating in an experiment than when they are not, a desire to please the experimental staff by providing socially desirable answers..." (p. 27) and such things as the natural regression of chronic pain.

The scientific research has established beyond a reasonable doubt that a key factor in the placebo effect is belief. The main component of this belief is the expectation of some benefit from the treatment. Belief also seems to play a large part in the attrition rate of a study: controls often figure out they're controls and quit.

Bausell does not consider all positive physiological effects that might result from reduction of stress or increase of exercise during CAM treatment to be part of the placebo effect. Any benefit from relaxation practices or exercise are not considered to be CAM-unique or CAM-specific. Even the practitioners of such voodoo as homeopathy, reflexology, iridology, chiropractic, or auricular acupuncture don't attribute the wonders of their craft to relaxation.

Another important factor in understanding the placebo effect is the role of suggestion and classical conditioning. Bausell speculates that we may be hardwired to be susceptible to suggestion and conditioning "because of their importance in facilitating learning" (p. 51). If you join together belief in a treatment, desire for it to bring relief, suggestion from authorities, anecdotes that it does work to relieve pain, classical conditioning, the natural history of a disorder, selective appeal to questionable studies, and the natural regression of most disorders involving chronic pain, then you have a recipe for magical thinking that will lead intelligent, highly educated persons down the path of no return from their favored CAM treatment. Bausell illustrates this point  by examining an article written by New York Times "Personal Health" columnist Jane Brody (pp. 287-289). Last year she wrote about the wonders of glucosamine and chondroitin sulfate supplements, a common CAM treatment for joint pain, and how both she and her dog are avid users of these substances. In a 1999 column, she wrote about her "arthritic spaniel's dramatic improvement" and her own "30 percent improvement," i.e., "less pain and stiffness and little or no swelling after activities that gave my knees a workout, such as tennis and ice-skating."

"A year later, my dog and I are still taking the supplement, though at lower daily doses," she wrote. "My dog, who will be 13 in June, appears to be free of pain and stiffness. He walks two hours a day, goes up and down stairs easily and regularly climbs a mountain road with me. I continue to play singles tennis two to four times a week and skate four or five times a week, and I have added a daily 3½-mile brisk walk to my activities."

Brody noted that her "mailbox has been stuffed with testimonials from others who have ventured into this form of alternative medicine to cope with their arthritis." She also noted that "anecdotes do not establish facts" and finished off her piece by citing several studies that look promising in support of the effectiveness of glucosamine and chondroitin sulfate supplements for joint pain. None of the studies would qualify as high quality, large scale, randomized, placebo controlled studies, however. Yet, when a high quality study involving 1,583 participants was published in the New England Journal of Medicine by Daniel Clegg et al. and reported that glucosamine and chondroitin sulfate alone or in combination are not effective in reducing pain in patients with osteoarthritis of the knee, Brody would have none of it. Yet, rather than reject the unbiased science in favor of her biased understanding of her personal experience, she offered this bit of rationalization and appeal to authority: "While not everyone who has tried this supplement has seen improvement, my sports medicine physician reports that at least a third of his patients have benefited."* In her 1999 article she used the following rationalization: "Not everyone improves on the supplements. If cartilage has completely worn away, it cannot be rebuilt. On average, about half of those who try the supplements report reduced pain and stiffness."* Again, the studies she cites in her earlier article are not of a very high caliber.

What is required of a high caliber study, the reader might wonder? According to Bausell, the most credible type of evidence available to us comes from "large [at least 25 in each group], double-blinded randomized, placebo-controlled clinical trials with low experimental attrition published in high-quality scientific journals that have been independently validated by other investigators" (104). Unfortunately, that eliminates most CAM studies from consideration. Even so, for the period February 2000 to January 2007, Bausell found twenty-two such studies involving such things as using acupuncture to treat cocaine addiction, St. John's wort for depression, chelation therapy for heart disease, ginkgo for memory and cognitive function, Echinacea for colds, glucosomanine/chondroitin for osteoarthritis, and magnets for heel pain. In every case but one, the results were negative (186-190). Bausell wonders out loud if the one positive isn't the one-in-twenty false positive expected given the statistical standards used in these studies (0.05 chance that the results are a statistical fluke).

Perhaps taking their cue from Dean Radin, many CAM supporters have turned to meta-analysis to try to make their case, although they seem to prefer the term systematic review. In any case, Bausell made his way through 98 systematic reviews of everything from using acupuncture to treat asthma to using therapeutic touch to heal acute wounds (pp. 210-244). This time he found 21% of the studies were positive. However, upon further review  (like removing those that didn't have placebo controls or were later trumped by large, high quality randomized control studies), Bausell came up with a figure of 5% positive, not very impressive, especially when you consider the problem of publication bias and the statistical probability that 5% of studies will produce a false positive.

Bausell thinks he can reconcile the lack of high quality scientific studies supporting the efficacy of CAM with the fact that billions of people have been swearing by these remedies for thousands of years. The answer, he says, is in people's ignorance of the placebo effect and a few other artifacts that accompany most CAM studies and treatments. Placebo effects "are ultimately built upon human frailty and they depend upon ignorance (or misconceptions) for their continued effectiveness" (66). Thus, we can probably count on CAM being around for a few more millennia.

While many CAM researchers either "totally disregard" or are "totally ignorant of" methodological quality, according to Bausell, research establishing the reality and the mechanism of the placebo effect has been first class. One such study was published in the journal Pain. It was conducted by Antonella Pollo et al. and demonstrated that placebos can help people with serious pain (pp. 139ff). Other researchers, such as Donald Price, have shown that placebos work to reduce pain only when the subject believes that the therapy is capable of reducing pain. "This belief can be instilled through classical conditioning, or simply by the suggestion of a respected individual that this intervention (or therapy) can reduce pain" (141). Martina Amanzio et al. demonstrated that "at least part of the physiological basis for the placebo effect is opiod in nature" (160). That is, we can be conditioned to release such chemical substances as endorphins, catecholamines, cortisol, and adrenaline. One reason, therefore, that people report pain relief from both acupuncture and sham acupuncture is that both are placebos that stimulate the opiod system, the body's natural pharmacy.

According to Bausell, complementary and alternative medicine (CAM) practitioners' greatest asset is their nourishment of hope (294). "Such therapies may be engendering nothing more than the expectation that they will reduce pain by elaborate explanations, promises, and ceremonies" (149). For billions of people everywhere, that seems to be enough.

While it may be unethical to knowingly package, prescribe, or sell placebos as magical cures, the CAM folks seem to think they are ethical because they really believe in their chi, meridians, yin, yang, prana, vata, pitta, kapha, auras, chakras, energies, spirits, succussion, natural herbs, water with precise and selective memory, subluxations, cranial and vertebral manipulations, douches, body maps, divinities, and various unobservable processes that mysteriously carry out every sort of analgesic and curative function imaginable.

Packaging placebos is big business and, for the reasons given by Bausell that have been outlined above, is likely to get even bigger. The only thing that could slow down CAM atavism would be the sudden appearance of horrible side effects issuing from toxic treatments like aura cleansings or homeopathic douches.

I'd say that there's only about a one in twenty chance of that happening any time soon.

I highly recommend Snake Oil Science. It's written for both the scientist and the non-scientist, has lots of examples and illustrations, and doesn't burden the reader with occult and arcane statistics. If the reader wants to expand his or her understanding of the intersection of science and the public interest, I'd also recommend two books that Bausell himself recommends: Bob Park's Voodoo Science and Christopher Toumey's Conjuring Science.

Robert Todd Carroll
December 26, 2007