Hearsay squared — acupuncture for breast cancer

You’ve no doubt heard the news of a study reportedly finding that acupuncture works to reduce the side effects of breast cancer treatment as effectively as conventional medicine, without the side effects. This is a hallmark news story worthy of lining the bird cage — did you catch why?

Certainly, women bravely undergoing breast cancer treatment are desperate for anything that promises relief from the adverse effects of the treatments. And every healthcare professional and loved one wants to be able to offer that help for them. But that is precisely why stories like these, that take advantage of the special vulnerability of cancer patients — with promises of healing modalities that work as well as, or better than, modern medicine — need to be viewed especially critically. It goes beyond the fact that countless women may start spending one-hundred dollars a pop for acupuncture or turn to alternative therapies at a time when they most need evidence-based care to save their lives. It also encourages the continued diversion of valuable and limited resources for cancer research to modalities that have already been shown ineffective and biologically implausible by every credible, fair test of them.

Science by press release — hearsay squared

Like me, you probably first heard about this study from news stories on Monday, when the alphabet media first began reporting on it. By this morning, at least 144 media outlets had reported the findings of this single study, with every report taken from the same press release — one issued on Monday from Henry Ford Health System.

A small study receiving widespread media coverage all on the same day — check

Not only has the media been reporting from a press release, it’s been reporting on the results of a study presented orally at a meeting of the American Society for Therapeutic Radiology and Oncology. There is no published research for a single journalist to report the findings first-hand or for any medical professional to peer-review the study’s data and methodology and respond in a helpful way. As we know, very few papers presented at meetings are strong enough to ever get published in peer-reviewed journals because a high percentage of them are biased, weak studies with more promotional than scientific merit.

But it gets worse. This wasn’t even research presented in-depth at the ASTRO meeting. It was a 10-minute reading of an abstract — that wasn’t held until 1:50 pm this afternoon! The press release was a forward-looking statement of the future. For days, hearsay-squared has been reported as news!

Reporting on an abstract presented at a meeting, not a published study — check

Whether it’s been from WebMD to the New York Times and every media outlet in between, the medical news has all simply repeated the script provided in the press release. ABC’s medical science reporter, John McKenzie, hadn’t even read the press release carefully to catch the date the abstract was to be presented and in his story published three days before it happened, he said the findings “were presented today at the ASTRO annual meeting,” as if he’d been there!

What are press releases? Marketing, of course.

What the public also didn’t hear was that Henry Ford Health System in Detroit, Michigan, has a Center for Integrative Medicine offering “healing traditions to help you achieve overall wellness and optimal health,” featuring energy medicine in the form of acupuncture. The co-authors of this study were with the Henry Ford Complementary & Integrative Medicine Program. As its website tells the public: “Acupuncturists believe that imbalances in chi are at the root of diseases and that acupuncture helps to restore proper energy flow along pathways, called meridians. Restoring normal energy flow can consequently relieve pain or treat disease.”

News taken from the same press release, giving only one side of the story — check

A closer look — the fair test challenge

But even the press release and the abstract, “Acupuncture for the Treatment of Vasomotor Symptoms in Breast Cancer Patients Receiving Hormone Suppression Treatment,” provided everything we need to see past the marketing and recognize why this study wasn’t a fair test of acupuncture or any medical intervention.

This study was led by Dr. Eleanor Walker, M.D., a radiology oncologist at Henry Ford. According to the abstract, the objective of this study was to “test the hypothesis that acupuncture reduces vasomotor symptoms in breast cancer patients receiving hormonal therapy and produces fewer side effects than venlafaxine.” The authors were working from a belief in acupuncture’s effectiveness for relieving the menopausal-like hot flashes and symptoms suffered by many women undergoing chemotherapy and/or receiving anti-estrogen hormones for breast cancer.

All we were told about the study’s methodology was:

Patients treated for breast cancer (Stages 0-III) receiving either Tamoxifen or Arimidex and having at least 14 hot flashes per week were randomized to receive a 12 week course of acupuncture or venlafaxine. 47 total patients completed the study (24 acupuncture, 23 venlafaxine). Patients logged daily the number and severity of hot flashes for one-week prior to treatment (baseline), during the course of treatment, and at regular intervals throughout 1 year following treatment. Other outcome measures assessed before, during, and after treatment included menopause-specific quality of life, general health status (SF-12), Beck Depression Inventory, and side effects.

Do you see the flaws so far and why this news story is one for the bird cage? Here are a few tips, using the principles of fair test:

Selection/allocation bias. There was no description of how the study participants were recruited or selected, or if they were representative of breast cancer patients. We have no information about them at all or how the two intervention groups differed, making it impossible to evaluate the factors that might bias the results. We don’t know if they were more likely to believe in alternative modalities, their age, socioeconomic situation, support systems, duration of cancer treatments and physical condition, types of previous interventions, other medications, etc.

Attrition bias. The number of women who began the study wasn’t revealed, only that 47 completed the study, but we don’t know the attrition rate or reasons for those who dropped out and how they differed from the women who completed the study.

Observation bias. This was not a blinded or a placebo-controlled trial. Acupuncture has been studied for decades and other recent trials have accomplished placebo (sham*) acupuncture that mimics real acupuncture while being physiologically inert, so that the participants don’t know if they’re actually getting acupuncture. Blinding of participants is imperative in human research to avoid the powerful unconscious effects of our own beliefs and expectations.

Not only was there no placebo control for the acupuncture arm, there was no placebo control group for the prescription medication. In a credible, fair test comparing acupuncture to standard medication, every study participant would have received “acupuncture” and a “pill” — with none of them knowing if either intervention was real or placebo/sham.

Intervention bias. There were numerous other weaknesses in the design of this study. There were no procedural controls to ensure the two groups received the same treatment and had similar study experience, as they would more likely have had in a blinded, placebo-controlled design as just described. When two groups are treated very differently, too many confounding factors get in the way of a fair test of the intervention itself.

In this study, the acupuncture group, for example, enjoyed twice weekly attention and human interaction for the first month and weekly thereafter, and may have been susceptible simply to the effects and expectations of the ritual of acupuncture. The other women were simply given a bottle of pills.

Nor do we know if acupuncture was compared to an effective treatment for hot flashes to begin with. The dosage of the prescription drug venlafaxine was not even mentioned!

We don’t know if this prescription medicine [venlafaxine, a selective serotonin reuptake inhibitor antidepressant] was given in a therapeutic, effective dose. We’ve seen that trick used before when a new drug is compared to low, untherapeutic doses of a competitor’s drug. This isn’t a moot point here, either. Just last year, the Oncologist reported on a randomized, double-blind, placebo controlled clinical trial of venlafaxine for hot flashes led by doctors at Indiana University. They found that venlafaxine resulted in modest reductions for acute hot flashes, but only at a higher dose. Subjective, but not physiological, hot flash measures showed placebo effects. But only women with a greater than 50% reduction in physiologically-measured hot flashes benefited in improved sleep quality, quality of life and less fatigue.

Measurement bias. There were no objective clinical endpoints that were measurable and confirmable. The women kept logs of their hot flashes and recorded their subjective feelings and “quality of life symptoms.” No physiological measures were recorded or observed by objective evaluators. As the abstract states, the women receiving acupuncture reported “increased energy, clarity of thought, sexual desire, and overall sense of well-being.” These results are subjective. Might these also have been more easily, honestly and inexpensively achieved with weekly loving massages, hugs or supportive meetings with a caring friend?

Reporting bias. The study reported no numbers or quantifiable results for how many or to what degree the women in each group saw effects. Nor was the study even designed to be of sufficient size to be adequately powered to derive any statistically or clinically meaningful differences between the groups. The authors reported findings too vague for any medical professional to credibly evaluate. Yet, despite the enormous methodological flaws which could easily tilt the results in favor of acupuncture, they still reported no difference:

Both acupuncture and venlafaxine groups exhibited significant decreases in hot flashes and other menopausal, quality of life symptoms, as well as decreases in depressive symptoms. These changes were similar in the two groups, indicating that acupuncture is at least as effective as venlafaxine in reducing vasomotor and other symptoms.

Scientific contribution — evidence to date

Research that is neutral and conducted in the interests of furthering scientific understanding and improving patient care, rather than promoting a specific modality, ends by carefully examining the strengths and weaknesses of the study and how it contributes to the body of evidence to date. The biological plausibility of any hypotheses are presented. These authors provide none of that. Instead, this was a small, methodologically faulty study that wasn’t designed to truly be a valid test of the hypothesis or convincingly help to resolve any scientific controversies.

Additionally, the significance of this study’s results were wildly overstated to suggest the establishment of acupuncture for breast cancer treatment regimens as being more effective than conventional care. They concluded:

The results of this study suggest that adding acupuncture to breast cancer treatment regimens may establish an integrative approach that is more effective in managing symptoms due to treatment with fewer side effects than conventional pharmacotherapy treatment.

It’s difficult to know what is most troubling: that medical professionals wouldn’t understand what makes credible clinical research or that they did. In either case, their far-reaching conclusions are not supported in this study. Nor does the body of scientific research to date support them.

The latest Evidence Report-Technology Assessment on the Management of menopause-related symptoms released by the Agency of Healthcare Research and Quality in March 2005 found only four small clinical trials examining acupuncture specifically for menopausal symptoms of hot flashes and well-being. It concluded:

Of these, one trial was rated fair-quality and three poor. The fair-quality trial found no differences between groups in any measures, while a poor-quality study [of only 18 patients who had received 6 treatments] reported improved mood in the treatment group, but no differences in menopausal symptoms or well-being. A poor-quality trial compared acupuncture treatment specific to menopausal symptoms to both nonspecific acupuncture and estrogen and found a more pronounced improvement in hot flashes with estrogen than acupuncture.

Dr. Edzard Ernst, M.D., Ph.D., FRCP, FRCPEd, chair of the department of complementary medicine at the University of Exeter, is Britain’s first and only professor of complementary medicine. As one of the world’s foremost experts on alternative modalities, he’s devoted more than fifteen years putting them to the test using the highest methodological quality, with a special concern for the safety and welfare of patients. He’s authored more than a thousand papers in peer-reviewed publications. [A review of his recent work can be found here.]

In the February 2006 issue of the Journal of Internal Medicine, he wrote a critical analysis of acupuncture, aiming to provide a balanced, critical examination of the existing evidence. As he said, “the effectiveness of acupuncture has been tested in several hundred controlled clinical trials, and numerous systematic reviews of these studies are now available.” Most reviews, however, “lack a critical stance and some are overtly promotional,” he said. He found that methodological rigor among clinical trials of acupuncture is more often than not low, most are underpowered, and “their conclusions range from optimistic enthusiasm to reserved skepticism.”

A 2001 review in the Scientific Review of Alternative Medicine had concluded: “Taken as a group, reviews of clinical studies published since 1990 on the clinical efficacy of acupuncture do not support the notion that acupuncture is effective for any variety of conditions and cast doubt on efficacy for some specific conditions for which acupuncture has been reported as effective.” But the clinical effectiveness of acupuncture continues to attract controversy, said Dr. Ernst.

The use of placebo needles (sham acupuncture), which enables researchers to appropriately control for the placebo effects of acupuncture, provides a way to help resolve some of the controversies. In examining all clinical trials available through July 2005 using these placebo devices, he concluded: “The majority of studies using such devices fails to show effects beyond a placebo response.”

For earlier research going back to 1973, the historical Position Paper on Acupuncture from the National Council Against Health Fraud provides a glimpse of the decades-old scientific concerns surrounding acupuncture.

There’s another feature of clinical trials missing in this small Henry Ford study in the news this week. A search at ClinicalTrials.gov through the National Institutes of Health and the National Center for Complementary and Alternative Medicine (NCCAM), finds that it was never registered and is not listed among the 49 completed or active clinical trials of acupuncture.

The abstract says the study was funded by the Susan G. Komen Foundation. This foundation’s Grants Program says it has awarded more than $400 million in research grants, devoted to “energizing science to find the cures.” A search of the research grants it has awarded, as well as the Detroit-area affiliate grant awards, going back through 2003, however, didn’t reveal a grant for this study. It’s uncertain if Susan G. Komen contributors knew that their donations had gone, if indeed they had, for a study that was so poorly science-based and wasn’t working towards a cure.

The authors declared no conflicts of interest, although the press release, as we learned, failed to say that they were with Henry Ford Health System’s Complementary and Integrative Medicine Program. Upcoming consumer Women’s Health classes at the Henry Ford center are already planned that will present this study’s “encouraging results” as support for acupuncture relief without side effects.

The level of media hype this week far outstripped the scientific merits of this research, but, no doubt, will encourage the spending of countless dollars on a modality that the strongest evidence suggests is little more than a placebo. That concerns us, too. At a time when everyone is talking about how expensive our healthcare system is, we can’t afford to waste public resources, or our insurance premiums. Dr. Walker was quoted in USA Today this morning lamenting that “many insurance plans don’t cover acupuncture,” while they do cover the medication. Surely, all of this marketing wasn’t about money.

Saddest is the false hope and misinformation that untold numbers of breast cancer patients heard this week, the subtle reinforcement of feelings of distrust and inferiority of their medical care and modern medicine, and the disservice these media stories provide by exploiting women at the most vulnerable time in their lives.

© 2008 Sandy Szwarc

* Sham acupuncture [from Dr. Edzard Ernst, M.D., Ph.D., FRCP, FRCPEd, in “Acupuncture—a critical analysis,” published in the Journal of Internal Medicine, February 2006]:

Sham devices allow patient-blinding and adequately control for placebo effects by closely mimicking real acupuncture and being physiologically inert. The common principle of these devices is that the sham acupuncture needle gives the impression of penetrating the skin whilst, in reality, it shortens under manual pressure similar to a ‘mini telescope’. Acupuncture-naive patients are therefore unable to tell the difference between these types of sham acupuncture and real acupuncture. Patient blinding in acupuncture trials is crucial not least because patients’ beliefs regarding the receipt of acupuncture bear a stronger relationship to pain perception than the specific effects of acupuncture. The new sham devices allow acupuncture trials to be conducted sham-controlled, patient-blind and evaluator-blind. The therapist will not, however, normally be masked as to treatment allocation, and this remains a source of potential bias.