Science says isn’t always what science found

We haven’t covered a bird cage news story in awhile, so let’s test our critical reading skills with one that’s been in the news. It’s reported that researchers have found that women who take probiotics during pregnancy can ward off postpartum obesity. What tipped you off that this news story was worthy of lining your bird cage with the newspaper it was printed on?

More than 246,000 news stories about probiotics during pregnancy for reducing fat already appear on Google. The claim made its way around the world in nanoseconds and is well on its way to becoming a truism.

The headlines have become increasingly sensational this past week — reporting that probiotics “may help ward off” postpartum obesity, to probiotics “can” reduce obesity, to probiotics do “cut belly fat” and even telling the public that probiotics “could spell end to need for obesity treatment!”

Every news report, however, has presented the same version the story — the one that appeared in the press release. When we see countless news stories at the same time all giving the same version of a study, it’s a dead give-away that some marketing department issued a press release. Releasing research directly to the media is not how credible science is reported. Press releases rarely present research objectively and fully. They typically not only fail to reveal a study’s limitations, they also exaggerate the significance of the findings, explained Drs. Steven Woloshin, M.D. and Lisa Schwart, M.D.. Science by press release isn’t science at all, it’s marketing.

No reporter read the research to report the findings first-hand. Not a single medical professional read the research to peer-review its methodology and provide critical analysis of its findings to offer balance to the news accounts. Why? There was no newly published study on probiotics and postpartum obesity to examine. The research in the news was from an abstract (T1:RS1.3) released in a 15-minute oral presentation at a conference, the European Congress on Obesity in Amsterdam. In other words, the news was anecdote squared: we’re hearing what a reporter says a researcher said her research found. As we know, abstracts presented at scientific meetings often generate lots of media coverage but are typically so flawed that they rarely get published in peer-reviewed publications — that’s the case with at least six out of ten clinical trials, according to Kay Dickersin, Director of the Center for Clinical Trials at Johns Hopkins University.

According to the press release, the study was presented by Kirsi Laitinen, a nutritionist and senior lecturer at the University of Turku in Finland. In the study described, 256 women were randomly divided into three groups during their first trimester of pregnancy. Two groups were given special dietary counseling for “healthy weight gain and optimal fetal development” along with special foods with “healthy” fatty acids and added fiber; half of those women were given a special probiotic supplement (Lactobacillus rhamnosus and Bifidobacterium lactis) to take during their pregnancy and during postpartum while breastfeeding; the other women on the diet were given a placebo. A third group ate their normal diet and received only a placebo. The women were weighed and measured at the start of the study and again during their third trimester; and at 1, 6 and 12 months postpartum. One year after childbirth, the women who had been taking the probiotic supplement “had the lowest levels of central obesity as well as the lowest body fat percentage,” said Laitinen.

Their study, she said, was “the first to demonstrate the impact of probiotics-supplemented dietary counselling [sic] on adiposity.” This, however, contradicted her closing remarks: “There is growing evidence that this approach might open a new angle on the fight against obesity, either through prevention or treatment.”

The press release suggested that probiotic supplements could play a role in a range of diseases. The only limitations noted about this study was that it failed to control for the mother’s pre-pregnancy weight or determine if the probiotic supplements had any effect on the health outcomes of the mothers or babies. These limitations had far greater significance than was realized by most reporters, most of whom failed to even mention them. Did you catch the significance? Laitinen reported that the women taking probiotics were less fat than the other groups one year after delivery — but it turns out that these women were less fat to begin with! (And there was no difference between the groups in the changes in their weights during pregnancy or after delivery.)

Filling in the blanks

This study was conducted by researchers at the University of Turku Functional Foods Forum. Their research is in nutrigenomics [reviewed here and here] and probiotic supplements. As Seppo Salminen, head of the University’s Functional Foods Forum, explained last year at the Universität Bodenkultur in Vienna, Austria, they are working on probiotics for preventing or reversing allergies and eczema in infants and children, autism, respiratory infections and obesity; plus for detoxification, binding “toxins,” genomics, and to enhance longevity and slow the rate of aging and age-related diseases.

Laitinen and Seppo, along with study co-author Erika Isolauri, are the inventors of an international patent (application date 4.7.08) of probiotic supplements for women during pregnancy for preventing diabetes, for glucose normalization, for improving insulin sensitivity and reducing the risk of metabolic syndrome. Any pill that the public believes will do all that is sure to fly off the shelves.

In the patent documents, filed in 2007, the authors supported their patent application based on the findings of their NAMI-tutkimusohjelma study, “Effects of Maternal Nutrition During Pregnancy and Breast Feeding on the Risk of Allergic Disease in Child.” This study of probiotic supplements during pregnancy for treating and preventing allergic inflammation and atopic diseases — as registered at Clinicaltrials.gov NCT00167700 — was due to be completed on an estimated 600 women in September 2010.

The findings on only 256 pregnant women were reported in their patent application, and the following year were published in the November 2008 issue of the British Journal of Nutrition. No explanation was given for what happened to the rest of the planned study participants. Starting out, the study data showed that the women in the probiotic group weighed about 13.4 pounds less at the start of the study than the diet/placebo group, but there was no statistical difference in the amount of weight the women in the different groups gained during pregnancy or in their weight at one-year postpartum, or in changes in their BMIs. The diet and probiotics had no effect on the birthweights of their babies. Nor was there any difference among the women in how long they breastfed. As the authors reported:

There was no distinction between the groups in terms of pregnancy weight gain, baseline adjusted weights during pregnancy or postpartum BMI. The mean duration of exclusive breast-feeding and thus the duration of probiotics/placebo intervention did not differ amongst the study groups.

Concerning high blood sugars during pregnancy (a marker of gestational diabetes), the risk in the probiotic group was “not statistically significant” and the “relative risk was not significantly lowered as seen on glucose challenge tests given to 45% of the women during their pregnancy. Similarly, “mean glycated haemoglobin A1C were comparable amongst the study groups at the third trimester of pregnancy and 12 months postpartum.”

By all objective measures, this was a null study and probiotics had no effect on pregnancy weight gain, postpartum weight loss, infant weight or the women’s blood sugars.

Once again, we’re reminded that studies’ conclusions don’t always reflect their actual findings. The patent authors concluded: “This study provides the first evidence of an active dialogue between the host and the gut microbiota in glucose metabolism… Probiotics appeared to bring about a more profound glucose lowering effect than dietary counselling alone suggesting that probiotics may be of particular importance.”

Among this study’s methodological shortcomings was that it wasn’t double blinded. When researchers know which particpants are on a study drug, subjective outcome measures are more difficult to interpret. There was also no documentation that exercise was controlled for, even while looking at body composition measurements.

Only selected data was provided on body measurements, biceps skin fold thickness and waist circumference, and no pre-pregnancy measurements were reported, making credible conclusions impossible. Average bicep skinfold thickness measurements were only given for 1, 6 and 12 months postpartum. All of the measurements were higher at six months and lower at 12 months, but the differences were miniscule and all of the changes seen during postpartum were less than the standard deviations:

● The probiotic group went from 1.03 cm to 1.08 cm at one-year postpartum. Their skin fold thickness was 0.05 cm higher one year after delivery.

● This compared to a 0.02 cm reduction seen in both the placebo/diet and control groups.

In other words, the skin fold thickness changes seen during postpartum differed between the probiotic and nonprobiotic groups by biologically meaningless amounts. It was the same story with the waist circumferences, only the data provided was even skimpier. With only the 6- and 12-month data reported, the changes seen among the groups differed by only half a centimeter.

When we see beyond the media’s coverage of a press release of a 15-minute presentation at a conference, “science by press release” marketing becomes a lot clearer.

The sad thing is, most consumers take all studies as being equal and when they hear that new research by a team of scientists found something, it’s natural to believe it is true. And people so want to believe that eating some special foods or taking some supplement can prevent obesity or chronic diseases of aging and help them live longer. But that isn't true.

© 2009 Sandy Szwarc