Evidence-based childhood obesity programs — another case of mistaken definition

We hear a lot about “evidence-based medicine” but we never seem to hear much of the actual evidence. Did you hear about the latest systematic review of the clinical evidence on treatments for childhood obesity?

It was just published the current issue of the Journal of Clinical Endocrinology & Metabolism, the journal of The Endocrine Society. The review was conducted by endocrinologists at Mayo Clinic College of Medicine in Rochester, Minnesota, and led by Dr. Victor M. Montori, M.D., a renowned researcher in diabetes and who edited the book on Evidence-based Endocrinology (Humana Press, 2005).

Evidence-based clinical decision-making

Dr. Montori has been an outspoken advocate for basing clinical decisions on sound, unbiased clinical trial evidence. Years ago, he led one of the earliest systematic reviews of randomized clinical trials that had been stopped prematurely when results favoring an intervention surfaced, and alerted the medical community to how this action can lead to flawed evidence.

For that study, published in the Journal of the American Association, twelve international experts in research methodology, working independently, reviewed the clinical trials published from 1975 to 2004 that had been prematurely stopped for favorable results. Dr. Montori and the other reviewers found that most were industry-funded drug trials in cardiology, cancer and HIV/AIDS. The reviewers found that increasing percentages of randomized clinical trials were being stopped early because of purported benefits, particularly common in studies published in the top medical journals, increasing from 0.5% in 1990-94 to 1.2% in 2000-4.

The trials had been stopped after recruiting fewer than two-thirds of the planned participants, after 13 months and a single interim analysis had been done. But when the results were published, 94% of the studies failed to report relevant information about the decision to stop the study early: the planned sample size, the interim analysis that led to the trial being stopped or the justification behind the decision, or do an adjusted analysis accounting for interim monitoring and truncation. Most reported small numbers of events, yet implausibly large effects of the treatment, they found. “These considerations suggest that clinicians should view results of RCTs stopped early for benefit with skepticism,” the reviewers concluded.

And last year, he led a critical review of the clinical evidence for preventing diabetes using drugs. Dr. Montori and his co-authors noted that they share a professional interest in “bias and spin in the medical literature and its role in informing and distorting practice.” In their introduction, they said that “the medicalisation of pre-disease states and risk factors has become increasingly common” and was making the marketing of drugs to otherwise healthy people an attractive idea for the drug industry as it greatly expands the market for drugs. But, it also risked increasing costs for society, increasing the use of healthcare resources, and potentially reducing quality of life for people by converting healthy people into patients. They found that the clinical trial evidence for the ability of drugs to prevent diabetes were of low methodological quality, with large numbers lost to follow-up, of short duration with even shorter follow-up, and none of the trials showed a reduction in diabetes complications.

They cautioned that diabetes and cardiovascular trials using composite endpoints could be misleading because endpoints of least importance for patients often contributed the most events of claimed benefit, making it important to look closely at what was being measured. The DREAM trial of patients with impaired fasting glucose or impaired glucose tolerance to evaluate if rosiglitazone could prevent diabetes, for example, reported that the diabetes drug was associated with fewer diagnoses of diabetes (“although it is unclear how long, if at all, that reduced risk will extend”). But it came with a 7-fold higher incidence of congestive heart failure, twice the bone fractures, and “had no effect on all-cause mortality, an outcome of great importance to patients.”

“Even under the most optimistic assumptions, patients offered rosiglitazone for prevention will end up taking more pills,” they concluded. “Thus, neither patients who value preventing diabetes in order to avoid taking drugs, nor a society concerned with cost minimisation, benefit from early use of rosiglitazone.”

The study found that a third of patients experienced distressing anxiety by being given early intensive treatment, suggesting that increasing patient anxiety and reducing their quality of life was a real possibility in preventive interventions. The body of evidence is growing showing serious side effects of glitazone drugs, from significant bone loss to serious cardiovascular complications, they wrote, while evidence linking the drugs with reduced diabetes complications “remains lacking.” When considering preventive interventions, they concluded, “because of the risk of harming people with no or minimal symptoms, the threshold for use of drugs in otherwise healthy people must be set high.” This requires large and long randomized controlled clinical trials.

The clinical uses of glitazones to prevent diabetes is currently impossible to justify, they said. Not only are benefits for important outcomes for patients or lasting effects on serum glucose unproved, but preventive medications increase the burden of being labeled as diseased, economic burdens, and increase risks for serious adverse effects.

This is a rather long-winded introduction to the lead author of this new review of the medical evidence on childhood obesity treatments. It seemed to help explain the tug and pull between the actual evidence reported in this careful systematic review and the interpretation of that evidence, especially as presented in the accompanying clinical practice guidelines written by the Endocrine Society.

Randomized trial evidence on child obesity interventions

In this systematic review, Dr. Montori and colleagues took an unusual step to write a paragraph in the introduction to explain that their review was done under a paid contract from the Endocrine Society that had formed a Task Force to develop its own guidelines for the management of pediatric obesity. This was followed by a paragraph at the beginning of the Methods section that said: “The Endocrine Society Pediatric Obesity Task Force commissioned this review, approved the review protocol, offered references, and provided insight into the interpretation of the results. We have produced this report in adherence with the Quality of Reporting of Meta-analyses (QUOROM) standards for reporting systematic reviews of randomized trials.”

Their evidential review of childhood obesity interventions began with a search of nine major electronic databases for published (in any language) randomized trials of overweight children and teens that were studying the effects of diet, lifestyle and pharmaceutical interventions on obesity. They also reviewed the suggested articles from the Endocrine Society’s Pediatric Obesity Task Force. Studies of young people with medical conditions (such as type 1 diabetes, eating disorders, and Prader-Willi) where obesity was merely part of the clinical outcomes, were not included. The validity and quality of the trials were evaluated, following the standard procedures recommended in the Cochrane Reviewers’ Handbook, for bias and fair test — examining allocation bias, blinding, randomization, attrition, etc. In the end, after evaluating 1,162 studies, 61 intervention trials had sufficiently complete data to include in their meta-analysis.

Overall, the Mayo researchers found the methodological quality of childhood obesity research poor, with nearly all trials failing to conduct or report on allocation and blinding. Also, 17% to 35% of the trials had lost 20% or more of the participants in followup, with no outcome data reported on those participants. Universally, the trials were all of too short a duration to credibly conclude that any intervention for weight loss or maintenance was effective. As we know, the FTC and obesity experts have said that studies must last at least 5 years before the efficacy of any weight intervention can be determined because regain is the norm. The 17 trials of pharmaceuticals were as brief as 6 weeks, lasting an average of 19 weeks. The 6 dietary intervention trials and the 23 trials that assessed a combination of lifestyle interventions (diet, activity and behavioral modification) lasted an average of only 23 weeks and 18 weeks, respectively.

In a supplement, the authors described in more detail each of the trials included in their systematic review. To summarize their findings: The drug trials on young people were all similar to those reported on adults, with small to nonsignificant changes in BMI or obesity status. Trials of Sibutramine in teens reported a loss in BMI of 2.4 kg/m² after 6 months, but was accompanied with higher rates of elevated blood pressure and pulse. More youngsters reported gastrointestinal side effects on orlistat with a BMI loss of 0.7 km/m²; and metformin showed no change in obesity after 6 months. Regardless of the dietary intervention, the Mayo authors said the various diet trials revealed a mere 0.22 mean difference from the placebo. Of the 20 trials using physical activity, the authors found no effect on BMI. And the comprehensive lifestyle interventions that included everything popularly believed to address childhood obesity showed small to moderate “nonsignificant trends on BMI.”

Despite the modest results demonstrated in the research, in summarizing the limitations of their review, the authors said it still likely represented “a high probability of reporting bias…i.e. overestimating the treatment effect.”

Overall, included trials have limited methodological features that protect their results from bias and therefore can only yield weak inferences. Pooling across trials with a high degree of clinical or statistical inconsistency increases the risk of spuriously precise estimates with limited clinical sense; the limited duration of the included trials weakens inferences about the long-term effect of the studied interventions.

In reviewing the findings of other major reviews, they said that while they included 42 more trials than the latest Cochrane Collaborative review of the evidence, their findings were similar. That 2003 Cochrane review had concluded that studies were too small to detect any effects of any childhood obesity intervention. They said their results were also consistent with a 1997 review of diet drugs in teens that reported only short-term effects.

They didn’t mention that their findings were also consistent with the largest, most comprehensive review of the evidence on interventions for childhood obesity published last year by the U.S. Preventive Services Task Force. This major undertaking had also examined 40 years of evidence, about 6,900 studies and abstracts, and concluded that there was no evidence for an effective intervention for childhood obesity in primary care settings. The USPSTF has also found no evidence to support dietary counseling in primary care centers for children or adolescents to promote a healthy diet. The USPSTF also reported that there is no quality evidence showing that diet or behavioral weight interventions in young people improves health outcomes or physiological measures, and that the limited clinical trial data on the effectiveness of all weight management interventions in young people was poor to fair.

Most importantly, the USPSTF Childhood Obesity Working Group cited the potentials for harm, adding: “The first principle of medicine is well known: primum non nocere (first, do no harm). If we forge ahead with an intervention (whether therapeutic, preventive, or even diagnostic) without knowing whether it is beneficial, we run the risk of causing unintentional harm.”

Dr. Montori and colleagues, after reviewing the clinical trials on pediatric obesity published worldwide since 1967, concluded:

In conclusion, limited evidence supports the short-term efficacy of selected pharmacological monotherapy, increased physical activity, and combined lifestyle interventions. The long-term impact of obesity treatments on the health of children and adolescents remains unclear.

So, after more than 40 years, there is no sound evidence that any child obesity intervention is effective and offers significant benefits** — with nominal to no short-term effects on BMI demonstrated to date, and no lasting effects for weight or health. As the authors wrote, “the available evidence should inform the practice of evidence-based obesity treatment in children.”

This is important information for healthcare professionals who care about evidence-based medicine and don’t want to subject their young patients to interventions that are ineffective, unproven or, worse, might cause them harm. Parents care about that, too. They all deserve to know that there is no quality evidence to date for the medical, dietary or behavioral treatments of child obesity.

But this review didn’t make the news. Instead, many may only hear about the clinical guidelines and patient recommendations from the Endocrine Society.

Clinical Practice Guidelines

The Endocrine Society published its evidence-based practice guideline for doctors for the treatment and prevention of childhood obesity in the same issue of the Journal of Clinical Endocrinology & Metabolism. It also published an abbreviated version, Patient Guide to the Prevention and Management of Pediatric Obesity, on the website of its Foundation.

Reading these, it is understandable how consumers and healthcare professionals can come to mistakenly believe that the evidence for childhood obesity interventions is stronger than it really is and to believe that obesity can be prevented.

The lead editor of the Patient Guide was Dr. Francine R. Kaufman, M.D., director of the Comprehensive Childhood Diabetes Center and head of the Center for Endocrinology, Diabetes and Metabolism at Children’s Hospital Los Angeles, who was appointed chair of the National Diabetes Education Program at the National Institute of Health (NIH) two months ago. Her financial disclosures are here. Dr. Kaufman authored Diabesity: the obesity-diabetes epidemic that threatens America and what we must do to stop it. JFS readers may be familiar with Dr. Kaufman who, while president of the American Diabetes Association, reported that type 2 diabetes is a new epidemic among American children and that “there is no doubt that the emergence of this epidemic in children and young adults is a major public health problem.”

The Endocrine Society presented its recommendations as being evidence-based, telling patients:

The clinical guidelines were developed after an extensive review of the best human research studies related to the diagnosis, treatment, and prevention of pediatric obesity. An expert panel of The Endocrine Society examined studies that were published in “peer-reviewed” medical journals (i.e., studies that were evaluated by other scientists). The panel rated the quality of the studies and gave the highest rating to randomized (assigned into treatment groups by chance) and placebo-controlled studies. Once the panel reached an agreement about their “recommendations” and “suggestions,” the guidelines were reviewed by the general membership of The Endocrine Society and approved by several of the Society’s committees.

There was no mention that the evidence behind the Panel’s recommendations was of poor quality and short term.

Instead, the Patient Guide told parents that childhood obesity is serious and “should not be dismissed as a condition the child will grow out of, something “in the genes,” or a merely cosmetic concern. Obese children become obese adults, with increased chance for early death.”

After those unsupported scares, parents were given the usual recommendations of diet and exercise; to encourage “healthy eating” in children, as defined as restricting calories and ‘bad’ foods; and to increase fiber, fruits and vegetables. Parents were told children should get one hour of “vigorous physical activity” and spend no more than one to two hours a day engaged in sedentary behaviors. The Endocrine Society Panel went on to recommend widespread community and intense school-based interventions to restrict access to unhealthy foods, educate children about healthy lifestyles, and increase activity.

There was also no mention that the country’s most comprehensive school intervention trial which combined all of these recommendations, in accordance with the U.S. Centers for Disease Control and Prevention’s Guidelines to Promote Lifelong Healthy Eating and Physical Activity, failed to have any effect on children’s BMI or weight status after 2 years.

The more detailed practice guidelines for clinicians were similar to the guidelines published by the CDC, AMA and HHS last summer, reviewed here. Reading the full 79-pages of recommendations, however, it doesn’t appear the Endocrine Society gave much weight to the Mayo researchers’ systematic review of the trial evidence. More than 250 studies that hadn’t made the cut in the Mayo review were added to its Evidence and favorably presented to support everything from suggesting Asians and certain ethnicities may need lower BMI cut-offs than other children; that diseases associated with children having BMI ≥85^th percentile were serious, “placing a lower value on avoiding medicalization and its costs and consequences;” that cutting sugar-sweetened drinks might be effective for reducing children’s weights; that eating more fruits and vegetables can prevent obesity and type 2 diabetes; to claims that breast feeding can prevent obesity.

The descriptions of those studies suggested each recommendation was backed by strong evidence and few busy practitioners probably looked up each reference to identify the incomplete information. Despite the findings of the Mayo review that the weight of evidence of comprehensive lifestyle interventions in pediatric populations had demonstrated insignificant effects on BMI, for example, the Endocrine Society authors said it was a common misconception that lifestyle modification is not efficacious. They supported this statement and their lifestyle interventions saying a weight management program for overweight children “had improved body composition and metabolic parameters” — failing to mention that in that study, all of the 8 children in a weight loss program had regained their weight by year one and that the nondieting control group had better health outcomes at year two.

In recommending BMI screening for children and extensive diet and behavioral interventions, the authors quoted the USPSTF as recommending “that clinicians screen all adult patients for obesity and offer intensive counseling and behavioral interventions to promoted sustained weight loss” — failing to clarify that it had specifically found no evidence to recommend screening, counseling or behavioral interventions for children and adolescents, or that the USPSTF had found no direct evidence that those interventions in adults had any effects on lowering mortality or for offering health benefits, and that sustained weight loss was defined as 3-5 kg loss at one year. The USPSTF had also concluded “the evidence is insufficient to recommend for or against the use of counseling of any intensity and behavioral interventions to promote sustained weight loss in overweight adults.”

The Endocrine Society used tiny symbols in the text to denote the strength of each of its recommendations and the quality of evidence for each, but how many readers looked closely for them?

Every recommendation in the guideline was rated as having very low quality evidence to support it, with only a couple having low quality evidential support. None of the recommendations from the Endocrine Society had quality evidence.

The Task Force explained that they made strong recommendations even when the evidence was of low or very low quality if they thought the trade-off would be a choice most patients would go along with. [Since when does that guide the efficacy of medical interventions and development of clinical guidelines?] They said they made weaker recommendations (which they termed “suggestions” in these guidelines) when the tradeoff was less clear and an individual patient’s values might affect the different choices. Only two recommendations were “suggestions” in these guidelines: criteria limiting the use of diet drugs and bariatric surgery.

Doctors were given these clinical practice guidelines this month, and many may come to find that following them in their practice is compulsory as part of their performance measures from third-party payers. But, as these show, just because expert recommendations and clinical guidelines issued by professional societies are said to be “evidence-based” doesn’t always mean quality evidence.

Poor quality studies can even be given greater weight than quality research. That’s a frightening thought.

© 2008 Sandy Szwarc

** Risk-benefit analysis

As Dr. Montori has written, the weight of the evidence must be set high when there are risks of harm. Evaluating the efficacy of any medical intervention includes a careful analysis of the potential adverse effects and a cost-benefit analysis to ensure the benefits for patients outweigh the risks. As noteworthy as this systematic review was, it was missing a critical section. Other than diet drugs and bariatric surgery, there was no mention of potential adverse effects or costs for any of the childhood obesity interventions. While it might seem intuitive that promoting “healthy eating,” lifestyle interventions and weight management is harmless, the body of evidence is showing this is not the case.

Healthy eating programs to prevent obesity or disease, focused on increased fruits and vegetables, or on diets low in fats, sugars or calories for children, are not only scientifically unsupported, but are actually causing harm. Nutrition-related health problems, growth and cognitive deficits from deficiencies brought on by dietary restrictions, overemphasizing certain foods to the exclusion of others, and undereating have been documented. Little attention is given to how much unsound and potentially harmful and age inappropriate health and nutritional information is being taught children, setting them up for lifelong dysfunctional eating and food fears. Nor was any mention made of how the frenzy to slim kids down has led to increasingly restrictive school meals that don’t even conform to minimal nutrient standards.

Childhood obesity prevention programs and untested health education messages can also stimatize fat children. As has been found, fat children endure stigma more severe than that faced by children with handicaps, amputated limbs or facial deformities. Bullying of children has been shown in the medical literature to have significant physical, academic and emotional health consequences. Fat teens experience nine times the weight-based teasing of their nonfat peers, and this weight stigma has been associated with disordered eating, anxiety, depression, suicidal ideation, and suicide attempts.

Significant adverse effects such as these were reported after the country’s largest and most comprehensive state-wide school-based child obesity initiative. While the program had no effect on obesity or increasing ‘healthy’ eating behaviors, it doubled concerns among young people about their weight and health, and increased dangerous weight control behaviors and eating disorders, especially among girls, to rates considerably above their peers nationwide.

Even when an entire community was organized around healthy eating, exercise and weight management, not only did it have no effect on reducing childhood obesity rates, but while the Somerville children’s school year was absorbed with healthy lifestyles, their average reading test scores were 15.4% below state average and math test scores were a whopping 26% below those of kids in the rest of the state.

Also not weighed in the systematic review were the costs to taxpayers, as limited health care resources are diverted from medical care to unproven diet and exercise programs. The CDC budget for healthy eating and activity initiatives has grown 2,000 percent just since 1999, with National Alliance for Nutrition and Activity (NANA), alone, receiving $65 million this year in funding, $5 million just for fruit and vegetable programs.

These are innocent children who deserve to trust that adults and healthcare professionals are doing what’s best for them and not needlessly putting them at risk with well-meaning interventions that have no sound evidence of benefit.

L. McGovern, J. N. Johnson, R. Paulo, A. Hettinger, V. Singhal, C. Kamath, P. J. Erwin, V. M. Montori (2008). Treatment of Pediatric Obesity: A Systematic Review and Meta-Analysis of Randomized Trials Journal of Clinical Endocrinology & Metabolism, 93 (12), 4600-4605 DOI: 10.1210/jc.2006-2409