No evidence that bariatric surgeries save healthcare costs or save lives

A study has been in all the business news this past week and its reported finding is already on its way to becoming a truism: that obesity surgery pays for itself in lower insurance costs in just two to four years. As with so many beliefs that are repeated over and over again, the real story behind its origin are lost.

Few people will hear that the study behind the cost savings of bariatrics was so faulty that even the economist behind the “costs of obesity” myth wrote a cautionary editorial pointing out conspicuous problems. Few people will hear that the study behind the truism was written by bariatric surgeons and staff from a consulting company that provides its corporate clients with data analysis for marketing, litigation and development strategies, and that the surgeon’s role in the paper “could be considered as payment from Ethicon-Endo-Surgery, Inc.,” which financed the paper. Few people will hear that it was published in a business-to-business trade journal, produced by a publisher that creates custom publishing products for health industry clients. Few people will hear that it used a database that’s been cited by the FTC for unfair and deceptive acts and practices, and the target of lawsuits from a state attorney general to the American Medical Association for being fraudulent. And virtually no one will go to the original paper and see how many ways it failed as a “fair test” of any scientific hypothesis.

Instead, the belief has already made its way around the world and become another marketing pitch. Bariatric surgeon and president of the British Obesity Surgery Society has been in the UK news, threatening to sue Primary Care Trusts for not funding bariatric surgeries for the 1.2 million British people who are eligible, saying the surgeries would pay for themselves within four years.

Study overview

This study was published in the American Journal of Managed Care and led by Pierre-Yves Crémieux, Ph.D., adjunct economics professor at the Université du Québec à Montréal. The introduction described the authors’ working assumptions and goals. They first uncritically presented the surgeries’ benefits as maintained by the bariatric industry, including reduced morbidity and marked resolution of comorbidities, longer lifespan, and cost healthcare utilization savings. The authors than stated that “bariatric surgery is an effective treatment for morbid obesity, however, payer coverage for these procedures has lagged because of cost concerns.” The purpose of this study, according to the authors, was to evaluate the return on investment of bariatric surgeries in the United States for private third-party payers.

In describing their methodology, the authors said that they examined the health insurance medical and drug claims, using ICD billing codes in the Ingenix private insurer claims database, linked to 3,651 adults who had bariatric surgeries from 1999-2005. They said this private database includes the private insurance claims of 5 million Americans from 31 large companies that provided health insurance to employers. The bariatric patients in this study were mostly women (86%) with an average age of 44. "All claimants in the study sample were required to have at least 6 months of continuous enrollment before the index date and 1 month following the index date."

For their control group, they selected another 3,651 obese people with medical claims in the database. The authors acknowledged that fat people who had medical claims were likely sicker than the general population of fat people who didn’t have recorded claims, so they said they selected people for their control group by matching them to the bariatric patients on ten comorbidities, also based on billing codes during that 6 month period, as well as geographic location, gender, age group, and healthcare costs during the 5 months prior to surgery.

The authors said they then calculated the number of months post-op before private insurers would recoup the surgery costs and see a return on their investment in savings. They used a Tobit model and estimated the cost differential by estimating the model on the pooled population of both surgery patients and the controls. In other words, the cost savings were estimated using a computer regression model. They reported that the total surgical costs for their cohort was $24,500 for all types of bariatric surgery combined ($26,000 for open surgery and $17,000 for laparoscopic procedures). Their results “estimated all costs to have been recouped within 2 years for laparoscopic surgery patients and within 4 years for open surgery patients:”

Based on the data available and on an assumption of constant savings after 19 months, we find that (for the combined sample) total surgery costs are fully recovered after 53 months (95% CI, –42 to 64 months). Costs of open surgery performed between 1999 and 2002 are fully recovered after 77 months (95% CI, –48 to 106 months), and, as expected, costs of open surgery performed between 2003 and 2005 are recovered after 49 months (95% CI, –35 to 63 months). Costs associated with laparoscopic surgery are fully recovered after 25 months (95% CI, –16 to 34 months). These returns on investment result from reductions in prescription drug costs, physician visit costs, and hospital costs (including emergency department visits and inpatient and outpatient visits).

In the discussion section, the authors added that private insurers could recover their costs in a shorter time frame with “any increases in copayments beyond those incurred by patients in our data (75% of patients had no copayment).”

Study co-author, Scott Shikora, M.D., a bariatric surgeon at Tufts-New England Medical Center in Boston and current president of the bariatric surgeon trade organization, American Society of Bariatric Surgery (now called the American Society for Metabolic and Bariatric Surgery), was quoted in Bloomberg as saying: “When you add the health benefits of bariatric surgery including resolution or improvement of significant diseases like type 2 diabetes, sleep apnea and hypertension to its cost benefit, you have a procedure that not only saves the health care system money, it saves lives.” [For critical examinations of the research behind these assertions, follow links to previous posts.] “Bariatric surgery is a unique field in that with one operation you can cure a wide range of different health conditions,” Dr. Shikora told Associated Press. “This paper demonstrates that you can do that and actually save money in the process.”

Did you catch this paper’s methodological flaws and why it was not a sound fair test of the hypothesis? Let’s take a closer look.

Selection bias

The single most problematic flaw is the easiest to identify: There was no randomization of the cohort. The bariatric surgical candidates for this analysis were hand-selected, as were the matched controls. Randomization, where study participants and controls are randomly selected, is the most important way to help ensure that intervention and control groups are comparable (even in ways researchers may not have considered) and haven’t been hand-chosen to manipulate the findings towards a predetermined conclusion. Randomization prevents stacking the deck, so to speak.

Randomization is critical to any sound, credible analysis of the balance of risks and benefits of any medical intervention. It is easy to select a surgical group that is healthier with fewer complications and healthcare expenses than most bariatric patients, and to select a nonsurgical control group that is sicker with more healthcare expenses than typical bariatric patients, to appear to show comparatively lower healthcare costs among a select surgical group.

Didn’t include 99.4% of bariatric patients. The authors provided no explanation of how or why they selected the 3,651 bariatric patients they used. An estimated 613,400 bariatric surgeries were performed during this study period (1999-2005), according to the American Society for Bariatric Surgery. Yet, this analysis only examined the private insurer claims on fewer than 0.6% of all bariatric surgical patients.

Didn’t include those who died. There were several clues that the surgical patients used in this analysis may not be representative of most bariatric patients. Since it looked retrospectively only at patients survived their surgeries, this paper automatically disregarded the estimated 58,778 bariatric patients in the U.S. from 1999-2005 who would have died. The healthcare expenses for these patients during their final months of life were likely to have been significant.

Atypically low surgical costs. The average surgical costs of $17,000 and $26,000 reported in this paper — based on medical claims one month before surgery, the surgery and to two months following surgery — are lower than other, objective estimates. These lower surgical costs would suggest that the patient data used may have been those with the rosiest outcomes and fewer complications during the post-op period. Other estimates based on everyone who had surgeries across entire state populations, for instance, have included costs associated with post-op complications. Such estimates using data from the same period as in this study (2004-2005) have shown hospitalizations more than doubled for most patients after surgery, and more than a third of patients had such severe complications they required second surgeries and hospital readmissions, bringing average per-patient costs over $44,000. The American Society for Bariatric Surgery even reported that while average costs for the initial surgery in 2005 was $25,000 apiece, complications can raise costs to $100,000.

Comorbidities not fully or appropriately controlled for. The control group that had been selected were matched to the bariatric patients using the following ten comorbidities that had been checked off as diagnoses on billing claims: asthma, coronary artery disease, diabetes mellitus, dyslipidemia, gallstones, gastroesophageal reflux, hypertension, nonalcoholic steatohepatitis or nonalcoholic fatty liver disease, sleep apnea, and urinary incontinence. Besides the fact that billing code check-offs have been shown to be inaccurate measures of medical diagnoses, there was no attempt to control for the severity of those conditions or even the degree of obesity.

In order to qualify for insurance coverage for bariatric surgeries, especially for those with BMIs under 40, prospective patients must have comorbidities noted in their medical records, but those may be minor and never previously have needed medical intervention. Patients seeking medical care specifically for those health problems, however, would be expected to experience more serious primary symptoms.

The authors also note that the guidelines of the American Society for Bariatric Surgery suggested that 18 comorbidities could cause imbalance between the two samples, but for this analysis they matched to fewer than half that number of comorbidities. They omitted matching on these eight conditions: breast cancer, congestive heart failure, lymphedema, major depression, osteoarthritis, polycystic ovary syndrome, pseudotumor cerebri, and venous stasis or leg ulcers — all conditions associated with potentially high healthcare costs.

As economist Eric A. Finkelstein, PhD, MHA, with RTI International, said in an interview reported in the Wall Street Journal, if the control group had had just “one really bad outcome, such as a heart transplant, that alone could be enough” to significantly throw the results to make the bariatric surgery appear cost effective.

No control for socioeconomics. The baseline healthcare expenses for 6 months prior to the study period reveals that the control group in this study had 15% more ER visits, 9% higher inpatient hospitalizations, and 11% fewer doctor office visits, which would suggest they might be underinsured, or not receiving care from a regular physician, both suggestive of socioeconomic differences. But, there was no matching to socioeconomic status.

No control for secondary insurance or deductibles. There was also no control for the amount of deductibles or out-of-pocket expenses paid by patients. Nor did the authors control for secondary insurance. Many bariatric patients, for instance, are eligible for Medicare and disability insurance or currently receive their surgeries paid under publicly-funded Medicare, which becomes the first payer of medical claims and their private insurance is secondary to pick up any remaining amounts. Since the healthcare costs in this study were based only on private insurer claims, they may fail to accurately reflect the full expenses encountered from bariatric procedures.

This study also didn’t include other expenses that employer-funded private insurance may encounter with bariatric coverage. The numbers of bariatric patients with poor outcomes who end up on disability insurance coverage, for example, appear to be significant. As Dr. Robert Anfield, vice president and medical director for UnumProvident, the largest provider of group and individual disability income protection insurance in North America, said in the March 2004 issue of Obesity Policy Report, the most common types of disability claims they are seeing where obesity is the primary diagnosis are for those who are out for bariatric surgery.

Analysis

The most problematic flaw in this report’s statistical analysis was the lack of transparency: it did not report any of the actual expenses among the bariatric surgical patients or the control group! There was no data to support its conclusion that bariatric procedures saved costs or for its purported break-even point. There was no actual expense data that could be examined by peer-reviewers. The data from both bariatric patient and control groups were lumped together into a single variable, derived from their computer model.

Not only that, but the cost savings was not due to actual reductions in medical costs among the bariatric patients, but a purported reduction in comparison to the higher expenses in their control group.

The mathematical calculations were so problematic, that professor Finkelstein and Derek S. Brown, Ph.D., wrote an accompanying editorial in the same AJMC issue explaining what the authors did and pointing out two important points they felt deserved attention:

First, the return on investment estimates are driven primarily by rising costs in their matched control group rather than by a reduction in costs from the bariatric sample. When the authors refer to “savings,” they are actually referring to the difference in costs between the surgery and control groups in the postsurgery period...

Because the monthly savings figures exceed the presurgery mean costs, and dramatically so for the laparoscopic sample, the return on investment estimates seem to result from substantially higher cost increases in the control group relative to the surgery group in the postsurgery period.

Second, ...the authors’ return on investment estimates are based on the assumption that the differential in costs between the 2 groups (ie, the savings) is constant after month 19 for the overall sample and after month 13 for the laparoscopic sample. Whether or not this assumption holds in reality will only be answered once additional follow-up data become available. If this assumption is not met or if another control group has a different cost profile, then the return on investment could be substantially increased.

In other words, these authors assumed that bariatric patients would not experience any long-term complications from their surgeries and that the initial health index changes resulting from rapid weight loss would be maintained, which defy the medical literature to date. The break-even point estimated for open surgery was also dependent on the assumption of a constant cost savings from month 19 onward.

Finkelstein and Brown illustrated their concerns based on claims data from Medstat’s proprietary MarketScan database, which show that during the first 54 months after bariatric procedures, there are no reductions in medical expenses from presurgery periods. As they explained:

[T]there are several reasons why costs are not reduced. First, we found that a few individuals experienced severe adverse events that required lengthy and expensive readmissions. Second,.. they required subsequent surgical procedures to have excess skin removed. There was even evidence of hip and knee replacements that likely resulted from formerly obese individuals’ becoming active...

[The data] also reveals substantial variability in the monthly cost estimates, suggesting that the assumption of constant savings after month 19 (or month 13) is unlikely to hold in reality.

This concurs with other studies which, for instance, counter claims of benefits based on reductions in medications. The Swedish SOS study reported in 2001 that the surgeries did not result in cost savings. The Swedish investigators said that “the use and cost of drugs in obese patients to be about the same, whether or not they had the surgery. Those who didn’t have the procedure needed medication for diabetes and cardiovascular disease, while those who underwent it needed treatment for gastrointestinal-tract disorders, anemia and vitamin deficiency.” Professors Finkelstein and Brown concluded:

As a parting note, we would like to add some caution for insurers and other payers. If coverage decisions are predicated on achieving returns on investment as short as those presented by Crémieux et al — and an internal ex post analysis produces results that are not as favorable because a within sample analysis akin to that in the Figure is conducted or because some of the assumptions in the study by Crémieux and colleagues do not materialize — then it is likely that the decision to cover the procedure will be reversed... For this reason, we have counseled bariatric device manufacturers, insurers, employers, and others against focusing on return on investment as a means for making coverage decisions for bariatric procedures....

The coverage decision should be based on whether or not the intervention can improve the condition in a cost-effective manner compared with other potential treatments.

Is the data from a reliable source?

The failure of this study to follow well-established scientific principles of a fair test of an hypothesis already give good cause to view its conclusions skeptically, but even the authors’ choice of data to use warrant pause. They used the Ingenix database, a private medical record electronic database established in 1996, owned by UnitedHealth Group, one of the nation’s largest health insurers. It compiles insurance claims with individuals’ prescription drug histories from contractual relationships with PBMs (pharmacy benefit managers) to create medical profiles on people and physicians. This database is sold to managed healthcare organizations, for instance, which use it for medical underwriting, to set rates for physician reimbursements, and determine the “usual and customary” out-of-network rates for insurance members.

Last fall, Ingenix ran afoul of the Federal Trade Commission, which has issued consent orders (DOCKET NO. C-4214) against Milliman, Inc. and Ingenix, Inc. for engaging in “unfair and deceptive acts and practices” and violating the Fair Credit Reporting Act.

Ingenix has come under legal action by doctors and New York State Attorney General, Andrew Cuomo, which have asserted that Ingenix reports artificially low price information which healthplans use to calculate out-of-network rates for reimbursement that leave consumers with higher bills, doctors and hospitals unpaid, and consumers less likely to seek the medical care they need. The American Medical Association sued UnitedHealth in March 2000, alleging the database is inaccurate. The case is still pending in U.S. District Court for the Southern District of New York, according to the Wall Street Journal.

Attorney General Cuomo told the press in February that health insurance company ownership of the billing database was a “gross conflict of interest.” In a lawsuit filed against UnitedHealth Group after a six-month investigation, his office argued that the insurer was committing consumer fraud by setting rates for paying out-of-network doctors and hospitals artificially low, sometimes manipulating data, leaving consumers with big bills.

The Attorney General also found that rates generated by Ingenix were much lower than the actual costs of typical medical expenses.

Disclosures

When we encounter studies with so many methodological concerns and conclusions that are so divergent from the medical literature to date, the potential conflicts and biases of the authors become more valuable in trying to make sense of it all. This study was published in a magazine produced by Intellisphere, LLC, which has produced online and print content and custom publishing products since 1999. The company acquired the b-to-b magazine in February from Ascend Media.

The study authors were employees of Analysis Group — including Arindam Ghosh, Ph.D., its manager; and the lead author, Pierre Y. Cremieux, its managing principal — along with two bariatric surgeons, Henry Buchwald, M.D., Ph.D. and Scott A. Shikora, M.D., who are the past and current president of the American Society of Bariatric Surgery, respectively.

Analysis Group is a consulting company which says it “applies analytical expertise in health care economics to litigation, outcomes research, managed care, and business strategies engagements.” It provides pricing and contracting strategies and data sourcing and analysis for its clients which include: “pharmaceutical manufacturers, biotechnology firms, medical device companies, managed care organizations, public and private insurers, health care providers, law firms, and federal, state, and local governments.”

The author disclosure statement at the end of the study states:

Drs Crémieux, Ghosh, and Yang and Ms Buessing report that their employer, Analysis Group, Inc received an unrestricted educational grant from Ethicon Endo-Surgery, Inc for the preparation of the manuscript. Dr Buchwald reports receiving grants from Ethicon Endo-Surgery, Inc and serving as a consultant and as the chair of their scientific advisory board. His involvement with the manuscript can be considered as payment from Ethicon Endo-Surgery, Inc. Dr Buchwald consults for Fulfillium, Inc in return for stock ownership and consults for EnteroMedics. Dr Shikora reports receiving honoraria for speaking engagements, consultancies, and advisory board activities and owns stock in a number of companies within the healthcare sector, including Ethicon Endo-Surgery, Inc, EnteroMedics, Synovis, BariMD, GI Dynamics, Coviden, and others.

This study has all the earmarkings of marketing, rather than solid science. Its focus on convincing private insurers to cover the surgeries is understandable, as the bariatric industry currently faces its greatest roadblock there. This study is a moot point for government-funded healthcare, of course, as the industry successfully convinced political officials to provide the surgeries at taxpayer expense through the Centers for Medicare & Medicaid several years ago. The public will also pay for bariatrics, should consumers be required to purchase health insurance that complies with government guidelines, or universal healthcare becomes a reality. But private third-party payers are less easily convinced than politicians.

Susan Pisano, the spokesperson for the trade association America’s Health Insurance Plans, told media that the cost of bariatric surgeries is a factor for most private insurers, as well as the medical quality and safety of the procedures and surgeons. But she was skeptical of its findings, saying they would need to be replicated in another study.

The media has been widely reporting that this study shows bariatric surgery to save healthcare costs and save lives, and will put pressure on insurance companies to cover the surgeries. But this study provided no sound evidence for any of those claims.

We’d never know that, though, if we didn’t examine the original study for ourselves and understand the concept of a fair test.

This story is not just about consumers paying higher taxes and insurance premiums because health policies aren't grounded in good science. This is about lives. When the myth that bariatrics reduces healthcare costs becomes widely believed, it's a step closer to fat people being coerced into have surgeries and condemmed when they don't.

© 2008 Sandy Szwarc