Loophole Part One: Who is writing clinical care guidelines and public health policies?

While financial disclosures don’t guarantee that research or conclusions of researchers are sound, they are invaluable in helping to alert us to potential biases that might have influenced the findings. Their role in helping to promote the integrity of medical research, as well as deter and detect misconduct, is so important that the Office of Research Integrity at the U.S. Department of Health and Services has made financial disclosures mandatory for all institutions receiving public funds. Most medical journals and expert committees setting clinical care guidelines also follow these financial disclosure regulations.

But there’s a major loophole in these financial statement requirements that you’ll want to understand because it can explain why we too often read research conclusions that depart radically from sound science, but are written by authors declaring no conflicts of interest... when a closer look can find that the most significant financial interests didn’t have to be reported.

Before describing how this discovery came about, the changes that have taken place in the agencies creating clinical care guidelines and health policies that have opened the door to greater abuses of this loophole, and recent examples of why this matters, let’s skip to the ending first.

Office of Research Integrity defines financial disclosures

In March of 1989, the U.S. Public Health Services created the Office of Scientific Integrity at the office of the National Institutes of Health director’s office and the Office of Scientific Integrity Review at the Assistant Secretary for Health’s office. These two offices were combined in 1992 into the Office of Research Integrity (ORI) at the HHS. This office oversees allegations of research misconduct at some 4,000 institutions that receive government agency funds for research. Since 2000, this office has also worked to educate and train investigators in the responsible conduct of research and to prevent misconduct in research and research review programs. It is also the source of disclosure regulations.

Federal regulation 42cfr50 specifically applies to financial interests and disclosure statements. Investigators are required to report ‘significant financial interests’. What that means, as with everything, is a matter of how it’s defined. These definitions have also been adopted by most professional publications and organizations. While reading this ORI policy definition, pay particular note to the financial interests excluded from required disclosures: According to the ORI financial disclosure policy:

Significant Financial Interest means anything of monetary value, including but not limited to, salary or other payments for services (e.g., consulting fees or honoraria); equity interests (e.g., stocks, stock options or other ownership interests); and intellectual property rights (e.g., patents, copyrights and royalties from such rights).

The term does not include:

(1) Salary, royalties, or other remuneration from the applicant institution;

(2) Any ownership interests in the institution, if the institution is an applicant under the SBIR Program;

(3) Income from seminars, lectures, or teaching engagements sponsored by public or nonprofit entities;

(4) Income from service on advisory committees or review panels for public or nonprofit entities;

(5) An equity interest that when aggregated for the Investigator and the Investigator's spouse and dependent children, meets both of the following tests: Does not exceed $10,000 in value as determined through reference to public prices or other reasonable measures of fair market value, and does not represent more than a five percent ownership interest in any single entity; or

(6) Salary, royalties or other payments that when aggregated for the Investigator and the Investigator's spouse and dependent children over the next twelve months, are not expected to exceed $10,000.

In other words, money from nonprofit foundations and professional organizations don’t have to be reported. Nor do salaries, grant funding or other financial benefits that come to researchers through their institutions or university programs. This is the first crucial point.

Changing role of USPSTF and clinical guidelines

The next key point is that changes have taken place at the U.S. Preventive Services Task Force and the Agency for Healthcare Research and Quality. Since its creation in 1984 by the HHS, the USPSTF has been seen as the leading source for objective, systematic reviews of medical research and advice to medical professionals on the evidence for various preventive healthcare measures. Its reviews of the evidence have also become integral to public health policies and initiatives, performance measures and public funding advanced by AHRQ, an arm of the HHS and National Institutes of Health.

The very first USPSTF panel consisted of 20 nonfederal experts, including 14 doctors, a dentist, nurse, researcher, educator, economist and sociologist. In other words, they had broad expertise in research methodology and medicine to allow them to address a range of topics “and to lessen the potential for conflicts of interest in any given issue,” according to a historical review of the USPSTF and the evolving role of preventive health, written by Steven H. Woolf, M.D., MPH. and David Atkins, M.D., MPH.

The USPSTF’s first Guide to Clinical Preventive Services was published in 1989 and evaluated 169 screening and preventive interventions. This first guide helped to establish the credibility of the USPSTF and support for evidence-based medical reviews. But its strict approach drew heated criticism from certain groups which ardently advocated services which the USPSTF found unsupportable. Its neutral position when evidence was lacking drew complaints for eschewing the opinions of experts as a basis for making its recommendations, said Woolf and Atkins.

When the USPSTF reconvened in 1990 to write its 1996 guide, this time its work was marked by stronger ties with both federal and nongovernmental partners, including various professional societies and foundations, said Woolf and Atkins. By the time of this second edition, the medical environment had also changed dramatically, and managed care was the dominant force for the delivery and paying of healthcare. In this cost-containment focus of managed care, its recommendations would have a greater impact and importance to purchasers, health plans and policy makers, they said. The USPSTF guide would be used by these interests to defend their health maintenance programs and benefits packages. Its recommendations also became part of HEDIS (Health Plan Employer Data and Information Set) “quality measures” (performance measures) developed by the National Committee on Quality Assurance for evaluating health plans.

“At the same time, specialty societies and professional organizations that had reorganized and invested in the early 1990s to establish their own practice guideline programs soon found their efforts eclipsed by guidelines developed by commercial vendors and sold to health plans and hospital systems with the promise of lower health care costs and lengths of stay.”

There was also tremendous promotion and commercial advertising among consumers for “healthy lifestyles” which had helped to create an interest among the public “in low-fat diets, exercise and weight management,” they explained. These intense preventive health campaigns had also led to new screening technologies and tests promoted heavily by celebrities and national organizations. “Health plans, in turn, recognized that including comprehensive health promotion and disease prevention was a valuable marketing tool for attracting patients.” Preventive health was becoming big business and a fiercely competitive market. In response to this need, the USPSTF was also tasked to create a Guide to Community Preventive Services, targeting communities and healthcare systems, rather than individual patients, with funding from RWJF to the American College of Preventive Medicine to promote the USPSTF strategies.

“By 1996, the enthusiasm for clinical practice guidelines and for evidence-based medicine had been tempered by a realization of their attendant practical and political challenges,” they wrote. Cost considerations took equal importance to quality care. Financial and political interests were also leading to re-examinations of the role of the USPSTF, as well as that of the Agency for Health Care Policy and Research (renamed AHRQ in 1999) in its role of developing guidelines and policies based on the USPSTF recommendations.

As experience with evidence-based guidelines grew, their shortfalls and limitations were becoming better recognized, including unintended harm, according to Woolf and Atkins.

By the time the third USPSTF was convened in 1998 to update the recommendations of the second Task Force, the Task Force members were selected differently, and were now chosen from nominations by national and specialty organizations.

The role of the AHRQ in developing guidelines changed the most. Congress redirected the AHRQ to not sponsor the development of guidelines, but to support the production of information to be used by outside partners, such as professional societies, who could in turn interpret it and develop guidelines and policies. The AHRQ established a network of 12 practice centers at universities and private organizations to produce reviews for the USPSTF and provide it with reports summarizing the evidence for specific preventive care interventions. These outside technical reports would then serve as the basis for brief summaries and recommendations from the USPSTF.

In other words, while clinical practice recommendations still officially come from the USPSTF and the Department of Health and Human Services, they are actually developed and written by outside centers. We can feel the growing influence of politics and stakeholder interests.

There are now “close relationships” between these academic centers, federal health agencies and professional organizations, which regularly attend Task Force meetings and provide draft review documents, explained Woolf and Atkins. This helps to ensure that “consensus is achieved when the position is supported by evidence.”

For the scientists reading, you’ve no doubt realized that this is now a different USPSTF than you’ve come to know. Consensus among private-public stakeholders is not the work of science. Consensus embodies compromise, after negotiations among various interests, rather conclusions based solely on the most rigorous, objective evidence. Dr. Michael Crichton, M.D., explained it best in his famous 2003 speech at the California Institute of Technology:

I regard consensus science as an extremely pernicious development that ought to be stopped cold in its tracks... Whenever you hear the consensus of scientists agrees on something or other, reach for your wallet, because you're being had. Let's be clear: the work of science has nothing whatever to do with consensus. Consensus is the business of politics... In science consensus is irrelevant. What is relevant is reproducible results...

There is no such thing as consensus science. If it's consensus, it isn't science. If it's science, it isn't consensus. Period.

The AHRQ also established the National Guideline Clearinghouse in 1998, in partnership with the American Medical Association and the American Association of Health Plans (now called America’s Health Insurance Plans, the trade and lobbying organization for health insurance companies). The National Guideline Clearinghouse provides public internet access to help distribute hundreds of clinical practice guidelines, not just those from the USPSTF. It includes clinical guidelines produced under the auspices of medical associations, professional societies, public or private organizations, government agencies, or healthcare organizations or insurance plans. Healthcare professionals must now closely evaluate the source for each of these dissimilar guidelines, even though they are found through the AHRQ.

The Woolf-Atkins paper made its most critical revelations when it described the “final frontier of preventive medicine.” It was referring to implementing the USPSTF clinical recommendations. As the authors wrote:

Practice guidelines are relatively weak tools for changing clinical practice... To effect change, guidelines must be coupled with strategies to improve their acceptance and feasibility. Such strategies include enlisting the support of local opinion leaders, using reminder systems for clinicians and patients, adopting standing orders, and audit and feedback of information to clinicians about their compliance with recommended practice. In the case of preventive services guidelines, implementation needs to go beyond traditional dissemination and promotion efforts to... ensure the delivery of recommended preventive care.

As they explained, factors other than scientific evidence shape public health policies. State legislatures, for example, have mandated screenings that the USPSTF hasn’t recommended. So, they said, a number of efforts have been put into place to “increase the impact of future USPSTF reports.” As they added:

The USPSTF convened representatives from the various audiences for the Guide — clinicians, consumer, and policymakers from health plans, national organizations and Congressional staff — about how to modify the content and format of its products to address their needs. With funding from the Robert Wood Johnson Foundation, the USPSTF and Community Guide Task Force [its Guide to Community Preventive Services was also funded by RWJF] conducted an audience analysis to further explore implementation needs...

National efforts such as the USPSTF face inherent limitations in trying to influence practice in individual physicians'offices. To be successful, approaches for implementing prevention have to be tailored to the local level and deal with the specific barriers at a given site, typically requiring the redesign of systems of care. Such a systems approach to prevention has had notable success in established staff-model health maintenance organizations, by addressing organization of care, emphasizing a philosophy of prevention, and altering the training and incentives for clinicians. Staff-model plans also benefit from integrated information systems that can track the use of needed services and generate automatic reminders aimed at patients and clinicians, some of the most consistently successful interventions. Information systems remain a major challenge for individual clinicians' offices, however, as well as for looser affiliations of practices in network-model managed care and independent practice associations, where data on patient visits, referrals and test results are not always centralized.

The door has been opened for special interests to fund and influence university and outside technical reviews to benefit their interests. The focus is no longer solely on providing healthcare professionals with careful reviews of the evidence. Compliance with those clinical guidelines is being made increasingly obligatory through performance measures and surveillance of electronic medical records. The sources of such funding and potential conflicts of interest, however, don’t have to be disclosed by the researchers when it comes to them through salaries or grants to their various universities and institutions. And they don’t have to disclose funding from nonprofit groups.

Finally, Woolf and Atkins described the inevitable tensions that result when translating the USPSTF recommendations to care provided by doctors in a variety of settings and situations. While one approach might be for the USPSTF to merely describe the quality of supporting evidence for specific situations and leave it up to healthcare professionals to apply appropriately, they said, feedback from stakeholders had clearly indicated they want the USPSTF to state explicit recommendations, that can be enforced.

For preventive care interventions without sufficiently robust evidence for the USPSTF to make evaluate and make conclusions about the benefits versus harms, some want the USPSTF to avoid making recommendations and leave it to clinicians to review the trade-offs with their patients. But, these authors suggested, “the USPSTF has a duty to proffer what that advice [to patients] should be.” Remembering that these guidelines are increasingly being used by healthplans and government agencies no longer as guides, but as mandates for healthcare providers, doctors may find this statement particularly troubling.

Which leads to their final insights. “The Partnership for Prevention convened a panel to develop methods for prioritizing services,” they wrote. It ranked the preventive interventions recommended in 1996 by the second USPSTF. The findings of the Partnership for Prevention, written with Robert Wood Johnson Foundation, Merck & Co., the AHRQ, Health Partners Research Foundation, American College of Preventive Medicine, CDC, etc. were published in the American Journal of Preventive Medicine and were developed to provide a basis for future priority-setting among policy makers to improve the execution of preventive guidelines.

This month has tied all of this together in what seems an almost surreal turn of events. While the ethics of public health has been in the news recently, some of the hardest questions and their answers aren’t at all popular to examine.

Part Two here, looks at a recent example that tests all of us.