Can most online health information be trusted?

In our connected society, it’s easy to overestimate people’s internet savviness. With more than 1.1 billion health websites on the worldwide web, it’s also easy to believe good information is abundant and simple to find. Recent studies have revealed how false those beliefs are and how vulnerable most people are, even healthcare professionals.

While nearly eight in ten people turn to the internet for health information, research has found that most people are putting themselves at risk from inaccurate information. We aren’t nearly as skilled in researching the soundest health information online as we may believe, either. In fact, a just published study found that the most popularly accessed source, Wikipedia, had incomplete information and errors of omission about drugs that could put people in danger of serious adverse reactions and drug interactions.

Looks can be deceiving

It is popularly and mistakenly believed that the reliability of health information online can be judged by the seeming quality of the website. Researchers from St. Mary’s Hospital in London and Maastricht University in Netherlands found, however, that even the most apparently credible websites didn’t necessarily provide higher levels of accurate health information.

The features that have been widely used by professionals as measures of the quality of health websites include the source of medical information, the date of the posting and updates, and hierarchy of evidence on each website (whether the attention given information is related to its validity and methodological quality, to enable visitors to understand the strength of the evidence). These medical researchers evaluated 121 health websites providing information on five common health topics and assessed their entire websites for the three credibility features. Only one quarter of the sites met at least two out of three quality measures and nearly half met less than one.

They then assessed the accuracy of the information provided on each of the health topics (chronic obstructive lung disease, ankle sprain, emergency contraception, abnormally heavy menstrual bleeding, and tubal ligation) using rigorously developed, peer-reviewed and published guidelines. They found that websites that appear to have more credibility are only weakly associated with greater levels of accuracy. “Apparently credible websites may not necessarily provide higher levels of accurate health information,” they concluded. In other words, health information on the internet has the potential of jeopardizing healthcare.

In that same issue of the British Medical Journal, maternal and child medical researchers from Milan, Italy published follow-up findings on their 1997 systematic evaluation of online health information, using the accuracy of information provided for managing fevers in children. Their initial published review, finding that the quality of information online was poor, had been widely mentioned in nearly 80 journals and should have reached a wide audience. Four years later, they repeated their study to see if the quality of those original websites had improved and how 40 newer health sites held up, using the same methods. Nearly half (46%) of those original inaccurate pages were still online, and only one in ten had replaced the content with more accurate information. Conversely, only one in four of the original web pages with the highest quality information were still around. Of the new pages, the quality of the health information had generally improved over the past four years, although only 5% of the new pages met all of the quality guidelines, compared to 8% of the original pages.

It’s imperative for consumers and healthcare professionals to be able to judge the soundness and completeness of the information, rather than trust the information based on perceptions of a website’s credibility. But do they?

A peek at what people really do online

Researchers with the Unit for Cybermedicine and e-Health at the University of Heindelberg, Germany performed a small usability study a few years ago to see how consumers searched the internet for health information and what criteria they used to judge whether a website and its information were trustworthy.

The investigators applied multiple methods used in research of human-computer interaction, video-taped observation and focus groups. Recruiting 21 computer-savvy participants, who ranged in ages from 19 to 71, with an average of about three years experience on the internet, they gave them eight health questions to research the answers for online. The questions were standard questions from the Heidelberg Database of Patient Questions and each participant had 20 minutes per question to find an answer they felt confident about. All started with an empty browser screen. They were to think out loud about their methodology and thought processes. Focus groups evaluated how they assessed websites.

The results revealed a troubling inability among nearly all participants to evaluate the merits of the information itself. Instead, most people used unscientific ways to judge the reliability of information, leaving them vulnerable to slick marketing.

The criteria participants said that they used to judge the merits of a site and its information was if:

● the information sounded professionally written

● there was a picture of the site owner or author and he/she “looked trustworthy” — “If there is a picture of a honestly smiling man, that would be very important.”

● the author had credentials and there was an email address

● the information “sounded scientific” and plausible

● if the information cited scientific references

● the site layout looked professional

● the site was associated with some official authority or was endorsed by a third party — information was considered more reliable if it came from a public institution, science publication, organization or association, or had a seal of approval from a federal agency or any authority.

While participants said they said they viewed the credibility of the source and who stood behind a site and their credentials as important in judging the information, none of them actually checked a source out, even though they were visiting sites unfamiliar to them. Nearly all of the participants trusted what they read and didn’t go beyond the page linked to by the search engine. They didn’t go to the home page of the website or the “about us” page to learn who stood behind the website, the credentials or source of the information, or the disclaimers or disclosure statements. They didn’t pay attention to the origin of the information, and more than a quarter weren’t even aware of the country of the website where they’d found the answers to their questions. In actuality, all sources held equal weight.

Rather than the science, most were influenced by things like the appearance of the design, photos and writing style. There was little to no cross-checking of facts. Nearly all participants went to the first site that came up on a search engine and stopped. Only half bothered with the second site listed by a search engine (like Google or Yahoo), and only one-third went to a third site. Only five even went to the second page of the search engine results, with one participant thinking that the first ten results that were displayed on the search engine results were all that existed.

The search techniques were similarly unsophisticated. Most didn’t use effective search terms, with only one in three using more than a single word, only one participant using Boolean operators, and only half knowing to use quotation marks. And none of the participants used the translation tool to find English-speaking sources, and the none of the English-speaking Germans entered English terms to find non-German sources. It turned out that nurses were no better at searching credible health information online as general consumers.

With frighteningly few consumers and even healthcare professionals critically evaluating the merits of health information or searching out the most credible information, it makes the soundness of the most popular websites vitally important. First up on the hierarchy of results that pop up on most web search engines nowadays is Wikipedia.

Wikiality

“Wikipedia is the source where you can be an authority even if you don’t know what the hell you’re talking about,” said comedian Steve Colbert in a hilarious, yet sadly correct, monologue last year [available here].

Concerns with Wikipedia include the fact that it is written and edited by anonymous authors, many of whom are young students with little understanding of the topics they are writing about, and others are those most invested in the topics, such as special interest groups, paid marketers and industry representatives; yet there are no disclosure statements — the authors are unnamed and their credentials are unknown. Anyone with an agenda can, and does, change an article to advance his/her cause, attempt to discredit or promote someone/something, or just make mischief, as fans of Colbert showed by changing an entry on elephants to incorrectly claim their population in Africa had tripled. Wiki neutrality is a myth.

Colbert created the term Wikilobbying, to illustrate that anyone with money can be paid to make an entry. He joked that he was sure people will start trusting Microsoft again, if Microsoft just pays someone to write an entry in Wikipedia about how people are trusting Microsoft again. “When money controls entries, reality has become a commodity.”

But this isn’t a joke and there are countless cases of vested interests writing and editing Wikipedia entries. Several drug companies, including AstraZeneca and Abbott Laboratories, have been caught deleting important safety information to downplay the risks of their drugs. While the original report has been wiped from the internet, a copy can be found in Pharma Marketing News here. [A later examination of the accuracy and completeness of drug information by pharmacist and pharmacy library professors at Ohio State University found that Wikipedia continues to have inconsistent quality of drug information, with most drugs they examined failing to give a complete list of FDA-approved indications, contraindications and drug and food interactions, increasing risks to consumers who may inappropriately rely on it. A 2016 review by a team of international pharmacology professors published in Annals of Pharmacology examined online, user-edited resources and found Wikipedia was still less complete with more omission errors compared to other sources and should not be used by consumers as an authoritative source.]

Wikipedia’s popularity has grown among a generally younger, less informed audience ready to believe whatever they read on the internet. It’s often claimed to be more democratic than peer-reviewed scientific or medical journals, allowing anyone with a viewpoint to edit and contribute to the final entry. But science isn’t democratic and facts aren’t about consensus. Comedian Steve Colbert also aptly coined the term Wikiality, saying: “When Wikipedia becomes our most trusted reference source, reality just becomes what the majority agrees upon.”

And that is a frightening thought, especially given the general poor scientific literacy among the public found by the National Science Foundation.

Encyclopedias may provide a cursory quick initial reference, but they are most used by elementary school children for class assignments. Wikipedia is just an online encyclopedia, written by who knows who. It is not a credible source of serious research and has been condemned by academics for being shoddy and full of inaccuracies, according to the Chronicle of Higher Education. The founder of Wikipedia, Jimmy Wales, told audiences at a 2006 conference at the University of Pennsylvania that he has no sympathy for college students who get Fs on their papers because they cited Wikipedia. “For God sake, you’re in college; don’t cite the encyclopedia,” he said.

Professors have become increasingly concerned about the increasing numbers of uncritical students citing Wikipedia. As Inside Higher Education reported, growing numbers of academics are taking stronger stands to discourage its use as an appropriate source for citation in papers or other academic work. Noting its lack of accuracy and completeness and the number of students citing the same incorrect information from Wikipedia, professor Don Wyatt, chairman of the history department at Middlebury College, said the department took a stand against the use of Wikipedia as a citation. “As educators, we are in the business of reducing the dissemination of misinformation,” he said, and students need to be taught to go for quality information, not convenience.

Dr. Alan Liu, an English professor at the University of California, Santa Barbara, adopted a similar policy after finding papers citing Wikipedia by students “with no awareness that there was any need to read a primary work or even a critical work.” Even if Wikipedia is considered by some to be better than other encyclopedias, it is still an encyclopedia and at the university level, an encyclopedia is not a credible source of research.

College students should be using more advanced sources of information and going to original sources, agreed Roy Rosenzweig, director of the Center for History and New Media at George Mason University, who had conducted an analysis of the accuracy of Wikipedia for The Journal of American History. The goal of educators is to better equip students with critical thinking skills to better judge the quality of information they read.

Incomplete and inaccurate information on drugs

Because of Wikipedia’s popularity, a study just published in The Annals of Pharmacotherapy is especially critical for consumers and healthcare professionals. This study was conducted by an international group of researchers, led by Kevin A. Clauson, PharmD, associate pharmacy professor from the College of Pharmacy-West Palm Beach at Nova Southeastern University.

More than a third of people who go online for health information are looking for information on prescription and over-the-counter drugs, they said. But turning to Wikipedia for drug information is not limited to the general public:

Alarmingly, its content is being cited as an authoritative source more frequently by students in the healthcare professions. Wikipedia links have also recently been included in the abstract of an article in the International Journal of Cardiology and thus similarly appear in MEDLINE.

The use of Wikipedia by healthcare professionals is disturbing because anyone with expertise on virtually any medical topic finds Wikipedia entries routinely filled with wrong and incomplete information, as this author has, and these pharmacology experts found that to be the case with drug information, too.

To evaluate the quality and completeness of drug information on Wikipedia, these authors took eight essential categories of drug information recognized as clinically important for patient safety and care: administration, adverse drug events, contraindications, dosage, drug interactions, indications, mechanism of action, and use in pregnancy and lactation. They included 80 question and answer pairs based on commonly used and potentially dangerous drugs in 8 categories. Evaluating this information on Wikipedia required a database to compare it to and regardless of which one is chosen, it, too, then becomes questioned. They chose Medscape Drug Reference (which includes content from the FDA, First DataBank and AHFS Drug Information^® published by the American Society of Health-System Pharmacists) as it met their qualifications for authoritative sources and traditional peer-review. Their findings for the safety of patients were never the less worrisome.

They found that Wikipedia entries mentioned half the drug information compared to the MDR. Wikipedia was significantly less complete (76% versus 95.5%). They found several errors, one regarding use for pregnant and nursing mothers, due to conflicted information and outdated information.

Wikipedia had nearly four times the errors of omission — leaving out information that would most directly impact patient safety, including contraindications, use in pregnancy and nursing, drug interactions and adverse drug reactions. The risks of errors of omission are often ignored, but can be as dangerous as blatant factual errors.

The overall completeness of Wikipedia pages also didn’t improve over 90 days, with prior page versions scoring 76.5% for completeness and current versions scoring 76%.

They said it will be especially important to have more informed and health-literate consumers over the coming years, with the increasing numbers of prescriptions are being written and the growth of online medication commerce.

The appeal of user-edited sites like Wikipedia is more due to their social networking aspects, they said. “They are characterized as self-correcting via a phenomenon called the “wisdom of the masses” and may be perceived to be edited by altruistic ‘peers’ who improve the content with each edit.” But this trust is misplaced, especially given the enormous commercial considerations that can drive content and increase the risk for abuses, they cautioned.

“An important drawback of user-edited sites is that free and anonymous editing potentially allows amateur or erroneous entries, fraudulent material, or conflict of interest (whether by inclusion or omission) to go unnoticed or unchallenged,” they said. This can impose serious risks for consumers turning to these type of sites for health or pharmaceutical information.

Today’s Web 2.0 environment requires that consumers learn a new set of skills and behaviors that include media literacy, health literacy, and ethical behavior online. Consumer search and appraisal skills are highly variable, but most adult online health consumers are not passive recipients of health information. They shop around, compare, and consider what peers are saying; in addition, many use information from the Web as a starting point for more information. There may be particular risk for persons, including youth, who are first using Wikipedia to obtain information on drugs and then ordering these drugs on ecommerce sites. Youth behavior in this new online context requires further research.

They concluded that Wikipedia is not authoritative and should not be the sole source of drug information. “Consumers relying on incomplete entries for drug information risk being ill-informed with respect to important safety features such as adverse drug events, contraindications, drug interactions, and use in pregnancy. These errors of omission may prove to be a substantial and largely hidden danger.”

Please be careful out there. Do your homework, go to original sources, and don’t believe what you read on the internet, especially from anonymous sources or those trying to sell you something.

© 2008 Sandy Szwarc

Drug information
Information on FDA-approved drugs in the U.S. is available from the FDA here.
Scientific analyses of the safety and effectiveness of drugs approved in Canada are available from Therapeutics Intitiative here.