Beyond Wikipedia

No surprise, these days more and more doctors are searching online for medical information. What is surprising, however, is that in a recent study, nearly 50% of physicians indicated that they use Wikipedia—the open-access encyclopedia that allows anyone to edit articles—as their source for medical information.

The study, conducted by Manhattan Research, and reported on here found that although physicians were visiting Wikipedia for medical conditions and other health information, only about 10% of the 1,900 physicians surveyed created new posts or edited existing posts on the encyclopedia.

“The number of physicians turning to Wikipedia for medical information has doubled in the past year alone,’ said Meredith Abreu Ressi, vice president of research at Manhattan Research. ‘Physicians, just like consumers, are heavily search engine reliant, and often Wikipedia results are what come up in the top of the organic results.’

Abreu Ressi noted the concern about accuracy regarding Wikipedia, which allows its users to create content for the site essentially without restriction. Articles are subject to perpetual editing by Wikipedia’s readers. Inevitably, false information sometimes slips through the cracks.”

Wikipedia is not a reliable source of medical information for doctors.

The fact that anyone can anonymously edit the content is the most obvious problem with the concept. For example, this article by Peter Rost, MD in BrandweekNRX provides details of how representatives from the pharmaceutical companies  Abbott and AstraZeneca removed negative information about the side effects of  their drugs from the site:

“The first drug company caught messing with the Wikipedia was AstraZeneca. References to claims that Seroquel allegedly made teenagers ‘more likely to think about harming or killing themselves’ were deleted by a user of a computer registered to the drug company.”

“[I]n July of 2007, a computer at Abbott Laboratories’ Chicago office was used to delete a reference to a Mayo Clinic study that revealed that patients taking the arthritis drug Humira faced triple the risk of developing certain kinds of cancers and twice the risk of developing serious infections. The study was published in the Journal of the American Medical Association in 2006.”

The irony of this situation is that doctors don’t have to resort to Wikipedia. There is a surfeit of more authoritative medical information that can be accessed electronically. Indeed there are scads of peer-reviewed  medical journals available online, as well as searchable treatment guidelines from the U.S. Preventive Services Task Force, the Agency for Health Care Research and Quality, and professional societies like the American College of Physicians, among other organizations. Private insurers publish their own treatment guidelines that they encourage network providers to use, and in the case of organizations like Kaiser Permanente and the Mayo Clinic, outside providers can gain access as well. But sifting through this overload takes time– something most doctors don’t have a lot of.

Thus, the appeal of Wikipedia: Aside from the sponsored sites that are bankrolled by drug companies, Wikipedia is often the first reference that comes up when doctors search the Web for information about a particular disease or condition. (Most search engines list the  most popular website first, and Wikipedia draws millions of eyeballs.) But doctors should be concerned about accuracy. And keep in mind that Wikipedia, like many of the medical sites out there, ultimately offers no guidance on which treatments are best for patients. It’s a barely-comprehensive overview.

I believe we would all be safer if a government agency consolidated all of the really good, independent research into one giant, searchable database that doctors could access easily. For this database to be truly valuable, it must be continually updated to include comparative-effectiveness studies that can guide practitioners in making informed decisions about care. Obama’s economic stimulus plan includes $1.1 billion for comparative effectiveness research—and a 15-member panel of government health experts was created to decide who will conduct these studies and how the new research will be applied.

For those quick to shout “rationing!,” the House conferees specifically stated that they did not intend for the comparative effectiveness research funding to be used to create mandates for coverage, reimbursement, or other policies for any public or private payer. It’s meant to be a resource for practitioners, offering “guidelines,” not “rules.”

Better Websites

When we will actually see such a database is hard to predict. There is still internal wrangling among Democrats about who should oversee comparative-effectiveness studies—and downright opposition to the idea from some conservatives. This week members of the New Democrat Coalition proposed legislation that would establish an “independent,” non-governmental office to oversee comparative effectiveness studies instead of the government panel.

Partially financed with economic stimulus money, this office would be overseen by a 21-member board that would include HHS officials, patients, physicians and private insurers, among others. So far, the industry-heavy group, Partnership to Improve Patient Care and drug-maker Merck have indicated that they support this legislation.

In the meantime, there are Web-based resources already out there that are far superior to Wikipedia for practitioners—and even motivated patients—to access. Kevin Pho of KevinMD says that doctors already have a good source of comparative effectiveness research in UptoDate:

“Over 360,000 clinicians, thousands of patients and the majority of academic medical centers in the U.S. consult UpToDate for answers to their clinical questions. For those who don’t know, UptoDate is a peer-reviewed, evidence-based, medical encyclopedia available via DVD or online that’s revised every 3 months. It does not carry advertisements, and is funded entirely via paid subscriptions. I am a big proponent, and like many other doctors, could not practice medicine effectively without it by my side.”

Medpedia, a free, health and medicine site that is modeled on Wikipedia—with the important difference that it  limits who can write and edit site information—came on line earlier this year. The goal of the site is to create and interlink content on about 13,000 drugs and 30,000 medical conditions. Medpedia’s editors must be invited to participate and have to be physicians or PhD’s. They are also required to disclose any conflicts of interest they have; i.e. whether they accept money from a drug company to conduct studies. One aim of Medpedia is to link practitioners and patients into “Communities of Interest,” to keep current on developments in particular disease areas. Whether it will be successful really depends on how much interest the site garners from doctors—and if they have enough time to edit and continually add to the knowledge base.

The fact that so many providers and consumers are going online to get medical information represents a real sea change in how knowledge is accumulated. As is the case with all web content, from parenting advice to politics to pornography; the range of medical information available online varies widely in quality and origin. It takes time to sift through the onslaught and pick out the really useful stuff. It’s encouraging that there are efforts, both public and private, to create up-to-date, meaningful content that can help drive better medical practice and maybe, just maybe, provide an impetus to improve patient care and cut down on unnecessary and costly treatments. Now the challenge is to get doctors to scroll past Wikipedia to find it.

Naomi Freundlich writes for the Century Foundation, where she works with THCB author Maggie Mahar on the HealthBeat project. Prior to joining the Century Foundation, she served as Science and Medicine Editor at Business Week from 1989 – 1997. Her work has appeared in numerous publications, including the New York Times,  Business Week, Real Simple and Parents magazine.

Categories: Uncategorized

Tagged as: ,

23 replies »

  1. I’m late to this conversation, but I’m very surprised to read that Up To Date is considered a good source by this audience in this advanced digital age. Even today, their own site says that their information is at least 4 months old. For broad searches that return many sources

  2. I’m now not positive where you are getting your info, however good topic. I needs to spend a while finding out much more or understanding more. Thanks for wonderful info I used to be searching for this information for my mission.

  3. I must say you made some good details there. I did research online about the issue plus located a number of people that definitely agree with your current Article

  4. I’m late to this conversation, but I’m very surprised to read that Up To Date is considered a good source by this audience in this advanced digital age. Even today, their own site says that their information is at least 4 months old. For broad searches that return many sources, try http://www.hakia.com. To search for the most recent information about specific diseases, symptoms, treatments, etc. published through PubMed, use MedInfoNow.com. Furthermore, MedInfoNow.com will send to you each week via email the latest abstracts and citations from PubMed in your chosen field.

  5. e-patient Dave has a valid criticism of the author’s exaggeration of the study results.
    What was written in this blog does not accurately represent the findings of the original source. Second line of the article: “nearly 50% of physicians indicated that they use Wikipedia—the open-access encyclopedia that allows anyone to edit articles—as their source for medical information”
    This implies that these doctors used wikipedia as the sole source of of medical information. It is not “their source”, it is “a site that they visit”. The author exaggerated the original study.

  6. I think the discussion has become heated here because the first line of the first comment was, indeed, condescending. People respond negatively to condescension. It tends to shut them down and cause them to disregard anything that comes after it.

  7. Many are aware of the embellishments/alterations that may occur with Wikipedia. However, there are references at the end of each subject or topic searched and located. The references are valid, which is why doctors go to wikipedia for information.
    Doctors in general are not naive. They look at the references more than they look at the body of the wikipedia entry.

  8. Please see http://www.wikidoc.org which does move beyond wikipedia. There are more images, expanded differential diagnoses. Over 1,200 contributors have edited the site over 400,000 times. Many pages have editors assigned who can be contacted.

  9. I really can’t see what the fuss is about. So physicians read Wikipedia and edit Wikipedia. Wow! I know several full professors who read (and edit) Wikipedia articles in their respective fields. Wikipedia provides a reasonably easy overview of a subject, and its articles can usually be improved by someone knowledgeable. The idea that professionals shouldn’t use Wikipedia simply because there’s “better” stuff out there is frankly ridiculous. There just aren’t enough hours in the day to read all the original research being published, especially not if you have to do something useful with your day as well. And lets not forget that you have to sort out the grain from the chaff with an original research paper, just as you do with a Wikipedia article. This post is simply intellectual snobbery, coupled with the dangerous fallacy that every patient could (or even should) receive the absolute latest idea in treatment.

  10. OK Dave, so this is what you said:
    “MedPedia reliability was pretty well assassinated in February over on the e-patients.net blog”
    “‘approved by a MedPedia editor’ is crap as a guarantee”
    On the enclosed link to your blog, I saw that e-patients want EQUAL (original caps) access to MedPedia as MDs. I assume that means editing and adding articles. Wouldn’t that turn it into a Wikipedia, which as you say “any fool knows can be edited by any fool”?
    I am NOT implying here that e-patients are fools.
    I have no idea really what e-patients are. We are all patients and many people search the web for clues and advice on their condition and then discuss that with their doctor. This is not new. What is new is the great availability of resources and lots of militant interest in being in control of one’s care and also one’s medical records. I personally think that’s a great idea, if the patient is able and willing.
    Just one question, are you also advocating for corresponding tort reform? After all, it is only fair…. Don’t you think?

  11. Maggie, if your suggestion that “a different name” is a covert way of suggesting I’m sexist, you have no freaking idea what you’re talking about. (And why not come out and say so?)
    Your assessment that I think the drug I got is The Drug has nothing to do with this conversation (plus I know it’s not).
    I pointed to Dr. John Grohol’s dissection of the Medpedia depression entry, not my own.
    Re Naomi’s credentials: did I impugn her credentials? No, I explicitly cited how she reported something that’s not in the piece she cited. Now, if you’re asserting that good credentials are supposed to grant a waiver on reportorial accuracy, then gracious.
    (Actually, that would be consistent with some people’s view that being an MD doesn’t mean you’re subject to fact-checking.)
    Margalit, you say a doctor’s experience “should add some weight to a doctor’s opinion.” Did I ever say it didn’t, or were you talking about someone else?
    Fwiw, I’m not going to respond to any more comments about *me*, only to comments about what I *said* to Naomi. Ad hominem diversions (intentional or unwitting) are not worth my time.

  12. Bravo Maggie!!
    I am all for patient involvement in treatment decisions and I believe most physicians appreciate the patient that can actively contribute to his/her own care. Just like rbar wrote, if one is dealing with a particular disease either for a long time, or if one is highly motivated and capable of research, that particular patient may very well be ahead of the physician on rare occasions and that is a good thing. However, to suggest that the clinical body of knowledge should be vetted by patients is a bit extreme.
    One thing that distinguishes the most motivated and educated patient from an MD, other than many years of medical school, is expertise – practicing the art and science every day, all day in a variety of conditions, with a variety of patients for a couple of decades should add some weight to a doctor’s opinion, I would think.

  13. JD wrote:
    “But what the internet has done (not just Wikipedia) is to make it clear that much of the time when it comes to the basics of a disease–causes, symptoms, treatments–there is nothing mysterious or even very difficult about what doctors know, and it is not outside of the grasp of reasonably intelligent people with a decent education. In retrospect, of course, that is obvious, but there has been a cult of the physician that has done much damage in our society and I do look forward to its unwinding.”
    I among the first to admit that at least in the past, doctors received (and mostly sought) a mythical status of superiority that is not always helpful and rarely, if ever justified. I will also happily admit that smart patients who read up on their problem/condition can engage in meaningful dialogue with the physicians, and at times, depending on the situation/doctor, be ahead of the physician.
    That, however, does not mean that physicians are not, in general, much better qualified to diagnose and treat illness, and not only with respect to the physical exam as you indicate. Even if you have a grammar and a dictionary of a foreign language, you cannot speak the language as well as someone who learned it.
    There are very many patients who incorrectly diagnose themselves, esp. with multifaceted and variable conditions like MS, SLE, Lyme disease (which is a whole other story), by virtue of obsessive googling. In my experience, the number of incorrect self diagnoses vastly outnumbers the hits (I would also venture to say that the number of unhelpful, erratic ideas outnumbers the number of helpful contributions). Not that you shouldn’t try (with caution and skepticism) to gain knowledge via the internet if you want to – but you should’t bank on the results.

  14. e-patient Dave–
    Your comment is a tad . . . condescending.
    I wonder, if the post had been written by someone with a different name, would you have written “Good heavens, Henry, be careful!”
    Somehow I think not.
    Before accusing Naomi of getting the facts wrong, you might want to check out her credentials at the end of hte post. She’s a seasoned jouralist and the post is accurate.
    But accuracy doesn’t seem to be your main concern. Your comment has very little to do with the post itself, which you seem to be using simply as a jumping off point to express your skepticism about websites that are vetted by doctors.
    Specifically, you wanted to call attention to the fact that, in your view, you have “annihilated Medipedia.” The website you cite (e-patients.net blog.) explains that you are a cancer paitent who believes that the drug that you received for your cancer is The Drug, and that oncologists who are not prescribing it simply don’t know as much about
    cancer as you do, after “a year’s research.”
    So, you think we would all be better off if medical websites were authored by a combination of doctors and patients–people like you who could correct the doctors. For may part, I would prefer to get my medical information from people who went to medical school.
    There are some very knowledgable patients out there (especially if they have suffered from the same disase for many, many years) But a site like Widipedia
    doesn’t identify the writers– or verify their credentials.

  15. I firmly believe that medical professionals and medical doctors are very necessary to a cohesive, comprehensive wellness medical model. Doctors provide necessary evaluations, screenings, tests, diagnoses, prescriptions etc. However, when it comes to getting adequate, timely, comprehensive and prescriptive nutritional advice many doctors fall short. Our medical system is devised to create a force of medical professionals adept at identifying conditions and treating these conditions with drugs. Most are not taught to teach patients how to remedy the condition, prevent the condition or follow a protocol of comprehensive (including natural mechanisms) prevention and management. It is a widely accepted fact that most traditionally trained doctors called allopathic medical doctors only receive very little nutritional training in the 12+years of training it takes to be an MD in the USA. As a matter of fact, according to a Senate investigation, they discovered that most doctors receive less than 3 hours of nutritional training in medical school. So it is no wonder why most doctor “pooh pooh” any nutritional approach to improved wellness, disease prevention, disease management, ailment management, psychological and physical dysfunction as a result of the things we are or are not eating. I suggest they consider reading books like http://livesmartbook.com and that new medical models incorporate preventative wellness. If this is the age of change then lets get with the program.

  16. I’ve been a systems analyst, software designer and consultant in the healthcare business since 1982. I also have had the rare privilege of being misdiagnosed and mistreated most of my life. The only reason I am here to write this today is that I solved my problem with the help of others who solved parts of that same problem and put the information on the internet. Over a 7 year period I filtered out the noise and refined the results increasing the predictability of results and identifying the important factors. In doing so I’ve jumped over perhaps 100 years of the usual pace of research. There is plenty of existing research I can cite for each step of the way, each one pointing out one small biochemical or biological piece of the puzzle, but nothing that puts it all together in a therapy protocol. There is also a much larger older body of research on which the existing diagnostic understandings, therapeutic protocols and insurance payments are based which as it turns out are wrong in the way that Newtonian physics understanding doesn’t handle relativistic or quantum events correctly. At the web forum on which this information and the ongoing patient led trials are happening, we get a flow of people some of whom are there because their physicians told them to look on the web for anything that looks promising. No physicians of whom I am aware have posted on that thread but quite a few nurses have. Some folks improve so dramatically, such as me, that it appears near miraculous. Many get strong enough fast enough results that they convince their physicians to take the next step with certain harmless but way off the established protocols prescriptions which insurance will rarely pay for despite very obvious results including substantial savings on other prescriptions.
    The trick in all this is to identify what works and who it applies to. There is a wealth of knowledge of precisely that in the medical records, if organized and analyzed correctly. The problem is so many factions fighting to keep their revenue stream from being diverted. Do you really think that the pharmaceutical companies want to prevent Alzheimer’s if it takes away from their revenues?

  17. Regardless of the current reliability of Wikipedia and exactly what percent of physicians use it, it’s clear that something huge is happening here and I think the medical profession is totally unprepared for what is to come. The democratization on Wikipedia is nothing compared to the impending democratization of medical knowledge.
    I can spend a weekend googling a condition and by the end of it understand its causes, symptoms and the latest word on recommended treatments nearly as well as a general practitioner (in some cases, better). There used to be an aura of esoteric expertise around physicians and to some extent there still is, but it is disappearing and will continue to do so at an accelerating pace. That will change how people interact with their physicians.
    What a physician will still have in spades over me is physical skill in detecting and treating. I don’t have experience listening to thousands of lungs to know what different sorts of problems sound like, for example. I certainly won’t have experience finding veins for injections, or have acquired the dexterity to slice open a human body without mangling it.
    But what the internet has done (not just Wikipedia) is to make it clear that much of the time when it comes to the basics of a disease–causes, symptoms, treatments–there is nothing mysterious or even very difficult about what doctors know, and it is not outside of the grasp of reasonably intelligent people with a decent education. In retrospect, of course, that is obvious, but there has been a cult of the physician that has done much damage in our society and I do look forward to its unwinding.
    And yes, I understand that most people are lazy, and many Americans are poorly educated. But it won’t take everyone looking at this stuff to make a difference in shifting the cultural assumptions.

  18. Well John, you picked one of my three points, twisted it, and then said I went a long way to make a weak argument. When was the last time you took a course in rhetorical analysis?
    I didn’t say doctors don’t use the Internet nor did I say they don’t use Wikipedia. I was expressing concern that the writer’s quoting of her source was imprecise; the article she cited didn’t say what she said. Worse, the way she phrased it could be taken to suggest there’s a real danger from docs reading Wikipedia (which any fool knows can be edited by any fool) as “using it as their source for medical information.”
    My concern is that patients need to scrutinize what they read. It applies to things they read on any blog, things they read on Wikipedia, things they read on Medpedia, and things they read in medical journals. And this post is a good example of how even a simple restating of a simple article can come out wrong.
    Caveat lector, if you will.

  19. John,
    yo are correct … kind of. On this very blog, there was some kind of poll or sampling that was interpreted that only a fraction of doctors use the internet … I already pointed out at that time/entry that I think that the poll was interpreted in a selective manner and that most (if not the overwhelming majority) of docs use the internet.
    Re. wikipedia use, I have seen tremendous progress in my specialty from the early 2000s to today. The quality of an entry is easy to estimate based on style, level of detail and number of references.
    However, the fact that authorship is anonymous makes me still very hesitant to use wikipedia for office use (at least not alone). There are quite reliable ressources such as up to date, mdconsult (both per subscription) and emedicine. It is not a guarantee for accuracy, but it feels good to see a name and affiliation next to an article.
    Doctors voting on articles (let alone patients) might indicate popularity/perceived helpfulness of a post, but does not guarantee accuracy. Someone with a knowledge level below the author of a post can only judge helpfulness, not accuracy … and an article can be felt to be helpful, but still be inaccurate.

  20. To “E patient Dave”
    You say to the writer, “Naomi, be careful”! While you are factually correct in your argument, the writer, Naomi, hardly made a mistake. All doctors use the Internet. Where on earth do you think they DO get their information… by consulting a library??? It is 2009 buddy.
    Again, you are factually correct, but it is HARDLY an argument. And certainly not a criticism of any measure. Who knows, perhaps most of the doctors who look at Wiki get value from it and ALSO consult with respected medical sources on-line. It is my guess that most Docs are not flunky idiots who think they are getting the latest info from Wiki. The Wiki link comes up first on a search, they take a peak and probably move on to a better source.
    You go a long way to make weak criticism.

  21. It depends what information you are using wikipedia to access. I use it occasionally to check anatomy facts which are not available on Up to Date. If I want diagnosis or treatment info, then Up to Date is my favorite source. I also use WebMD to supplement Up to Date for patient information. There are many sources, so it’s a matter of picking the appropriate one.

  22. Wikipedia isn’t going to disappear soon. It will continue to be the top search result in most queiries regardless, and patients and less-thorough physicians will continue to refer to it. Spending fortunes in building proprietary alternatives to Wikipedia may not be the best use of funds.
    For many reasons, it’s in the interest of Wikipedia’s longevity to allow a process of peer review — it reduces the Wikimedia Foundation’s exposure to liability, and it gives it greater claim to public service. The challenge is mostly technical: Wikipedia as an application is entirely democratic and it doesn’t appear to have peer review built in just yet.
    However, the bulk of Wikipedia’s content is maintained by the 1,000 most dedicated registered users. They register themselves as editors on Wikipedia, contribute edits to the content, and those edits go live. If another editor disagrees with the edits, then they will post updates to the article, or roll it back to an earlier revision. If enough people disagree or fail to provide adequate citations, then the article is flagged.
    If the Medical Powers-that-Be want patients to have quality content, then they should manage and contribute to the content. In the world of Free, a little budget can go a very long way. Here are two practical ways to improve Wikipedia:
    1) Contact the Wikimedia Foundation and support/fund an intitiative to add peer review to medical/academic content. Here’s a link to the folks you’d want to contact: http://wikimediafoundation.org/wiki/Staff (By the way, this is how things get done in the open source world)
    2) Out-edit and out-maintain the current folks editing medical content on Wikipedia. it’s a matter of divvying up the content among paid staff, much as you’d do for MedPedia. With tools like Google Alerts and RSS, it’s not difficult to track others’ changes.
    Denouncing Wikipedia for poor content isn’t going to productive, but improving the quality of its content is.

  23. Good heavens, Naomi, be careful!
    First, good to meet you. I don’t know you and I’m sure you mean well, so pardon me, but: Your logical slip is the kind of error we want to avoid when a writer misinterprets what another article said. In fact it’s a perfect example of why patients need to be cautious about what they read in someone’s interpretation of any source.
    Citing a write-up about a study, you said “nearly 50% of physicians indicated that they use Wikipedia … as their source for medical information.”
    What that source http://is.gd/CHY2 actually said is “Nearly 50% of US physicians **[who go] online for professional purposes** are **visiting** Wikipedia for health and medical information.”
    (1) the source said 50% of docs who go on line for professional purposes – not 50% of all docs
    (2) the source said the docs *visit* Wikipedia, and you said they “use Wikipedia as their source”! As if those docs consider Wikipedia authoritative, when the article you cited says nothing of the sort.
    Would you agree that the source doesn’t say what you said? (Again, I don’t know you, so I’m trying to be cautious.)
    And since the original study http://is.gd/CId9 isn’t public, we can’t even assess whether that write-up of the study was accurate. For instance, what was the specific wording of the question about going online?
    Meanwhile, MedPedia reliability was pretty well assassinated in February over on the e-patients.net blog. I raised questions (“Who will vet the vetters?”) and commenters piled on with evidence of MD-vetted info that was just plain wrong. I see that happen every week in the cancer listserv I belong to on ACOR: patients getting out-of-date info from their MDs, even their oncologists.
    I don’t assert for a moment that patients know best, but it’s very clear that “approved by a MedPedia editor” is crap as a guarantee. It’s no assurance at all that the info is current and reliable. For starters, consider this dissection http://is.gd/kLuf of the MedPedia entry on depression. (As far as I can tell, nobody has fixed the errors points out in that comment.)
    Never one to complain (I hope) without proposing something, I suggested a two-channel Amazon-style ratings system, in which patients get to give their thumbs up/down and (separately) professionals get to give theirs.
    As I say, it’s important for patients who read things like this to go back to the original article and confirm that it says what the writer reported. In this case, not so.
    What do you think?