How to read articles about health and health care

If you’ve just read a health-related headline that’s caused you to spit out your morning coffee (“Coffee causes cancer” usually does the trick) it’s always best to follow the Blitz slogan: “Keep Calm and Carry On.” 

On reading further you’ll often find the headline has left out something important, like “Injecting five rats with really highly concentrated coffee solution caused some changes in cells that might lead to tumors eventually. (Not to mention that the study was funded by The Association of Tea Marketing)The most important rule to remember: “Don’t automatically believe the headline.” It is there to draw you into buying the paper and reading the story.  Would you read an article called “Coffee pretty unlikely to cause cancer, but you never know?” Probably not. Before spraying your newspaper with coffee in the future, you need to interrogate the article to see what it says about the research it is reporting on.  Bazian (the company I work for) has interrogated hundreds of articles for "Behind The Headlines" on the NHS health news Web site, and we’ve developed the following questions to help you figure out which articles you’re going to believe, and which you’re not.

Does the article support its claims with scientific research?

If an article touts a treatment or a lifestyle factor that is supposed to prevent or cause a disease, but doesn’t give any information about the scientific research behind it, or refers to research that has yet to be published, then treat it with caution.  A lot of caution, like balling the article up and throwing it in the (recycling) bin.Is the article based on a conference abstract?

Another area for caution: news articles based on conference abstracts.  Research presented at conferences is often at a preliminary stage and usually hasn’t been scrutinised by experts in the field.  Also conference abstracts rarely provide full details about methods, making it difficult to judge how well the research was conducted.  For these reasons, articles based on conference abstracts should be no cause for alarm.  Don’t panic or rush off to your GP. Was the research in humans?

Quite often the “miracle cure” in the headline turns out to have only been tested on cells in the laboratory or on animals.  These stories are often accompanied by pictures of humans, creating the illusion that the “miracle cure” came from human studies.  Studies in cells and animals are crucial first steps and should not be undervalued.  However, many drugs that show promising results in cells in laboratories don’t work in animals, and many drugs that show promising results in animals don’t work in humans.  If you read a headline about a drug or food “curing” rats, there is a chance it might cure humans in the future, but unfortunately a larger chance that it won’t.  So no need to start eating large amounts of the “wonder food” featured in the article.How many people did the research study include?

In general, the larger a study the more you can trust its results.  Small studies may miss important differences because they lack statistical “power”, and small studies are more susceptible to finding things (including things that are wrong) purely by chance.  You can visualise this by thinking about tossing a coin.  We know that if we toss a coin the chance of getting a head is the same as that of getting a tail – 50/50.  However, if we didn’t know this and we tossed a coin four times and got three heads and one tail, we might conclude that getting heads was more likely than tails.  But this chance finding would be wrong.  If we tossed the coin 500 times – gave the experiment more “power” – we’d be much more likely to get an even number of heads and tails, giving us a better idea of the true odds.  When it comes to sample sizes, bigger is usually better.  So when you see a study conducted in a handful of people, proceed with caution.Did the study have a control group?

There are many different types of studies, and they are appropriate for answering different types of questions.  If the question being asked is about whether a treatment or exposure has an effect or not, then the study needs to have a control group.  A control group allows the researchers to compare what happens to people who have the treatment/exposure with what happens to people who don’t.  If the study doesn’t have a control group, then it’s difficult to attribute results to the treatment or exposure with any level of certainty.

Also, it’s important that the control group is as similar to the treated/exposed group as possible. The best way to achieve this is to randomly assign some people to be in the treated/exposed group and some people to be in the control group.  This is what happens in a randomised controlled trial (RCT) which is why they are considered the “gold standard” way of testing the effects of treatments and exposures.  So when reading about a drug, food or treatment that is supposed to have an effect, you want to look for evidence of a control group, and ideally evidence that the study was an RCT.  Without either, retain some healthy scepticism.Did the study actually assess what’s in the headline?

This one is a bit tricky to explain without going into a lot of detail about “proxy outcomes”.  To avoid doing that, here is the key thought: the research study needs to have examined what is being talked about in the headline and article.  (Somewhat alarmingly, this isn’t always the case.)  For example, you might read a headline that claims “Tomatoes reduce the risk of heart attacks”.  What you need to look for is evidence that the study actually looked at heart attacks.  You might instead see that the study found that tomatoes reduce blood pressure.  This means that someone has extrapolated that tomatoes must also impact heart attacks, as high blood pressure is a risk factor for heart attacks.  Sometimes these extrapolations will prove to be true, but other times they won’t.  So if a news story is focusing on a health outcome that was not examined by the research, treat it with a grain of salt.Who paid for and conducted the study?

This is a somewhat cynical point, but one that’s worth making.  The majority of trials today are funded by manufacturers of the product being tested – be it a drug, vitamin cream or foodstuff.  This means they have a vested interest in the results of the trial  which can affect what the researchers find and report in all sorts of conscious and unconscious ways.  This is not to say that all manufacturer-sponsored trials are unreliable.  Many are very good.  But it’s worth looking to see who funded the study to sniff out a potential conflict of interest for yourself.How can I find out more?

It’s not possible to cover all the questions that need to be asked about research studies in a short article, but we’ve covered some of the major ones.  For more, go to Behind the Headlines at www.nhs.uk/news for daily breakdowns of health care stories in the media.

See also: THCB review of health care journalist Robert Davis' book, "The Healthy Skeptic."

Dr. Alicia White is Bazian's Health Research Reviewer Manager. She received her doctorate from the University of Manchester and worked as a Postdoctoral Research Fellow in the Department of Pediatrics and Child Health at University College London before joining Bazian.

Categories: Uncategorized

Tagged as:

17 replies »

  1. Hi. I am Linda Smith, a 30 years old women suffered from a painful disease of mouth named Burning Mouth Syndrome. It is a condition that causes pain in the mouth, lips or tongue without any clear reason. BMS afflicts seven times more women than men, affecting between 1.5% and 5.5% of middle-aged to elderly women. I have lost a lot of weight because I am unable to eat anything. Nothing seems to have any taste. Then I visited a site http://burningmouthsyndrome.boomja.com/ which contained detailed description of every aspect of BMS. All my queries were solved. It helped me to take a better care of myself and recover faster.

  2. Maggie Mahar has written about our HealthNewsReview.org
    project on this site. https://thehealthcareblog.com/the_health_care_blog/2008/06/how-the-mainstr.html
    So has Sarah Arnquist: https://thehealthcareblog.com/the_health_care_blog/2008/06/hyping-health-s.html
    I’ve also been afforded some space to write about the project: https://thehealthcareblog.com/the_health_care_blog/2007/07/healthnewsrevie.html
    We appreciate this site’s attention to these issues about the quality of health journalism. We have now reviewed more than 700 stories in three years and this dynamic database provides the industry a very clear data-driven picture.
    Gary Schwitzer
    Publisher, HealthNewsReview.org
    Publisher, Schwitzer Health News Blog

    Associate Professor
    University of Minnesota School of Journalism & Mass Communication

  3. Terrific piece!! At the number of talented, experienced health and science journalists at newspapers and magazine declines care of the media transformation underway, the tips are even more important to help readers, patients, consumers, all better digest the substance (or lack there of) behind the prose.

  4. I recently came across your blog and have been reading along. I thought I would leave my first comment. I don’t know what to say except that I have enjoyed reading. Nice blog. I will keep visiting this blog very often. Tammy M.

  5. Just to clarify, I’m actually not an MD doctor, but a PhD doctor. (My parents will be pleased though – they never even had to pay tuition fees!)

  6. There are definitely a lot of people out there with only a headline/soundbite amount of depth of knowledge of the issues plaguing our healthcare system. I must admit, I’m worried that these people will have influence in reform efforts.

  7. The post is spot on and the blitz slogan is sound advice.
    There’s also a great resource that I would suggest checking out:
    HealthNewsReview.org is a website dedicated to:
    • Improving the accuracy of news stories about medical treatments, tests, products and procedures.
    • Helping consumers evaluate the evidence for and against new ideas in health care.
    Health News Review supports and encourages the ABCs of health journalism.
    • Accuracy
    • Balance
    • Completeness

  8. Good article, and I’m a fan of “Behind the headlines”, but just one point. It’s not true that if you toss a coin 500 times you’re more likely to get “an even number of heads and tails” than if you toss it 4 times. In fact, 250 heads and 250 tails out of 500 tosses is a very improbable result.
    500 tosses is more likely to give a result that’s close to even, and enables you to say that the coin is probably unbiassed with a higher degree of confidence.

  9. Over the last 25 years, clinical research has been largely privatized. The overwhelming majority of clinical studies published in the three most respected medical journals (NEJM, JAMA and the Lancet) are commercially funded. As a result, our medical knowledge grows not in the direction that best improves our health but toward corporate profits, the way that plants grow toward sunlight.
    Before 1980, most medical studies were publicly funded, and most academic researchers scorned industry support. Now, the vast majority of clinical trials are commercially funded, and with the financial stakes so high, there is so much mounting evidence of individual scientists and corporations manipulating their findings.
    Even our most trusted journals are dependent on drug-company money. Drug makers don’t just buy advertising in their pages. According to Richard Horton, editor of the Lancet, they also pay up to $1.75 million for reprints of articles favorable to their drugs, which sales reps then hand out to doctors.
    And many journal articles are biased in favor of their sponsors’ products. So many times, the type of sponsorship that is available for randomized controlled trials is strongly linked to the results and conclusions of those studies, even when other factors are taken into account.
    A 2003 report in the Journal of the American Medical Association found that clinical studies funded by drug companies are three times more likely to conclude that the sponsor’s drug is the treatment of choice, compared to studies of the same drug that were not commercially funded.
    Sometimes drug companies may deliberately choose lower dosages for the comparision drug when they carry out head-to-head trials, making the company’s product do better in the trial. And sometimes trials produce unfavorable results that are not published, or that unfavorable outcomes are suppressed.
    The disturbing conclusion is that most of the evidence in what doctors believe to be “evidence-based medicine” is more infomercial than dispassionate science.
    Real-world studies are not being performed under real-world conditions. No one is publishing real-word studies, except private laboratories performing cell-based analysis, which can only do real-world studies, because their studies require fresh, viable human specimen, which must be accessioned and tested in real-time under real-world conditions.
    Patient outcomes need to be reported in real-time, so patients and physicians can learn immediately if and how patients are benefiting from new diagnostics and therapies.

  10. Excellent post for the poor laymen (and sometimes doctors too!) reading these articles. Now THIS needs to be published in the newspapers!