statistics – The Health Care Blog

Statistical Certainty: Less is More

Dec 26, 2018• 3

By ANISH KOKA MD

The day after NBC releases a story on a ‘ground-breaking’ observational study demonstrating caramel macchiatas reduce the risk of death, everyone expects physicians to be experts on the subject. The truth is that most of us hope John Mandrola has written a smart blog on the topic so we know intelligent things to tell patients and family members.

A minority of physicians actually read the original study, and of those who read the study, even fewer have any real idea of the statistical ingredients used to make the study. Imagine not knowing whether the sausage you just ate contained rat droppings. At least there is some hope the tongue may provide some objective measure of the horror within.

Data that emerges from statistical black boxes typically have no neutral arbiter of truth. The process is designed to reveal from complex data sets, that which cannot be readily seen. The crisis created is self-evident: With no objective way of recognizing reality, it is entirely possible and inevitable for illusions to proliferate.

A Guide to Making Machine Learning Work in Health Care

Sep 19, 2018• 4

Apologies on the hiatus for posting on THCB. As many of you know, I was running around getting Health 2.0 in order this past weekend. Today we are featuring a piece on understanding how machine learning can actually work in health care today-Matthew Holt

By LEONARD D’ AVOLIO, PhD

There’s plenty of coverage on what machine learning may do for healthcare and when. Painfully little has been written for non-technical healthcare leaders whose job it is to successfully execute in the real world with real returns. It’s time to address that gap for two reasons.

First, if you are responsible for improving care, operations, and/or the bottom line in a value-based environment, you will soon be forced to make decisions related to machine learning. Second, the way this stuff actually works is incredibly inconsistent with the way it’s being sold and the way we’re used to using data/information technology in healthcare.

I’ve been fortunate to have spent the past dozen years designing machine learning-powered solutions for healthcare across hundreds of academic medical centers, international public health projects, and health plans as a researcher, consultant, director, and CEO. Here’s a list of what I wish I had known years ago.

Google Flu Trends Shows Good Data > Big Data

Mar 26, 2014• 3

By Kaiser Fung

In their best-selling 2013 book Big Data: A Revolution That Will Transform How We Live, Work and Think, authors Viktor Mayer-Schönberger and Kenneth Cukier selected Google Flu Trends (GFT) as the lede of chapter one.

They explained how Google’s algorithm mined five years of web logs, containing hundreds of billions of searches, and created a predictive model utilizing 45 search terms that “proved to be a more useful and timely indicator [of flu] than government statistics with their natural reporting lags.”

Unfortunately, no. The first sign of trouble emerged in 2009, shortly after GFT launched, when it completely missed the swine flu pandemic. Last year, Nature reported that Flu Trends overestimated by 50% the peak Christmas season flu of 2012. Last week came the most damning evaluation yet.

In Science, a team of Harvard-affiliated researchers published their findings that GFT has over-estimated the prevalence of flu for 100 out of the last 108 weeks; it’s been wrong since August 2011.

The Science article further points out that a simplistic forecasting model—a model as basic as one that predicts the temperature by looking at recent-past temperatures—would have forecasted flu better than GFT.

In short, you wouldn’t have needed big data at all to do better than Google Flu Trends. Ouch.

In fact, GFT’s poor track record is hardly a secret to big data and GFT followers like me, and it points to a little bit of a big problem in the big data business that many of us have been discussing: Data validity is being consistently overstated.

As the Harvard researchers warn: “The core challenge is that most big data that have received popular attention are not the output of instruments designed to produce valid and reliable data amenable for scientific analysis.”

The amount of data still tends to dominate discussion of big data’s value. But more data in itself does not lead to better analysis, as amply demonstrated with Flu Trends. Large datasets don’t guarantee valid datasets. That’s a bad assumption, but one that’s used all the time to justify the use of and results from big data projects.

Statistics – Using the Truth to Mislead

May 12, 2010• 14

By DR. PULLEN

My daughter is an accountant. She took a statistics class in high school, and another as a requirement for her major. My son has taken a statistics course, and he is an English Literature major. I was a chemistry major in college and have an an MD and have never taken a statistics course. I don’t even recall a lecture on statistics in medical school. Mark Twain quoted Disraili as saying, “There are three kinds of lies: lies, damned lies and statistics.” Reading medical journal articles reporting on the benefits and lack of benefits when reported statistically can be really challenging. Reading a report of these, or worse listening to an interested party, like a sales rep or sponsored speaker talk about a study, requires being a skeptic. Here are some examples of how true statistics can be worse than a lie, and how what would seem to be common sense does not pay off.