By MATTHEW HOLT

A decent amount of time in recent weeks has been spent hashing out the conflict over data. Who can access it? Who can use it for what? What do the new AI tools and analytics capabilities allow us to do? Of course the idea is that this is all about using data to improve patient care. Anyone who is anybody, from John Halamka at the Mayo Clinic down to the two guys with a dog in a garage building clinical workflows on ChatGPT, thinks they can improve the patient experience and improve outcomes at lower cost using AI.

But if we look at the recent changes to patient care, especially those brought on by digital health companies founded over the past decade and a half, the answer isn’t so clear. Several of those companies, whether they are trying to reinvent primary care (Oak, Iora, One Medical) or change the nature of diabetes care (Livongo, Vida, Virta et al) have now had decent numbers of users, and their impact is starting to be assessed. 

There’s becoming a cottage industry of organizations looking at these interventions. Of course the companies concerned have their own studies, In some cases, several years worth. Their  logic always goes something like “XY% of patients used our solution, most of them like it, and after they use it hospital admissions and ER visits go down, and clinical metrics get better”. But organizations like the Validation Institute, ICER, RAND and more recently the Peterson Health Technology Institute, have declared themselves neutral arbiters, and started conducting studies or meta-analyses of their own. (FD: I was for a brief period on the advisory board of the Validation Institute). In general the answers are that digital health solutions ain’t all they’re cracked up to be.

There is of course a longer history here. Since the 1970s policy wonks have been trying to figure out if new technologies in health care were cost effective. The discipline is called health technology assessment and even has its own journal and society, at a meeting of which in 1996 I gave a keynote about the impact of the internet on health care. I finished my talk by telling them that the internet would have little impact on health care and was mostly used for downloading clips of color videos and that I was going to show them one. I think the audience was relieved when I pulled up a video of Alan Shearer scoring for England against the Netherlands in Euro 96 rather than certain other videos the Internet was used for then (and now)!

But the point is that, particularly in the US, assessment of the cost effectiveness of new tech in health care has been a sideline. So much so that when the Congressional Office of Technology Assessment was closed by Gingrich’s Republicans in 1995, barely anyone noticed. In general, we’ve done clinical trials that were supposed to show if drugs worked, but we have never really  bothered figuring out if they worked any better than drugs we already had, or if they were worth the vast increase in costs that tended to come with them. That doesn’t seem to be stopping Ozempic making Denmark rich.

Likewise, new surgical procedures get introduced and trialed long before anyone figures out if systematically we should be doing them or not. My favorite tale here is of general surgeon Eddie Jo Riddick who discovered some French surgeons doing laparoscopic gallbladder removal in the 1980s, and imported it to the US. He traveled around the country charging a pretty penny to  teach other surgeons how to do it (and how to bill more for it than the standard open surgery technique). It’s not like there was some big NIH funded study behind this. Instead an entrepreneurial surgeon changed an entire very common procedure in under five years. The end of the story was that Riddick made so much money teaching surgeons how to do the “lap chole” that he retired and became a country & western singer.

Similarly in his very entertaining video, Eric Bricker points out that we do more than double the amount of imaging than is common in European countries. Back in 2008 Shannon Brownlee spent a good bit of her great book Overtreated explaining how the rate of imaging skyrocketed while there was no improvement in our diagnosis or outcomes rates. Shannon by the way declared defeat and also got out of health care, although she’s a potter not a country singer.

You can look at virtually any aspect of health care and find ineffective uses of technology that don’t appear to be cost effective, and yet they are widespread and paid for.

So why are the knives out for digital health specifically?

And they are out. ICER helped kill the digital therapeutics movement by declaring several solutions for opiod use disorder ineffective, and letting several health plans use that as an excuse to not pay for them. Now Peterson, which is using a framework from ICER, has basically said the same thing about diabetes solutions and is moving on to MSK, with presumably more categories to be debunked on deck.