I was at the VIVE conference in Miami last week and caught up with a number of CEOs & execs for some quickbite interviews — around 5 mins getting (I hope) to the gist of what they & their companies are up to. I am going to dribble them out this week.
First two up are Julia Hu, CEO of Lark, a conversational AI program for chronic & behavioral health that works primarily with health plans, and Adnan Iqbal, CEO of Luma Health, a patient messaging system mostly used by providers. — Matthew Holt
The conflict between Ukraine and Russia has been called many things. To most of the world, of course, it’s considered an invasion, a war between the two countries. To Russia, it’s a “peacekeeping” mission. The description that I can’t get out of my head, though, is one that I believe The Washington Postfirst used: it’s the world’s first crypto war.
“There is something about the war in Ukraine that feels different,” a former U.S. intelligence official told Nick Bilton. “We’ve seen wars documented on Twitter and images shared on the internet before, but this time it isn’t just bombs and bullets; this war is digital from the top to the bottom.” And, Mr. Bilton says: “At the center are cryptocurrencies.”
If crypto has come to war, can healthcare be far behind?
Infermedica is a company that started by creating symptom checking and chatbot functionality in Poland back in 2012. It’s spread to delivering that patient-facing diagnosis functionality via API and now as preparation for a physician visit. Today they announce a $30m series B and demo their new product which helps prepare a visit, and integrates into the clinician workflow. I spoke with CEO Piotr Orzechowski and Chief Product Officer Tim Price–Matthew Holt.
We’re already living in an era of unprecedented misinformation/disinformation, as we’ve seen repeatedly with COVID-19 (e.g., hydroxychloroquine, ivermectin, anti-vaxxers), but deepfakes should alert us that we haven’t seen anything yet.
ICYMI, here’s the 60 Minutes story:
The trick behind deepfakes is a type of deep learning called “generative adversarial network” (GAN), which basically means neural networks compete on which can generate the most realistic media (e.g., audio or video). They can be trying to replicate a real person, or creating entirely fictitious people. The more they iterate, the most realistic the output gets.
Like many industries that serve consumers, healthcare has long been envious of Disney’s success with customer experience. Disney even offers the Disney Institute to train others in their expertise with it. Disney claims its advantage is: “Where others let things happen, we’re consistently intentional in our actions.” That means focusing on “the details that other organizations may often undermanage—or ignore.”
You’d have to admit that healthcare ignores too many of the details, allowing things to happen that shouldn’t.
One of the things that Disney has long included in its parks’ experience were robots. It has had robots in its parks since the early 1960’s, when it introduced “audio-animatronics” – mechanical figures that could move, talk, or sing in very life-like ways. Disney has continued to iterate its robots, but, as Mr. Barnes points out, in a world of video games, CGI, VR/AR, and, for heaven’s sake, Atlas robots doing flips, its lineup was growing dated.
Mr. Barnes quotes Josh D’Amaro, chairman of Disney Parks, Experiences and Products, from an April presentation: “We think a lot about relevancy. We have an obligation to our fans, to our guests, to continue to evolve, to continue to create experiences that look new and different and pull them in. To make sure the experience is fresh and relevant.”
Enter Project Kiwi.
In April, Scott LaValley, the lead engineer on the project, told TechCrunch’s Matthew Panzarino: “Project KIWI started about three years ago to figure out how we can bring our smaller characters to life at their actual scale in authentic ways.” The prototype is Marvel’s character Groot, featured in comic books and the Guardians of the Galaxy movies (he is famous for only saying “I am Groot,” although apparently different intonations result in an entire language).
I recently interviewed Subha Airan-Javia, the CEO of CareAlign. CareAlign is a small company that is working to fix the clinician workflow by creating a tool for all those interstitial gaps that the big EMRs leave, and now get moved to and from paper by the care team. In this interview she tells me a little about the company and shows how the product works. I found it very impressive
Tens of millions of Americans rely on consumer experience apps to help them find the best new restaurant or the right hairdresser. But while relying on customer opinion might make sense for figuring out where to get dinner tonight, when it comes to picking which doctor is best for you, AI might be more trustworthy than the wisdom of the crowd.
Consumer apps provide us with rich data categories that often take into account preferences, from location to free wi-fi, to help users narrow down choices. Navigating your health insurer’s network of physicians is a different proposition, and some of the popular ranking systems reportedly have significant limitations. Doctors are often categorized by specialty, insurance, hospital, or location, which may be effective for logistics, but fail to take into account a patient’s unique health conditions and say very little about what an individual patient can expect in terms of health outcomes. Research from my company Health at Scale shows that 83% of Medicare patients seeking cardiology care and 88% of cases seeking orthopedic care may not be choosing providers that are highly rated for best predicted outcomes based on each patient’s individual health conditions.
Deep personalization is exactly what physicians, health systems, and insurers need to offer patients to improve outcomes and lower costs across the board. A study using our data recently published in the Journal of Medical Internet Research sought to quantify how consumer, quality and volume metrics may be associated with outcomes. Researchers analyzed data from 4,192 Medicare fee-for-service beneficiaries undergoing elective hip replacements between 2013-2018 in the greater Chicago area, comparing post-procedure hospitalization rate, emergency department visits, and total costs of care at hospitals ranked highly by popular consumer ratings systems and CMS star ratings as well as those ranked highly by a machine intelligence algorithm for personalized provider navigation.
Anyone who has read my blog or tweets before has probably seen that I have issues with some of the common methods used to analyse the performance of medical machine learning models. In particular, the most commonly reported metrics we use (sensitivity, specificity, F1, accuracy and so on) all systematically underestimate human performance in head to head comparisons against AI models.
This makes AI look better than it is, and may be partially responsible for the “implementation gap” that everyone is so concerned about.
Disclaimer: not peer reviewed, content subject to change
A (con)vexing problem
When we compare machine learning models to humans, we have a bit of a problem. Which humans?
In medical tasks, we typically take the doctor who currently does the task (for example, a radiologist identifying cancer on a CT scan) as proxy for the standard of clinical practice. But doctors aren’t a monolithic group who all give the same answers. Inter-reader variability typically ranges from 15% to 50%, depending on the task. Thus, we usually take as many doctors as we can find and then try to summarise their performance (this is called a multi-reader multicase study, MRMC for short).
Since the metrics we care most about in medicine are sensitivity and specificity, many papers have reported the averages of these values. In fact, a recent systematic review showed that over 70% of medical AI studies that compared humans to AI models reported these values. This makes a lot of sense. We want to know how the average doctor performs at the task, so the average performance on these metrics should be great, right?
In a recent podcast about the future of telehealth, Lyle Berkowitz, MD, a technology consultant, entrepreneur, and professor at Northwestern University’s Feinberg School of Medicine, confidently predicted that, because of telehealth and clinical automation, “In 10-20 years, we won’t need primary care physicians [for routine care]. The remaining PCPs will specialize in caring for complicated patients. Other than that, if people need care, they’ll go to NPs or PAs or receive automated care with the help of AI.”
Berkowitz isn’t the first to make this kind of prediction. Back in 2013, when mobile health was just starting to take hold, a trio of experts from the Scripps Translational Science Institute—Eric Topol, MD, Steven R. Steinhubl, MD, and Evan D. Muse, MD—wrote a JAMA Commentary arguing that, because of mHealth, physicians would eventually see patients far less often for minor acute problems and follow-up visits than they did then.
Many acute conditions diagnosed and treated in ambulatory care offices, they argued, could be addressed through novel technologies. For example, otitis media might be diagnosed using a smartphone-based otoscope, and urinary tract infections might be assessed using at-home urinalysis. Remote monitoring with digital blood pressure cuffs could be used to improve blood pressure control, so that patients would only have to visit their physicians occasionally.
You may have missed it, but the Association for the Advancement of Artificial Intelligence (AAAI) just announced its first annual Squirrel AI award winner: Regina Barzilay, a professor at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL). In fact, if you’re like me, you may have missed that there was a Squirrel AI award. But there is, and it’s kind of a big deal, especially for healthcare – as Professor Barzilay’s work illustrates.
The Squirrel AI Award for Artificial Intelligence for the Benefit of Humanity (Squirrel AI is a Chinese-based AI-powered “adaptive education provider”) “recognizes positive impacts of artificial intelligence to protect, enhance, and improve human life in meaningful ways with long-lived effects.” The award carries a prize of $1,000,000, which is about the same as a Nobel Prize.
Yolanda Gil, a past president of AAAI, explained the rationale for the new award: “What we wanted to do with the award is to put out to the public that if we treat AI with fear, then we may not pursue the benefits that AI is having for people.”
Dr. Barzilay has impressive credentials, including a MacArthur Fellowship. Her expertise is in natural language processing (NLP) and machine learning, and she focused her interests on healthcare following a breast cancer diagnosis. “It was the end of 2014, January 2015, I just came back with a totally new vision about the goals of my research and technology development,” she told The Wall Street Journal. “And from there, I was trying to do something tangible, to change the diagnostics and treatment of breast cancer.”