This is the part two of a three-part series. Catch up on Part One here.
Preetham Srinivas, the head of the
chest radiograph project in Qure.ai, summoned Bhargava Reddy, Manoj Tadepalli, and
Tarun Raj to the meeting room.
“Get ready for an all-nighter, boys,”
Qure’s scientists began investigating
the algorithm’s mysteriously high performance on chest radiographs from a new
hospital. To recap, the algorithm had an area under the receiver operating
characteristic curve (AUC) of 1 – that’s 100 % on multiple-choice question
“Someone leaked the paper to AI,”
“It’s an engineering college joke,”
explained Bhargava. “It means that you saw the questions before the exam. It
happens sometimes in India when rich people buy the exam papers.”
Just because you know the questions
doesn’t mean you know the answers. And AI wasn’t rich enough to buy the AUC.
The four lads were school friends from
Andhra Pradesh. They had all studied computer science at the Indian Institute
of Technology (IIT), a freaky improbability given that only hundred out of a
million aspiring youths are selected to this most coveted discipline in India’s
most coveted institute. They had revised for exams together, pulling
all-nighters – in working together, they worked harder and made work more fun.
Despite an area under the ROC curve of 1, Cassandra’s
prophesies were never believed. She neither hedged nor relied on retrospective
data – her predictions, such as the Trojan war, were prospectively validated. In
medicine, a new type of Cassandra has emerged –
one who speaks in probabilistic tongue, forked unevenly between the
probability of being right and the possibility of being wrong. One who, by conceding
that she may be categorically wrong, is technically never wrong. We call these
new Minervas “predictions.” The Owl of Minerva flies above its denominator.
Deep learning (DL) promises to transform the prediction
industry from a stepping stone for academic promotion and tenure to something
vaguely useful for clinicians at the patient’s bedside. Economists studying AI believe that AI is revolutionary,
revolutionary like the steam engine and the internet, because it better predicts.
Recently published in Nature, a sophisticated DL algorithm was able to predict acute kidney injury (AKI), continuously, in hospitalized patients by extracting data from their electronic health records (EHRs). The algorithm interrogated nearly million EHRS of patients in Veteran Affairs hospitals. As intriguing as their methodology is, it’s less interesting than their results. For every correct prediction of AKI, there were two false positives. The false alarms would have made Cassandra blush, but they’re not bad for prognostic medicine. The DL- generated ROC curve stands head and shoulders above the diagonal representing randomness.
The researchers used a technique called “ablation analysis.”
I have no idea how that works but it sounds clever. Let me make a humble
prophesy of my own – if unleashed at the bedside the AKI-specific, DL-augmented
Cassandra could unleash havoc of a scale one struggles to comprehend.
Leaving aside that the accuracy of algorithms trained
retrospectively falls in the real world – as doctors know, there’s a difference
between book knowledge and practical knowledge – the major problem is the
effect availability of information has on decision making. Prediction is
fundamentally information. Information changes us.
How easy is it for physicians to choose wisely and reject low value care? Who decides what’s wise and what’s unwise? In this episode Saurabh Jha (aka @RogueRad) speaks with William Sullivan MD JD. Dr. Sullivan is an emergency physician and an attorney specializing in healthcare issues. Dr. Sullivan represents physicians and has published many articles on legal aspects of medicine. He is a past president of the Illinois College of Emergency Physicians and a past chair and current member of the American College of Emergency Physicians’ Medical Legal Committee.
Recently, my niece gingerly
confided that she was going to study engineering rather than medicine. I was
certain she’d become a doctor – so deep was her love for biology and her
deference to our family tradition. But she calculated, as would anyone with
common sense, that with an engineering degree and an MBA, she’d be working for
a multinational company making a comfortable income by twenty-eight. If she
stuck with tradition and altruism, as a doctor she’d still be untrained and
preparing for examinations at twenty-eight.
Despite the truism in India that
doctors are the only professionals never at risk of starving, the rational case
for becoming a physician never was strong. Doctors always needed a dose of the
irrational, an assumption of integrity and an unbridled goodwill to keep going.
Once, doctors commanded both the mystery of science and the magic of
metaphysics. As medicine became for-profit, the metaphysics slowly disappeared.
Indians are becoming more
prosperous. They’re also less fatalistic and expect less from their gods and
more from their doctors. In the beginning they treated their doctors as gods, now
they see that doctors have feet of clay, too. Doctors, who once outsourced the
limitations of medicine to the will of Gods, summarized by the famous Bollywood
line “inko dawa ki nahin dua ki zaroorat hai” (patient needs prayers not
drugs), now must internalize medicine’s limitations. And there are many –
medicine is still an imperfect science, a stubborn art, often an optimistic breeze
fighting forlornly against nature’s implacable gale.
Can we reduce over diagnosis by re-naming disease to less anxiety-provoking makes? For example, if we call a 4.1 cm ascending aorta “ecstasia” instead of “aneurysm” will there be less over-treatment? In this episode of Radiology Firing Line Podcast, Saurabh Jha (aka @RogueRad) discusses over diagnosis with Ian Amber, a musculoskeletal radiologist at Georgetown University, Washington.
What does it take to create a decision rule? In this episode of Radiology Firing Line podcast Saurabh Jha (@RogueRad) has a discussion with Robert W. Yeh MD MBA about the deep thought and complex statistics involved in creating a decision rule to guide therapy which have narrow risk-benefit calculus, specifically a rule for how long patients should continue dual anti-platelet therapy after percutaneous coronary intervention. They also discuss the motivation behind the legendary, and satirical, parachute RCT published in the recent Christmas edition of the BMJ, which delighted satirists all over the world.
In this episode of Radiology Firing Line Podcast, I speak with Bishal Gyawali MD, PhD. Dr. Gyawali obtained his medical degree from Kathmandu. He received a scholarship to pursue a PhD in Japan. Dr. Gyawali’s work focuses on getting cheap and effective treatment to under developed parts of the world. Dr. Gyawali is an advocate for evidence-based medicine. He has published extensively in many high impact journals. He coined the term “cancer groundshot.” He was a research fellow at PORTAL. He is currently a scientist at the Queen’s University Cancer Research Institute in Kingston, Ontario.
What are the challenges of bringing advanced imaging services to India? What motivates an entrepreneur to start build an MRI service? How does the entrepreneur go about building the service? In this episode, I discuss radiology in India with Dr. Harsh Mahajan, Dr. Vidur Mahajan and Dr. Vasantha Venugopal. Dr. Harsh Mahajan is the founder of Mahajan Imaging, a leading radiology practice in New Delhi, and now a pioneer in radiology research in India.
Listen to our conversation on Radiology Firing Line Podcast here.
Saurabh Jha is an associate editor of THCB and host of Radiology Firing Line Podcast of the Journal of American College of Radiology, sponsored by Healthcare Administrative Partner.
I have a wide ranging conversation with Dr. Nicole Saphier for JACR’s Firing Line Podcast. Dr. Saphier is a radiologist specializing in women’s imaging. We discuss screening mammograms and the breast density law. Dr. Saphier, a frequent contributor to multiple major media outlets, tells us what it means for a radiologist to opine on health policy in the national media.
About the author:
Saurabh Jha is a contributing editor to THCB. He’s the host of JACR’s Firing Line Podcast. He can be reached on Twitter @RogueRad