Physicians

Who Is the Better Radiologist?

flying cadeuciiThere’s a lot of talk about quality metrics, pay for performance, value-based care and penalties for poor outcomes.

In this regard, it’s useful to ask a basic question. What is quality? Or an even simpler question, who is the better physician?

Let’s consider two fictional radiologists: Dr. Singh and Dr. Jha.

Dr. Singh is a fast reader. Her turn-around time for reports averages 15 minutes. Her reports are brief with a paucity of differential diagnoses. The language in her reports is decisive and her reports contain very few disclaimers. She has a high specificity meaning that when she flags pathology it is very likely to be present.

The problem is her sensitivity. She is known to miss subtle features of pathology.

There’s another problem. Sometimes when reading her reports one isn’t reassured that she has looked at every organ. For example, her report of a CAT scan of the abdomen once stated that “there is no appendicitis. Normal CT.” The referring physician called her wondering if she had looked at the pancreas, since he was really worried about pancreatitis not appendicitis. Dr. Singh had, but had not bothered to enlist all normal organs in the report.

Dr. Jha is not as fast a reader as Dr. Singh. His turn-around time for reports averages 45 minutes. His reports are long and verbose. He meticulously lists all organs. For example, when reporting a CAT of the abdomen of a male, he routinely mentions that “there is no gross abnormalities in the seminal vesicles and prostate,” regardless of whether pathology is suspected or absence of pathology in those organs is of clinical relevance.

He presents long list of possibilities, explaining why he thinks a diagnosis is or is not. He rarely comes down on a specific diagnosis.

Dr. Jha almost never misses pathology. He picks up tiny lung cancers, subtle thyroid cancers and tiny bleeds in the brain. He has a very high sensitivity. This means that when he calls a study normal, and he very rarely does, you can be certain that the study is normal.

The problem with Dr. Jha is specificity. He often raises false alarms such as “questionable pneumonia,” “possible early appendicitis” and “subtle high density in the brain, small punctate hemorrhage not entirely excluded.”

In fact, his colleagues have jokingly named a scan that he recommends as “The Jha Scan Redemption.” These almost always turn out to be normal.

Which radiologist is of higher quality, Dr. Singh or Dr. Jha?

If you were a patient who would you prefer read your scan, the under calling, decisive Dr. Singh or the over calling, painfully cautious Dr. Jha?

If you were a referring physician which report would you value more, the brief report with decisive language and a paucity of differential diagnoses or the lengthy verbose report with long lists on the differential?

If you were the payer which radiologist would you wish the hospital employed, the one who recommended fewer studies or the one who recommended more studies?

If you were a hospital administrator which radiologist would you award a higher bonus, the fast reading Singh or the slow reading Jha? This is not a slam dunk answer because the slow-reading over caller generates more billable studies.

If you were hospital’s Quality and Safety officer or from Risk Management, who would you lose more sleep over, Dr. Singh’s occasional false negatives or Dr. Jha’s frequent false positives? Note, it takes far fewer false negatives to trigger a lawsuit than false positives.

I suppose you would like hard numbers to make an “informed” decision. Let me throw this one to you.

For every 10, 000 chest x-rays Dr. Singh reads, she misses one lung cancer. Dr. Jha does not miss a single lung cancer, but he recommends 200 CAT scans of the chest for “questionable nodule” per 10, 000 chest x-rays. That is 200 more than Dr. Singh. And 199/ 200 of these scans are normal.

I can hear the siren song of an objection. Why can’t a physician have the sensitivity of Dr. Jha and the specificity of Dr. Singh? The caution of Jha and the speed of Singh? The decisiveness of Singh and the comprehensiveness of Jha?

You think I’m committing a bifurcation fallacy by enforcing a false dichotomy. Can’t we have our specificity and eat it?

Sadly, I’m not. It is a known fact of signal theory that no matter how good one is, there is a trade-off between sensitivity and specificity. Meaning if you want fewer false negatives, e.g. fewer missed cancers on chest X-ray, there will be more false positives, i.e. negative CAT scans for questioned findings on chest X-ay.

Trade-off is a fact of life. Yes, I know it’s very un-American to acknowledge trade-offs. And I respect the sentiment. The country did, after all, send many men to the moon.

Nevertheless, whether we like it or not trade-offs exist. And no more so than in the components that make up the amorphous terms “quality” and “value.”

Missing cancer on a chest x-ray is poor quality (missed diagnosis). Over calling a cancer on a chest x-ray which turns out to be nothing is poor quality (waste). But now you must decide which is poorer. Missed diagnosis or waste? And by how much is one poorer than the other.

That’s a trade-off. Because if you want to approach zero misses there will be more waste. And if we don’t put our cards on the table, “quality” and “value” will just be meaningless magic talk. There, I just gave Hollywood an idea for the next Shrek, in which he breaks the iron triangle of quality, access and costs and rescues US healthcare.

If I had a missed cancer on a chest x-ray I would have wanted Dr. Jha to have read my chest x-ray. If I had no cancer then I would have wanted Dr. Singh to have read my chest x-ray. Notice the conditional tense. Conditional on knowing the outcome.

In hindsight, we all know what we want. Hindsight is just useless mental posturing. The tough proposition is putting your money where your mouth is before the event. Before you know what will happen.

This is the ex-ante ex-post dilemma. In case you want a clever term for what is patently common sense.

Dr. Singh is admired until she misses a subtle cancer on a chest x-ray. Then Risk Management is all over her case wondering why? How? What systems must we change? What guidelines must we incorporate?

Really? Must you ask?

Dr. Jha, on the other hand, is insidiously despised and ridiculed by everyone. All who remain unaware that he is merely a product of the zero risk culture in the bosom of which all secretly wish to hide.

The trouble with quality is not just that it is nebulous in definition and protean in scope. It can mean whatever you want it to mean on a Friday. It is that it comprises elements that are inherently contradictory.

Society, whatever that means these days, must decide what it values, what it values more and how much of what it values less is it willing to forfeit to attain what it values more.

Before you start paying physicians for performance and docking them for quality can we be precise about what these terms mean, please?

Thank you.

 

Livongo’s Post Ad Banner 728*90

27
Leave a Reply

14 Comment threads
13 Thread replies
0 Followers
 
Most reacted comment
Hottest comment thread
14 Comment authors
Chris ChuAnne JackMark A. RaineyHapLegacyflyer Recent comment authors
newest oldest most voted
Chris Chu
Guest
Chris Chu

This is a fantastic article outlining the dilemma radiologist face in clinical practice. We cannot be perfect and have 100% sensitivity and 100% specificity. I have seen radiologists in both the Singh and Jha schools of diagnostic radiology. I personally try to strike a balance between the two schools for my practice but naturally I am still inclined to be in the Singh side because I feel that I am helping the patients and giving a more specific and definitive answer to referring clinicians. I have seen some radiologists in the Jha school of radiology overcalling everything you can possibly… Read more »

Anne Jack
Guest

These are some awesome tips to find the best radiologist. Thanks for sharing this information. We provide back surgery treatment and care in West Palm Beach. You can contact us for assistance.

Mark A. Rainey
Guest
Mark A. Rainey

Excellent article. The diagnostic imaging for a multitude of problems has exploded. Insecure primary care providers, including nurse practioners, order studies with reckless abandon. The vast majority of MRI scans for low back pain are useless or harmful. Over-reads on disc changes promote medicalization of routine problems. In my area, Dr. Jha easily does more harm than good. Incidentalomas are an epidemic. Get out of the darkroom and look at the whole clinical picture. The priority should be on patient care. I know it is a complex issue, but let’s keep quality patient care on top.

Hap
Guest
Hap

So far I have seen many thoughtful comments and suggestions. However, the answer is in plain sight…..BOTH and ALL…get paid less. The fight is all about how much less, and for which one, and by whom?

Saurabh Jha
Guest
Saurabh Jha

“Sometimes in error, but never in doubt”

That’s absolutely correct Legacy. I’ve found that clinicians favour certainty and consistency over doubt and accuracy.

Which means that radiologists might have to internalize some risk in order to be considered useful.

Legacyflyer
Guest
Legacyflyer

Legacyflyer signing in. My signal is strong but my content is weak ….. or something like that. Dr. Jha is absolutely correct and he has explained this concept well. When I was in training I heard the following two statements which have had a great impact on me: “Don’t be a ditzel caller!” From a professor I admired greatly (John Diaconis -and if you knew him, you knew that most everything he said had an exclamation point behind it.) The point of his statement is that there are many ill defined gray (not black and white) object on an image.… Read more »

Barry Carol
Guest
Barry Carol

I would love to hear from Legacyflyer on this one. Since absolute perfection is an impossible and unreasonable expectation, if you’re hiring a radiologist for a large group practice, how many missed positives per 1,000 or 10,000 readings are acceptable? As a patient, I would opt for Dr. Singh. A 1 in 10,000 chance of missing a lung cancer is an acceptable risk to me in exchange for cost-effectiveness and the avoidance of unnecessary additional testing. If I’m the primary care doctor, I would prefer the radiologist who can give me the information I need in a clear and concise… Read more »

David Lawrance
Guest
David Lawrance

For a single radio receiver, if I turn up its sensitivity, I am more likely to detect a weak station I want to hear and I am also more likely to detect others that I don’t (1-specificity.) The relationship between these two characteristics can generally be plotted as a continuous function, a receiver operator curve (ROC.) You describe the tradeoff beautifully. But, then you make a mistake. Each radio has its own ROC. Similarly, so does each radiologist. ROCs can intersect, resulting in the tradeoffs you describe between Dr. Jha and Dr. Singh. The other possibility is that their curves… Read more »

Saurabh Jha
Guest
Saurabh Jha

I agree that there are better and worse ROC curves. But even within an ROC curve there is a trade off.

One of the problems is discerning between radiologist moving along an ROC curve and radiologist moving to a higher (greater AUC) curve. Often, the two are conflated.

Fergus
Guest
Fergus

There is an inverse relationship between clarity of information shared with the radiologist prior to reading the study, and amount of gibberish and pointless waffle in the report.
One comment here suggested the radiologist read the study, then track down and ring the referrer, which could take 10-15 minutes, to ask what they were concerned about. How about letting the radiologist know the concern when the scan was booked by filling the clinical question box correctly, doesn’t the patient deserve that basic courtesy?

Peter1
Guest
Peter1

Fergus, I think I would prefer to let the Rad interpret the picture without outside information that may lead them down a biased path. I have had experiences with dentists, who I was sent to for second opinion, just agreeing with the first doc as a matter of professional courtesy. Not saying that a Rad would do that but just putting a thought in their mind may be a subconscious influence. I’d prefer a blank slate analysis. Another point here relating to so called “consumer directed health plans” (CDHP) is that how is the consumer supposed to evaluate cost/quality, especially… Read more »

Saurabh Jha
Guest
Saurabh Jha

Actually I, too, would prefer the blank slate approach. The problem is that there are so many shades of gray (possible adrenal cancer, possible this and possible that) that context helps reduce the noise, particularly when there will be an after the fact expert witness who will tell me how I should have known better all along.

Perry
Guest
Perry

My wife describes Radiology as making black and white decisions based on shades of gray. My opinion is that making a radiographic diagnosis without complete information on the patient is like making a clinical diagnosis without taking a history. This is why the best radiologists make themselves readily available to consult by phone or in person when the clinician has a question on the result. The other problem with not knowing the history is that the wrong test may be ordered for the potential condition, making the test worthless. The most recent issue of JAMA actually has an interesting Opinion… Read more »

Saurabh Jha
Guest
Saurabh Jha

Yes, I agree. If imaging was an exact science, and there is a movement within imaging increasing more quantification, then no context would be necessary.

Fergus
Guest
Fergus

Peter, my comment was the general blight of booking advanced imaging studies with a one word entry on the “Clinical Details” section. In my institution in North America, it is considered acceptable to perform a CT pulmonary angiography study with the words “CTPA protocol” entered in the clinical details section, and patients get CT brains, carotids and circle of Willis with the words “hot stroke protocol” entered. ER patients often get near full body x ray and CT scans with only the word “trauma”. The radiologist has the option of dialing for every study he/she is reading, to get more… Read more »

Pat
Guest
Pat

I have heard that a computer program can be more accurate than an expert cardiologist in interpreting EKGs. Is there a prospect of something similar occurring in radiology?

Saurabh Jha
Guest
Saurabh Jha

There is computer aided detection. But that must decide as well where it wishes to draw the line between being wrong (false positive) or being really wrong (false negative).

Peter1
Guest
Peter1

Don’t both radiologists need to go back to school? Is there anything here justifying peer review and additional training?

Perry
Guest
Perry

Actually, there is peer review, Peter, the rads in the group review a percentage of each other’s cases. You can also bet if there’s a miss, that one will be reviewed extensively.
I can also tell you that misses will happen whether reading fast or slow. Busy day, multiple interruptions, radiologist on night call, being human. I bet even the ficitonal Dr. Jha will eventually have a miss despite his overcaution.

Peter1
Guest
Peter1

Well I guess in an imperfect world that’s all we can hope for, just as long as the peer review does not favor billing volume over patient safety.

Perry
Guest
Perry

Absolutely not, it’s serious business for the rads.

Saurabh Jha
Guest
Saurabh Jha

It is easier to make the decisive radiologist over cautious than it is to make the over cautious radiologist decisive.

That is it is easier to improve sensitivity at the expense of specificity than it is to improve specificity at the expense of sensitivity.

That’s my conjecture. I have no empirical evidence to back that up.

Rob
Guest

You nailed it. The problem is that the overly-cautions doc is not clinically helpful a lot of times to the doctor who ordered the test. I often get radiology reports that give so much information that it makes it quite hard to find the answer to the question I was asking. The ideal situation is when the radiologist calls me and gives me a verbal report. Then I can ask if they see the thing I am looking for (or hoping to rule out). Of course, I need to be sure I send enough clinical information to the radiologist so… Read more »

Perry
Guest
Perry

I agree, great piece. Being married to a radiologist, I can appreciate the nuances in how these two are perceived. I like to think my wife is a nice combination of the two, which does happen. Dr. Jha will be appreciated by the specialists, but also perhaps by the hospital because his hedging reports mean more revenue from more studies. Dr Singh will be a favorite of Primary Care for her succint reports, and fast TAT will endear her to the hospital admin as well. The radiology group will really like Dr. Singh over Dr Jha because she will produce… Read more »

Saurabh Jha
Guest
Saurabh Jha

Thanks for reading Perry.

Specificity, like good Scotch, comes with age and experience!

Vik Khanna
Guest

This is a GREAT essay. Thank you, Saurabh for painting the picture very nicely of the kinds of trade-offs we have to eventually deal with but pretend don’t exist. The healthcare industry is the Gordian knot that it is because these confounders have been ignored as we added even more layers of complexity figuring that someone would eventually sort it all out. There is no someone and eventually is now…or, maybe never, because I believe the entire construct of American healthcare is irretrievably broken. By the way, I vote for Dr. Jha, not only because he is apparently careful to… Read more »

Saurabh Jha
Guest
Saurabh Jha

Now Vik, you are allowing your single malt bias to creep in!

The problem is that the system is killing the decisive Singh. And that’s ultimately bad care.

I’m all for checking the blind spot of the rear view mirror without exception, and punishing those that don’t. But the safety culture has gone out of control.