Medicine is obsessed with numbers. Or rather, journalists and medical administrators are. Here are two related examples of how large a grain of salt one must put on numbers.
Cardiac surgical procedures, like everything else in medicine, have quality indicators. One of these is what we doctors call “30-day mortality”. What this term means is that surgeons are evaluated in part on how many of the patients they operated on died within a month of having surgery. Presumably a surgeon whose patients rarely die within 30 days is a better surgeon than one whose patients die all the time. The American Academy of Hospice and Palliative Medicine, whose members deal frequently with the elderly, thinks this number, 30, harms old people. http://nyti.ms/1AR3OqB. The problem, according to Paula Span of the New York Times, is that surgeons refuse to operate on people who are more likely to die within 30 days, and that they keep patients alive in ICUs until day 31 to keep their numbers up. Bad doctors!
The problem with the number 30 is not that it’s to short or too long, it is that it is a terrible metric for quality. Patients die despite everyone’s best efforts, especially patients who are at higher risk for dying to begin with. We need to find a metric that actually reflects quality of care. Of course doctors are going to be leery of operating on really sick people, if their jobs are at stake! I know people would like for doctors to be saints who take care of everyone all the time with nary a pecuniary thought, but I’m sorry, doctors are not saints. Neither are patients.
Speaking of risk, here’s number two reason numbers are evil. A recent article in the Journal of the American Medical Association reviewed the current literature on how accurate patients assessments of risks and benefits are. The authors found that 65% of the time patients overestimate the benefits and 67% of the time they underestimate the risks. The problem, according to Austin Frakt and Aaron Carroll of the New York Times, is that doctors don’t give patients adequate information about risks and benefits. http://nyti.ms/1wJ8LwC. Bad Doctors!
The problem is not that doctors don’t give people the numbers. The problem is that the numbers don’t influence patient’s decisions. Reams of research as well as best-selling books by people like Nobel prize winner Daniel Kahneman tell us that risk assessment has little to do with statistics. Humans estimate risk based on things like what is most prominent in the news, how they feel about the risk in question, and how closely they compare to others who have undergone the event in question. For example, women who have had bilateral mastectomies after a breast cancer diagnosis were asked how much the surgery had decreased their risk of recurrence. The average response was women felt their risk had gone from 76% to 11%. The actual risk before surgery is actually only 17%, so the surgery reduces the risk of recurrent breast cancer six percentage points. (This is for women who don’t have the BRCA gene). I’m sure women are told what the risk of recurrence is and how much the surgery decreases the risk. I’m sure they are. But the numbers are being told to women who are scared out of their minds about breast cancer and just want it to go away. They don’t hear nor care what the statistics are. That’s called being human. Pick a subject. Vaccinations – gross overestimation of risk because the guy down the street has an autistic kid. Dying in a plane crash – driving in your car is way more dangerous but the newspaper just had a big story about a horrific plane crash. Ebola – one case in the US but everyone is afraid they will get infected because it’s a really bad disease.
Please. No more numbers.
Shirie Leng is an anesthesiologist. She has worked in health care both as a nurse and as a doctor for 15 years. She writes about the processes, redundancies, red-tape and pure pointlessness of much of medicine, so that you can make decisions and navigate for yourself. She also a mother with three young girls. More posts are available on her medicine is real blog site.
Categories: Uncategorized
nice your aticle sharring thank for all
“CV.” Good.
Lotta people don’t even know what that is. “Coefficient of Variation,” a.k.a. “RSD” — “Relative Standard Deviation.” The ratio of std devation to the mean. An estimate of assayed variability. Sometimes also called “Percent Std Deviation.” Taking it into account helps accuracy of analysis — assuming distributional normality, which is not always the case, particularly as target analyte concentrations are diluted down toward the “LLD” (Lower Limit of Determination).
But, no, we get some number, and blithely assume it to be “accurate.”
And, as Earlw points out, beyond individual assay precision, multivariate “context” matters significantly (in particular over time, in the n-dimensional “flow sheet” sense).
Great point. Cholesterol is a metric. It can vary by season, by time of day, and by what was eaten the night before. If the timing of checking the metric can be accommodated to maximize these factors, you can game it to your favor without actually improving anything about the patients risk profile. So if I load my patient with oatmeal for 2 days straight, I could produce some great looking metrics. Probably get a bonus. They may be some other issues, but who cares? If I get penalized for not meeting a metric, I will call for a recheck. Certainly, there should be some standard of deviation for metric meeting. What is the SD for each measure? Entire civilizations can be destroyed on just a few unchallenged assumptions.
I wish policy folks would listen to you, Bobby.
Chemistries, hematology, serology, urinalysis, et al.., are just approximations. CV’s very often @10% on repeat testong on same specimen. It is OK and often good to repeat them. Micro is a little different, but not altogether. Repeat cultures CAN show different bugs but not that often.
The unappreciated fact is that lab tests almost always cost pennies to do. They look expensive on our bills because the hospitals want to make them a profit center. Example: woman comes to hospital outpatient department wanting to know is she was pregnant. Cost of chorionic gonadotropin test to find out: $2.19. Charge to woman as she left hospital: $125.
“All prices in health care are bogus”…Abe Lincoln said this. 🙂
Think of lab testing as a chemical history of the patient. It is good to get a lot of history as it is lab testing as long as it is priced around the marginal cost of the test, but our brains have been ruined by making it seem wasteful to do much testing. Docs have to learn to ignore some testing and mentally integrate results so that they don’t go off on goose chases.
The best docs get lots of testing early after admission so they arrive at diagnosis fast and accordingly shorten LOSs. But that was in the past…sadly.
@1/7 of US is eating off of health care. We are making too big a deal out of it.
Your “We need to find a metric that actually reflects quality of care.” contradicts “No more numbers.”
Numbers make it easier to compare different things but they need context to make them useful.
Good (decimal) point, Bobby.
Yeah. Also ignores that fact that EVERY lab result is a proxy point estimate having a statistical distribution around it (with the distribution getting increasingly non-Gaussian as the target analyte concentration levels go down). In the forensic lab world where I cut my professional teeth in the ’80’s, you put “8.1” in a client report, you have better be able to demonstrate to a horde of auditors and regulators that you can discriminate empirically between “8.0” and “8.2” Otherwise, you gotta lose the decimal point (“significant figures rounding”).
Take a serum specimen, break it out into 9 equal aliquots, send sets of 3 to 3 different labs — blinded — and you might be amazed at the range of divergence coming back, across the breadth of lab slip parameters.
We get some lab number back, we simply assume it’s “exact.”
I don’t. It’s not.
“There is no true value of anything” — Deming (he qualified that by noting “beyond the enumeration of discrete objects”)
Meaningless Use?
i had a diabetic and got his hgba1c down from the 13 to 8.1 and was a bad doctor, had another patient and his hgba1c went from 6.5 to 7.9 and i got a pat on the back for being a good doc. The quality metric was 8.0. yippee.
Yes. There is an epidemic of producing numbers with no insight attached to them. Good points.
Not saying we should Shirie, but with health costs getting beyond affordability keeping someone alive for 30 days no matter the cost is just not good economics for either the poor or the rich – especially using Medicare, and as you note, probably not “quality of care”.
It might be true that having some skin in the game monetarily would change a person’s decision. It certainly used to in the nineteenth century, when wealthy people called the doctor and the poor died unattended. Clearly we can’t go down that road.
I find most docs useless in helping patients assess risk/outcome. They usually say something like; “you can do this or you can do that, it’s up to you”.
One more point about risk is who pays. When it’s free everyone will take that chance at a small % of success.
Yes, actually, Dr. Palmer. A surgeon who has a great reputation operating on old folks is exactly who I want operating on my mother or father. That kind of quality isn’t measured in 30-day mortality metrics.
Good point, Shirie. Maybe a doc’s reputation actually encodes more bits of information about quality than any number. Think of all the parts to a reputation: from the way the doc dresses to the way he speaks and what he drives as well as his failures and successes in his profession, which ooze out of the hospital and his office in thousands of rumored bits into the community where they are believed or not believed. His kindness and bedside manner are surrogate placebo medications that most of us would count in quality measures and these are in there too, along with how crowds assess his intelligence and skill.
But think of the alternatives too. Do you want surgeons who have poor 30 day survivals, but who have great reputations, operating on old folks?