An article in this week’s New York Times called Will This Treatment Help Me? There’s a Statistic for that highlights the disconnect between the risks (and risk reductions) that epidemiologists, researchers, guideline writers, the pharmaceutical industry, and policy wonks think are significant and the risks (and risk reductions) patients intuitively think are significant enough to warrant treatment.
The authors, bloggers at The Incidental Economist, begin the article with a sobering look at the number needed to treat (NNT). For the primary prevention of myocardial infarction (MI), if 2000 people with a 10% or higher risk of MI in the next 10 years take aspirin for 2 years, one MI will be prevented. 1999 people will have gotten no benefit from aspirin, and four will have an MI in spite of taking aspirin. Aspirin, a very good drug on all accounts, is far from a panacea, and this from a man (me) who takes it in spite of falling far below the risk threshold at which it is recommended.
One problem with NNT is that for patients it is a gratuitous numerical transformation of a simple number that anybody could understand (the absolute risk reduction – “your risk of stroke is reduced 3% by taking coumadin“), into a more abstract one (the NNT – “if we treat 33 people with coumadin, we prevent one stroke among them”) that requires retransformation into examples that people can understand, as shown in pictograms in the NYT article. A person trying to understand stroke prevention with coumadin could care less about the other 32 people his doctor is treating with coumadin, he is interested in himself. And his risk is reduced 3%. So why do we even use the NNT, why not just use ARR?
I cannot answer that question, because the NNT is a gratuitous transformation for physicians too. Referencing the article that introduced NNT as a concept in 1980, we find scant justification for transforming the ARR to its reciprocal. The authors state that “because the expression of the absolute risk reduction as a decimal fraction may not seem sensible to practicing physicians, this measure may be difficult for them to remember and incorporate into clinical practice,” and in their conclusion that “[the ARR’s] reciprocal, the number needed to be treated, expresses the absolute risk reduction in a manner that is easily understood by clinicians.”
Thirty-five years later, I find these statements to be both condescending and false. What justification is there to say that physicians cannot understand and remember decimals and subtract them? To the contrary, many physicians do not remember how to calculate the NNT or to retransform it back to the ARR. I am going to posit that the NNT took on a life of its own because it was published in the NEJM, and it became academically vogue to prattle on about it without really thinking about whether it added anything to a basic understanding of simple math.
Back to patients and their making decisions on the basis of data. Suppose a patient is discussing taking aspirin as primary prevention with his physician who presents him with the ARR for MI of 0.05% (1/NNT) over two years. A patient considering this (or any small number or its NNT transformation with associated analogies and diagrams) may elect to “take their chances” because “0.05% ain’t much, Doc.” Is this person rational? One way to judge this is to evaluate the patient’s decision to defer aspirin for congruence with other behaviors that demonstrate his revealed preferences. Does this person smoke, does he exercise, drive the speed limit, wear his seatbelt, own a firearm, drink excessively, etc? Is he a person who takes chances on a daily basis (and has during his lifetime) that are on par with the chances he takes by foregoing aspirin for primary prevention? If so, not taking aspirin (or another therapy) may be a congruent, rational decision for this individual.
(We demonstrated in an article in Medical Decision Making that physicians, too, have an intuition that smaller ARRs (yet much larger than 0.05% are not worthy of their endorsement. When evaluating a trials with identical characteristics except ARR (which was manipulated as the independent variable in a randomized case vignette study) willingness to adopt a new therapy with a ARR of 3% was 20-35% less than willingness to adopt the same therapy if the ARR was 10%. And in our study the outcome of the ARR was short-term mortality and the treatment had no opportunity costs and minimal/no side effects.)
But researchers, public health types, and policy activists are less concerned with the nuances and abstractions of individual decisions than they are with decisions that affect entire populations. At thepopulation level, thousands of heart attacks will be prevented each year, (THOUSANDS!) if several million people at risk take aspirin, or many other treatments with smallish ARRs. What one may deem negligible at the individual level is not negligible at the population level.
Suppose the above patient’s physician convinces him to take aspirin for primary prevention of coronary disease. I will call this situation the Therapeutic Paradox, borrowing from the Voter’s Paradox, which is the observation that, since the likelihood that any individual’s vote will influence the outcome of an election is small, and since it is costly in terms of time and effort to vote, it is paradoxical that anybody votes. (The paradox of voting is related to the Tragedy of the Commons.) In the Therapeutic Paradox, a patient is taking a drug or other therapy that benefits the population but that may not benefit him either because he is not one of the patients who gets the benefit, the risk reduction is not congruent with his other behaviors, or because his value system makes the side effects or costs unacceptable to him, even though the epidemiologist thinks the population as a whole would be better off if he (and everybody else) were to take the treatment. (Perhaps the biggest Therapeutic Paradox is the use of INH prophylaxis for positive PPDs [tuberculin tests], which have may net negative benefit for the individual but positive benefit for the population, but I digress.)
This patient’s risk of dying in a motor vehicle accident this year, assuming he will drive 12,000 miles, is 0.014%. It is small, but certainly not negligible on the individual level, and on the population level it is magnified to on the order of 40,000 deaths per year in the United States. Yet this patient does not awaken and think “I should drive fewer miles today to reduce my risk of death in a car crash.” (Nor does he think of the other negative externalities of driving 12,000 miles a year, and maybe he should be more mindful of both.) If he does not, it is not irrational for him to forego aspirin for primary prevention of cardiovascular disease, and perhaps many other therapies that may be offered to him if he knew the actual ARRs associated with those therapies. Thus, I think that candid discussion of risk and its reduction may have unanticipated paradoxical effects that are the opposite of those that are intended.
So how do we get patients to take the therapies that we (physicians and public health types) believe are good for them and for the population? One way is to nudge the pendulum away from the autonomy model of decision making back towards the paternalistic model – the doctor (or Mr. Bloomberg) simply says “I think you should take aspirin” – and the patient does it because the doctor knows best, just like he always votes because grandpa always said you should exercise your right to vote. Thus, the physician serves a dual role as a public health practitioner. It is abundantly clear that “shared decision making” is a utopia that few doctors or their patients have the luxury of, or even want to have the luxury of. Many and perhaps most patients don’t want to split hairs about taking an aspirin, or consider the small numbers and all their transformations and the analogies necessary for a proper understanding of risk and its reduction – they just want their doctor to tell them what is best, and if she says “take an aspirin a day” they are more than happy to oblige, whatever their Framingham score or their appraisal of the raw numbers and how negligible they may seem. On the other hand, some patients are aversive to medications with much larger ARRs than aspirin and there is little that can be done to convince them otherwise. Indeed, exploration of patients’ perceptions of risk may do little more than expose the Therapeutic Paradox and allow them to realize that many of the medications they are taking have what they would consider to be negligible benefits.
One thing is clear: the NNT is not any kind of magical transformation that makes the concept of risk and its reduction more accessible to patients (or physicians). Quite the contrary.
Scott Aberegg, MD is a physician based in Salt Lake City, Utah. He blogs regularly at The Medical Evidence Blog, where this blog post first appeared.
Categories: Uncategorized
New dx, “Randilosis”
https://twitter.com/BobbyGvegas/status/563012147358490624
Interesting.
You need a population to know the individual risk reduction: smaller the benefit, the larger the population. The obverse is not true: the individual does not need the population to benefit. Sample size for a clinical trial is confused by many for population health.
NNT, to me, is a simple way of quantifying diminishing marginal returns of a treatment.
If you need to treat 1000 people to save one life, what that should tell the patient is that the risk reduction is negligible, the treatment benefit small.
NNT does assume a binary state which is misleading for cause-specific mortality.
The only way your motorcycle crash risk of 3% for each track visit is cumulative would be if you decided that you are going to the track no matter how many broken bones you have or the number of bandages or concussions. Otherwise, a crash at visit #7 might keep you from visiting 8,9, ot 10 times.
What about 1–( 97^10/10^10)?
Dr Mike; interesting comments; I have become perhaps too comfortable bashing the “population health” idea as I think only patients should make choices. I can’t mathematically come up with a better population outcome than when we allow individuals to make choices based on individual variations in risks and values. For example, an average ARR is a summary of many ARRs. Suppose an average ARR of 5% results from 1/3 of patients with 0; 1/3, 5% and 1/3, 10%. A population approach will treat them all and harm them all; an individual approach that would lead to the 0% ARR not taking the intervention would preclude harm in 1/3. The best population health is the best individual patient health, isn’t it?
But, the paradox is that population data are used to estimate individuals’ risks which is both necessary, but impossible. One of my motives is to push people to believe they can decide so they will push us to do better studies in better ways to get closer to individual risk estimates; genetic stuff may help, but we will have to see. We never plan studies, presently, to adequately include subgroups or ARR variations and poorly extrapolate the average when it would be better to not. In addition, the discussion of the 10% risk of heart disease above is a “point estimate of points estimated”, so it is not even a real number.
Last, and now the point where I will get into trouble. I simulate studies and can say that we do not have a science of numbers like 1 in 10,000. We are overpowered to detect false positive associations.
You point out some limits on my thinking, though. I like your conceptualization; when an intervention treats me AND you, it is population health. When it doesn’t we should plan for individual health concepts. Thanks.
@ J Kirschman: Your lottery example is poignant. It comes down to values. If you value the entertainment of a lottery ticket, it may be worth a dollar a day, year in and year out. But your values should be informed values. Here’s some information for those people (who remind me of the bumper sticker “The Lottery: A Tax for People Who Don’t Get Math”):
Your expected return/value for that $1 ticket is $0.50. Alternatively, you could save that $365/year and invest it in your IRA. If you use a Vanguard Target Retirement Date fund, you will have negligible costs, and an expected tax deferred return of 10% per year, give or take. Over 30 years, the account balance will grow to $60,000.
On the other hand, it does not surprise me that people play the lottery, or don’t diet or exercise, or do many things that they “should” do if they want to maximize LONG TERM outcomes, in favor of short term ones.
@William Palmer: Whether you want to reduce healthcare costs or not, most ARRs ARE, in fact, puny, and that was a major point of the NYT article. $0.03 is, after all, $0.03. RELATIVE to $0.02 it’s HUGE! But relative is irrelevant. We can make any difference look larger than it is by introducing an irrelevant denominator.
@Michael Millenson: I’m not sure if there is any substance here that merits a response. The confidence with which one expresses himself has no bearing on the fundamentals of his arguments. A perusal of my blog makes it abundantly clear that, quite contrary to your contention, I carefully evaluate a diverse range of data and opinions and logical arguments from all sides. The aim of the blog is to point out shortcomings in logic or clinical trials or perspectives and to challenge the conventional wisdom, and the “party line”. I think this article, like the others, does so in a fair, evenhanded manner, even if, to make a point or to entertain, it comes across as forceful or incisive.
If a person likes the NNT, by all means use it! But recognize that it is a simple arithmetic transformation of the ARR to its reciprocal. It is not magical. It just gets you to think about the ARR in another way.
Finally, I am a pointy headed intellectual…..with a knack for self-deprecation.
@PaulSlobodian: I just had an argument with a cardiologist friend who was in my MPH class over the weekend about whether the 3% ARR for stroke with warfarin is a population risk reduction or an individual risk reduction. That argument could go on ad infinitum, but that would not be practical. The practical thing is for patients to think that warfarin reduces THEIR risk of stroke by 3%. And it does.
Warfarin is a ticket for entry into the treatment group where strokes occur at a rate of 2% per year from the placebo group where it occurs at a rate of 5% per year. When you switch groups you switch risks.
Think of it another way. Suppose I have a 3% chance of crashing my motorcycle each time I take it to the track (experience suggests that it is actually higher than that – ask me how I know). I take it to the track 10 times per year. Since, like annual stroke risk, the risk is cumulative, my annual risk of crashing on the track is 30%. Each time I elect to “sit out” during one of the 10 annual track days, I reduce my annual track crash risk by 3% because I have removed myself from the “track” group (3% risk) and put myself in the “spectator” group (0% risk).
We get too caught up in these abstractions about stroke being a binary event for the person, and the population risk and whatnot. I think they’re the same thing. Even if they are not, I don’t think there is a superior way of communicating them, in terms of accuracy or understandability.
@Slobodian
I agree, but disease is also not just the tip of the island that we see. It is almost always astonishingly complex chemistry. Eg some two thousand metabolic reactions have been diagramed on one chart (one should glance at this for fun) involving glycolysis and the citric acid cycle. See Kanehisa Laboratories. To deal with only the island tips–the phenotypes of disease–of these vast systems of chemistry and feel that we know very much is hopelessly naive. When we take aspirin, eg, it is like sticking a needle through an i-phone and hoping that we can fix it. To treat 10,000 people with aspirin to achieve five no-show anti-MIs also means that we have changed the cellular machinery in probably 99% of those folks. This is knowledge we are not studying.
True, Dr. Mike. As we have seen in the recent measles outbreaks, people still choose not to vaccinate, despite the fact that links to autism and brain damage have been debunked.
I don’t think Americans are ready or willing to participate in population medicine. The just want to know what is best for themselves as individuals.
Why should a patient care about population health? Unlike with vaccinations, which clearly are for population health and not for individual health, whether or not a patient takes his statin has zero impact on his neighbor.
Ah the nanny state. Your concern for improving the numbers does not make me believe for one minute that you care about me, only I care about me. Therefore I will decide what I do with my body for the choices that have zero impact on you.
The risk (odds) of winning Powerball is 1 in 175,223,510 (ARR = 5.7 x 10e-9), and yet individuals choose to participate. Of course, there is no significant side-effects of participation. People will participate when the reward is great enough.
Tell these folks that they can reduce their risk of stroke by 3% by taking warfarin, they may participate until you explain the chances of a significant side effect that may reduce the benefit to 0.
Failing to tell them that your Big Brother Data Reviewer evaluates and incents you only how many of your patients are on the “appropriate” medical treatment and not how many suffered side effects of your medical treatment, I believe you might have entered special space regarding conflict of interest. .
Just some thoughts stimulated by the article.
But a baby aspirin a day? What the heck….I’ll take that lottery ticket.
Re your statement: ” so that in the long term, way beyond two years, you have much better long term survival.”
I think it is dangerous to justify an intervention on the basis of a postulated benefit that has not been strong enough to be seen in the research….as it may well be that there are offsetting (or greater) harms that also haven’t been seen in the research.
If that long term survival rate is there, lets find it by research….too many commonly accepted interventions that have been well researched show no long term clinical benefit over non intervention.
You have to consider another dimension in risk. Think of a seamount as being an island that didnt make it to the surface. All or most islands sit on former seamounts.
Now imagine the wide genetic and matabolic bases of many diseases. Eg cancer or diabetes or rheumatoid arthritis. Dozens of associated genes and proteomes and methylated epigenes are involved. Extreme complexity.underlies the bases for most common diseases. The average CA patient has about 50-60 mutations or tandem repeat problems of introns.
An island is to a seamount as a disease is to its metabolic and genetic basis. When you look at the ARR for aspirin for MI you are seeing that five in ten thousand (.05%) of these potential islands are not popping up after ASA therapy. This appears to be a trivial improvement. But consider that you also may be improving a large number of seamounts (metabolic and genetic bases) that you don’t see or study… so that in the long term, way beyond two years, you have much better long term survival that you can not see on a two year study. Of course, it all depends upon the range and three dimensional effectiveness of your drug or intervention.
Folks who want to reduce costs in health care love to use ARRs and NNTs because it causes nearly all interventions to look puny. But if we look at this other deep dimension of intervention we may be having a profound effect on the metabolic and genetic bases of future disease with our aspirin or whatever. If it affects the physiologic “seamount” beneath the disease.
I finished reading this article sure of only one thing: Dr. Aberegg may sometimes be wrong, but never in doubt. Like Mr. Solobian above, I believe the NTT is more understandable, and rhetoric about “they don’t care about the other people” and the equivalent of rants about pointy-head intellectuals is disturbing. Dr. Aberegg may or may not be right, but he certainly doesn’t want to listen to anyone else’s opinion. In my opinion.
Well, I will have to ponder this article….as it completely contradicts what I think. I believe NTT is much more understandable….and much better in that it explains that most people will not see a benefit from the tx….and likely are subjecting themselves to risk by partaking of tx…..
….as most people when they hear a tx reduces risk by 3% think that they personally will see that benefit.