On the Non-Effectiveness of Cost Effectiveness Analyses

Nortin Hadler MD
Robert McNutt MD

Take the example of a middle-aged woman undergoing chemotherapy for breast cancer. Month after month she receives a bill for $16,000. This purchases a monthly infusion of one chemotherapeutic agent.  Much of the bill is paid by her insurance, but her personal checking account will cough up about $1000 per month until she pays down her deductible.

The invoice, however, is an illusion. The amount is not the actual number of dollars required to pay for services and materials rendered. Most of the money is diverted in accordance with contractual agreements between the hospital and various agents, brokers, and insurers. The total transfer of those dollars is known but not readily accessible to any who was not privy to the negotiations. The $1000 co-pay is a tithe with totally obscure added value to be paid no matter how painfully.

Few of us know the direct cost of any medical intervention. The term “cost” is the money needed to produce a service. For example, the cost of a single chest-x-ray is amazingly cheap, just a few dollars. The materials are inexpensive, the x-ray machine can spit out exam after exam, and one person can service a multitude of patients, making the unit cost a bargain. The cost of many a test is so low that it could be given away without wobbling a hospital’s budget.

Researchers have evaluated the true costs of some classes of medications, before marketing, distribution and middle-man charges are added. The cost to provide a pharmaceutical pale in comparison to what we pay. Parsing spread sheets in search of cost is very challenging. We do know that however the billing is construed, Americans pay far more for the same service than do residents of similarly resource-advantaged countries.  Determining cost should not be so onerous that reproducible numbers are elusive.  We suspect that few business leaders want us to know. The markups might astound if not infuriate us.

Markups from true cost are communicated as “charges”. Because charges are not costs, one person’s $16,000 charge may not be another’s. Charges are subject to differences in local markets, negotiating expertise; availability of insurance programs in a local area, and myriad other influences. Because charges are inaccurate proxies for true cost, charges are a misleading measure of patient care. This realization is the flaw in all the efforts and finger-pointing that surrounds debates over the costliness of healthcare in America. We misconstrue the “buck”, and, worse, do we know which “bang”, if any, we are purchasing?

Undaunted by the inaccuracies in the measures of cost of care, econometricians’ turn to cost-effectiveness analyses (CEA) as a means to make sense of the senseless. CEA explores whether expenditure for one intervention is as likely to result in a particular outcome as the same expenditure for another intervention. If not, one can calculate the cost/per added outcome. For example, if the outcome is a difference in longevity, a CEA may reveal the cost per year of a life saved. If there isn’t a difference in outcomes between two interventions, paying more for one would be folly. But if there is a difference, one cannot simply assume that the less costly intervention is the better choice.

To better understand CEA, consider this question; “What addition to standard care is costlier: a heart transplant or enteric-coated aspirin?” Students usually scoff and say, heart transplant, of course. But the answer is not so obvious. The question forces one to assume that there is no alternative to heart transplant since the standard of care has already been prescribed. Without the transplant death will ensue. The heart transplant will engender costs and charges in excess of doing nothing, but it has a potential for benefit. There will be a CEA ratio to ponder.

Enteric-coated aspirin provides no marginal benefit over regular aspirin; so even a penny more for enteric-coated aspirin would create a CEA ratio of infinity. Offering enteric-coated aspirin as an alternative to regular aspirin would not be cost-effective. Alas, such a violation is common. It occurs whenever we order tests that yield redundant information or continue treatments with long-term potential for a disease likely to be lethal in the short-term.

That’s why CEA offers little insight into the $16,000 bill this patient received, let alone the fraction that was consumed in the deductible. Any attempt at an explanation is thwarted; 1) determining the true costs of care is never easy and often not possible, 2) whether codified from Electronic Health Records or not, charges are a sorry representation of cost, and 3) using CEA to audit the healthcare system does not limit higher expense care. Furthermore, the notion of “how much is too much” exemplified by the heart transplant versus aspirin question is where CEA breaks down even further. If a CEA finds added benefit of one intervention over another, there must be a point when the marginal cost is too much for the marginal benefit.

But, what is this amount? We don’t know, and there is no consensus. In a review of 109 CEA studies, about 50% of the time, the higher expense strategy was chosen. The range of CEA ratios for those interventions were from $400 to $166,000 per added benefit. For interventions that cost between $60,000 to $166,000, calculations of added value lead to highly variable results, biased toward CEAs sponsored by industry. Without a defined “threshold” of how much is too much for a given added benefit, CEA becomes merely a smokescreen, which obscures debates on what matters most to patients and the practitioners treating them: the magnitude of benefit and the risk for harm.

We are purposely being diverted from what we should be doing in healthcare by debating dollars. The margin of benefit and harm of compared options for care should dominate discussions. In addition, who decides the value of a benefit and harm should be the patient, not the healthcare system. In our view, the more we explain benefits, the more people will tell us what the interventions should cost.

Robert McNutt, MD has been an associate editor at the Journal of the American Medical Association for 12 years and before associate editor at the Journal of General Internal Medicine. He is a professor of Medicine at the University of Wisconsin and Rush University Medical Center.

Nortin M. Hadler, MD is a graduate of Yale College and Harvard Medical School. He joined the faculty of the University of North Carolina in 1973 and has been a professor of medicine and microbioogy/immunology since 1985. 

Categories: Uncategorized

7 replies »

  1. Quotes from Barrons March 22 interview with health-care analyst Tom Tobin ” if Tobin is correct in his forecasts, today’s trend could mark the start of a profound shift for consumers and investors alike. ”
    “On the commercial side, high-deductible health plans and rising co-pays are reducing utilization.”
    “you’re going to see rising frustration turn into greater transparency on price. That can hurt some of the bigger players, but unlock a lot of value for innovators, and for consumers.”

  2. Steve 2., A Few Points:
    1. I agree patients have much room for improvement in choosing/evaluating value…in part because few Drs. are are able or inclined to practice like Hadler and McNutt.
    2. It Is well established high deductible employer plans reduce utilization and save $.
    3. This has happened in spite of the system constraints such as price opaqueness.
    4. Poor outcomes due to forgoing services are a concern, but haven’t been shown to have a net negative effect (note much preventive care is generally acknowledged to be uesless..such as annual physical).k

    Here is a brief sampling of thought, including critical pieces:

    “While spending on unnecessary services did not significantly change after a patient switched plans, the high-deductible plan did result in an average, annual $231 decrease on outpatient spending”….this in a Rand/USD study.

    “survey found that an ongoing shift toward high-deductible plans will continue, with 40 percent of employers offering one as the only plan option next year — an increase from last year. Nearly all employers — Washington post Aug 8 2017
    plans may be associated with a reduction in appropriate preventive care and medication adherence. Current evidence suggests that HDHPs are associated with lower health care costs as a result of a reduction in the use of health services, including appropriate services. “Health Affairs Oct 2017

    “More recent studies have shown that HDHPs and HSAs lead to spending about 5-7 percent less on medical care per enrollee. Most of these reductions come from reducing the amount of care consumed – not from shopping for cheaper providers. There is also evidence that individuals delay care, do not comply with doctors’ treatment plans and are unaware of free preventive services. Salon by HAEDER, THE CONVERSATION” (a piece arguing against such plans)

    “One firm that switched from a preferred provider organization to offering only a HDHP saw a reduction of between 11.79 percent and 13.80 percent in firmwide health care spending. This may be why 61 percent of large employers offer a HDHP, and about 9 percent offer it as their only plan option. The evidence thus far shows that consumers decrease their use of health care services under HDHPs. However, this reduction includes both care that is potentially wasteful and care that may be valuable to the consumer.”
    Suzanne F. Delbanco (Leapfrog founding CEO) Health Affairs blog Oct 26, 2017

  3. Patients are not currently good at choosing how to spend health care dollars. It will take a huge cultural change to make that happen.


  4. Paul, he also called it a ” tapeworm on the economic system” which is an apt description. Unfortunately, I worry that he wishes to feed the tapeworm his own mixture of sustenance with top-down control when that is exactly what has caused the tapeworm to grow so large.

    What we need are things like liquidity constraint. Considering a deficit of 21 Trillion dollars and growing (http://www.usdebtclock.org) it doesn’t appear that Washington knows or understands what such terms mean. If “every dollar has a constituency” that means we can’t have a solution for every dollar and every person. We need the patients to control the dollars. In general, the patient can better choose how to better spend his healthcare dollars than Warren Buffett can choose to spend those healthcare dollars for him.

  5. Re “Determining cost should not be so onerous that reproducible numbers are elusive. We suspect that few business leaders want us to know. The markups might astound if not infuriate us.”
    Yes, if you mean healthcare business leaders. As a corporate H.R. Guy I wanted to know, and I assumed our nonprofit insurer was working on our behalf to lower costs. But once on the Board I found out this was not their main interest. Healthcare eonomist H.E. Frech explained to me that the Blues were a funding mechanism for hospitals more than a purchasing agent for employers/employees,
    As Warren Buffett recently commented re his entry to reduce. healthcare expenditures: “every dollar has a constituency”….who will fight to protect their dollar while calling for cuts to the other guys dollar.
    In my opinion the essential element is for patients to have a direct financial benefit from thoughtful and curious and prudent use of healthcare services….guided by thoughtful, caring and prudent doctors (such as Drs. Hadler and McNutt).

  6. The expression of Parkinson’s Law is borne out by this scenario. It is driven by the large institutions that pursue market-share leadership priorities with an institutional co-dependency between the Complex Healthcare providers and the major payers. As a side-light, there is a small group of persons who pay first dollar, no matter what the ultimate charges represent. For a time (? current status), the Mayo Clinic in Scottsdale, Arizona did not accept persons who used Medicare for health insurance. The number of persons with foreign citizenship who paid first dollar was enough to fill their intake queuing requirements.

  7. “The materials are inexpensive, the x-ray machine can spit out exam after exam, and one person can service a multitude of patients, making the unit cost a bargain. The cost of many a test is so low that it could be given away without wobbling a hospital’s budget.”

    This stuff kind of bothers me. While there is some I can agree with in the rest of the article, have you watched X-ray techs do Chest X-rays recently? Maybe in the old days when we got them on everybody so you were mostly doing healthy people there might have been some real truth here, but now you have techs doing films on sick people. Not so easy getting that sick 80 y/o to cooperate to get a good picture. They you need an IT support structure to make sure the picture goes into the system properly and people can access it. You need a radiologist to read it in a timely fashion, and be available for questions. You need to make enough on these films so that you can support techs being able to go take emergent portable films as needed.

    Also, if you really want to talk about CEA, it would be nice to have it with the kinds of real choices we face, not the equivalent of a straw man like with transplant vs enteric ASA.