How Come Comparative Effectiveness Research is All the Rage?

Comparative Effectiveness Research (CER) is suddenly a hot topic at all the health care conferences. How come? Everybody agrees that we have to decrease per-capita cost and increase quality. Why? Government programs like Medicare and Medicaid foot more than 50% of our nation’s health bill, and if everything stays the same these programs will go belly up (bankrupt) in 8 years. Big problem.

Health and Human Services (HHS) has defined comparative effectiveness research as conducting and synthesizing research comparing the benefits and harms of different interventions and strategies to prevent, diagnose, treat, and monitor health conditions in “real world” settings. In other words, CER is figuring out what treatments, tests, and drugs work and which ones don’t work.

John E. Wennberg spent a whole career at Dartmouth studying American medicine, and he comes to the startling conclusion that 60% of Medicare is spent on supply sensitive care (physician visits, consultations, imaging exams, and hospital and ICU admissions) and 25% on preference sensitive care (PSA tests, mammography, and elective surgery). Although we assume that this care is based on solid scientific evidence, Wennberg states that “medical science is virtually silent on such matters” as how often to see a patient, what test to order, and whether to admit a patient to the hospital or ICU. Some evidence based medicine experts state that only about 20% of what physicians do is based on sound science.

The American Recovery and Reinvestment Act of 2009 contained $1.1 billion for CER, and the Patient Protection and Affordable Care Act of 2010 put in place a structure including a Patient-Centered Outcomes Research Institute to provide a continuous stream of funding and oversight to CER.

So we just need to do the research, figure out what works, and then have Medicare only pay for treatments and tests that work. That approach will solve the health care budget crisis and pay for care that is evidence-based. Right? Wrong. In the current legislation is language that states that CER findings may not be “construed” as mandates regarding payment or treatment or to deny or ration care.

A quick history of CER in the United States reveals how intense the politics around health care can become. Senator David Durenberger of Minnesota in the 1990s encouraged the government to fund Patient Outcomes Research Teams (PORT) to study the best ways to treat angina, low back pain, cataracts, and benign prostatic hypertrophy. When the 23 member expert PORT panel found little science to support surgery as a first line treatment for low back pain, the back surgeons lobbied Congress. The result was Congress cut CER funding for the PORT; one man’s waste is another man’s revenue.

One way to analyze the intensity of health care in the United States is to take a look at Medicare data for the last two years of life. The Dartmouth Atlas project that Wennberg founded had done just that. In the last two years of life, per-capita Medicare spending at UCLA is $93,842 per patient and $53,432 per patient at the Mayo Clinic. Many have suggested if we could get the entire country to treat such patients like the Mayo Clinic we could save $700 billion a year. Another study looking at the last two years of life found that patients in Newark, New Jersey spend about 35 days in the hospital; patients in Cleveland and San Francisco spend about 20 days in the hospital; and patients in Portland, Oregon, and Salt Lake City, Utah spend only 12 days in the hospital. If the doctors in Portland and Salt Lake City could teach the rest of us how they do it, much of our budget problems would be gone.

If CER is just trying to figure out what is scientifically the best way to diagnose and treat human disease how can anyone be against it?

Princeton health care economist Uwe Reinhardt writing in the New York Times economics blog identifies two groups opposing CER.

“The first group includes individuals or enterprises that book other people’s health care spending as their own health care income.”

“The second group…includes individuals who sincerely believe that health and life are ‘priceless’ – for them cost should never be allowed to enter clinical decisions.”

What seems clear is that American society needs to have a frank and honest discussion about CER, waste, and the American budget deficits. CER itself is not controversial. It is what you do with the results that create political tension and heat. The Kaiser Family Foundation stated the obvious when they wrote recently: “Ultimately, however, conducting research and gaining knowledge about what is clinically effective is only valuable if the findings are used by the health care system.”

Kent Bottles, MD, is past-Vice President and Chief Medical Officer of Iowa Health System (a $2 billion health care organization with 23 hospitals). He was responsible for the day-to-day operations of a large education and research organization in Michigan prior to his work with in Iowa with IHS. Kent posts frequently at his new blog, Kent Bottles Private Views.

18 replies »

  1. What you say about QALY’s is NOT true of comparative effectiveness research. It IS true of COST effectiveness analysis. Lots of CER studies (a shame they have the same acronym) is done with no attention to cost, although I personally believe that we need to get serious, as Kent says above, about the value/cost of our health care. I actually believe that this post reflects the general fear of comparative effectiveness, i.e. that it will lead not only to cost effectiveness analyses being done, but actually used to make, for example, coverage decisions.

  2. I also heard today that hidden in this bill is medicaid eligibility for anybody that collects an unemployment check.The Health Policy Brief, produced in conjunction with the Robert Wood Johnson Foundation examines some of the key issues concerning comparative effectiveness research.

  3. Where is the research comparing CER to traditional clinical trials, in other words, the CER on CER vs. other methods of medical study?

  4. For a discussion of the conflicting arguments about the way the QALY is used in England, see J G Taylor’s paper in Clinical Ethics at –http://ce.rsmjournals.com/cgi/content/abstract/2/1/50.
    He looks at the clash between utilitarian and egalitarian approaches to health care.
    The British government has recently announced a pilot of an approach to the pricing of medicines that will change the role of the National Institute for Clinical Excellence: http://bit.ly/bq9Ufr.
    An article by Polly Toynbee comments from a perspective well to the left of most US opinion on QALYs –
    Marc Taylor

  5. MG, as I’m sure you know, the term cost-effectiveness research (CER) does not specifically refer to cost-effectiveness analyses. CER is basically a synedoche.
    Dr. Bottles, let’s be clear about what you’re citing here, it’s not a peer-reviewed scientific article appearing in the prestigious NEJM. Nor is it a consensus view of the NEJM editors or some medical association. This is an opinion piece written by two Harvard CER researchers (Neumann was at Harvard for 10 years) whose careers (and, no doubt, funding) are built upon cost-effectiveness research utilizing the QALY. They are hardly objective on this issue.
    The ISPOR “Building a Pragmatic Road: Moving the QALY Forward” Consensus Development Workshop convened a Consensus Development Group (Michael Drummond, DPhil, Diana Brixner, BSPharm, PhD, Marthe Gold, MD, MPH, Paul Kind, Alistair McGuire, PhD, and Erik Nord, PhD) which published its conclusions in the 2009 Value in Health article “Toward a Consensus on the QALY.” In it they say:

    Given the current level of disagreement about the preferred approach [for measuring QALYs], the group considered that a QALY “reference method” that would inform the “reference case” concept … is needed if cost-effectiveness analyses were to be made more comparable across diseases and interventions.

    In other words, making the kinds of across disease/intervention comparisons Drs. Weinstein and Neumann extol as being made possible by the QALY aren’t really wise using the QALY. Why? Because a QALY isn’t necessarily a QALY.
    To researchers like Neumann and Weinstein, these are interesting questions to debate while the funding spigots flow. To policy makers, those of us practicing medicine, or citizens concerned about the future of health care in this country, they are fundamentally important questions that have not been widely debated.
    Academics/technocrats (heck, people in general) overestimate their understanding of complex systems and their ability to intervene in complex systems to effect desired change. Caveat emptor.

  6. “These aren’t proposals–these provisions are now the law of the land.”
    What’s the difference between a law and a proposal? If a law is not enforced then it is nothing more then a proposal. We have had 45 years of laws to control Medicare Spending, and yet here we are today.
    It is the law that your not allowed to file fradulent Medicare claims, yet we pay $60 billion or more a year of them. Certain people need to stop patting themselves on the back for suggesting we fix things and wait till they are actually fixed. Medicare has NEVER performed as favorably as “the Law” said it would, sooner or later we should acknowledge the trend.

  7. There is a huge difference between a cost-effectivness analysis and a cost-utility analysis that incorporates QALYs. QALYs are subjective and potentially problematic for the reasons you mentioned along with several other reasons including the method used to gather the QALY. There are also simple a number of medical conditions where there is no valid/recent QALY data available.
    Cost effectiveness analysis has its own set of limitations and problems but it is a pretty different tool than cost-utility analysis.

  8. Comparing the total Medicare reimbursements per patient during the last two years of life at UCLA and Mayo is problematic. These two hospitals serve very different patient populations in very different geographic settings.
    Go to The Dartmouth Atlas and look at the Medicare Spending and Patient Experiences listed there for the two groups of hospitals below. For convenience, I list below the total reimbursements per patient during the last two years of life at each hospital (i.e. the same statistic Dr. Bottles uses to compare Mayo Saint Mary’s and UCLA):

    Large Urban Hospitals

    UCLA – $93,842
    Rush University Medical Center in Chicago, IL – $82,075
    University of Texas M.D. Anderson Cancer Center in Houston, TX – $81,839
    Jackson Memorial Hospital in Miami, FL – $81,695

    Large Midwestern Hospitals in Small Cities
    Mayo Saint Mary’s Hospital in Rochester, MN – $53,432
    Sioux Valley Hospital University Medical Center in Sioux Falls, SD – $41,393
    University of Iowa Hospitals and Clinics in Iowa City, IA – $48,427
    BryanLGH Medical Center in Lincoln, NE – $52,359

    As you can see, the total Medicare reimbursements per person the last two years of life are very similar within each group but not between groups. This is true for Patient Experience data also.
    Does Dr. Bottles actually mean to say that all the large urban hospitals lack the dedication to cost-effective care that all the large rural hospitals seem to have? Occam’s Razor makes that conclusion vanishingly unlikely.
    Dr. Bottles’ dramatic comparison between UCLA and Mayo says more about their respective geographic settings and populations served than it does about their relative commitment to “cost effective” care.

  9. “Princeton health care economist Uwe Reinhardt writing in the New York Times economics blog identifies two groups opposing CER.”
    I believe there is a third group. This group is not opposed to CER (as I don’t believe the second is either). Just like the physician survey indicated, CER is welcome if it is done with the purpose of informing physicians instead of informing payers.
    And just like all other scientific research, CER should be debated, peer reviewed, repeatable and tested by time. If it passes muster, it will be incorporated into practice.
    As to the Dartmouth research, it is a bit different than most. Customarily, a cohort is selected, followed, measured and outcomes are recorded. You may then draw conclusion on comparative effectiveness of treatments. Selecting the outcomes and retrospectively comparing costs to achieve those outcomes in various locals with no consideration for cohorts and different outcomes is a bit counter intuitive for most people, particularly when the outcomes are death. In other words, the price of failure at UCLA is almost double that of Mayo. I am always left wondering about the price and prevalence of success at these institutions…..

  10. At the root of most comparative effectiveness research is the quality adjusted life year (QALY). The QALY is a dubious thing.
    In 2009, the International Society For Pharmacoeconomics and Outcomes Research (ISPOR) produced a supplement for its journal Value in Healthcare dedicated to the QALY. Health economist F. Reed Johnson, PhD wrote in that issue:

    The simplicity of the QALY as a universal health-care metric certainly has lead to its broad acceptance, but it comes at the cost of several limiting assumptions. The significance of these assumptions generally are poorly understood, or simply ignored, among practitioners and policymakers despite the assumptions’ repeated failure in careful tests of validity and reliability.

    The QALY, the foundational metric used by the majority of practitioners and policymakers for CER, requires accepting assumptions which have repeatedly failed tests of validity and reliability!

    CER itself is not controversial.

    If CER is not controversial, it sure outta be.

  11. ” . . . 25% on preference sensitive care.” Wow, I would think that should be the other way around. No wonder health care is so expensive and too often wasteful.

  12. Kent,
    Why don’t we just give Dartmouth the $1.1B allocated for CER and call it a day? They seem to have all of the answers (…well for end of life care anyway, that’s everything right?), and then they can manage the US healthcare system from Lebanon, N.H.. They can turn the entire state of New Hampshire into a call center where all physicians can be electronically linked to a motivational electronic shocking system. Every time a physician violates their research they will get jolted back into reality and prevented from making patient-centric decisions. Meaningful Use and the HITECH funds can all be diverted into making sure every doctor has a buzz-o-meter attached to their…well, keester. Wind farms will be needed to power the zapping grid and some physicians may need to be removed from the grid, but not treated unless CER calls for a Band-aid brand bandage.
    This will all be highly motivational and improve health care to be patient-centered, high quality, cost-effective with access and excellence for all!
    See, because I am sure they have figured out how to get 750,000 independent physicians, 5,000 hospitals and 15,000 nursing homes (that’s all, right?) to do what they believe is right…it’s a “no brainer”.
    Lastly we will all be saved from having to listen to their PR spinster talk about how great she/they are, and that will save (them) money too! It’s a win/win scenario, because apparently it is the only solution!

  13. Corrections to above comments:
    The second sentence in the first paragraph should read, “However, your final comment by the Kaiser Family Foundation is the key, and CER’s findings will NOT find translation into mainstream practice if they don’t have the heft of financial incentives behind them.”
    Also, the Annals of Internal Medicine article may be found at http://www.annals.org/content/153/8/551.extract. A Medscape article on it is at http://www.medscape.com/viewarticle/730805.
    Sorry for the inconvenience.

  14. Kent,
    It goes without saying that I fully support your comments. However, your final comment by the Kaiser Family Foundation is the key, and CER’s findings will find translation into mainstream practice if they don’t have the heft of financial incentives behind them.
    To this point, please see the letter by Keyhani et al. in the current issue of Annals of Internal Medicine reporting on the results of a national survey of physician views on the uses of CER. Most physicians (56% said they thought that CER would improve care quality. But 2/3 (66%) of physicians nationally agreed with the statement “Comparative effectiveness data will be used to restrict my freedom to choose treatments for my patients.”
    This is the perspective of a cottage industry, where each shop keeper makes his own rules, not an industrial perspective that is focused on identifying processes that deliver the best possible results.
    Sad but true, so long as American physicians have this mindset, without disincentives for poor results, it will be a long, uphill slog to improve safety, quality or cost.
    On the other hand, there are a slew of market-based approaches – medical destinations, minimally invasive surgeries, data collaboratives, value-based benefit designs, primary care empowerment, Health 2.0 applications – that are all rapidly gaining traction and that demonstrably drive down cost while improving quality. As purchasers become less and less able to shoulder the burden of cost that the old guard has insisted on, the market will change, and the kinds of progress you’ve been envisioning will become more and more possible and realized.

  15. Kent–
    Good post!
    Just one quibble–you write “If everything stays the same, Medicare goes bankrupt in 8 years.”
    But everything hasn’t stayed the same. Under the Affordable Care ACt Medicare will be trimming Increases in payments to hospitals, nursing homes and home health care by 1% a year. It also is slashing overpayments to Medicare Advantage.
    These aren’t proposals–these provisions are now the law of the land.
    According to Medicare Trustees, these savings give Medicare an extra 12 years before it’s hostpial fund begins to run out of money. (The hospital fund was the part of Medicare that was in danger.)
    MedPAC reports show that hospitals should easily be able to abosrb the 1% cut in payment increases. Hospitals under financial pressure have shown that when they have to, they can reduce waste, and redesign systems to become more efficient–so that they turn a profit on Medicare payments.
    Using comparative effectivenss reserach would help them go even further in lifting quality while cutting unnecessarily high costs.