ACO Winners and Losers: A Quick Take

Last week, CMS sent out press releases touting over $1 billion in savings from Accountable Care Organizations.  Here’s the tweet from Andy Slavitt, the acting Administrator of CMS:

The link in the tweet is to a press release.  The link in the press release citing more details is to another press release.  There’s little in the way of analysis or data about how ACOs did in 2015.  So I decided to do a quick examination of how ACOs are doing and share the results below.

Basic background on ACOs:

Simply put, an ACO is a group of providers that is responsible for the costs of caring for a population while hitting some basic quality metrics.  This model is meant to save money by better coordinating care. As I’ve written before, I’m a pretty big fan of the idea – I think it sets up the right incentives and if an organization does a good job, they should be able to save money for Medicare and get some of those savings back themselves.

ACOs come in two main flavors:  Pioneers and Medicare Shared Savings Program (MSSP).  Pioneers were a small group of relatively large organizations that embarked on the ACO pathway early (as the name implies).  The Pioneer program started with 32 organizations and only 12 remained in 2015.  It remains a relatively small part of the ACO effort and for the purposes of this discussion, I won’t focus on it further.  The other flavor is MSSP.  As of 2016, the program has more than 400 organizations participating and as opposed to Pioneers, has been growing by leaps and bounds.  It’s the dominant ACO program – and it too comes in many sub-flavors, some of which I will touch on briefly below.

A couple more quick facts:  MSSP essentially started in 2012 so for those ACOs that have been there from the beginning, we now have 4 years of results.  Each year, the program has added more organizations (while losing a small number).  In 2015, for instance, they added an additional 89 organizations.

So last week, when CMS announced having saved more than $1B from MSSPs, it appeared to be a big deal.  After struggling to find the underlying data, Aneesh Chopra (former Chief Technology Officer for the US government) tweeted the link to me:


You can download the excel file and analyze the data on your own.  I did some very simple stuff.  It’s largely consistent with the CMS press release, but as you might imagine, the press release cherry picked the findings – not a big surprise given that it’s CMS’s goal to paint the best possible picture of how ACOs are doing.

While there are dozens of interesting questions about the latest ACO results, here are 5 quick questions that I thought were worth answering:

  1. How many organizations saved money and how many organizations spent more than expected?
  2. How much money did the winners (those that saved money) actually save and how much money did the losers (those that lost money) actually lose?
  3. How much of the difference between winners and losers was due to differences in actual spending versus differences in benchmarks (the targets that CMS has set for the organization)?
  4. Given that we have to give out bonus payments to those that saved money, how did CMS (and by extension, American taxpayers) do? All in, did we come out ahead by having the ACO program in 2015 – and if yes, by how much?
  5. Are ACOs that have been in the program longer doing better? This is particularly important if you believe (as Andy Slavitt has tweeted) that it takes a while to make the changes necessary to lower spending.

There are a ton of other interesting questions about ACOs that I will explore in a future blog, including looking at issues around quality of care.  Right now, as a quick look, I just focused on those 5 questions.

Data and Approach:

I downloaded the dataset from the following CMS website:


and ran some pretty basic frequencies.  Here are data for the 392 ACOs for whom CMS reported results:

Question 1:  How many ACOs came in under (or over) target

Question 2:  How much did the winners save – and how much did the losers lose?

Table 1.

Number (%)

Number of Beneficiaries

Total Savings (Losses)


203 (51.8%)




189 (48.2%)




392 (100%)




I define winners as those organizations that spent less than their benchmark.  Losers were organizations that spent more than their benchmarks.

Take away – about half the organizations lost money and about half the organizations made money.  If you are a pessimist, you’d say, this is what we’d expect; by random chance alone, if the ACOs did nothing, you’d expect half to make money and half to lose money.  However, if you are an optimist, you might argue that 51.8% is more than 48.2% and it looks like the tilt is towards more organizations saving money and the winners saved more money than the losers lost.

Next, we go to benchmarks (or targets) versus actual performance.  Reminder that benchmarks were set based on historical spending patterns – though CMS will now include regional spending as part of their formula in the future.

Question 3:  Did the winners spend less than the losers – or did they just have higher benchmarks to compare themselves against? 

Table 2.

Per Capita Benchmark

Per Capita Actual Spending

Per Capita Savings (Losses)

Winners (n=203)




Losers (n=189)




Total (n=392)





A few thoughts on table 2.  First, the winners actually spent more money, per capita, then the losers.  They also had much higher benchmarks – maybe because they had sicker patients – or maybe because they’ve historically been high spenders.  Either way, it appears that the benchmark matters a lot when it comes to saving money or losing money.

Next, we tackle the question from the perspective of the U.S. taxpayer.  Did CMS come out ahead or behind?  Well – that should be an easy question – the program seemed to net savings.  However, remember that CMS had to share some of those savings back with the provider organizations.  And because almost every organization is in a 1-sided risk sharing program (i.e. they don’t share losses, just the gains), CMS pays out when organizations save money – but doesn’t get money back when organizations lose money.  So to be fair, from the taxpayer perspective, we have to look at the cost of the program including the checks CMS wrote to ACOs to figure out what happened.  Here’s that table:

Table 3 (these numbers are rounded).


Total Benchmarks

Total Actual Spending

Savings to CMS

Paid out in Shared Savings to ACOs

Net impact to CMS

Total (n=392)

$73,298 m

$72,868 m

$429 m

$645 m

-$216 m

According to this calculation, CMS actually lost $216 million in 2015.  This, of course, doesn’t take into account the cost of running the program.  Because most of the MSSP participants are in a one-sided track, CMS has to pay back some of the savings – but never shares in the losses it suffers when ACOs over-spend.  This is a bad deal for CMS – and as long as programs stay 1-sided, barring dramatic improvements in how much ACOs save — CMS will continue to lose money.

Finally, we look at whether savings have varied by year of enrollment.

Question #5:  Are ACOs that have been in the program longer doing better?

Table 4.

Enrollment Year

Per Capita Benchmark

Per Capita Actual Spending

Per Capita Savings

Net Per Capita Savings (Including bonus payments)





















These results are straightforward – almost all the savings are coming from the 2012 cohort.    A few things worth pointing out.  First, the actual spending of the 2012 cohort is also the highest – they just had the highest benchmarks.  The 2013-2015 cohorts look about the same.  So if you are pessimistic about ACOs – you’d say that the 2012 cohort was a self-selected group of high-spending providers who got in early and because of their high benchmarks, are enjoying the savings.  Their results are not generalizable.  However, if you are optimistic about ACOs, you’d see these results differently – you might argue that it takes about 3 to 4 years to really retool healthcare services – which is why only the 2012 ACOs have done well.  Give the later cohorts more time and we will see real gains.

Final Thoughts:

This is decidedly mixed news for the ACO program.  I’ve been hopeful that ACOs had the right set of incentives and enough flexibility to really begin to move the needle on costs.  It is now four years into the program and the results have not been a home run.  For those of us who are fans of ACOs, there are three things that should sustain our hope.  First, overall, the ACOs seem to be coming in under target, albeit just slightly (about 0.6% below target in 2015) and generating savings (as long as you don’t count what CMS pays back to ACOs).  Second, the longer standing ACOs are doing better and maybe that portends good things for the future – or maybe it’s just a self-selected group that with experience that isn’t generalizable.  And finally, and this is the most important issue of all — we have to continue to move towards getting all these organizations into a two-sided model where CMS can recoup some of the losses.  Right now, we have a classic “heads – ACO wins, tails – CMS loses” situation and it simply isn’t financially sustainable.  Senior policymakers need to continue to push ACOs into a two-sided model, where they can share in savings but also have to pay back losses.  Barring that, there is little reason to think that ACOs will bend the cost curve in a meaningful way.

Spread the love

Categories: Uncategorized

21 replies »

  1. The accountable care idea catches us before we have had a time to think. What could be wrong with giving us an incentive to try to reduce costs in our population of patients? What could be wrong in rewarding ourselves with a little from these savings?…as long as these patients showed a little improvement in the ways we measure quality?

    What-is-wrong is that it keeps us from being nimble and exerting beneficence at scientific breakthrough times. After all, there a many times when it has been proper to increase costs in a large number of patients all at once. This ACO ideas, when you think about, is very regressive and conservative. If we were entangled in these incentives at the time when MRIs or CT scans rushed into medicine, how would we have given the best patient care? What about when cardiac echoes came along? Or fiberoptic examinations or surgery? Or when new testing allowed us to more firmly diagnose C. difficile? It’s easy to list many more occassions here when good medical practice demanded that we increase costs in many patients all at once. Now, e.g., we face the critical need to do certain genomic studies while treating malignancies: Her2, checkpoint inhibitors, etc.

    There have been many times when it would have been harmful to many of our patients to have been tethered to incentives that tell us to try to save money. It is a selfish idea that is an insurance or payer idea that has been shucked upon us when we were not thinking. I admit that it is also a selfish idea on our part to believe that can advance care willy nilly of costs. But we are agents of the patient and we have to at least TRY to serve his beneficence.

  2. To echo some of the points already made:

    1. Dr. Jha’s findings confirm findings from the NEJM paper that there are NO net cost savings to the tax payer (despite the PR coming from the most transparent admin ever)
    2. Of the 32 pioneer ACO’s that started in 2012, more than 50% have dropped out. Only analyzing those that remain and looking at savings is not intention-to-treat – its a very selected population. You absolutely cannot conclude from this that it just takes a while for folks to get it right!
    3. The point that is not addressed frequently enough is that there is a cost of administering/running an aco
    4. Evaluating quality of care delivered is very, very hard. Cardiologists are currently locked in a heated battle about appropriate blood pressure targets. Who should we screen for breast cancer? prostate cancer?
    5. The achilles heel of all of this is not focusing on the unit price side of the equation. Why does medicare pay ~$400 for an echo (from medicare)? Why does basic bloodwork cost so much? why is there one epi-pen manufacturer that has been FDA approved?

  3. Margalit hit it out of the park, as usual. The savings are phony. No one anywhere is saving money in a healthcare system growing faster than GDP. No one. The winners are likely those entities (because we are not dealing with physicians any more) who can game the “our patients are sicker” angle along with snarfing data. If the cost to benefit ratio gets too high, as it is flirting with now, it will disappear. There was really nothing wrong with FFS in principle except that it was corrupted by the self interested. There is nothing protecting any other “care models” from the same. Anyone want to try the doctor-patient relationship for a change? Remember when that used to work?

    • “want to try the doctor-patient relationship for a change? Remember when that used to work?”

      gee Leo, guess it’s up to me to break it to you that that never worked so well as in your honeyed imagination. But please, do go on fantasizing about the good old days when doctors were gods

  4. “Simply put, an ACO is a group of providers that is responsible for the costs of caring for a population while hitting some basic quality metrics. This model is meant to save money by better coordinating care. As I’ve written before, I’m a pretty big fan of the idea – I think it sets up the right incentives and if an organization does a good job, they should be able to save money for Medicare and get some of those savings back themselves.”

    From 2012 – 2015, cumulative expenditures on Medicare approximate $2.4 trillion. During this time, CMS says that cumulative savings from ACOs approximate $1.29 billion. That represents a savings of about 0.05% of cumulative expenditures over 4 years.

    As I watch my insurance premiums and out of pocket expenditures continue to rise, I cannot help but think there has to be a better way…

    • “I think it sets up the right incentives and if an organization does a good job, they should be able to save money ”

      Scott, the hope is the ACO will be able to save money and get a bonus by not getting an MRI when it isn’t needed. That is a good thought, but the other side of the coin exists as well. In order to get that bonus, they may err on the side of not getting that MRI or that cardiac workup etc. That is a dangerously placed incentive and known to be dangerous. Just look at the court cases from a number of years back when this perverse incentive was touted years ago.

  5. Kudos and thanks to Dr. Jha for venturing into the weeds of CMS data in search of reality. No matter how it is pitched, CMS has little reason to strut. Even Rand wonks could find little to crow about in the Pennsylvania Chronic Care Initiative.

    Far be it for American policy experts to learn from the experience of others. Britain plowed a great deal of thought, effort and largesse into their form of ACO experiment which commenced a decade before CMS decided to re-invent this wheel. The most striking outcome was in the behavior of physicians who found this practice structure a “distraction, diverting their gaze onto limited parts of clinical practice and reducing the focus on the patient’s agenda during the consultation” (NEJM 2014;370:1944-9). As a result, the NHS is re-tooling in hopes of extricating humanism from the flood of metrics (doi:10.1136/bmj.15539).

    Perhaps we should place our efforts into converting “healthcare” administration and bureaucracy into an infrastructure designed to facilitate a trustworthy, rational patient-physician interaction. It’s far less costly on the moral high ground (http://uncpress.unc.edu/books/12931.html)

  6. Thanks for the great article Ashish! While your analysis does not bode well for the ACO / MSSP model to-date, it’s an important first step towards getting us where we need to go. A two sided risk model will be quite a bit more palatable once organizations develop the necessary processes. The successful systems who formed Pioneer ACOs, the ones who stuck with the program at least – had been at this for decades. Value-based care is not something you just pick up in 2-3 years – the entire organization needs to refocus its way of thinking. So it’s no surprise that after just 3 years we’re not seeing stellar results. Let’s look at the results in 5-10 years, assuming we are able to stick it out for that long – and then see what conclusions we can draw. While they have not proven to be a magic bullet, the ACO/ MSSP model may have helped put participating providers on the right track to a value-based care system and preparing them to move away from the FFS world.

    • My understanding is that those that developed the equivalent of an ACO did so organically and all or at least most of the companies used internal models very different from one another.

      You are looking towards results in 5-10 years based upon those organically grown groups. I think that is erroneous for these newer groups are not being grown organically rather they are responding to regulations. The successful ones you refer to that had organic growth didn’t go through years of failure rather they demonstrated some measure of success. Those that didn’t do well disappeared.

      • Allan – You may be right – I currently work at one of those organically grown population-health focused systems, but it’s clear that there are many specialty-care focused, reputation based systems that are more than content to just pay lip service to population health and have done very well in the FFS world. It remains to be seen whether these gradually changing payer policies will be incentive enough to create agility in organizations that are notoriously resistant to change. If legacy health systems can’t change we’ll see one of two things happen – either the hospital and healthcare lobby will prove strong enough to slow or reverse the movement towards value based care, or 20 years from now we’ll have an entirely new crop of dominant players in the healthcare provider space. Lastly, let’s not forget that the movement of providers into the payer space is another way we can get there ACOs aren’t the only way to put financial risk in the hands of provider. Though this movement does little to address Medicare/Medicaid spending, there may be spillover benefits.

        • “ACOs aren’t the only way to put financial risk in the hands of provider.”

          Robert, I better understand where you are coming from. It appears you are asking physicians to become insurers. I don’t believe that is a good idea unless perhaps it happens organically and is in competition with other models where the physician is the direct agent of the patient.

          Placing financial risk in the hands of a physician can lead to denial of treatment which is the fundamental problem of the HMO model and all other models using the same incentive. The HMO denial process can be so subtle that no one knows when patients are being killed or injured. At least in the FFS model, there is a paper trail that is too frequently absent in the other model. That paper trail permits retrospective analysis. Further, it places the physician, hospital, insurer and government on one side leaving the weakest party, the patient, left all alone.

          I think to control costs one needs the patient to be substantially involved.

          • I don’t view the term ‘Provider’ as equivalent to ‘Physician’. I mean that term to represent the hospitals, healthcare systems, and physician groups. And yes, I do think that these organizations will assume financial risk for the care of their patients. The providers that can do more with less and provide cost effective care will win under this system. It’s true that the patient needs to be involved – I think if you look at the Pioneer ACOs you’ll see systems that understand that concept and are ready willing and able to engage patients and try to influence behavior – particularly among the so-called frequent fliers.

          • “The providers that can do more with less and provide cost effective care will win under this system. ”

            In this discussion, I am only including physicians whether they work singularly or in a group. I consider them different from other providers as in many respects the physician acts as an agent for the patient. To add to the complexity neither the physician nor the patient can be treated like widgets that are all identical.

            The ACO is using the same incentive as the HMO. How does one eliminate that incentive, the denial of necessary care? FFS leads to over treatment and the HMO incentive leads to denial of care. In general, an extra echo or MRI causes no harm to the patient and at times can be beneficial. Denial of the procedure can lead to a missed diagnosis and death. There frequently is no paper trail when something is denied and a large paper trail including the observations of individuals where over treatment is concerned. Denial of care is much more dangerous and not as obvious or detectable as over treatment.

    • Why should they “move away from the FFS world”? Is there any evidence that the FFS model is wasteful? Or is this just a “gut feeling” of some people who happen to be in a position of power to do whatever they want with (our) 3 trillion dollars?
      What are they going to do when this whole thing comes crashing down? Apologize (as one recently did)?
      Is it possible that the FFS model is just fine, but the pricing is failing, so the first F does not represent the value of the S very well? Wouldn’t that be a lot easier to address?

      • “Is there any evidence that the FFS model is wasteful?:”

        Margalit, I’d ask if you are serious, but sadly, I know you really mean this

  7. The annual cost of our nation’s healthcare industry continues to increase faster than the expansion of our nation’s economy. Adding to that economic anomaly is the fact that our medicare eligible group of citizens will dramatically increase over the next 15 years. We are realistically facing severe and painful rationing without a completely revised strategy for healthcare reform. Annually, we continue to borrow money to pay for the cash-flow requirements of the Federal government. Within ten years, our credit won’t be recognized as dependable, and we’ll be left borrowing money just to pay the interest on our international debt.
    CMS continues to play the shell game to the discredit of our nation’s ‘autonomy’ within the world’s market-place arenas of RESOURCES, KNOWLEDGE and HUMAN DIGNITY. When will it ever stop? Well, the next recession is looming, and we will soon learn how many health systems go-under from our financing of the ‘economic mandate’ for complex healthcare needs of each citizen while we neglect the ‘social mandate’ for the basic healthcare needs of all citizens.
    Before Medicare-Medicaid, our nation’s healthcare represented 6% of the national economy. It’s now 18%. Assuming an efficient cost at 13% of the national economy instead of 18%, the difference last year represented an excess cost of $800 Billion. Of the excess cost, the Federal government paid 40% or $320 Billion. It represented the cost of 3 Iraqi wars fought simultaneously.
    My deep gratitude to Ashish for his clarity about the ACOs.

  8. I don’t quite understand the conclusion, or final thoughts.
    First, I don’t think the longer standing ACOs are doing better. Savings are an artificial measure. The actual cost per patient is higher in the veteran ACOs, so they are more expensive. It’s hard to believe that somehow people signed up for the program precisely in inverse order of the average health of their patients.
    Second, whatever the results, I don’t see how imposing two sided risk will change anything for the better. Either ACOs will drop out, go bankrupt, or shortchange patients, with the latter being the most likely case. As others said, the infrastructure and operations costs to the ACO are not negligible, and I seriously doubt any of the vendors they are forced to engage will accept two-sided risk.

    If CMS decides to force doctors into becoming miniature insurance companies, I would want to see clear COI disclosures posted in every clinic and every hospital, so patients know that the patient-doctor relationship is now essentially adversarial.
    Let’s see how that goes over with the public…

  9. Without a doubt the BIGGEST flaw in all these calculations is NO ONE is telling us how much it COSTS to setup and maintain their ACOs. This is a ENORMOUS issue. If we are talking a few bucks in savings per beneficiary, and it costs practices many times that to setup and run an ACO, then it will NEVER work. Also, it is false to assume just because the ACOs that started in 2012 means the longer you are in, the better you do, that is just silly. There are many more possible reasons that the 2012 group has a tiny savings.
    All in all we are missing a HUGE factor in whether ACOs will work. The COST to implement and run an ACO. Will ANYONE talk about that? It appears to me that is the MAIN reason Pioneer failed and many others have left the ACO game.

  10. When you do look at the two sided model, you can’t just look at the bottom line for CMS AND THE TAXPAYERS, you have to look at the take home pay for the professionals and other employees of the ACO, counting all the new infrastructure costs that the ACO has demanded.

    If the ACO saved the government a few hundred dollars per patient, and the ACO gained half of this for its professionals, it would appear a win, but it might not be a true win if the group had to hire new accountants and work longer hours.

Leave a Reply

Your email address will not be published. Required fields are marked *