Making Accountable Care Organizations Great Again

Test your Accountable Care Organization knowledge:

  1. Select all of the options that are True
  • If you belong to an ACO you qualify for the alternative payment model track in MACRA
  • Most ACO’s are enrolled in to a two sided risk model – (They share profits and losses with Medicare)
  • In 2015, ACO’s saved Medicare more than a billion dollars
  • None of the above

I have been mystified by Accountable Care Organizations ever since I first heard of them almost a decade ago.  ACO’s have had a hallowed place in the world of health care policy for some time now.  Everyone knew they were coming, and everyone knew they would be the answer.  Traditional fee for service medicare that paid based on volume was thought to be the driver of rising health care costs.  Regional variation in medicare expenses even when controlling for underlying population risk suggested that lower costs were possible without sacrificing quality.  Imposing a capitated model of reimbursement tied to quality metrics seemed to be the answer and ACO’s were the vehicle to make this happen.

I was on board, though I freely admitted to everyone who asked, that I had no idea how they would work.  To be fair, I was far more stressed about learning coronary anatomy in the cath lab, than I was about health care policy.  By the time ACO’s finally started in 2012, I had been in practice for a few years, and I was paying slightly more attention.  The passage of time now allows for the assessment of the value delivered by ACO’s and is a health care policy researchers dream.  Clinicians, however, continue to be blissfully unaware of the construct. ACO’s exist in some alternate universe that is interesting and worthy of name dropping to establish your policy-cred, but even the coarse details are shrouded in mystery for most clinicians and patients.  What follows is a brief primer and analysis of some recent data that doesn’t require an understanding of linear regression modeling.

Basic Background

ACOs are organizations that consist of groups of providers made responsible (accountable) for the cost of patients, and the quality of care provided.  There are two ACO programs: Pioneer and the Medicare Shared Savings Program (MSSP).  The Pioneer program is an initiative that springs from the Innovation center at CMS (CMMI) – it was created by the Affordable Care Act (passed in 2010) to test new models of health care delivery and payment.  And test they have.  32 high performing organizations were carefully selected to participate in this model in 2012.  By 2015, only 12 remain and consequently the only news I see about Pioneer ACO’s is about the latest dropout from the program.  The lessons of the failure of the Pioneer program, however, are quite instructive.

The Pioneer Program

The Pioneer programs were set up to test how to break the the-more-you-do-the-more-you-make fee for service, volume based approach that has been the hallmark of Medicare since its inception.  To do this, ACO’s were assigned a capitated amount per attributed patient – a so called benchmark.  This benchmark was calculated based on CMS expenditures for patients in the 3 years prior as well as expenditures for the national Medicare population.  At the end of the year, actual medicare expenditures for each ACO would be compared against their benchmark.  The Pioneer model was testing two sided risk – spend less money than the benchmark and you received a share of the savings, spend more money than the benchmark and you may owe money to CMS.  In order to tie value to the cost side of the equation, Pioneer ACOs were also graded via ~30 performance metrics.  The results on the financial front were troubling from the start.

In 2012, 19/32 (60%) of the carefully selected high performing organizations ended up spending more money than their expected benchmarks.  Because of rules I do not fully understand only one of the ACO’s that ‘overspent’ relative to their benchmark had to share the losses and pay CMS.  In 2013, 23 ACOs remained, 17/23 spent more than their expected benchmark, and 6 had to pay medicare $ values ranging from $1 million to $2.8 million (Figure 1).  Faced with the possibility of these types of losses, health care systems did what you would do if you were playing a game where you had the potential to lose a few million dollars at the end of the year – they stopped playing the game.  By 2015, only 12 ACO’s remain, and even in this best of the best group 1 ACO spent $14 million more than their benchmark said they should.


Figure 1. 2013 Pioneer Data (Source: CMS)

This is not what healthcare systems signed up for.  The health systems were probably under the impression that providing high quality care as measured by metrics would result in financial rewards.  They were gravely mistaken.  Lowering costs has little to do with hiring a performance improvement team to make sure that ACO metrics were being completed (ACO metric #18 – screening for clinical depression).   Not surprisingly, health systems complained as they were exiting the pioneer  program about how benchmarks were calculated – a relevant complaint that I’ll discuss shortly.

Medicare Shared Savings Program (MSSP)

Unlike the shrinking Pioneer program, the other major ACO program – the Medicare Shared Savings Program (MSSP) has been adding participants by leaps and bounds.  It should be easy to guess why.  MSSP programs introduced two tracks initially.  Track 1 involves a one-sided risk model. I call this the heads-I-win-tails-you-lose-track.  ACO’s in this track may share in what they save relative to their benchmarks, but have no penalty if they overspend.  This is like going to a casino where you can only win money – losses are on the house.  What a delicious world that would be!

Track 2 was the non fantasy world of the Pioneer ACOs – a 2 sided risk model where the shared savings were slightly higher than track 1, but spending too much meant paying millions of dollars back to Medicare.  Track 1 was always meant as an on-ramp to track 2. The idea was that organizations could test and refine their care models without significant downside risk initially before maturing eventually to take on the greater downside risk in track 2.  Of the 393 ACO’s data is reported on in 2015, only 3 chose track 2.

The same model that applied in the Pioneer model applied here: ACO’s were compared to benchmarks to decide on savings, and quality metrics were required to be reported.  The Pioneer experience should have predicted the outcome here.  If mature organizations cherry picked by Medicare largely failed, what chance would the MSSP programs have?  The 2015 MSSP performance on the currently participating 392 ACO’s is available here for perusal.  Ashish Jha, a prominent public policy researcher also provides an excellent summary of the data on his blog here.

MSSP 2015 Results

There were 203 ACOs that spent less than their medicare benchmark and thus ‘saved’ money (winners).  189 ACOs spent more than their medicare benchmark and lost money (losers).  There were more winners than losers in 2015, and, on the face of it, ACO’s in a one sided risk model appear to have saved the taxpayer $429 million (Table 1)

Savings (Losses) ($)
Winners (203) 1,568,222,249
Losers (189) -1,138,967,553
Total 429,254,696

Table 1. MSSP 2015 ACO Savings/Losses

Recall, however, that Medicare was to share their savings with the health care systems.  This meant that about half of the $1.5 billion saved was returned to the winners in the form of an end of the year bonus.  Medicare paid out $645 million dollars to ACOs in shared savings in 2015.  Account for these payments and the taxpayer is in arrears to the tune of $216 million dollars in 2015 because of ACO’s (Table 2).

Savings (Losses)
Winners 1,568,222,249
Losers -1138967553
Shared savings payments to ACO’s -645543866
Net Medicare savings (loss) -216,289,170

Table 2. MSSP 2015 Savings/Losses inclusive of Medicare shared savings payments

Numbers should bring certitude, but in this case that certitude would be misplaced. The savings are somewhat of an illusion.  Winners and losers are declared based on a made up number – the benchmark.  This benchmark is Medicare’s best guess of what the assigned/attributed patients should have cost.  No one can truly attest to the veracity of the benchmark.  A health system could invest in all the right infrastructure, hit all the quality metrics asked of it, but still come out a loser if the benchmark gods give them a bad  number.  It is of interest to note that your chances of being declared a winner are much higher if your benchmark is high.  In other words, the traditionally more expensive per capita regions were the most likely to demonstrate savings.  Conversely, low cost regions were much more likely to lose money. (Figure 2).

My advice to health care systems in regions where medicare benchmark expenditures are < $10,000 / capita? Stay in a one sided risk model (MSSP track 1) as long as you can!


Figure 2. Winners and Losers by per capita benchmark

The Delaware Valley ACO

There is no better example of the capriciousness of the selection of winners and losers than the former Fresh Prince of Bel-Air’s home turf – Philadelphia.  This now happens to be my neighborhood, and the home of 9 ACO’s.  The results for 2015 for the group were fairly poor.  Only three of the ACO’s saved money – two by razor thin margins – and no one qualified for a bonus.  The largest ACO – the Delaware Valley ACO (DVACO) – with ~70,000 patients attributed – spent more than $15 million over their benchmark in 2015.  This, ignobly, put them in the top 20 losers nationally.  This was in stark contrast to the news in 2014, when the DVACO received a bonus from Medicare for coming in below their benchmark.

What happened? The statement from the DVACO blamed the losses on doubling the number of beneficiaries from one year to the next, and working with different populations of physicians and providers.  Interestingly, the DVACO actually spent less per beneficary in 2014 than they do in 2015.  In a health care cost climate of ever rising healthcare costs, this should be cause for celebration.  The problem, of course, was that the calculated benchmark was almost $800 lower per capita from one year to the next.  So even though the DVACO spent less per capita than the prior year, they overspent $211 per attributed patient.  Certainly, this could be because the expanded patient population of the DVACO in 2015 had a lower acuity level than the prior year.  Or it could be because the expanded patient population don’t have their acuity documented as well.  Who really knows?

Regardless, this is where the size of the ACO works against it.  The same size that should theoretically allow for economies of scale in the form of better care coordination and more control over expenditures now ratchets up the level of risk astronomically.  A ~$211 per capita loss with 70,000 patients means a total loss of $14 million.

Benchmark/capita ($) Expenditure/capita ($) Savings/loss ($) Total savings (loss)
DVACO 2014 11,852 11450 402 $13,402,585
DVACO 2015 11046 11257 -211 -$14,894,475

Figure 3. Comparison of Delaware Valley ACO expenditures (2014-2015)

Thank goodness the person in charge of selecting the MSSP track chose the right box. Recall that almost every ACO chose Track 1 for this very reason – to insulate themselves from the risk of having to write a check to medicare if they overspent relative to their benchmark.  Clearly calculating the benchmark correctly is vital. A discussion of how exactly the benchmark is calculated is beyond the scope of this post and likely beyond the scope of my frontal lobe, but with the number of variables in play in predicting future per capita health costs, the odds of landing on the right number remind me of my unsuccessful attempts at the roulette table.

While track 1 is good for ACO’s, it is not good for Medicare.  Medicare pays shared savings bonuses without recouping any money from ACO’s that spend more than their benchmark.  It is thus little surprise that Track 1 of the MSSP is not considered an Alternative Payment Model (APM) by the new proposed payment raj that begins in 2017. Track 1 was always supposed to be an introduction to ACOs that would ultimately give way to a 2 sided risk model.  This is a real problem for health care systems that in total lost $1.1 billion dollars in 2015, and would face massive liabilities under a two sided risk model if nothing changes.

But those concerned with the poor performance of ACOs will be rescued from their doldrums by Dr. Michael McWilliams, a Harvard policy guy who in a recent paper takes aim at benchmarks to defend the maligned performance of ACOs. He gives 3 explanations

  1. Provider responses to ACO contracts affect care of patients served by ACO’s but not attributed to these organizations
  2. ACO spending reductions reduce ACO benchmarks because the lower the spending growth rate used to update benchmarks each year
  3. Spending reductions by ACOs lower Medicare Advantage spending because payments to medicare advantage plans are tied to local fee for service spending. Declines in fee for service spending means lower payments per capita to medicare advantage plan

To summarize his argument : ACO’s are actually saving lots of money – its just that we can’t tell methodologically because the benchmarks aren’t right.  If you think that this sounds like grasping at straws, that was my first impression as well.  The policy folks have  a problem – their world rests on a conceptual framework that at its core places the ills of health care on the fee for service (FFS) model Medicare was born into.  Unfortunately for them, growth in spending in the traditional all you can eat buffet of Medicare FFS has had a sustained surprising slowdown.


Figure 4. Medicare spending per beneficiary trends

This was a completely unexpected phenomenon.  Inexplicably, providers that get paid per encounter, procedure, and test are simply doing less.  I am one of those providers, and I can unequivocally say that we do less today than we did ten years ago.  We place fewer stents, do fewer stress tests, and admit fewer people to the hospital. (Figure 5)


Figure 5. Changes in volume of health care delivered over time (Source – Congressional Budget Office)

Is this happening in my locale because of the Delaware Valley ACO that started a few years ago, or because of a larger cultural shift driven by conversations on cost that began years prior?  It’s definitely not the ACO, but I don’t know the right answer, and really no one can because of how broadly alternative payment schemes were implemented.  My very strong belief is that the simple act of paying attention and discussing cost has resulted in changes in provider (hospital/physician) practice patterns.  Perhaps the beauty of the ACO is not getting anything right, but simply threatening to get it right.  The Hawthorne effect – when individuals modify or improve an aspect of their behavior in response to their awareness of being observed – is certainly a well described phenomenon.

Academics, of course, in a feat of circular reasoning, are now trying to prove that the slowdown in traditional medicare is because of alternative payment schemes.  This is the crux of an argument you will hear over and over again for all the alternative payment schemes constructed – if ACOs are reducing traditional medicare costs, it isn’t fair to judge the success of ACOs relative to traditional medicare.  This certainly may be true, but how do you go about proving this?  You can’t.  But you can publish from the high castles of academia in peer reviewed journals, demonstrate correlations, imply causation, and then point to evidence when it comes time to ask the taxpayer to fund said programs.  Understand that by these yardsticks, I could probably ‘prove’ to you that we are spending less because I wear black pants to work on most days.

I get the sense that Dr. McWilliams already knows alternative payment schemes are responsible for moderating health care costs – he just has to prove it.  His solution is a better benchmark.  I remain unconvinced.  Despite the ‘savings’ that are said to have accrued with alternate payment schemes that relate to the ACA, health care cost moderation clearly precedes the passage of this bill, and 2014 data shows health care costs on the rise again due largely to the expansion of coverage the ACA triggered.  It has to be said, if not for the larger overall moderation in health care costs – the actual rise in health care costs would have been significantly higher.

The recent growing ‘evidence base’ emerging to allow alternative payment schemes to take credit for cost moderation will  make a lot of people feel good.  Physicians who bear much of the regulatory, administrative and cost burden of these schemes shouldn’t be fooled.  Open your eyes.  Zoom in.  And feel free to roll your eyes when you run into the next acolyte of our new world talking breathlessly of Accountable Care Organizations.

Anish Koka is a Cardiologist in private practice in Philadelphia.  He has developed an allergy to alternative payment models recently.  Answer to multiple choice question: None of the above.


Categories: Uncategorized

Tagged as:

8 replies »

  1. I don’t understand how CMS sets the benchmark for the second year of ACO participation. While I can understand an adjustment based on any change in the average risk score of the ACO population from one year to the next, it seems that the primary factor should be the average increase in Medicare per capita spending for the region where the ACO is located. So, if per capita spending grew 1.5% in the region which was the average nationwide growth rate for 2008-2014, then the benchmark should increase by 1.5% as well if the average ACO population risk score remained constant. What am I missing here?

  2. To clarify, if you wanted to do this then there are more efficient ways of doing this. Clearly incentivizing physicians to do less has its own problems..

  3. My understanding is that you’re paid through the year in a ffs manner (not all your patients will be aco patients), and at the end of the year, depending on how you did per capita on your aco patients, you get a bonus check (or write a check if you foolishly chose track 2). Of course, I’m not sure most folks know who exactly their ACO patients are – ideally everyone should send their non aco participants for lots of tests,to drive up the benchmark, and only give aspirin to the ACO folks! In reality, physicians prob try to be parsimonious to everyone, which is Dr. Mcwilliams point about ACOs effecting the benchmark. I’d argue there are less convoluted, less expensive ways of doing this..

  4. Right. Regarding FFS. I think the FFS is origin of all evil is a mantra that everyone chants. I say over and over again, there are 2 sides to incentivizing lots of service. Of course, It would be nice if we as a physician group had some mechanism to deal with outliers? Lastly, meltoots, do you think I need a pseudonym to keep writing ? (Serious question – are there real concerns I’m missing by being transparent about who,where I am?)

  5. Figure 5 would argue, – and it certainly feel this way as well – that we are doing less, that the culture is shifting because of the conversation – or because of the threat of value based care? Who knows. Regardless, and not surprisingly, everyone wants to take credit for this changing practice pattern..

  6. The issue is essentially culture. For cardiologists, the cath was the “gold standard” Why would you not get one for your patient? Not doing one during the hay day was derelict. See a blockage: put a stent in it. Hear a murmur: get an echo. Why would a physician not evaluate a problem using the latest literature supported technology? Never mind that CMS has paid handsomely for all this over utilized technology. Do they really think that a group of managers in an ACO is going to control this? They are going to have to dig in much more deeply to control culture. Perhaps control the literature, silence “thought leaders”, stop paying for advances in technology, punish hospitals for hosting such things. The same thing is happening to mammography right now. If you didn’t order a mammogram for your patient, you were a war criminal. Or were you being judicious? No one can tell until the pendulum stops swinging.

  7. Excellent article. Very complicated attribution and math, and still cannot make the numbers positive. Spin all they want, but ACOs are obviously spawn from HMOs and those in policy circles that want to use buzzwords like Accountable, Value, Quality, etc. The learn to teach to test, click the right boxes for quality (all self reported) to show quality and then IRS the heck out the costs, and still no win.
    Turns out, in the US, paying for a service, i.e., FFS, is really the best method. I think DC, thinks that a majority of MDs are out there running up the bills. Not true at all. Always a few outliers, but they get caught and shut down. Without any help of CMS, we work tirelessly to improve our workflows, care, outcomes. Its what we do.
    CMS wants to spin that the longer the ACOs stay in the game, the better chance of winning, but again, its skewed stats. The losers already left. Soon the others will as the floor shifts and there is no way to ratchet down costs more.
    In the end, play it out, ACOs are trying to keep patients from spending money on care. Period. And that always rears its ugly head at some point.

  8. As capitation is the method of paying the doc, is it also the method of paying for their subsidiary referrals to other non-member docs, to the hospital and imaging centers, etc? Who pays the claims that the MRI or lab sends and where are they sent? Are these sent to Medicare or are they sent to the ACO? Are the docs paid every month by capitation or is there a yearly payment and accounting of savings and over-expenses? …and if the latter, what do the docs live on in the meantime?

    It must take months to calculate the benchmark by looking at old claims and payments. How do the docs get dough while they are waiting for the benchmark?