ACO Performance Year Three: What Happened and What Does It Mean?

ACO Performance Year Three: What Happened and What Does It Mean?


Since Accountable Care Organizations (ACOs) are CMS’s largest pay for performance model delivering care to over 7.5 million beneficiaries, each August Medicare policy analysts await CMS’s press release summarizing ACO financial and quality performance for the previous year.  This past August 25, CMS announced 2015 results.  Like performance year one (2013) and two (2014), performance year three again produced marginal results.  Largely because, inexplicably, CMS is not evaluating the ACO program, once again analysts are left to decipher what performance results mean for the program, how success was achieved and what ACO performance means in context of the agency’s overall efforts at lowering Medicare spending growth.   

Summary of 2015 Performance Results

In CMS’s August 25th press releases, the agency noted 120 out of 392 2015 ACOs earned share savings. 1  CMS also releases annually a data file summarizing ACO participants, participation track, number of assigned beneficiaries, financial benchmark and quality measurement data.  Based on the data file, of 392 ACOs, 115 ACOs earned shared savings, 114 in Track 1 and one in Track 2, or 29% of all ACOs. 2  This compares to 26% in 2014. 3  Of the remaining 276 ACOs, one ACO earned shared savings but did not meet their quality performance standard, 86 ACOs produced savings but did not exceed their Minimum Loss Ratio (MLR) and 95 received reimbursements beyond their benchmark and fell within their negative MLR. The worst performing ACOs were the 95, or 24%, that received reimbursements that exceeded their negative MLR.  Had this last group not been in Track 1 they would have owed CMS half of their above benchmark reimbursements.  As in 2014 earned shared savings was highly concentrated.  Ten of the 115 successful ACOs earned $220 million in shared savings, or more than one-third of total $645 million in shared savings payments. 

Concerning quality performance, as in 2014, ACOs earning shared savings did not outperform all other ACOs.  The mean quality scores for the four subgroups ranged from 90% for those within their positive corridor to 92% for those that earned shared savings.  Once again there was no correlation between financial success and quality performance.  Despite high overall mean quality scores, ACOs dramatically underperformed on some measures.  For example, depression remission at 12 months, all condition re-admissions and stewardship of patient resources all had mean performance scores under 30%.  Among the 115 successful ACOs, 21 left money on the table since they did not have perfect, or above 90%, quality scores.  That is their quality multiplier was less than 1.0.  Another 19 successful ACOs were in their first agreement year, therefore, quality performance for them was measured simply as pay for reporting. 4    

What largely explained success was, again, having a comparatively higher financial benchmark.  Those that earned shared savings in 2015 had a per-member-per-year benchmark approximately $1,500 higher than the worst performing ACOs, again those that exceeded their negative MLR, or $11,413 versus $9,961.  Because of this successful ACOs were able to comparatively spend only $39 less per-member-per-year to be successful, or $10,557 versus $10,596.  (In 2014 successful ACOs underspent the worst performing ACOs by $29.) 5            

Success Was Achieved How?

While CMS details financial and quality performance results the agency does not explain, at least not publicly, how results, favorable or unfavorable, were achieved.  This is not a minor point.  The ACO, formally the Medicare Shared Savings Program (MSSP), is simply a payment model based on providers achieving a predetermined spending outcome.  Program participants are expected to translate the model into practice or find solutions that satisfy the new performance standard.  The MSSP lends further evidence to Henry Mintzberg’s observation that policy is frequently formed without being formulated.  Instead of providing formal evaluative results, what CMS does instead is conduct a webinar, presented by the Research Triangle Institute (RTI), for ACO participants only that summarizes performance year results.  The webinar slides, particularly the few that provide utilization trend information, are valuable because this is the only CMS analysis explaining what ACO providers are doing differently and because CMS does not make this information publicly available. 6

Not surprisingly the slides show ACOs that earned shared savings had markedly different utilization in certain categories compared to other ACOs, particularly the worst performers.  Short and longer term inpatient hospital expenditures along with ED visits that led to a hospitalization were down -9%, -14% and -12%.  For the worst performers these were up 6% and 28% and slightly down at -0.5%.  Total Part B expenditures for successful ACOs was up only 0.7% compared to 9% for the worst performers.  Skilled nursing, home health and hospice utilization were down by -21%, -8% and -10% respectively compared to 11%, 14% and 9% respectively for the worst performers.  For all four subgroups outpatient expenditures were up significantly between 13% and 21% percent as was rehabilitation hospital utilization or up from 5% to 20%.  Primary care visits with a nurse practitioner, physician assistant and certified nurse specialists were all also substantially up from between 43% to 53% while primary care visits with specialists increased only 1% to 3%.  Utilization in 2014 and 2013 was very similar to 2015.  Clearly, ACOs are working to reduce higher intensity care, over-all utilization including imaging and exploit mid-level professionals. 7


RTI explains “what” ACO providers did but not “how” they did it.  Since CMS is, again, not evaluating the program the “how” remains black box.  This also makes it at least difficult to improve program regulations.  In response to ACO provider criticisms, CMS did this past June revise ACO financial benchmarking to phase in regional expenditures and in the July proposed physician fee schedule rule, CMS offered allowing beneficiaries to voluntary attest participation in an ACO. 8  (ACO providers would still like to see program changes regarding risk adjustment, payment waivers and quality performance.)  The benchmark change would alleviate providers from having to improve upon historical performance but the migration to regional benchmarks may chase away those providers the program needs most, comparatively high spenders.  Beneficiary attestation is a sensible idea but by increasing assigned beneficiaries by, CMS estimates 0.2% to 2.7%, the agency is not going to alleviate the continuing problem of unstable assignment or patient churn.  RTI calculated on average only 71% of 2014 ACO beneficiaries remained in the same ACO in 2015. 9  Absent ongoing evaluation findings it’s anyone’s guess how ACO providers are innovating and/or how the program needs to be improved after three years of data that show savings are insignificant and financial and quality performance are unrelated.   

More broadly, the question is how does the ACO program compliment or align with the agency’s overall pay for performance efforts.  At best the answer is mixed.  CMS will allow ACO providers to  simultaneously participate in the agency’s Comprehensive Primary Care Plus (CPC+) demonstration. 10  This affords ACOs the opportunity to leverage or exploit their experience under another performance based payment model.  Under the recently proposed cardiac care bundled payment demonstration, CMS will allow ACOs the opportunity to gain share with demonstration hospitals. 11 This is a change in policy from last year’s Comprehensive Care for Joint Replacement (CJR) bundled payment demonstration that excluded ACOs from gain sharing.  These benefits are however far outweighed by the fact the agency’s bundled payment demonstrations, including Bundled Payment for Care Improvement (BPCI), compete with or undermine the MSSP because these demonstrations are redundant, both ACOs and bundlers are trying to reduce Part A and B spending growth, create lost opportunity costs for the ACO and oddly leave ACOs less accountable.  (As an aside whether bundled payment or some incremental form of capitated payment is the better path to sustainable health care financing was debated by Michael Porter and Brent James in the July/August issue of the Harvard Business Review. 12)  In addition, trying to equitably account for spending and savings for ACO beneficiaries also receiving care under a bundled payment arrangement is likely impossible.  While CMS recognized this problem in stating in the proposed cardiac demo rule, “it would be difficult for CMS at this time to provide standard program or model rules that would fairly distribute savings among different models & programs for overlapping periods of beneficiary care,” it’s difficult to be sympathetic since CMS created the problem. 13  Lastly, CMS will likely not recognize the vast majority of ACOs, or the 95% of ACO provider groups in Track 1, in the final Medicare Access and CHIP Reauthorization Act (MACRA) rule because Track 1s do not, per CMS’s definition, bear financial risk that’s in excess of a nominal amount. 14  Making matters worse, CMS recently announced providers can effectively choose to not participate in MACRA’s Merit-Based Incentive Payment System (MIPS) in 2017. 15  Since the MIPS-calculated payment update to Part B reimbursement is zero sum or cost neutral, Track 1 ACOs that thought they’d score comparatively well under MIPS and receive up to a four percent payment update in 2019 will now likely not because those that choose not to participate in 2017 are those who would have received a negative MIPS payment update in 2019.      

Thinking more comprehensively if ACOs, or the entirety of CMS’s pay for performance models, succeed to the extent the Department of Health and Human Services (DHHS) Secretary can say in 2018 DHHS met its goal of having 50% of Medicare payments tied to quality or value through Alternative Payment Models (APMs) like ACOs, the accomplishment will be compromised by the fact fee for service is becoming an increasingly smaller portion of the Medicare program. 16  Since the 2010 passage of the ACA the Medicare Advantage (MA) program has grown by 50 percent and most of MA plan growth, or well over 50 percent, is attributed not to commercial insurers but to providers or hospitals or health systems. 17  The MA program now accounts for approximately one-third of all Medicare beneficiaries and if growth continues at its current pace, MA will account for approximately half of all beneficiaries in the near future.  The Congress did make an attempt at incorporating MA in MACRA by requiring the DHHS Secretary to produce a study examining the feasibility of incorporating MA in the MACRA APM pathway.  The Secretary’s report, published this past July, was largely unilluminating because CMS is prohibited from, the report stated, dictating MA provider payment terms.  That is the Secretary cannot tell MA plans to have its physicians bear risk beyond a nominal amount. 18  MA plans, on there own, have given no indication they’re interested in either incenting their physicians such that they their MA contracts de facto constitute an APM or make publicly known their contractual terms such that the Congressional Budget Office (CBO) can score MA plans for budgetary savings. 

In sum, because ACO performance remains solidly mediocre particularly when you consider Part D  drug costs are not factored in to the program, CMS would benefit from formally evaluating the program and do so in a transparent manner.  CMS would also benefit by working to correlate quality  performance with financial results such that providers are better able to succeed.  The agency should work toward knitting together, or finding synergy between, the MSSP and the agency’s other pay for performance initiatives particularly its bundled payment demonstrations.  Per MedPAC’s 2014 recommendation CMS, should either find a way to level the playing field between the MSSP and MA or work with equal fervor to measurably improve MA quality and value beyond its current MA Value-Based Insurance Design (MA-VBID) demonstration. 19   

David Introcaso is a health policy analyst based in Washington DC.                                          

End Notes

1. CMS’s press release is at:

2. The ACO performance year three (2015) data file is at :

3. David Introcaso and Gregory Berger, “MSSP Year Two: Medicare ACOs Show Muted Success,” Health Affairs Blog (September 24, 2015), at:

4. See note 2.

5. “Medicare Shared Savings Program Webinar: Performance Year 2015 Quality Performance and Financial Reconciliation Results for ACOs with 2012, 2013, 2014 and 2015 Start Dates,” September 7, 2015.

6. Note 5 notes the title of RTI’s webinar slide presentation.  Again, these are not publicly available, the authors obtained a copy from an ACO participant.    

7. See note 5.  RTI’s performance year one slides are titled similarly, “Medicare Shared Savings Program Webinar: Performance Year 1 Quality Performance and Financial Reconciliation Results for ACOs with 2012 and 2013 Start Dates,” and performance year two slides are titled, “Medicare Shared Savings Program Webinar: Performance Year 2015 Quality Performance and Financial Reconciliation Results for ACOs with 2012, 2013 and 2014 Start Dates.”  The authors also have copies of these slide decks. 

8. The June 2015 ACO final rule is at:  The proposed 2017 physican fee schedule rule is at:

9. See note 5.

10. Information concerning CPC+ is at:

11. The proposed cardiac care demo rule is at:

12. Michael E. Porter and Robert S. Kaplan, “How to Pay for Health Care,” and Brent C. James and Gregory P. Poulsen, “The Case for Capitation,” both in the July/August 2016 issue of the Harvard Business Review, at: and

13. See note 11 at page 50919.

14. The MACRA proposed rule is at:

15. Andy Slavitt, “Plans for the Quality Payment Program in 2017: Pick Your Pace,” The CMS Blog (September 8, 2016), at:

16. DHHS Press Release, “Better, Smarter, Healthier: In historic announcement, HHS sets clear goals and timeline for shifting Medicare reimbursements from volume to value,” (January 28, 2015) at:

17. Avalere, “Nearly 60 Percent of New Medicare Advantage Plans Are Sponsored by

Healthcare Providers,” (January 26, 2016), at: file:///C:/Users/Owner/Downloads/1460404643_20160126_MA_Provider-Sponsored_Market_Entrants_Press_Release.pdf.

18. CMS, “Report to Congress, Alternative Payment Models and Medicare Advantage,” at:

19. Concerning MedPAC’s 2014 recommendations, see:  Information concerning CMS’s MA-VBID demonstration is at:

Leave a Reply

1 Comment on "ACO Performance Year Three: What Happened and What Does It Mean?"

Oct 2, 2016


Thanks for this write-up. I was, however, perplexed at your dismal view of ACO-EPM Overlap.

ACO concerns about overlaps in patient attribution rest on pricing issues, which can be addressed with risk-sharing arrangements between providers and refinements to EPMs. The current overlap rules allow ACOs and EPMs to co-exist, leveraging the strengths of each model.

With risk-sharing, this tool should (and does) allow ACOs to achieve market-by-market arrangements with EPM participants, avoiding the need for broad regulatory exclusions that may not fit each circumstance.