By DAVID INTROCASO

In his THCB essay, “Why We Have So Little Useful Research on ACOs,” Kip Sullivan correctly notes we know surprisingly little about the ACO program. (While he identifies Medicare, Medicaid and commercial plan ACOs, here I’m referring specifically to the Medicare Shared Savings Program (MSSP) ACOs that account for two-thirds of all ACOs.)  Why there is little useful research is however not due to the two reasons Mr. Sullivan proposes.  To understand why we lack useful ACO research look no further than the agency that manages the MSSP.

Mr. Sullivan’s explanations are: since ACOs have been defined amorphously or aspirationally they cannot be assessed based on a prescribed set of activities or services; and, policy analysts have been “cavalier” program performance-related evidence.  Neither explanation is correct.  Medicare ACOs are defined regulatorily in great detail. This fact is made obvious by the, to date, 430-relevant Federal Register pages.  Generally defined MSSP ACOs are a model of care delivery that increasingly shifts financial risk from the payer to the provider in order to reduce spending growth and, though less definitively determined, improve care quality and patient health outcomes.  An MSSP ACO’s “prescribed activities” are simply to provide beneficiaries all necessary Medicare Part A and B services.  To define them beyond that or to expect the same precision or efficacy in delivering timely, comprehensive, population-based health care as administering a single prescription drug, as Mr. Sullivan would like, is impossible.  Arguing ACO researchers or stakeholders are “cavalier” about how best to define and measure the program ignores, among other things, the fact CMS received over 1,670 comment letters in response to the agency’s 2011 and 2014 proposed MSSP rules.

In arguing MSSP ACOs have no prescribed set of activities, Mr. Sullivan cites L&M Policy Research’s evaluation of the Medicare Pioneer ACO demonstration funded by CMS.  L&M’s primary purpose was to determine if the 32 Pioneer ACOs reduced Medicare expenditures in their first two performance years, or in 2012 and 2013.  L&M found they did.  How they did this was less clear.  As Mr. Sullivan notes, the best L&M could determine was lower spending growth appeared to have some relationship to greater provider engagement.  This does not demonstrate L&M’s research was, as Mr. Sullivan states, “useless,” or prove ACOs are amorphously defined.  L&M’s research simply lends further evidence to the fact how providers achieve spending efficiency varies as widely as their patients and the quality of their interactions with them.  Performance also varies because, among numerous other variables, providers range widely in their risk contracting experience, health information technology expertise and care coordination capabilities.  To assume spending efficiency can be predetermined or attained via a prescribed set of activities is to assume reasons are causes.  Any set of prescribed activities or services will not work equally well between and among providers.

Consider, instead, what CMS has done to enable providers to inform CMS about how best the agency can redefine the MSSP and improve providers’ own understanding of the program.

Recently, CMS published the agency’s third proposed MSSP ACO rule. The program’s first rule, that established the program, was finalized in November 2011.  The second rule was made final last June.  The current proposed rule, published in January and will likely be finalized this summer, attempts to improve how ACO financial benchmarks are calculated – the single most important factor in determining the program participation and success.  CMS has also revised twice the MSSP ACO program’s quality performance measure set via annual physician fee schedule rule making.  All this rule making is to CMS’s credit.  The agency is engaged in the difficult task of creating an Affordable Care Act-mandated program that will attract substantial provider participation, improve quality and substantially reduce Medicare spending growth.  The program is based on one related demonstration, the physician group practice demonstration, that achieved at best modest success.

Credit aside, CMS nevertheless has made it inexplicably difficult for providers and policy analysts to understand how the MSSP model works operationally thereby compromising stakeholder ability to comment intelligently on how best to improve the model.  For example, the current proposed rule provides not a single example to illustrate the many proposed computational changes to resetting and updating a MSSP ACO’s financial benchmark.  Specifically, how the agency proposes to use a regional/historical spending ratio to reset and update benchmarks and several other changes including how exactly CMS will recalculate a historical benchmark during an agreement period when a MSSP ACO’s provider composition changes thereby changing the ACO’s assigned beneficiary population.  Since there are no examples it’s impossible for providers to model with any confidence what these changes would mean to them.  In addition, CMS provides no evidence for commenters to evaluate how or why the agency’s proposed financial benchmarking and related changes will improve the model, facilitate improved provider performance and save the Medicare program money.  In addition, CMS is also not planning to conduct any public forums to discuss the proposed rule.  Well aware providers have complained for years about not having the ability to do their own financial analysis, CMS finally last October began to make available historical benchmark expenditure, average expenditure by enrollee type and risk score data to allow providers and policy analysts to trend or impute future financial performance benchmarks.

Apart from the black box aspect of ACO rule making, it appears CMS is not conducting an ongoing evaluation of the MSSP.  The agency to date has made no mention of conducting one.  Beyond the absence of an ongoing evaluation, CMS has made known little evaluative evidence about the program.  Of the limited number of MSSP public use date files CMS has posted, the two files that provides performance year (PY) data for 2013 and 2014 (PY 2015 data will be available late this summer) only provide generated savings and losses and quality score performance information.  CMS has conducted two webinars, advertised to ACO participants only, to discuss PY1 and PY2 results.  What performance assessment data is presented during these webinars is very limited.  For example, ACO utilization data amounts to a few slides comparing 12 utilization variables by ACOs that earned shared savings to those whom did not.  CMS provides no supportive narrative and the slides presented are made available to ACO program participants only.  CMS’s MSSP web portal and the agency’s MSSP newsletter, that provides program updates and upcoming webinar announcements, is also only accessible or available to MSSP participants only.

The MSSP has become the backbone of Medicare and payment reform generally.  ACOs are Secretary Burwell’s best hope to accomplish the Medicare program’s quality and value payment goals.  More broadly, the Medicare program is considered the market maker.  Medicare CMS’s unwillingness to be more transparent is therefore disappointing.  It is also difficult to reconcile how a program that is about moving Medicare providers to risk contracting is managed in such a way that, ironically, provider participants and participant hopefuls have been left unable to determine their risk capabilities.  As a result there has developed an apprehension about the program’s future. This likely explains why a third of MSSP participants dropped out this past December, why providers willing to participate almost uniformly refuse to sign an at risk MSSP contract despite CMS’s continuing efforts to offer them payment waivers and other incentives and why providers, knowing the Medicare program is inexorably moving to risk contracting, are increasingly interested in CMS’s other risk bearing program, Medicare Advantage.

David Introcaso is a healthcare policy consultant based in Washington DC. David’s acute care experience is via DC General Hospital and in post acute via in part his work with National Hospice and Palliative Care Organization, he consulted with a wide array of clients including the American Heart Association and the American Public Health Association and has taught as an adjunct at the University of Chicago and George Washington University. Over the past four years David has interviewed over 90 health care experts in producing independently a health care policy podcast