The verdict is in: All three of CMS’s “medical home” demonstrations have failed

Between September of 2016 and last month, CMS released “final evaluations” of all three of its “medical home” demonstrations. All three demos failed.

This spells bad news not just for the “patient-centered medical home” (PCMH) project, but for MACRA. The PCMH, along with the ACO and the bundled payment (BP), is one of the three main “alternative payment models” (APMs) within which doctors are supposed to be able to find shelter from the financial penalties inflicted by the MIPS (Merit-based Incentive Payment System) program which was recently declared to be unworkable by the Medicare Payment Advisory Commission. Medicare ACOs and virtually all Medicare BP programs are also failing. Thus, we may conclude what some predicted a long time ago – that neither arm of MACRA (the toxic MIPS program and the byzantine APM program) will work.

In this post I describe each of CMS’s three PCMH demos, review the findings of the final evaluations of the three demos, and then explore the reasons why all three demos failed. I’ll conclude that the most fundamental reason is that the PCMH is so poorly defined no one, including the doctors inside the PCMHs, knows what it’s supposed to do. That’s not to say that the hopes and dreams of PCMH proponents were never clear. They have always been clear. PCMH proponents have said over and over the PCMH is supposed to lower costs and improve care. But a clear expression of hopes and dreams is not the same thing as a clear description of what it is you’re dreaming about.

I will close with the recommendation that policy makers should eliminate sprawling “reforms” like the PCMH and the ACO which target entire populations and replace them with pilot projects that focus on specific types of patients.

Review of the three demos and their final evaluations

The first “home” demonstration out of the chute was the Federally Qualified Health Centers Advanced Primary Care Practice Demonstration (referred to hereafter as the FQHC demo). It ran from November 2011 to November 2014. CMS published the final evaluation of this demo in September 2016. That evaluation, written by scholars at the RAND Corporation, concluded that the demo raised costs, had no impact on quality, and was associated with a worsening of burnout among staff. Here is an excerpt: “Demonstration sites were associated with significant increases in total Medicare expenditures during Year 3 relative to comparison sites and cumulatively when care management fee payments were excluded from the analysis (p<.01). When the fees were included, total Medicare expenditures were significantly higher in demonstration sites (p <.05) … [B]eneficiary surveys identified few significant differences in outcomes for demonstration and comparison FQHC patients.” (p. xxii) The report also noted that PCMH staff experienced severe stress. “[C]linicians and staff reported significant reductions in overall professional satisfaction and corresponding increases in stress, burnout, chaos, and likelihood of leaving their practices,” noted the report. (p. 124)

The other two PCMH demos started in October 2012, roughly one year after the FQHC demo began. The first of the two to receive a final evaluation was the Multi-payer Advanced Primary Care Practice (MAPCP) demo. This evaluation, written by RTI International, was published in June of 2017. RTI concluded that the demo raised Medicare’s costs but not by a statistically significant amount, and had “unimpressive effects” on quality (p. ES11). Here’s how they put it: “Medicare expenditures for the MAPCP Demonstration beneficiaries were … nearly $171 million more than the non-PCMH comparison beneficiaries” (after taking into account CMS’s payments to the PCMHs) (p. ES8), and “there were no consistent impacts by the MAPCP Demonstration on quality of care, access to care, utilization, or expenditures within or across states….” (p. ES12) RTI reported the same results for Medicaid. RTI did not attempt to assess staff morale, but the report did note, “Many practices across the MAPCP Demonstration states found it challenging to fund extended hours and to find staff to work the hours.” (ES6)

The last of the three demos to receive its final evaluation was the Comprehensive Primary Care Initiative (CPCI), which, like the MAPCP demo, ran from October 2012 to October 2016. The final report, written by Mathematica and published last month (May 2018), was as dismal as the evaluations of the other two demos. Mathematica concluded the CPCI had no impact on Medicare spending and had a “minimal” impact on the tiny handful of quality measures CMS used. “After including care management fees, Medicare expenditures increased by $6 PBPM [per beneficiary per month] more for CPC practices than for comparison practices,” the report stated. “The difference was not statistically significant.” (p. xviii) Moreover, the report found that the CPCI “had minimal effects on the limited claims-based quality-of-care process and outcome measures examined” and “had little impact on beneficiaries’ experience of care.” (p. xviii) Mathematica reported that the burdens associated with “transforming” clinics into PCMHs had no impact, good or bad, on burnout among PCMH staff.  However, they did note that “some … care managers felt overwhelmed” (p. xxxii) and that “[R]espondents reported that participation throughout the CPC initiative was burdensome…. In addition…, several respondents reported that they had overall change fatigue.” (p. 135)

If the three final evaluations had taken into account the costs that PCMH clinics incurred in order to do whatever it is PCMHs do, all three evaluations would probably have concluded PCMHs raise total spending. A tiny body of research indicates these costs are well in excess of $100,000 per physician per year. However, only one of the contractors that CMS hired to write evaluations (RAND) attempted to determine how much PCMHs spent, and they came up empty. None of the three contractors attempted to determine how much CMS contributed in the form of services (webinars, data that allegedly helped clinics assess their “performance,” etc.). And in the case of the MAPCP and CPCI demos (which involved multiple payers, not just Medicare), Mathematica and RTI did not report what non-CMS payers contributed in the form of subsidies and services. [1]

It is now obvious the PCMH does not work. What went wrong?

A laundry list of aspirations is not a plan

I’m sorry to do this to you, but I must begin my exploration of why the CMS demos failed by asking you to look at the mandala below. You’re not looking at the Great Wheel of Life. You’re looking at CMS’s PCMH “change package.” According to Mathematica’s final and interim evaluations of the CPCI, this mandala – this collage of vague and manipulative phrases – expresses CMS’s muddled “understanding” of the PCMH. As Mathematica put it in the final evaluation, “The CPC [comprehensive primary care] change package describes the underlying logic of CPC, including the primary and secondary drivers to achieve the aims of CPC and the concepts and tactics that support the changes.” (footnote 1, p. xvii) Do you see any “underlying logic” in this “change package”? The title CMS chose for this mandala was, “Comprehensive Primary Care Initiative Logic Diagram.”


The wheel has four concentric rings around a circle in the middle with the words “patient and family.” Let’s ignore for now the sanctimonious assumption that patients were never the primary concern of health care professionals. Let’s concentrate on the phrases in the four rings and ask whether they’re understandable and whether any of them are evidence-based. What is “an engaged community,” an “environment to support comprehensive primary care,” “strategic use of practice revenue,” “enhanced accountable payment,” “coordination of care across the medical neighborhood,” “a culture of improvement,” or “optimal use of HIT,” to take the more oleaginous examples? How would anyone – doctors, CMS or evaluators – define and operationalize flabby phrases like that? And where is the evidence that any of the activities hinted at by these vague phrases lower costs or raise quality, much less do both simultaneously?

I have asked you to look at CMS’s “change package” because it illustrates at a glance why the PCMH has failed. The two most fundamental reasons are: The “definition” of the PCMH consists of multiple elements, nearly all of which are unsupported by evidence; and most of the elements are so vaguely defined they are impossible to operationalize. It’s that combination of (a) multiple elements unsupported by evidence and (b) multiple elements that elude clear definition that renders the PCMH concept so useless.

Note that the multiplicity of elements attributed to the PCMH is not by itself lethal. The Mediterranean diet and the lifestyle arm of the Diabetes Prevention Program are examples of things that are testable and usable even though they are defined by multiple components – whole grains, olive oil and fish in the case of the former, and exercise and diet classes in the case of the latter. All human beings know what “olive oil” and “exercise” mean, and those terms are, therefore, easy to operationalize (define clearly and in a uniform manner), and once that is done for all elements, the bundle of elements is testable. But “medical neighborhood,” “culture of change” and the other phrases in the CMS mandala are too imprecise to reduce to testable definitions. It’s the indefinability of nearly all of the multiple elements of the PCMH that dooms it. The multiplicity of elements alone is not fatal.

In fairness to CMS, the dual defects I’m discussing – multiple components vaguely defined, and lack of evidence supporting the components separately or as a package – were not created by CMS. They were invented by the four primary care groups who propelled the PCMH from esoteric idea to stardom in 2007 with the publication of a document entitled “Joint Principles of the Patient-Centered Medical Home”. This statement purported to define the PCMH with seven vague components called “principles.” Principle 1, for example, was “ongoing relationship with a personal physician.” How is CMS supposed to operationalize that notion? How would CMS employees or anyone else know when a “relationship” has progressed from “not ongoing” to “ongoing”? Principle 5, to take another example, was entitled, “Quality and safety are hallmarks.” According to this principle, doctors who wish to conjure said “hallmarks” must “support the attainment of optimal, patient-centered outcomes.” How would anyone know when Doctor X has progressed from not supporting “patient-centered outcomes” to supporting them? [2]

The seven principles in the Joint Statement are laudable, but they are merely vague expressions of aspiration. They do not define anything. They read like the Boy Scouts’ Oath — a scout is “trustworthy, loyal, helpful, friendly, courteous, kind,” etc. The Oath consists of wonderful aspirations, but do those aspirations constitute a testable program? Of course not. The seven principles in the Joint Statement don’t either.

If CMS had decided that the “home” as defined by the Joint Statement and other PCMH manifestoes is not testable, and had instead tested one or two of its more definable components, I would have no criticism of CMS. But CMS didn’t do that. They took the PCMH proponents’ lists of aspirations at face value, concocted their ridiculous “change package,” and then tried to operationalize portions of the “package” with requirements for clinics they called “milestones.” Examples of CPCI “milestones” enforced in 2016 included a requirement to “risk stratify all patients,” “[a]ssess patient experience through patient surveys or patient and family advisory council meetings…,” and “[p]articipate in regional and national learning offerings….” “Learning offerings”? As you can see, these “milestones” exhibit the defects we saw in the pious phrases in the “change package” – they are evidence-free and very vague. [3]

“Learnings” from the PCMH debacle

The contractors who wrote the evaluations for CMS failed to tell readers what I have just told you – that the PCMH is failing because it is so poorly defined no one, including PCMH staff, knows which activities PCMHs should engage in, and because evidence does not support the assumption that the poorly defined components of PCMHs lead to lower costs or higher quality. The evaluators instead offered a few speculative explanations.

The most common explanation was that the PCMHs just ran out of time. This excuse might be believable if we had any evidence to support the claim that three to five years is not long enough for at least some of the vaguely defined components of “homes” to kick in and produce results. But as I noted above, we have no research indicating the components do anything to lower costs or (with the possible exception of shared-decision-making tools, an element of some definitions of the PCMH) improve care and lower costs over any period of time. The contractors offered little or no evidence for this claim.

A second justification, offered by the authors of the evaluations of the MAPCP and FQHC evaluations, was that the PCMHs might have been underfunded. This may very well be true. Payments by Medicare and other insurers under the CPCI, for example, amounted to $70,045 per clinician in 2013 and $50,189 in 2016 (p. xxiv), far below the estimate of $105,000 for only partial PCMH implementation reported by Macgill et al.  [4] But the insufficient-funding diagnosis presents a brain-twister. If the subsidies to the PCMHs were, say, doubled, that would raise total costs and make net savings even less likely.

A related argument offered by the Mathematica authors in an article just posted by Health Affairs is that PCMH doctors “might need stronger value-based financial incentives.” This is becoming a common excuse for the failure of ACOs as well. No evidence supports this speculation.

No, the fundamental defect in the PCMH is its flabby definition and the resulting lack of focus. It is time to replace the sprawling PCMH experiment with focused pilot projects aimed at improving the quality of care for specific types of patients. If lower costs result, we can count our blessings. As one “lead clinician” told Mathematica during an interview about the CPCI, CMS “tried to fix everything in one program, rather than pick one high-value target area, start it, assess it, and then build from there.” (p. 135) That is precisely the right explanation for the failure of the PCMH.

Too bad Mathematica didn’t think to endorse it.


[1] The failure of all three of CMS’s “home” demos is consistent with the failure of earlier “home” experiments, including the Southeastern Pennsylvania Chronic Care Initiative and the Veterans Health Administrations PCMH program . It is consistent as well with the literature on PCMHs. A 2013 review of 19 studies found “no evidence for overall cost savings.”

[2] Here is an edited version of the list of the seven “principles” enumerated by the 2007 statement on the “medical home” by the American Academy of Family Physicians and three other primary care specialty groups. Every one of the principles is vague and aspirational.

  1. Personal physician: each patient has an ongoing relationship with a personal physician….
  2. Physician directed medical practice: the personal physician leads a team of individuals at the practice level who collectively take responsibility for the ongoing care of patients.
  3. Whole person orientation: the personal physician is responsible for providing for all the patient’s health care needs….
  4. Care is coordinated and/or integrated across all elements of the complex health care system (e.g., subspecialty care, hospitals, home health agencies, nursing homes) and the patient’s community (e.g., family, public and private community-based services). Care is facilitated by registries, information technology, health information exchange and other means….
  5. Quality and safety are hallmarks of the medical home: Practices advocate for their patients to support the attainment of optimal, patient-centered outcomes ….
  6. Enhanced access to care is available through systems such as open scheduling, expanded hours and new options for communication between patients, their personal physician, and practice staff.
  7. Payment appropriately recognizes the added value provided to patients who have a patient-centered medical home.

[3] Mathematica made a feeble effort to defend CMS’s “milestones.” In their first annual report on the CPCI, Mathematica admitted, “While the Milestones themselves are not evidence based, they are rooted in strong conceptual thinking about what activities a practice needs to pursue to achieve comprehensive primary care.” (p. 81) Mathematica offered no support for this statement.

[4] Although the final evaluation of the CPCI reported that the payments to PCMHs were far below the $105,000 figure reported by Macgill et al., Mathematica stated, “More than three-quarters of practices reported on the CPC practice surveys in 2014, 2015, and 2016 that CPC payments – including care management fees and, when relevant, shared savings payments – were adequate or more than adequate relative to the costs of implementing CPC.” (p. xiv) It is not clear from Mathematica’s report that the respondents had any idea what they’re clinics, or the systems their clinics were part of, were actually spending on “home” activities.

Kip Sullivan, J.D., is a member of the Health Care for All Minnesota Policy Advisory Committee and the legislative strategy committee of the Minnesota Chapter of PNHP

Categories: Uncategorized

Tagged as: ,

10 replies »

  1. “All human beings know what “olive oil” and “exercise” mean, and those terms are, therefore, easy to operationalize (define clearly and in a uniform manner), and once that is done for all elements, the bundle of elements is testable.”

    This is the crux of our entire health system, not a “medical home”.

    To change our culture of sickness we need change our culture, and tax those behaviors that are unhealthy. We tax cigarettes and alcohol to pay for the known societal consequences of their use. Tax sugar, fat, calories, stop subsidizing corn and transfer those subsidies to fresh fruits and vegetables. Allocate the taxes on sugar, fat and calories to help and pay for health care costs like diabetes and heart disease.

    If you want a real reduction in health spending go to Medicare-for-all single pay.

  2. Thank you, thank you, Mr. Sullivan for giving us your take on all three programs in one succinct article.

    Though the other primary care societies came along for the ride, this dismal cluster f*** belongs 100% to the morons at the AAFP.

    And, based on long personal experience, I have to strongly disagree with Mr. Goldsmith’s description of them as “well meaning.” These people are profoundly anti-science and therefore anti-physician; they are close-minded and intolerant of any disagreement; they are supremely self-righteous and contemptuous of those who actually provide medical care ; they simply are not bothered by the enormous harm they have done to our specialty, our practices and our patients.

    They are vile human beings.

  3. This farce has now been repeated over and over.

    One of the things that is becoming ever more clear is that the progenitors of these rackets (is there really a better word?) presume zero effect on the work we do – you know, take histories, do physicals, order tests, figure out what’s wrong, and treat it?

    I’m old now (44 years in the game), and I find that the new EHR, the rev cycle, the meaningful use, the patient education (not the kind I do), the end of visit form, the freaking CODING, adding unnecessary elements to meet my RVU target, all of it – all of it soaks up time and energy that I should be spending doing what I am supposed to do, and doing it well.

    I had a revelation recently while discussing more documentation required of the MAs before and after I see the patient.

    The con artists who dream up this stuff must think what I do is similar to magic. I go in the room, I close the door, I open the door and walk out – et voila! It takes no time, it takes no skill, it takes no effort – it just somehow HAPPENS.

    That being the case, why NOT add on a question about domestic violence? Or whatever? None of it matters because somehow, with the actual visit compressed to three minutes or less, the magic doctor will still manage to do some good – and, if not, at least it will code out as a 99232 instead of a 99231 – which is, of course, what all of this is about.

  4. We should take a lesson from MIPS: Punish those who promote farcical fad management. Have their funding dwindle year over year until they go away and get real jobs where they can’t just sit back and parasitize patient care.

  5. So much of this management fad is about becoming the thing, proving you are the thing, and then maintaining the thing. It has almost nothing to do with the patient. Peculiar that they put the patient at the center of their cartoon. I guess seed money needs to be planted somewhere. Way too many people had their hand in the kitty during this misadventure: medical societies (AAFP), think tanks (PCPCC), certification cartels (NCQA), and anyone else snorting the grant money dole. These management fad pushers had to take the direct to legislation path and force physicians and patients to accept these models exactly because of the financial infeasibility and stunning lack of evidence on outcomes. No industry, physician, or patient would ever elect to clog up our clinics with this nonsense. So it is insulting to see how much proving I have to do concerning my performance to organizations that have absolutely no validity concerning their own performance, and no relationship to outcomes or reduced costs. Time to end the farce. Pull the plug on the money flow to these corrupt shill groups and it should go away quickly.

  6. Some policy people pay a cost if their ideas don’t work out. For example, this is obvious in
    the sectors of aviation and space technology and in taxation design and capital investment.

    Others seem to get off scot free if their ideas don’t work: education fads galore–only the kids seem to pay here–lots of politics is in there, and nearly all sectors of health care.

    It seems as if we have been living in this dreamland of vague health policy guesses and ideas and hopes and yearnings since forever….well, at least since the PSRO (remember THAT?–the Professional Standards Review Organization).

    We really need to find some way to exert a personal cost on all these bum ideas. Maybe insist that the policy promoter has to bet some of his own dough in a futures market about the success of the ideas? [A trifle fanciful….but we do need some accountability here….we have OD’d on BS for too long]

  7. From Kip Sullivan to Jeff Goldsmith

    Thanks, Jeff. I agree with your phrasing of the problem — not everyone in Medicare (or under age 65 for that matter) needs a “medical home,” whatever it is.

    I couldn’t tell from your reply whether you think the Independence at Home demo should be considered a “medical home” demo. I would say it should not, but I haven’t read the evaluations of it so I should withhold a final opinion till I have. It makes more sense to describe the demo as a pilot test of new SERVICES — or at least newly reimbursed services — namely, home visits for the chronically ill, rather than a test of a new STRUCTURE or ORGANIZATION (which is how we think of HMOs, ACOs, and “homes”). There was never any evidence supporting the rush to push doctors and hospitals into new structures or organizations, and now that we’ve got 50 years of experience fooling around with entities with “O” in their names (HMO, PPO, ACO), it’s time to admit it was all a mistake and to focus on specific services and specific subsets of patients.

    The Independence at Home program may turn out to be a good example of the programs I think CMS should be experimenting with. It targets a defined subset of high-cost patients.

    What’s unclear to me at the moment is whether the savings CMS claims the Independence at Home demo has achieved take into account the additional costs incurred by CMS and by the participating clinics.

  8. To join the chorus, I would add the extensive analysis published in the Milbank Quarterly for March of this year. Lawton Burns and Mark Pauly from The Wharton School, University of Pennsylvania evaluated our nation’s current health care transformation commitment. I cite their “Findings.” “Data suggest a low prevalence of provider risk payment models and slow movement toward new payment and organizational models. Evidence suggests the impact of both on cost and quality is weak.” Verily, a modest statement for a 57 page report by two economists.

    Meanwhile, the cost and quality problems of our nation’s HEALTH remain basically untouched.
    Is there an underlying connection between mass shootings, worsening suicide, depression, chemical dependency, entrenched poverty, wide-spread obesity and worsening maternal mortality that is associated with the fundamental cost and quality problems of healthcare? No doubt its a stretch. I quote from Eleanor Roosevelt: “Its better for everybody when it gets better for everybody.” The Design Principles for managing a Common Pool Resource successfully (the portion of our GDP allocated to health spending) have already been defined. Fortunately, the connection to solve this very large Paradigm Paralysis requires Social Capital rather than Financial Capital, community by community.

  9. Thank you, Kip, for another workmanlike deconstruction of a poorly framed and poorly executed ObamaCare demo. Not everyone on Medicare needs a medical home, whatever it actually is. Some of us just need help with a bum hip and are fine otherwise.

    PCMH was a fifty year old idea that originated to solve the problem of seriously ill children with a lot of specialists and major demands on overwhelmed parents. It got resurrected by well meaning folks who really want to redefine primary care, and delivered into the hands of a well meaning but poorly focused new agency, the CMS Innovation Center, assembled out of spare parts, and with a serious shortage of managed care operating experience.

    Beyond the Independence at Home demo, did we get anything scalable out of this agency’s $5 billion check writing festival?

  10. What good timing.
    Here are a few tidbits I gathered a few hours ago from my Twitter feed.
    Andy Slavitt

    In case you missed it: Yesterday the Medicare Trustees announced that Medicare is due to run out of funds 3 years earlier. As someone who sat on the Trust Fund, here are the implications in a few tweets.

    The trust fund are funds set aside when we pay payroll taxes to cover hospital care for seniors & ppl w disabilities. When Obama became President, it was due to be depleted in 2018. The ACA added 10 years to the life of the fund. Yesterday’s news is it lost 3 of those years.

    Employment, wage increase, and sufficient taxes improve the life of the fund. If wages don’t keep up with medical expenses or if employment lags retirees, this hurts Medicare. The actuaries judge our current policies accordingly. If nothing changed, the Trust Fund wouldn’t be empty in 2025. But it would mean that the reserves would be gone and one of two thins would have to happen: benefits would need to be cut or more money would need to be added.

    This is where there is an important debate…There are things both parties believe, solutions historically favored by Republicans, and solutions historically favored by Democrats. Worth reviewing simply for those with the interest…

    Both parties believe we should do a better job controlling medical costs. Obama years cut Medicare cst growth to 2% from 6% during GWB years. But there is no silver bullet and the fund actuaries won’t credit new savings until proven. The major differences are that Rs generally (Paul Ryan of note) believe that cutting benefits, or capping them, is the answer. Whether a voucher to replace the defined benefits, changing the age, or some other test.

    Democrats treat the promise of Medicare made in 1965 as a sacred promise and reforms should come from sufficient taxes on high earners and from reducing payments to “middle men” like insurers. This should not be portrayed as or dealt with as a crisis. It would be smart for the public to weigh in— in this election and more generally. Candidates should articulate their views.

    If you believe like I do that the Medicare promise is sacred, and that a promise to allow Americans to care for their families should offered to everyone— and be done without effecting the Trust Fund, there are few more important divides between the political parties. /end
    Here’s a link to Part 2 of a Dallas Morning News investigation, hot off the press…
    “As patients suffer, companies profit — Years of poor state oversight have allowed companies to skimp on essential care for sick kids and disabled adults”
    And finally, here is a Twitter message I penned myself.
    It’s a tiny spitwad in a contest being waged by
    • lawyers,
    • accountants,
    • economists,
    • politicians,
    • cyber-experts,
    • financial advisors aaand
    every other kind of expert you can think of
    …other than (or in addition to) medical professionals.
    => Unless & until Medicare beneficiaries get better rates for meds, treatments, rehabs & aftercare no “adjustments” or “privatizing” will yield meaningful benefit.
    Too much tax money is used for NON-MEDICAL COSTS — admin redundancy, ads, rebates, corporate profits, exec comp, &c.
    That’s all I got, boss.
    Take it away.