A state the size of Vermont claiming savings of $120,000,000 through a patient-centered medical home (PCMH) program should raise eyebrows.
North Carolina made similar claims about its PCMH model, only to have the results so thoroughly debunked that consulting firm, Milliman, was forced to retract its key assertion.
I expect more from Vermont, if only because I’m a Democrat and Vermont has turned so “blue” that in 2008 John McCain received only 10,000 more votes than Calvin Coolidge garnered in 1924. However, it turns out red states don’t have a monopoly on invalid PCMH data.
A brief summary of the Vermont Blueprint for Health, as described in the enabling legislation, would be: “a program for integrating a system of healthcare for patients, improving the health of the overall population…by promotion health maintenance, prevention, and care coordination and management.”
This is to be achieved by emphasizing the usual suspects — patient-centered medical homes and various support mechanisms for them. The idea is to achieve “a reduction in avoidable acute care (emergency visits and inpatient admissions).”
Growth in participation has been phenomenal. In 2009, only a few practices and a dozen employees were involved, so we can call that the baseline year. The report’s findings take us through 2012, by the end of which two-thirds of the state’s primary care practices (104) and population (423,000) were involved, along with 114 full-time employees.
The State’s Analysis
Through the end of 2012, the state — by using the classic fallacy (also embraced by the wellness industry) of comparing participants to non-participants — was able to show savings of $120,000,000 and a double-digit ROI.
Fallacious comparisons always generate questionable findings. For instance, double-digit ROIs like theirs are never found in care management programs of any stripe, and should be seriously vetted before being released. Further, in almost every cohort studied, the PCMH population handily outperformed the non-PCMH population in acute care utilization while also spending less on preventive utilization than the non-PCMH population, including an “unexpected reduction” in pharmacy spending.
And that – plus declines in outpatient, doctor visits, and “other” – is the clearest red flag. Every component of cost can’t decline – something has to increase in order to save money elsewhere. As Why Nobody Believes the Numbers observes: “Insulating your house will save money on heat, but not on insulation.” Note the absence of “insulation expense” in Vermont’s graph above on the left.
It bears a striking resemblance to the discredited Mercer analysis of North Carolina on the right – every category of resource use in the PCMH outperforms the respective benchmark
So What Really Happened?
Just like North Carolina’s consultants, Vermont’s analysts didn’t perform a simple plausibility test, using statewide admission and ER visit tallies collected by the government for the very purpose of doing this exact type of analysis.
It would have taken them an hour to enter in all the diagnosis codes for preventable admissions (the Blueprint is aimed at avoiding preventable admissions, and the hyperlink provides the official ICD9 codes for them) to see if $120 million worth of preventable admissions were avoided.
Just click through on the 2009 baseline and 2012 study years and see for yourself: they weren’t avoided.
Quite the contrary, total admissions climbed ever-so-slightly faster than population, with the greatest increase being in Medicare. This is no surprise given the rapid aging of Vermont’s population. The noticeable reduction in Medicaid admissions, combined with an increase in Medicaid beneficiaries statewide, probably generated about $4-million in savings, which would not have even covered the costs of the program, let alone accounted for the $120-million claimed.
This admissions reduction figure, covering all of Medicaid, is also curious because: (1) many if not the majority of Medicaid beneficiaries still were not enrolled and/or had not visited a doctor in the Blueprint program to establish a preventive care plan by the midpoint of 2012; and (2) the unavoidable admissions in Medicaid also declined, albeit not quite as much as the avoidable ones did.
Plus, even in the state’s own analysis, Medicaid ER visits climbed, partially offsetting that $4-million in savings. This matches the government data, showing that between 2009 and 2011, Medicaid ER visits climbed a little faster than the Medicaid population (2012 is not available yet).
Lesson Learned: Patient-Centered Medical Homes Do Not Save Money
Three statewide “natural control” experiences now show that a PCMH model doesn’t come close to breaking even. Illinois would be the third, having concocted huge and immediate savings simply by having their consulting actuaries – Milliman again – project a high trend and then “outperforming” it.
(There is also a fourth state which found the same result, but for which I am under a confidentiality restriction, having done the analysis for them.)
A recent RAND Report reached a similar conclusion. Offsetting these results are a number of studies comparing participants to non-participants, or starting out with only patients known to have chronic disease, which creates regression to the mean by not enrolling people who are ignoring their chronic disease and are hence more likely to crash but be counted in the control group rather than the participant group.
Additionally, most of these other studies were done by people whose livelihoods are tied to the PCMH movement.
Notwithstanding the financial results, Vermont should be applauded for trying—and I’m sure there are benefits to the PCMH other than savings. I myself appreciate all the attention my own PCMH gives me, despite the risk of overtreatment from all that attention.
And by definition experiments can fail—that’s why they are called “experiments.” This one does not seem to have worked, so far. Perhaps the state should try one more year, and then re-allocate the resources elsewhere. But the answer is not to generate implausible analyses that show results where none exist.
Even so, the Department of Vermont Health Access doesn’t merit the same disparagement as Community Care of North Carolina. Vermont didn’t pay consultants to lie for them four times, or even once. They didn’t claim savings in excess of spending, and they haven’t driven the cost of Medicaid to a level 40% higher than surrounding states, causing their state to cut back on other Medicaid benefits to finance their boondoggle like North Carolina did.
Vermont’s was a good faith effort combined with some wishful thinking, and, speaking of wishing, we wish them the best the next time around.
Disclosure: Consistent with comments to the previous posting I did with Vik Khanna, on Saturday we sent an email to the Vermont Department of Health Access offering the opportunity to fact-check or comment and indicated that publication was happening. This email can be forwarded upon request.
Al Lewis is the author of Why Nobody Believes the Numbers, co-author of Cracking Health Costs: How to Cut Your Company’s Health Costs and Provide Employees Better Care, and president of the Disease Management Purchasing Consortium.
Categories: Uncategorized
It looks like the Vermont governor may have paid for a study to tell
them almost nothing….VT Gov Shumlin
hired a Maine consultant BerryDunn to tell them almost nothing….according to a story below. The Vermont health care experiments look like massive wishful thinking and fads being implemented with other peoples’ money (the feds seem to be underwriting much of this).
http://vtdigger.org/2014/03/31/outside-review-health-care-exchange-launch-finds-problems-inside/
Honestly, no idea why this book cost them so much to produce. The book cover is white, so I know they didn’t budget much for ink. But after that nice plug, if you’d like to contact me offline I’ll send you one from my private reserve collection for $27 including s/t/h.
Al,
After your shameless plug above I went to Amazon to look up Why Nobody Believes the Numbers….great reviews! But it is a bit pricey even used or Kindle….must have been a limited run by Wiley?
Paul
Fair enough — I should have given them more time. But I’m making progress. I used to not send them in advance at all, so I’m making progress. If he had gotten back and just said they wanted more time, we would have given them more time, but we never heard anything — which as I say is probably the right answer from their viewpoint.
Also, I wouldn’t call this “slinging mud.” By my admittedly none-too-lofty standards this posting is the ultimate in graciousness. If you want to see slinging mud, I’d invite you to visit the Intelligent Design Awards on my website, http://www.dismgmt.com Even then, no one has ever threatened to sue me.
In all fairness though, Al, you didn’t give them an appropriate amount of time to respond given the facts you’ve laid out – you contacted a state government agency on a Saturday for an analysis-driven response to an article you intended to post on Monday, Had they even seen your analysis before the email on Saturday?
Generally speaking, I like your basic ‘sniff test’ type of analysis, but you come off petty and vindictive when you vilify the state so readily when you haven’t given them a real opportunity to respond (based on the facts you outlined). Your argument is not without merit and I don’t think it’s necessary for you to sling mud to make your point.
They were sent a copy on Saturday and offered the opportunity to fact-check or rebut. The correct answer is what they did, which was nothing. Only once has a perp rebutted. Rebutting just raises the visibility and the likelihood of a pickup from the lay media, which is the last thing they want to see.
And the one time someone did rebut, which was in vik’s and my Propeller Health posting, the rebuttal consisted mostly of the principal investigator throwing Propeller under the bus
I noticed this time you gave the subject of the article advance notice. How much advance notice and did you hear back from them at all?
Amen and Amen.
Yeah.
We need to hear form Doc Gurley on this.
and the darnedest part, Perry, is that if they had just SAID that — what you said in the first few sentences — it wouldn’t be an issue. But that’s not what they said. You know the old saying: “Pigs get fed. Hogs get slaughtered.”
Or to mix metaphors (really, cliches), they flew about $120,000,000 too close to the sun.
The concept of the PCMH is really not bad. If you have a high proportion of sick, indigent patients with poor social support systems, you will need an interdisciplinary model of health care to provide for them. You have to weigh the costs of extra staff and support personel against the likelihood that you will eventually make enough progress in treatment design to keep people healthier, on less meds and out of the hospitals. It makes sense that our sicker patients are often poor, homeless, elderly with little family or social network support, and the illness itself is almost a social problem as much as a medical one.
However, the policy makers like to “standardize” everything, but it makes little sense to undergo complete transformation of the medical care system for suburban areas and most semi-rural areas where these social issues are less prominent.
I swear this comment is not a setup and I don’t even know this guy but since you brought it up, I do have three books covering outrageous claims.
Surviving Workplace Wellness (so far only in Kindle — hardcopy a couple of weeks off), co-authored with Vik Khanna, published by this very same Health Care Blog.
Cracking Health Costs, with Tom Emerick (Wiley, 2013)
Why Nobody Believes the Numbers (Wiley, 2012), which contains a very specific smackdown on North Carolina’s PCMH and the consultants they hired to lie for them. The first line: “I’ve seen worse analyses than North Carolina’s, and I’ve seen more expensive analyses than North Carolina’s, but I’ve never seen an analysis that was both worse and more expensive than North Carolina’s.”
You could really do a whole book just on the PCMH and the ocean of outrageous claims and broken promises made.
It’s fitting that, on opening day, Al Lewis hits another outcomes shattering home run. Liberal or conservative, D or R, in this space it’s all the same: stupid is as stupid does.
John, that can be and is easily done for most states. Dan Quayle once said: “The role of the Vice President can be summed up in one word: To Be Prepared.”
Likewise, I can give you the answer in one word: a statewide program designed to avoid preventable admissions can be evaluated simply by taking the list of preventable admission ICD9s compiled by AHRQ and then entering them into the HCUP database maintained by AHQR. (I have already collected those ICD9s if anyone wants them. )
Obviously you should see an inflection of some kind in that rate over the course of the program, that is not matched by changes in unavoidable admissions. Or in control states. Like I wanted to use NH as a control for VT but NH for some reason stopped sending data to HCUP. No need — VT’s results were so plausible and expectable, and so far off their claimed results, that no double-check was necessary,
There are some other subtleties involved in these analyses but “subtle” is not in the lexicon of most states. They claim such massive savings that the debunking can be done blindfolded. Like North Carolina’s consultants didn’t even bother to note that the amount they claimed in savings exceeded the amount spent in admissions…and by the way their admission rate didn’t decline at all anyway. At least Vermont garnered a small decline in avoidable admissions for Medicaid, even though there was some squirreliness even in that small decline.
Hmm, would be interesting to see a chart showing costs savings claimed by various state programs and run them through the sniffer …
Yeah.
apropos of that analogy, to the extent that “insulation” involves reducing the escape of heated air into the outside ambient environment, the subtle (but real) adverse health effects of more stagnant air cannot be ignored. You necessarily have to exchange stale air with fresh air. That new air has to be heated (or cooled) as well.
“Efficiency” has both limits and unintended side-effects.
Graphics above (Vermont on the left and NC on the right) both violate the “insulation rule” that every component of spending can’t decline. Insulating your house will save money on heat but not on insulation