Patient-Centered Medical Homes in statewide populations have unstoppable momentum and major constituencies in support of them, so valid analysis of their outcomes is probably as futile as it will be unwelcome. However, the math speaks for itself, at least in the mother of all statewide medical homes, North Carolina Medicaid’s Community Care Access Model.
I write this after having analyzed the actual data from this project’s outcomes report, rather than the stated conclusion of the report, a conclusion that continues to be cited in support of the many states considering or implementing medical home models for their Medicaid populations.
The conclusion makes North Carolina looks like a huge win for PCMH: $300-million in reported savings. However, readers should have (but largely didn’t) observe a number of curiosities about the data in support of that conclusion:
(1) Every element of resource use declined. People have to be getting their care from somewhere, but inpatient, ER, outpatient, physician, drug, and other expenses somehow all declined vs. trend.
(2) The decline in physician practice expense is especially counterintuitive: Why are the doctors so supportive if they are working harder but making less money?
(3) Even though somehow savings were shown in physician expense, per capita doctor visits did indeed increase. More concerning was that specialist visits –which are supposed to decline in a PCMH model – also increased.
(4) Inpatient expense fell 47%. This was achieved despite the fact that all the AHRQ’s “Ambulatory Care –Sensitive Conditions” total to about 20% of admissions in most populations.
(5) The evaluators (William M. Mercer) are on record as saying that “choice [emphasis mine] of trend has a large impact on estimates of financial savings.” Perhaps it is possible that Mercer, having given themselves this latitude, “chose” a trend that would make the study look good.
Those observations merely suggest that the study was done wrong. But one other “finding” invalidates the entire study: the 54% reduction in spending on babies under one year of age, accounting for the majority of the entire $300-milllion in spending. (Any nontrivial savings whatsoever in this category should have raised eyebrows since PCMH is mostly about managing chronically ill patients.) The components of spending in that category include physician expense, which should rise since doctors get paid more to be more accessible, and drug expense, which should rise for the same reason. This means that the entire 54% savings across the category must be concentrated in neonatal expense. Since neonatal expense is about half of total spending in the age category, it would have to decline by a mathematically impossible 100%+ in order for the category to average a 54% reduction.
But perhaps the neonatal savings came close to 100%. To determine that, let’s do something the investigators didn’t do: Look at the actual data. Neonatal admissions and days of care, by state, are freely available at http://hcupnet.ahrq.gov/HCUPnet.jsp. It turns out that the percentage of neonatal admissions and days fell only 1%, from 27.5% in the baseline to 26.5% in the last year studied—a decline two orders of magnitude less than was required to make Mercer’s math work. (The argument that neonatal utilization would have risen absent a PCMH program is also invalid – South Carolina enjoyed a relative decline in neonatal utilization as well over that same period, without a medical home.)
Therefore, without even needing to “challenge the data,” but rather just looking that the data that the consultants presented, as well as other public sources of data, one must conclude that the study is fatally flawed.
What do proponents of this study say in defense? In a private conversation with one of the major proponents of a PCMH model (whose company benefits greatly from adoption of electronic health records), the response was that he “believed” the Mercer outcomes. Math, however, is not a belief system. It’s as proof system, and every assumption about the savings from widespread implementation of statewide medical homes in the Medicaid population has just been proven wrong.
Categories: Uncategorized
I understand your gripes about this particular study. However, there is a bigger picture here. This is a good piece to look at to think about the underlying issues more clearly- and the real possibilities and limits of Accountable Care Organizations that intent to create a more integrated care format http://healthpolicyandreform.nejm.org/?p=13937&query=home.
thanks for letting me have the last word. Not sure what more I can add. I looked at quite a number of DRGs (and the DRGs in total) and would suggest that anyone else do the same. There were none with significant numbers of admissions (I didn’t look at low-frequency ones) where the savings in Medicaid admissions (vs. usual care as represented by the rest of the state’s admissions) even approached the claimed “average” of 47% better.
I looked the hardest at the DRGs most likely to show savings. First, the neonates due to the very high claimed savings.
Next, I focused on the asthma example for the reasons given above. Asthma was the most widely touted part of their program (and I think did reduce ER visits by a noticeable amount…but the consultants said most of the state’s savings were in inpatient).
For other DRGs, I just looked at NC Medicaid vs NC non-Medicaid. It’s not worth listing them all — they all show the same thing.
This DRG analysis is easily replicable using public data, and if anyone can find even a single DRG with 1000 admissions or more in which Medicaid admissions fell 47% vs. non-Medicaid (remember, that is the average savings claimed–I am saying I was not able to find one number even close to the average), they are more than welcome to anything — including Critical Outcomes Report Analysis Certification — free from DMPC.
Al
No sarcasm here, you strike me as a pretty sharp guy, but based on your assertions, I would expect that with your above work in this domain, a fuller look at the data was in the cards–at least as inpatient expenses go. I wont be searching HCUP–your post, your task:)
Needless to say. asthma wont cut it for me alone, and to validate your conclusions, I need to see more. i did not think I was asking for air-speed velocity of an unladen swallow, but that is just me.
Anyway, you can certainly have the last word.
cordially
Brad
PS–?
Well, the ones I looked at in depth were the ones most likely to decrease (but for the most part, didn’t decrease even by double-digits when adjusted for usual care changes, let alone 47%).
In particular, asthma is the most instructive since (1) NC made such a big deal of their asthma program and (2) asthma admissions are possibly the most avoidable common admission among common diagnoses.
Using the state population in 2001 (the middle year of the baseline) and 2006 (study year endpoint), and separating out the Medicaid and the non-Medicaid populations, I then went to the HCUP database and looked at asthma discharges for the Medicaid and non-Medicaid populations. (I had originally done this with FOIA but was able to do it this morning with public data available to anyone free.)
It turns out that over that period asthma admissions in the Medicaid population declined by 21% per capita, an outstanding performance until one considers that in the non-Medicaid population, they declined by 22.5%.
This analysis can easily be replicated by you.
Conclusion: both the largest age-related reduction (neonates) and the largest resource-related reduction (inpatient) are as invalid as numbers can be. The Mercer report served the state’s purpose (more matching funds) and served Mercer’s purpose (look how much PR they’ve gotten) but once someone other than me reviews the actual numbers, this will be a setback for statewide Medicaid PCMH as its numbers will start to be challenged across the board. (More tightly controlled and likely successful PCMH projects, paid for by the private sector, should not be affected.)
Note: I am not pushing DM as an alternative in states. With the exception of Wyoming and Colorado, statewide Medicaid DM numbers are made up too. For instance, Mercer validated an outcome in Georgie equivalent to more than a 100% reduction in avoidable admissions. This is paradoxical because he state ultimately settled with the vendor for exactly the opposite reason — they didn’t do enough to make any difference at all.
And in Texas, the state changed its DM RFP (Mercer also the consultant) in order to make the metrics easier to achieve, so as not to embarrass itself by not hitting the cost reduction numbers, as happened in the previous contract cycle when they were using valid metrics.
Al
Would you be so kind as to list the top 5 (or 10) DRGs that WERE reduced, if not ambi sensitve designated.
It might be semantics (“ambi sensitive”), but it is possible that through systemic changes in NC program, reductions were real, perhaps not accounted for by providers alone, and a greater look at what is being done, by whom, and at what cost (ROI) is necessary?
There is a reason inpat costs went down IF they bucked secular trends. If so, that occurrence deserves consideration.
Thanks
Brad
Brad
Yes, thank you for pointing that out. I would call that omission a major oversight, sort of like the 1831 Bible that left the word “not” out of its commandment on adultery. John, if you have an pencil and eraser rhandy, would it to be late to insert the words “statewide Medicaid” before Patient-Centered Medical Homes?
Two other observations I failed to address. First, coordinated care, which might bring down overall costs, does not bring down the cost of preventive care, any more than insulating your house brings down the cost of insulation.
The most successfuly, most widely studied (by me and others) coordinated care company, Quantum Health, will share some data with you. Preventive elements of care — labs for screening, drugs, and PCP visits go up, while everything else declines. The lesson from Quantum: Preventive care costs money too
Let’s dig into the 47% inpatient cost reduction again. The asthma ED admisson reduction was 16%, not 6% as I had originally reported. This refutes three claims. First, it is a highly preventable condition AND was subject to a comprehensive program –it should be bringing up the avergae, not pulling it down. Second, it is a low-cost condition, refuting the argument that it is possible to reduce claims much faster than one reduces admissions. Third, what drugs were they given to prevent their asthma? Over the course of a year, those cost more than an ER visit.
Finally, kudos to the state of NC, which I beleive did this study on their own. They not only avoided the classic mistake of looking only at previously diagnosed asthmatics (and showed the dramatic effect on results if they had done that), but also reduced admissions! That is the point of PCMH and it worked, albeit at a cost.
“… I have no problem with those PCMH results you’ve described in group health plans. Those settings lend themselves to PCMH success and I wish hem the best of luck.”
The title of your post certainly doesn’t support that statement.
This is my favorite comment so far…:)
My policy (so it doesn’t look like i am pitching a product) is if I blog on something, to give away the report to anyone except close competitors and (in this case) the state of NC, which having spent $500,000 can probably afford $500 more. So even close competitors can have it but they need to pay for it. It is at dismgmt.com.
This vignette is featured in Case Studies in Invalid Outcomes, which is available for the asking. I might also recommend (but you gotta pay for it) its theoretical companion volume, Outcomes Measurement for Dummies…and Smarties.
I might also recommend (also on dismgmt.com) Critical Outcomes Report Analysis (CORA) certification. CORA Certification will make you a pro at deconstructing outcomes reports. The 200 people who have passed it are listed on the website and any one of them may be called as a reference. Analytic job searches now often require or prefer it.
thanks for the opportunity for the 30-second shameless plug.
Is the entire report available anywhere for me to read, or is its just for “internal use”?
The neonatal data link (to the federal governement’s data, not my data).is in the article.
Neonatal days and admissions declined only marginally — that’s what the data says. Mercer didn’t look at this database or else looked at it, realized they couldn’t refute it, and didn’t bring it up. Either way is substandard. (To be fair, there was a noticeable mix change between the ultra-preemies and the other preemies, so that perhaps the cost reduction exceeded the 1% utilization reduction, but remember, they need a 100% reduction to make their conclusion valid.)
The ACSC list also comes from AHRQ and is a list of the admissions that they feel can be avoided through better ambulatory care. Perhaps their list is too narrow but they put a lot of thought into it.
It is theoretically possible that “a small reduction in inpatient admissions” could generate a 47% reduction in expense, but that would require the leap of faith that the admissions that were avoided were the most expensive ones. even then you would need a sizable reduction.
I now have a better understanding of your assertion that it is impossible to reduce inpatient (expenses) by 47% when only 20% of admissions are sensitive to ambulatory care”. You don’t seem to be accounting for the fact that you are comparing inpatient expenses ($) to admissions (a patient count). Inpatient care is expensive compared to ambulatory care so even a small reduction in the number of admissions and severity of admitted patients can lead to large savings (even 47%). I also don’t understand your assertion that only 20% of admissions are sensitive to ambulatory care… could you explain? I would take the view that just about any hospital admission is a failure of ambulatory care.
Also, I did address your “neonate” point which is your “smoking gun to invalidate the entire study”. You still haven’t explained where you data comes from vs theirs and how yours is “better” and at the same time comparable.
thank you to many of you for keeping an open mind and drawing their own conclusiions from the data and, as one person pointed out, the incentives of the evaluators..
I will try to address the other comments but I don’t expect to change many minds. As Upton Sinclair once said, “It is impossible to prove something to someone whose salay depends on believing the opposite.”
(1) It isn’t the expenses incurred by physicians that are declining. it is the expense of paying physicians. In other words, according to Mercver they are making less now than they would have. that’s why it is surprising that they are supportive despite seeing more patients and (according to Mervcer) making less money.
(2) It is impossible to reduce inpatient by 47% when only 20% of admissions are sensitive to ambulatory care, period. Another study of NC — one done with integrity and competence instead of a self-promoting flair for the dramatic — showed that asthma (among the most ambulatory-care sensitive conditions) delcined by a believable 6%, that I independently confirmed in my FOIA. What set of DRGs declined more than 47% to offsett the small asthma decline? Answer — looking at the AHRQ data: none.
(3) I do have some FOIA info on MD visits that shows both PCP and specialist visits rising. I’ll send it to any health plan or state that would like to contact me.
(4) Steve, I have no problem with those PCMH results you’ve described in group health plans. Those settings lend themselves to PCMH success and I wish hem the best of luck.
(5) I noticed that none of the critics took on the neonate point, which is the smoking gun that invalidates the whole thing.
(6) The most valid criticism was the absolutely correct insight that I do indeed have “an axe to grind.” I will admit to being jealous that I only charged my clients $5000 to do this analysis correctly when Mercer (I am told) charged 100x that amount to NC taxpayers to do it wrong. I may not be able to learn much from them about analysis, but I can learn a ton from them about writing proposals….
Al,
I understand that you do results benchmarking and ROI plausibility testing for Disease Management Vendors. Presumably it’s possible to do something similar for the larger patient centered medical home initiatives.
I’d love to see how findings from primary care-redesign initiatives like those at Geisinger, Group Health and BCBS of Michigan stack up against yours and vice versa!
It may be simple, perhaps PCMH are avoiding the complexly ill and are managing care of a more “normal” population of pregnant women and children. Without knowing the nature of the NC Medicaid enrollees it’s hard to tell.
I always welcome the chance to get a little learnin’ about PCMH, disease management, case management, care coordination, cost and quality initiatives, and the like. I read the author’s post, did a preliminary data search, and re-read the author’s post.
Lessons learned: The author’s post certainly does not carry the weight of such an antagonistic title.
Whether it’s the structure, the conclusions without support, the lack of data source citation (excepting the neonatal comparison), or the lack of author disclosure/conflict of interest, this post surely has a lot of its own ‘curiosities’.
Before supporting or refuting the authors assertions, I’d certainly like to see:
1) source citation
2) logical support of claims. Idea (although not a million dollar one): start with ‘curiosity’ #1.
3) author disclosure/conflict of interest statement
My preliminary data search included:
http://www.communitycarenc.com/PDFDocs/Sheps%20Eval.pdf
http://www.communitycarenc.com/PDFDocs/Mercer%20SFY03.pdf
http://www.communitycarenc.com/PDFDocs/Mercer%20SFY04.pdf
http://www.communitycarenc.com/PDFDocs/Mercer%20SFY05_06.pdf
http://www.communitycarenc.com/PDFDocs/Mercer%20SFY07.pdf
http://www.communitycarenc.com/PDFDocs/Mercer%20SFY08.pdf
Echo Jonathan’s points.
1) Every element of resource use declined. People have to be getting their care from somewhere, but inpatient, ER, outpatient, physician, drug, and other expenses somehow all declined vs. trend.
What’s the problem here? Can’t you believe that PMCH could possibly reduce expenses by reducing unnecessary care?
(2) The decline in physician practice expense is especially counterintuitive: Why are the doctors so supportive if they are working harder but making less money?
Why wouldn’t doctors be happy that their practice expenses are declining?
(3) Even though somehow savings were shown in physician expense, per capita doctor visits did indeed increase. More concerning was that specialist visits –which are supposed to decline in a PCMH model – also increased.
Perhaps these doctors are doing fewer high cost procedures and using the time to see more patients?
(4) Inpatient expense fell 47%. This was achieved despite the fact that all the AHRQ’s “Ambulatory Care –Sensitive Conditions” total to about 20% of admissions in most populations.
This sentence makes no sense. It needs more explanation.
(5) The evaluators (William M. Mercer) are on record as saying that “choice [emphasis mine] of trend has a large impact on estimates of financial savings.” Perhaps it is possible that Mercer, having given themselves this latitude, “chose” a trend that would make the study look good.
It appears that this study looks at the savings versus the “trend” which I assume is the expected increase without PMCH. You can quibble about their chosen trend but the fact that PMCH has been able to “bend the curve” is significant.
Also, you claim to have found data which “invalidates” the study. Just a brief look at this shows that your chosen data has different numerators, denominators and probably different time series so it’s not comparable.
You belief that PMCH cannot work is not sufficient. You need some real arguments and real data, not just a personal bias.
Die quickly and be Happy!
LOL. In another context, it’s called “Creation Science.”
When it comes to these surveys of the government, the conclusion is written first and the data massaged to support that conclusion.
Brad, didn’t see yours before I replied. Looks like we had the same reaction.
Those are truly stunning numbers from North Carolina and they deserve close scrutiny. I would like to see a more even-handed treatment, though. It feels like there is an axe being ground in this piece.
It may turn out that some of the data is suspect (Inparticular, Al may have a good point about inpatient and neonatal savings), but that doesn’t throw everything into question. There are also some critical asides that are way too quick. Why should we necessarily be worried if physician payments went down for both primary care and specialties while visits went up. That could be a wonderful result, depending on the details. I would love to see a response to this post from someone who knows the data well.
Al,
Please clarify:
Docs could be happier because of QOL and happiness quotient improved in this setting?
Did the primary care docs earn more, ie, did their salaries go up?
Specialists: How were they incentivized, and even with increased volume, payment structure might have changed, reducing overall costs? Additionally, with ? improved care coordination, other costs may have dropped beyond whats apparent through referrals.
AHRQ ambi-sensitive condtions not the final word. Did you look at what DRG admits actually decreased; what % of ambi sensitive patients were reduced, and at what savings. Need $$$ data.
What are other costs trends that could be used for comparison and by what magnitude do they differ from Mercer?
I could go on. Anyway, seeing as NC PCMH foray widely quoted, I would want to see a part II post where you substantiate your interesting findings. You have put some interesting stuff up for thought, but part I needs to ripen. Please post again or expand here.
Brad