The Cleveland Experiment

There have been a number of research studies published that question the value of Electronic Health Records (EHRs), particularly as it pertains to improving quality of care and ultimately outcomes. Chilmark has always viewed these reports with a certain amount of skepticism. Simple logic leads us to conclude that a properly installed (including attention to workflow and thorough training) of an enterprise software system such as an EHR will lead to a certain level of standardization in overall process flow, contribute to efficiencies and quality in care delivery and ultimately lead to better outcomes. But to date, there has been a dearth of evidence to support this logic, that is until this week.

Last week the New England Journal of Medicine published the research paper: Electronic Health Records and Quality of Diabetes Care, which provides clear evidence, albeit a little fuzzy around the edges, that physician use of an EHR significantly improves quality metrics over physicians who rely on paper-based medical record keeping processes.

The research effort took place in Cleveland as part of Better Health Greater Cleveland from July 2009 till June 2010 and included 46 practices representing some 569 providers and over 27K adults with diabetes who visited their physician at least twice during the study period. Several common quality and outcome measures were used to assess and compare EHR-based care to paper-based. On composite standards of quality, EHR-based practices performed a whooping 35% better than their paper-based counterparts. On outcome measures, which are arguably more difficult for physicians as patients’ actions or lack thereof are more integral to final outcomes, EHR-based practices still outperformed their paper-based peers by some 15%. The Table below gives a more detailed breakout.

While the authors claim that insurance coverage has little bearing on the final analysis (i.e., Medicare, commercial and Medicaid patient metrics are similar) there is a surprisingly high percentage of patients in paper-based practices who do not have any insurance which makes one wonder: Will future Health Insurance Exchanges (HIX) and the individual mandate, should it survive the Supreme Court, have some bearing on what physician a patient may chose in the future? Will patients migrate to those doctors that use more advanced technologies (EHRs)? Also, there was an abnormally high percentage of patients in paper-based practices that were “Nonwhite” which raises another question: Could those practices that still rely on paper-based processes be in more disadvantaged neighborhoods? If that is indeed the case, will HITECH and its incentives trickle down to this strata of the healthcare sector? All in all though, these are relatively minor points in relation to the broader implications of this paper.

This research paper could become a seminal piece in support of the current administration’s efforts to reform the healthcare sector as it not only supports efforts to digitize the healthcare sector via EHR adoption, but may also provide an added incentive that goes beyond HITECH Act incentive payments.

Throughout the healthcare sector reimbursement models are changing from fee for service to value-based contracts. Such value-based contracts, be they ACOs, PCMHs, P4P, or whatever other acronym you want to throw at it, are accelerating coming not only from the government but also commercial payers. A key component of these value-based contracts is achieving certain quality metrics and moving from episodic care to continuous care models. This research paper is one of the first and most comprehensive that has come across our desks here at Chilmark Research that clearly shows the use of an EHR has a significant impact on key quality measures, in this case diabetes care. While virtually all hospitals are on the HITECH bandwagon, it is less clear just how many private physician practices are jumping in and adopting EHRs for their practices. For many such practices, the HITECH incentive payments may not be enough of a reward for the numerous Meaningful Use hurdles that a physician needs to jump through. But if you hit these physicians directly in their wallet with value-based contracts and they see that EHRs provided demonstrably better quality care metrics, then we may see broader EHR adoption in the ambulatory sector. Bit of a crystal ball forecast, but the logic is there.

Ultimately, what we are seeing happen in the healthcare sector is not dissimilar to what we saw occur in the manufacturing sector. In manufacturing the lag between adoption of enterprise software systems and subsequent increases in productivity has a special term: the “Productivity Paradox,” wherein it was some ten years after wide spread adoption and deployment of these enterprise systems that improvements in productivity metrics could be measured.

Might the healthcare sector have its own paradox? We think so and from this point forward will refer to it as the Quality Paradox.

John Moore is an IT Analyst at Chilmark Research, where this post was first published.

8 replies »

  1. My main question is whether the compliance to standard protocol is the real driver to outcomes difference rather than EHR.

    EHR –> helps with getting on standard protocl –> better outcome, not
    EHR = better outcome.

    Like to see two groups, both using EHR, but compliancy to the protocol being the variance to see how the outcomes measure up.

  2. The study is flawed by selection bias and investigator bias.

    This report simply reaffirms that wealthy and educated people are more compliant and achieve better outcomes.

    The pro HIT bias of the NEJM and the ONC is more than evident.

  3. I hope they follow these patients long-term to determine the effect on common diabetes complications, such as heart disease, blindness, renal failure, amputations. That will go a long way to show the value of managing these metrics – to individuals and to society.

  4. I have three issues with this report:

    1) The demographics are horrendously skewed, as pcp noted above

    2) The EHR cohort consist of well over 24,000 patients and over 500 doctors, while the paper practices are represented by 2660 patients and 53 doctors. Are these paper numbers even statistically significant?

    3) What was the situation before EHRs were introduced? Were all the practices doing equally poorly and the EHR changed that? Or were the relative numbers similarly distributed before EHR?
    In other words, does introduction of an EHR improve quality, or quality driven practices tend to install EHRs?

  5. Key is that, Quality of input always affects quality of outcome. The government had planned to mandate that all physicians would be using EHR by 1997. I know there were earlier mandates but that was when they had planned to review billing, utilization review and quality outcomes of medicare and medicaid from physician’s offices via the internet. It was otherwise cost ineffective to send auditors out to that many doctor’s offices. So, they had to dump that program. Doctors are still dragging their heels, gripping about computer compatibility when actually alll of it could be done over the internet via the web so cost is not is not the factor they say it is. To race base it is the same old excuse that everyone uses from education (there are many studies that show all students can get a quality education like they use to when I was a girl without much money. Computers are not costly, it is that administration is too top heavy) to zoos (again too top heavy. Shake all those administrative branches and you can get many more banana trees for your money to feed the monkeys)

  6. Oh, the OP answered my concerns.

    The patients in the practices using paper records are uninsured and non-white at much higher rates than in the EMR records. “These are relatively minor points,” he tells us, so it’s fine to pay their docs less under “value-based contracts.”

  7. The demographic differences between the two patient groups are so enormous as to make it hard to see how they can be “adjusted” for. Studies like this could well be used to discriminate against those few remaining docs who care for the neediest members of society.