There have been a number of research studies published that question the value of Electronic Health Records (EHRs), particularly as it pertains to improving quality of care and ultimately outcomes. Chilmark has always viewed these reports with a certain amount of skepticism. Simple logic leads us to conclude that a properly installed (including attention to workflow and thorough training) of an enterprise software system such as an EHR will lead to a certain level of standardization in overall process flow, contribute to efficiencies and quality in care delivery and ultimately lead to better outcomes. But to date, there has been a dearth of evidence to support this logic, that is until this week.

Last week the New England Journal of Medicine published the research paper: Electronic Health Records and Quality of Diabetes Care, which provides clear evidence, albeit a little fuzzy around the edges, that physician use of an EHR significantly improves quality metrics over physicians who rely on paper-based medical record keeping processes.

The research effort took place in Cleveland as part of Better Health Greater Cleveland from July 2009 till June 2010 and included 46 practices representing some 569 providers and over 27K adults with diabetes who visited their physician at least twice during the study period. Several common quality and outcome measures were used to assess and compare EHR-based care to paper-based. On composite standards of quality, EHR-based practices performed a whooping 35% better than their paper-based counterparts. On outcome measures, which are arguably more difficult for physicians as patients’ actions or lack thereof are more integral to final outcomes, EHR-based practices still outperformed their paper-based peers by some 15%. The Table below gives a more detailed breakout.

While the authors claim that insurance coverage has little bearing on the final analysis (i.e., Medicare, commercial and Medicaid patient metrics are similar) there is a surprisingly high percentage of patients in paper-based practices who do not have any insurance which makes one wonder: Will future Health Insurance Exchanges (HIX) and the individual mandate, should it survive the Supreme Court, have some bearing on what physician a patient may chose in the future? Will patients migrate to those doctors that use more advanced technologies (EHRs)? Also, there was an abnormally high percentage of patients in paper-based practices that were “Nonwhite” which raises another question: Could those practices that still rely on paper-based processes be in more disadvantaged neighborhoods? If that is indeed the case, will HITECH and its incentives trickle down to this strata of the healthcare sector? All in all though, these are relatively minor points in relation to the broader implications of this paper.

This research paper could become a seminal piece in support of the current administration’s efforts to reform the healthcare sector as it not only supports efforts to digitize the healthcare sector via EHR adoption, but may also provide an added incentive that goes beyond HITECH Act incentive payments.

Throughout the healthcare sector reimbursement models are changing from fee for service to value-based contracts. Such value-based contracts, be they ACOs, PCMHs, P4P, or whatever other acronym you want to throw at it, are accelerating coming not only from the government but also commercial payers. A key component of these value-based contracts is achieving certain quality metrics and moving from episodic care to continuous care models. This research paper is one of the first and most comprehensive that has come across our desks here at Chilmark Research that clearly shows the use of an EHR has a significant impact on key quality measures, in this case diabetes care. While virtually all hospitals are on the HITECH bandwagon, it is less clear just how many private physician practices are jumping in and adopting EHRs for their practices. For many such practices, the HITECH incentive payments may not be enough of a reward for the numerous Meaningful Use hurdles that a physician needs to jump through. But if you hit these physicians directly in their wallet with value-based contracts and they see that EHRs provided demonstrably better quality care metrics, then we may see broader EHR adoption in the ambulatory sector. Bit of a crystal ball forecast, but the logic is there.

Ultimately, what we are seeing happen in the healthcare sector is not dissimilar to what we saw occur in the manufacturing sector. In manufacturing the lag between adoption of enterprise software systems and subsequent increases in productivity has a special term: the “Productivity Paradox,” wherein it was some ten years after wide spread adoption and deployment of these enterprise systems that improvements in productivity metrics could be measured.

Might the healthcare sector have its own paradox? We think so and from this point forward will refer to it as the Quality Paradox.

John Moore is an IT Analyst at Chilmark Research, where this post was first published.

8 Responses for “The Cleveland Experiment”

  1. pcp says:

    The demographic differences between the two patient groups are so enormous as to make it hard to see how they can be “adjusted” for. Studies like this could well be used to discriminate against those few remaining docs who care for the neediest members of society.

    • pcp says:

      Oh, the OP answered my concerns.

      The patients in the practices using paper records are uninsured and non-white at much higher rates than in the EMR records. “These are relatively minor points,” he tells us, so it’s fine to pay their docs less under “value-based contracts.”

  2. eppie ward says:

    Key is that, Quality of input always affects quality of outcome. The government had planned to mandate that all physicians would be using EHR by 1997. I know there were earlier mandates but that was when they had planned to review billing, utilization review and quality outcomes of medicare and medicaid from physician’s offices via the internet. It was otherwise cost ineffective to send auditors out to that many doctor’s offices. So, they had to dump that program. Doctors are still dragging their heels, gripping about computer compatibility when actually alll of it could be done over the internet via the web so cost is not is not the factor they say it is. To race base it is the same old excuse that everyone uses from education (there are many studies that show all students can get a quality education like they use to when I was a girl without much money. Computers are not costly, it is that administration is too top heavy) to zoos (again too top heavy. Shake all those administrative branches and you can get many more banana trees for your money to feed the monkeys)

  3. I have three issues with this report:

    1) The demographics are horrendously skewed, as pcp noted above

    2) The EHR cohort consist of well over 24,000 patients and over 500 doctors, while the paper practices are represented by 2660 patients and 53 doctors. Are these paper numbers even statistically significant?

    3) What was the situation before EHRs were introduced? Were all the practices doing equally poorly and the EHR changed that? Or were the relative numbers similarly distributed before EHR?
    In other words, does introduction of an EHR improve quality, or quality driven practices tend to install EHRs?

  4. sr says:

    I hope they follow these patients long-term to determine the effect on common diabetes complications, such as heart disease, blindness, renal failure, amputations. That will go a long way to show the value of managing these metrics – to individuals and to society.

  5. Curly Harrison, MD says:

    The study is flawed by selection bias and investigator bias.

    This report simply reaffirms that wealthy and educated people are more compliant and achieve better outcomes.

    The pro HIT bias of the NEJM and the ONC is more than evident.

  6. Bobby Lee says:

    My main question is whether the compliance to standard protocol is the real driver to outcomes difference rather than EHR.

    EHR –> helps with getting on standard protocl –> better outcome, not
    EHR = better outcome.

    Like to see two groups, both using EHR, but compliancy to the protocol being the variance to see how the outcomes measure up.

Leave a Reply


Founder & Publisher

Executive Editor

Editor, Business of Healthcare

Contributing Editor

Contributing Editor

Business Development

Editor-At-Large, Wellness

Editor-At-Large, Europe



The Health Care Blog (THCB) is based in San Francisco. We were founded in 2003 by Matthew Holt. John Irvine joined a year later and now runs the site.


Interview Requests + Bookings. We like to talk. E-mail us.

Yes. We're looking for bloggers. Send us your posts.

Breaking health care story? Drop us an e-mail.


We frequently accept crossposts from smaller blogs and major U.S. and International publications. You'll need syndication rights. Email a link to your submission.


Op-eds. Crossposts. Columns. Great ideas for improving the health care system. Pitches for healthcare-focused startups and business.Write ups of original research. Reviews of new healthcare products and startups. Data-driven analysis of health care trends. Policy proposals. E-mail us a copy of your piece in the body of your email or as a Google Doc. No phone calls please!


Healthcare focused e-books and videos for distribution via THCB and other channels like Amazon and Smashwords. Want to get involved? Send us a note telling us what you have in mind. Proposals should be no more than one page in length.

If you've healthcare professional or consumer and have had a recent experience with the U.S. health care system, either for good or bad, that you want the world to know about, tell us about it. Have a good health care story you think we should know about? Send story ideas and tips to

REPRINTS Questions on reprints, permissions and syndication to



Affordable Care Act
Business of Health Care
National health policy
Life on the front lines
Practice management
Hospital managment
Health plans
Specialty practice
Emergency Medicine
Quality, Costs
Medical education
Med School
Public Health

Electronic medical records
Accountable care organizations
Meaningful use
Online Communities
Open Source
Social media
Tips and Tricks


Health 2.0
Log in - Powered by WordPress.