Harvard Study Gets it Wrong on EHRs and Quality

America’s hospitals are a triumph of modernity, stocked as they are with PET scanners, ECMO machines, and ICUs bedecked in eye-popping gadgetry.

They are also the most complex organizations ever created by man. The seemingly simple process of delivering a drug from the pharmacy to the bedside for example, typically involves a 30-step process executed by a half-dozen people on 3 floors. There are hundreds of ways it can fail.

It often does, and that’s just half the story. Each hospitalized patient requires a unique combination of services including lab tests, physical therapy, a discharge plan and so forth. Since a complex process must be executed to produce each service, the hospital becomes a job shop.

By contrast, the processes used to produce cars and silicon chips are relatively unfettered. That is why piston rods can be produced in batches with every item meeting specs to the micron, while hospital processes often feature error rates of 10-20%.

This explains why hospitals have struggled for decades to improve quality. It also explains why a study by Ashish Jha and colleagues at Harvard has shown that hospitals using electronic health records (EHRs) don’t have better quality.

Jha’s team divided 3,000 US hospitals into 3 groups based on the extent to which they had adopted EHRs. One group had deployed “comprehensive” EHRs throughout the facility. A middle group had implemented “partial” EHRs. A third group did things the old-fashioned way.

The scientists compared the groups using federally approved quality measures for congestive heart failure, pneumonia and surgical infections, and found minimal differences.

In the heart failure patients for example, hospitals that had deployed full-blown EHRs met best-practice standards 87.8% of the time. Hospitals in the middle group did so 86.7% of the time and the EHR laggards clocked-in at 85.9%. Similarly trivial differences were noted for other conditions and length of stay, a measure of hospital efficiency.

These differences were “really, really marginal,” Jha told the New York Times. “The way electronic medical records are used now has not yet had a real impact on the quality or cost of health care.”

Jha and many others that have commented on his research have it all wrong.

EHRs aren’t intended to fix the complex care systems which cause quality problems in hospitals. In fact, operations researchers have told us for decades that automating error-prone processes will, perversely, produce more errors in the same amount of time.

Outpatient Settings are Different

Things are different in solo and small group practices. Although harried physicians and office staff in such venues might not believe it, the processes in which they work are less complex than those in hospitals.

For one thing, fewer people are involved. This means there are fewer information hand-offs and the errors that accompany them.

In addition, most people in solo and small group practices work within shouting distance of each other. This means process failures can be corrected in real time. And if that doesn’t work, the process can often be redesigned over lunch.

Furthermore, in solo or small group practices the key aspects of clinical quality usually do come down to decisions made by physicians. Did the doctor order the colonoscopy for the 61 year old male? Did she account for drug allergies when she wrote the prescription?

Anyone that has participated in a hospital-based effort to reduce length of stay will understand that physicians cannot be held accountable for quality in such complex systems.

It follows that in small outpatient settings (but not hospitals), EHRs can favorably impact the quality of care. The EHR can provide decision support to the physician at the time she needs it. The informed physician makes a better decision and quality improves.

Unfortunately, Jha’s study didn’t focus on outpatient settings. In part, this is because EHR penetration has been poor in those venues, with the major barriers being cost and the inability to accommodate office workflow.

The advent of inexpensive, easily modifiable Web-based EHRs ought to change all that.

Livongo’s Post Ad Banner 728*90

Categories: Uncategorized

Tagged as: , ,

Leave a Reply

8 Comment threads
0 Thread replies
Most reacted comment
Hottest comment thread
6 Comment authors
marcMark ScrimshirerbarMichael Kirsch, M.D.Regina Holliday Recent comment authors
newest oldest most voted

Dr. Jha has taken his observations from siloed EMR utility (today’s world – and his base for the study) and extrapolated with sophomoric fervor and a flawed design of a study, the benefits of meaningful use of EHRs – QI. He seems to have discounted generously and with certain obfuscation by design, the fact there is not much data yet on meaningful use of EHRs, which by definition needs to be interoperable and exchanged. HIE leads to better information at point of care – the edge – ambulatory environment – keeping people away from the very same hospital that Dr.… Read more »


Every sane person should try to “stay out of the hospital”, as the hospital is a place of suffering, death, nosocomial infection (regardless of optimal hygiene measures) etc.
It is unique to the US to what extent hospitals have, at least partially, degenerated into semicomfy places with waterfalls and pianos in the lobby, superfluous hysterectomies and back surgeries … it is not only the docs who are to blame for this development. This is a cultural issue.

Mark Scrimshire

One way to read this study is: “Stay out of hospital”
If the Health care system is so complex then patients need to get engaged in their health and “Trust but Verify”.


I get the feeling that Dr. Laffel, had the Harvard group chosen to examine small private practice, would have made the inverse argument: that it is complex hospital care that benefits from EMR use. Intuitively, the inverse argument makes more sense: it is complex hospital where we need to have the notes, pics and labs always at hand, and allergy checking matters most in inpatients. In fact, I am convinced that EMR matter most in urban MSG settings; there you have a high density of chronically ill, complex patients that could show to the ER at any time and often… Read more »

Michael Kirsch, M.D.

Sebelius was shameful. She should have embraced the USPSTF revised guidelines instead of trashing them for patently political reasons. The USPSTF is under her own agency. The biggest casualty is comparative effectiveness research, which lost a great opportunity to show the nation how it would work. Instead, our bloated health care system took on additional weight. More at http://bit.ly/656CwP

Regina Holliday

John has avery valid point about the ten year window of implementation. IT is not about only the day to day operation of a facility, it also about data history. This is where real benefits can often be seen. A fully accessible and interoperable and transferable EMR/EHR system would have amazing time saving value. We have seen this revolution in business. Sure, retail businesses love the speed of check-out with a computer based register, but the real time saver is behind the scene. That sales history effects buying decisions, staffing decisions and inventory levels. The IT in place gives the… Read more »


All medical records, elecronic or otherwise, are full of errors and misinformation. Records cannot be used reliably to improve care. Only better people can improve care.


Geez, this is such a silly argument as to almost be laughable. Healthcare loves to say they are different, that this differences prevents them from adopting best practices from other industries. Yes, healthcare is different, I’ll give you that much, but can it learn from other industries, absolutely. Stop the whining. One aspect of Jha’s research that is not discussed is that in any industry sector, it takes a significant amount of time before one actually sees the results in productivity, quality etc from the deployment of IT. This has been studied extensively where in mfg, where IT was adopted… Read more »