Harvard Study Gets it Wrong on EHRs and Quality

America’s hospitals are a triumph of modernity, stocked as they are with PET scanners, ECMO machines, and ICUs bedecked in eye-popping gadgetry.

They are also the most complex organizations ever created by man. The seemingly simple process of delivering a drug from the pharmacy to the bedside for example, typically involves a 30-step process executed by a half-dozen people on 3 floors. There are hundreds of ways it can fail.

It often does, and that’s just half the story. Each hospitalized patient requires a unique combination of services including lab tests, physical therapy, a discharge plan and so forth. Since a complex process must be executed to produce each service, the hospital becomes a job shop.

By contrast, the processes used to produce cars and silicon chips are relatively unfettered. That is why piston rods can be produced in batches with every item meeting specs to the micron, while hospital processes often feature error rates of 10-20%.

This explains why hospitals have struggled for decades to improve quality. It also explains why a study by Ashish Jha and colleagues at Harvard has shown that hospitals using electronic health records (EHRs) don’t have better quality.

Jha’s team divided 3,000 US hospitals into 3 groups based on the extent to which they had adopted EHRs. One group had deployed “comprehensive” EHRs throughout the facility. A middle group had implemented “partial” EHRs. A third group did things the old-fashioned way.

The scientists compared the groups using federally approved quality measures for congestive heart failure, pneumonia and surgical infections, and found minimal differences.

In the heart failure patients for example, hospitals that had deployed full-blown EHRs met best-practice standards 87.8% of the time. Hospitals in the middle group did so 86.7% of the time and the EHR laggards clocked-in at 85.9%. Similarly trivial differences were noted for other conditions and length of stay, a measure of hospital efficiency.

These differences were “really, really marginal,” Jha told the New York Times. “The way electronic medical records are used now has not yet had a real impact on the quality or cost of health care.”

Jha and many others that have commented on his research have it all wrong.

EHRs aren’t intended to fix the complex care systems which cause quality problems in hospitals. In fact, operations researchers have told us for decades that automating error-prone processes will, perversely, produce more errors in the same amount of time.

Outpatient Settings are Different

Things are different in solo and small group practices. Although harried physicians and office staff in such venues might not believe it, the processes in which they work are less complex than those in hospitals.

For one thing, fewer people are involved. This means there are fewer information hand-offs and the errors that accompany them.

In addition, most people in solo and small group practices work within shouting distance of each other. This means process failures can be corrected in real time. And if that doesn’t work, the process can often be redesigned over lunch.

Furthermore, in solo or small group practices the key aspects of clinical quality usually do come down to decisions made by physicians. Did the doctor order the colonoscopy for the 61 year old male? Did she account for drug allergies when she wrote the prescription?

Anyone that has participated in a hospital-based effort to reduce length of stay will understand that physicians cannot be held accountable for quality in such complex systems.

It follows that in small outpatient settings (but not hospitals), EHRs can favorably impact the quality of care. The EHR can provide decision support to the physician at the time she needs it. The informed physician makes a better decision and quality improves.

Unfortunately, Jha’s study didn’t focus on outpatient settings. In part, this is because EHR penetration has been poor in those venues, with the major barriers being cost and the inability to accommodate office workflow.

The advent of inexpensive, easily modifiable Web-based EHRs ought to change all that.

Categories: Uncategorized

Tagged as: , ,

8 replies »

  1. Dr. Jha has taken his observations from siloed EMR utility (today’s world – and his base for the study) and extrapolated with sophomoric fervor and a flawed design of a study, the benefits of meaningful use of EHRs – QI. He seems to have discounted generously and with certain obfuscation by design, the fact there is not much data yet on meaningful use of EHRs, which by definition needs to be interoperable and exchanged.
    HIE leads to better information at point of care – the edge – ambulatory environment – keeping people away from the very same hospital that Dr. Jha beholds in his study of “half-truths”. The end-result of “real” meaningful use is QI – something which has yet to arrive.
    Dr. Jha and the NY Times (the real culprits in this propagation of half-truths) have tried in vain to pollute the real discourse in meaningful use. Give this study an “F” grade without reservation.

  2. Every sane person should try to “stay out of the hospital”, as the hospital is a place of suffering, death, nosocomial infection (regardless of optimal hygiene measures) etc.
    It is unique to the US to what extent hospitals have, at least partially, degenerated into semicomfy places with waterfalls and pianos in the lobby, superfluous hysterectomies and back surgeries … it is not only the docs who are to blame for this development. This is a cultural issue.

  3. I get the feeling that Dr. Laffel, had the Harvard group chosen to examine small private practice, would have made the inverse argument: that it is complex hospital care that benefits from EMR use.
    Intuitively, the inverse argument makes more sense: it is complex hospital where we need to have the notes, pics and labs always at hand, and allergy checking matters most in inpatients.
    In fact, I am convinced that EMR matter most in urban MSG settings; there you have a high density of chronically ill, complex patients that could show to the ER at any time and often might give you no (or an incomplete) history.
    I have worked with 3 different EMR over a decade, in OP and IP settings, and I personally believe that EMR improve quality and efficiency significantly in the care of complex patients, but all in all, the gains are modest and the effort considerable.
    If we want to improve quality and efficiency in the US, we should vigorously enhance:
    -teach the population/patients what EBM is and that more is not always better (that would mean an organized campaign, which of course is not necessarily in the campaigners financial interest)
    -change the medicare fee schedule that is heavily tilted towards procedures
    -hand washing in the hospital
    But of course computer use must be paramount, right? Since IT gave us Email, facebook, wikipedia and kitten videos on youtube, they MUST revolutionize health care, right?

  4. Sebelius was shameful. She should have embraced the USPSTF revised guidelines instead of trashing them for patently political reasons. The USPSTF is under her own agency. The biggest casualty is comparative effectiveness research, which lost a great opportunity to show the nation how it would work. Instead, our bloated health care system took on additional weight. More at http://bit.ly/656CwP

  5. John has avery valid point about the ten year window of implementation. IT is not about only the day to day operation of a facility, it also about data history. This is where real benefits can often be seen. A fully accessible and interoperable and transferable EMR/EHR system would have amazing time saving value. We have seen this revolution in business. Sure, retail businesses love the speed of check-out with a computer based register, but the real time saver is behind the scene. That sales history effects buying decisions, staffing decisions and inventory levels. The IT in place gives the sales staff and manager more time on the sales floor to help customers.
    What an amazing idea to apply to medicine. On the sixth floor a nurse grabs a catheter kit, scan the barcode and the system prints inventory level reports or is set for auto re-order. Down in admission Patient X does not have to waste time and try to recall their medical history because it is already in the computer. Doctors and nurses do rounds with WIFI computers and are able to access records and use drop down windows to see test results before talking with the patients. With integrated IT the barriers to patient access to data fall down. The patient/caregiver can also read their record. Just as we read over our bank statement looking for errors and staying informed about our financial health, we would do the same with medicine. This alone will help with medical errors in the record.
    The IT functions I mentioned above took 20 years to fully implement and integrate in business world. It also contained a generational shift in employees as people who refused to work on the new computerized systems had to find other employment. I am well aware of the time-consuming stress of implementing a new system, but it is well worth the work

  6. All medical records, elecronic or otherwise, are full of errors and misinformation. Records cannot be used reliably to improve care. Only better people can improve care.

  7. Geez, this is such a silly argument as to almost be laughable. Healthcare loves to say they are different, that this differences prevents them from adopting best practices from other industries. Yes, healthcare is different, I’ll give you that much, but can it learn from other industries, absolutely. Stop the whining.
    One aspect of Jha’s research that is not discussed is that in any industry sector, it takes a significant amount of time before one actually sees the results in productivity, quality etc from the deployment of IT. This has been studied extensively where in mfg, where IT was adopted in earnest beginning in the mid-80s, we did not see a corresponding rise in productivity until a decade later. In healthcare, where IT adoption has been anemic, it is not too surprising what Jha and his research team found.
    As for ambulatory care, Jha has also conducted extensive research here as well. Really do not understand your argument as in other developed countries, ambulatory care’s adoption of HIT is far higher than what we have in the US and it is working. Sounds like just another excuse and I think the public at large is getting tired of the excuses.