By Ashish Jha, MD

Should we hold hospitals accountable for what happens after a patient leaves the hospitals’ doors? A year ago, I thought the answer was no. A hospital’s job was to take care of sick patients, make them better and send them on their way. With more thought and consideration, I have come to conclude that I was probably wrong. It may be perfectly reasonable to hold hospitals accountable for care beyond their walls, but we should be clear why we’re doing it. Readmissions are not a good quality measure – but they may be a very good way to change the notion of accountability within the healthcare delivery system.

The debate around the readmissions measure has come to the forefront because of the CMS Hospital Readmission Reduction Program, which penalizes hospitals for “greater than expected” readmission rates. It has raised the question — does a hospital’s 30-day readmission rate measure the “quality of care” it provides? Over the last three years, the evidence has come in, and to my read, it is unequivocal. By most standards, the readmissions metric fails as a quality measure.

Why do I say that readmissions are a poor measure of hospital quality? First, we have to begin by thinking about what makes a good quality measure. Quality is about the essence of the thing being produced – a good or a service. The job of a car is to get you from place A to place B and a high-quality car may be one that does the job reliably, safely, or maybe even comfortably. The job of a restaurant is to provide a meal that you don’t have to cook — and a high quality restaurant may provide food that is fresh, tasty, or with an attention to service that you enjoy. What is the job of a hospital? When you get sick and require hospital services, a high-quality hospital should give you the right treatments, attend to your needs while you’re there, and make sure nothing bad (i.e. a new nosocomial infection) happens along the way. That’s how we measure hospital quality.

Quality measures for healthcare come in three flavors – structural measures (do you have enough intensivists manning your ICU?), processes measures (did you give the heart attack patient his or her aspirin?) and outcomes measures (did the pneumonia patient die?). The elemental part of both structural measures and process measures is that they have to be tied to an outcome we care about. If having more intensivists in the ICU does not lead to lower ICU mortality (or lower complication rates), we wouldn’t think it’s a particularly good quality measure. We know that giving aspirin to heart attack patients lowers their chances of dying by 25%. We have multiple randomized trials. We don’t need much more evidence. Hospitals that have the right structures in place and reliably deliver the right treatments can reasonably be called high-quality hospitals.

What about outcomes measures? It gets trickier. An “outcome” is a health state – and the goal of healthcare is to maximize the health of the patient. Death is clearly a bad outcome (with the caveat that for someone with a terminal illness, death may be an expected outcome). Nosocomial infections are bad outcomes. Is being readmitted a bad outcome? It’s a little funny – hospitalization or re-hospitalization is a not a health state. It’s a clinical service provided to people who are sick. While no one likes to be hospitalized or readmitted, being admitted doesn’t make you sick. Being sick gets you admitted (or readmitted). If we had a sick patient in the ER who was last discharged from the hospital 28 days prior, we don’t make them better by sending them home instead of admitting them. Yes, sending them home avoids a readmission – but the goal is not to avoid readmission, the goal is to make people better.

When we develop “quality” measures, we also look for external validity. We know that there are terrific hospitals and mediocre ones. Arguably, the gold standard for measuring hospital quality is condition-specific mortality rates. Almost by definition, good hospitals are those that have low risk-adjusted mortality rates while bad ones have high risk-adjusted mortality rates. So when we consider other potential quality measures, we may begin by asking if they are correlated with the gold standard – mortality rates. Do process measures have external validity (i.e. do hospitals that give aspirin to their heart attack patients tend to have better mortality?)

We know that they do and although the relationship between process measures and the gold standard outcome (mortality) is not very strong, it adds one piece of evidence that process measures may be picking up something good. Quality. What about patient experience scores? You could argue that higher satisfaction is a health state unto itself and doesn’t need to have external validation – and that’s a reasonable argument. Further, we do know from studies that hospitals with better patient experience scores tend to have better mortality rates. No one thinks that patients are happier at these hospitals because these hospitals are keeping patients alive. Instead, these hospitals are generally well run, and therefore, have better performance across the board.

So if one measure of quality is external validity – being at least somewhat correlated with the gold standard (mortality rates) — how does the readmission measure do? In a paper published recently in JAMA, we see that readmission rates don’t do so well at all. Readmission rates are un-correlated with mortality rates. In fact, for one of the three conditions, the readmission rate seems to go the wrong way: the best hospitals for heart failure (i.e. those with the lowest mortality rates) have readmission rates that are actually higher. Not perfect. Readmissions seem to have little external validity as a quality measure. Readmissions are, however, correlated with two things: how sick your patients are, and how poor your patients are. We now have good data that the Hospital Readmission Reduction Program disproportionately penalizes big academic teaching hospitals (that care for the sickest patients) and safety-net hospitals (that care for the poorest). See table 1 below.

So, given its poor test characteristics, can we justify using the current hospital readmissions measure to grade hospitals on quality? I don’t think we can. However, here’s where my own ideas have evolved (OK – another way of saying that I was probably short-sighted in the past), the 30-day readmission measure may be a good way to promote accountability in healthcare. Let’s think about what that might mean.

Right now, in our very fragmented healthcare system, no one seems accountable for what happens to the patient after they leave. No one is responsible for ensuring that patients get good follow-up or even that someone calls the patients a few days after they are discharged to make sure they are doing well. There are probably a dozen other things that some creative entrepreneur can come up with to make sure that patients who leave the hospital do well, recover quickly, and don’t get sick again in a way that requires them having to come back to the hospital. Unfortunately, our system generally doesn’t reward any of that. No one gets rich keeping patients healthy and well (and therefore, not needing to be hospitalized). The Accountable Care Organization (ACO) program begins to do that, and I am hopeful it will make a difference. But the truth is that most patients are not in an ACO and won’t be anytime soon.

In conversations with colleagues and friends, the readmissions penalty program seems to have gotten some hospitals to think outside of their four walls. Hospital leadership has started to rethink the role of the hospital. Hospitals are building relationships with community-based organizations. Some are creating follow-up clinics while others are calling all the patients who are discharged to make sure they are doing OK at home. Of course, some hospitals are taking the opposite approach – admitting more patients to “observation” status (so they don’t get penalized for readmissions), a move that saddles many patients with extra healthcare bills. Others are even sending patients home from the ER if they are in the 30 day window when clinically, they should have been admitted. This is the consequence of using a measure that is very narrowly tailored. If the patient is sent home and dies, the hospital will be penalized much less than if the patient is readmitted. Not very patient-centered.

Getting hospitals to change their business model – to start thinking about what happens to a patient when they head out the door – is a good thing. But if we’re asking hospitals to change, and be responsible for everything that happens in the first 30 days after discharge, then we should rethink how we pay hospitals. Here is the promise of “bundled payments” and personally, I’d be much happier if we just bundled all the services that a patient receives after discharge into the initial hospitalization. We should also make sure that we pay hospitals that care for the sickest patients a lot more, because they will need a lot more post-discharge care. Finally, we should measure the quality of care that patients receive in those days after discharge. If CMS wants to use the readmissions program as a way to rethink the hospital episode – I’m all for it. But, it should be far more comprehensive than whether the patient was readmitted to the hospital within 30 days or not.

The readmissions program seems to be, for some hospitals, having a positive effect. Will it pay off? Will we see a real, sustained change in the way they provide care to patients after they are discharged? I hope so. But remember – some of the best hospitals in America have the highest readmission rates, almost surely because they care for sicker, poorer patients. See table 2 below. In the current business model, they are doing things right – taking good care of the patient while the patient is in the hospital. It’s fine to ask these hospitals to change their business model and to become accountable for what happens to their patients after they are discharged. But, let’s not call them bad hospitals or suggest that they are providing poor quality care. There is no evidence that they are.

Table 1. Hospital Characteristics by Penalty Group from the Hospital Readmissions Reduction Program*

*From Joynt and Jha, Characteristics of Hospitals Receiving Penalties under the Hospital Readmissions Readuction Program, JAMA 2013; 309(4) 342-343

Table 2. U.S. News Top 50 hospitals* on readmissions and mortality rates

*Top 50 in cardiovascular or pulmonary disease

Ashish Jha, MD, MPH is the C. Boyden Gray Associate Professor of Health Policy and Management at the Harvard School of Public Health. He blogs at An Ounce of Evidence.He will serve as a Senior Editor-in-Chief for Healthcare: The Journal of Delivery Science and Innovation. Submission is open now, and the first issue will be released in late spring 2013.