The Measurement Question

I sat at home with a sense of relief. I had just finished my first month of residency – a grueling inpatient hospital month where I was pushed to new limits.  I now finally had my first “golden weekend” (meaning I had both Saturday and Sunday off). More importantly, I had survived my first month without any patient deaths on my service. Given how sick people are when they come to the hospital, I felt pretty good about this result.

That feeling lasted less than 24 hours. As I logged in from home onto the electronic medical record to finish some documentation, I realized one of my patients was in coma due to a sudden stroke. This patient had few clinical symptoms and appeared the healthiest amongst all the patients I managed the entire month. A heavy knot quickly developed in my stomach, as I could not shed the feeling that perhaps I did something wrong. I scoured the medical records, retracing my management. Over the next couple of days I discussed the case with other colleagues and experts in the field, and read in depth on the management of this condition.  To my relief it was clear that I did not nor did anyone involved in the patient’s care make an error in management. Unfortunately, however, this patient eventually passed away.

As I reflect on the experience, an important point stands out in my mind. This patient exhibited few signs of being “sick” and was managed very well by all the physicians during the course of the hospital stay, but died. On the other end of the spectrum are patients who appear incredibly sick, and despite a poor prognosis survive against odds. One of the goals of residency is to learn to assess a patient and quickly identify who is in imminent danger and may need immediate attention. Unfortunately, however, physicians cannot predict everything, as situations similar to the one above are not uncommon scenarios. Given this fact it makes the discussion about measuring healthcare and pay for performance very cloudy.

Measuring the success of an organization or individual is critical in any field. For some industries or organizations this is a relatively simple process. For example, most private companies in for-profit industries measure their success on profit and increasing shareholder value (though this is becoming more complicated as there has been an increasing number of debates on accountability and the role of business).  Measuring success in any public service or non-profit industry where profit is not the main driver is much more challenging. Healthcare measurement is even more complex because there is so much unpredictability in healthcare outcomes.

Despite the difficulty in measuring health outcomes, it is important that we continue to try to find appropriate yardsticks for measurement. The world renowned economist and father of modern strategy, Michael Porter, has dedicated a significant portion of his time to healthcare lately. He believes that while Healthcare costs are important, there has been too much of a preoccupation on costs instead of driving and improving outcomes. Porter and many others point to the improvement in organ transplant outcomes across the entire nation after measuring outcomes were initiated.

Outcomes and processes are two important indicators often used when one thinks of measuring physician performance. These terms are also used in a slightly different manner to measure health organization performance, but I’ll specifically focus on the physician measurement aspect in this discussion.  While I agree whole-heartedly with the importance of measuring health outcomes, I disagree with the idea of tying physician performance and payments too tightly to health outcomes of patients. Analyzing the patient I mentioned above illustrates that too often a patient’s outcome is not determined by the physician’s actions alone. While physicians play a key role in patients’ health, factors such as patient compliance, genetics, access to transportation, eating habits, education, stress, access to ancillary services, and exercise lifestyle play roles. Ultimately, it is the social determinants of health that will impact the patient’s health outcomes. Even the delivery of purely medical care itself is a team process: doctors, nurses, social workers, pharmacists, physical therapy and so on all coordinate medical care. Physicians have far less control over one’s health than most people perceive. You could put the greatest clinician in the world in the worst system and conditions, and I can almost guarantee the health outcomes of his/her patients will be sub par. For any Detroit Lions football fans who followed Barry Sanders throughout the 1990s, you know exactly what happens when you have a star performer in a below average system. Contrary to what most people think, we need to learn how to find a way of measuring physician performance INDEPENDENT of the patient’s outcomes.

While processes may not be the ideal way to measure physician performance, it does serve some utility. We are simply not at the point where a doctor can guarantee a patient will have their diabetes controlled. What we can do is assure that the doctor’s actions are appropriate – that they check the Hemoglobin A1c regularly, educate the patient, check renal function, inspect the feet and eyes, manage medication appropriately, etc. Measuring processes is often criticized, but this is in fact how most employees in most industries are paid – pay primarily for the employee’s direct actions (process) and a bonus for the success of the organization or final product (outcome). The percentage of that bonus is dependent on the opportunity for the employee to influence the outcome. For instance, some organizations such as those in the fast food industry probably do not want to have too high a company bonus for its employees doing straightforward tasks since they have a relatively small influence on the bottom line, however, a senior VP at a small startup probably may have a large bonus tied to the performance of the organization. The bonus must be commensurate to the potential to influence.

Many organizations have prematurely tied outcomes too tightly to the performance of physicians. In some medical scenarios where there is an exact formula for success (which is rare in medicine) this reimbursement formula may work, but on the aggregate it doesn’t. The ultimate goal of a doctor is to make the patient healthy, and thus having a small bonus for doctors tied to health outcomes may yield a healthy and aligned incentive structure. However, once that bonus grows too large it begins to affect the entire morale of physicians. You end up with a frustrated population of physicians who become upset by the fact that they are measured and reimbursed by things beyond their control. I have long supported the idea that the culture of medicine is one that resists change; however, in this case I think the resistance by physicians to many of the pay-per-performance schemes arising is somewhat justified. Most doctors would love to be in a position where they can guarantee the health of their patient. I plan to spend much of my time dedicated to underserved patients with many barriers to improved health, and thus I would love if there were a full-proof formula to assure my patients’ health. At that point I would certainly welcome an incentive system that reimbursed me for patient outcomes. But until then, I’d prefer to not to bind my hard earned dollar to results that I have minimal control over. I would rather gamble and take my money to Vegas.

Vipan Nikore, MD, is an Internal Medicine Resident Physician at the Cleveland Clinic. He received an MBA from the Yale School of Management, an MD degree from the University of Illinois-Chicago, and a BS in computer science and software engineering from the University of Western Ontario. He has led projects at UNICEF in India, the WHO in Geneva, IBM, Sun Microsystems, Citibank, UCLA, and the Ontario Ministry of Health. He is the President and Founder of the youth leadership non-profit Urban Future Leaders of the World (uFLOW). His posts are personal views and do not necessarily reflect the opinions or positions of the Cleveland Clinic.

12 replies »

  1. When will we understand that our Human Body is such a highly complex structure with many external and internal variables at interplay, that define our constantly changing Physiology and Pathology ,that lumping payments to outcomes[and who defines the best outcomes]is as ridiculous as stating that a lawyer should be paid on the outcome of the case.

  2. Doc…I know, we can help & yes, it is Free & No Gimicks! This is how:
    At http://www.Chicago.MD or http://www.LosAngeles.MD or http://www.NewYork.MD or any of our other 500 .MD city websites in the USA (see all cities at http://www.MyCity.MD), all licensed health care professionals automatically receive and are set up already (knowingly or not over 1 million accounts are up live online) with a free website presence, including patient ratings and free private mitigation tools, for issues/problems, many other free physician desktop tools and free Internet presence and ratings. (No catch!). Read the June 2008 article on one of the Cincinnati.MD doctors that had their entire surgical teams mal-practice fees lowered 3% using .MD survey tools: http://www.healthleadersmedia.com/content/PHY-214175/Turning-Online-Physician-Ratings-into-Lemonade##
    Readers also need to know that the city .MD sites are built for the new transparency acts for health care and include ratings and feedback of not just doctors, dentists, chiro’s etc but also hospital and nursing home compare stats, as well as free hospital digital exit survey apps. These apps and programs include but are not limited to the HCAHPS surveys (in http://www.Cincinnati.MD up live now and coming to all hospitals in January 2011) and our just introduced, “Rate Your Health Insurance Company” feature available today in all 500 .MD cities. See this news article: http://www.bizjournals.com/cincinnati/news/2010/12/23/yourcitymd-adds-new-rating-feature.html
    Try it now in any city from the homepage search box.
    Isn’t it time we all had a say regarding our health insurance providers too. After all, turnabout is fair play! We rate and leave feedback for our doctors and hospitals.
    Many extremely expensive survey and rating/Coaching companies like Press Ganey and/or Health Grades, etc. will likely hate the new business model but we feel it is for the good of our health care communities to provide all these services, FREE to all of our patients, doctors and hospitals, which have been taking the brunt of the economic health care crunch to date that we’ve been experiencing in U.S. Health Care marketplace. If we help the providers reduce costs, then we believe, the patient will benefit greatly in the long run with better services and much lower costs for themselves as insurance coverage pays less and less each year on the new plans!
    Your thoughts?
    Joe Benza, Jr.
    CEO YourCity.MD LLC

  3. My understanding is that, at the population level, compliance is lower among patients with low socioeconomic status than among the middle class, upper middle class and wealthy. It doesn’t matter how good the doctor’s diagnostic and communication skills are. This is one of the main reasons why, in the UK, doctors are allowed to remove some patients from their panel for quality assessment purposes. Frail elderly patients with multiple co-morbidities may also be less compliant as a group, especially if they happen to be poor.

  4. @twa,
    I don’t understand what you are talking about. Patient comes in with Problem X. I advise him to do Y. If he does it, then he is compliant. If he chooses not to do it, then he’s non-compliant.
    If the doc recommends what he’s supposed to, then he’s doing good and should be rewarded. This is regardless of whether or not the patient actually does what he’s been told.
    Does this help you? You seem to be implying that I should be “selling” or attempting to convince the patient to do what I tell him. Sorry, I’m a doctor, not a salesman. A patient can either accept my advice, or reject it. I’m not in the business of “selling” anyone or anything. The used car lot is that way –>

  5. This “compliance” issue fascinates me. So Dr. Vickstrom what accounts for the fact that two groups of patients, with two different physicians/or physician groups, can have dramatically different care and outcomes? What are the factors that lead to “compliance”? Does the physician practice have the potential to impact “compliance”? If more “compliant” patients have better outcomes, might physicians be interested in the dynamics that impact “compliance”? Might some doctors be better than others at communicating and working with patients to achieve “compliance”? And might the market want to reward these practices that have more “compliant” patients? But wait, maybe your patients are sicker too…

  6. > We need to start driving.
    Amen, Dr. Bev. And start laying the track. If docs lay the track any fool MBA can drive the train.

  7. @Bev
    I don’t know about meaningful metrics. As long as there is this thing called patient free choice involved, I think it’s going to be rather hard. Now, if the metrics are on what is offered, as opposed to what actually gets done and/or patient agrees to, that might be more realistic. How about measuring patients as to their compliance, and making non-compliant patients pay more?

  8. In the UK, primary care doctors are measured on no fewer than 146 separate metrics with points assigned to each adding up to a maximum possible score of 1,050 points. Scores are used to determine bonus compensation. Under certain circumstances, doctors are allowed to exclude some patients from their panel for measurement purposes. The system is not perfect and can be gamed to some extent. At the end of the day, though, the NHS seems satisfied with it and PCP’s are making considerably more money than they did before the measurement approach was implemented. No system is perfect but we should be able to do better than what we have now. As bev M.D. says, doctors should take the lead in developing reasonable metrics they can live with rather than just complaining and resisting change.

  9. Measurement of physician performance is inevitable. One can either board the train and help drive it (e.g. help find meaningful metrics) or complain about the engineer while the train is hurtling down the track. We need to start driving.

  10. What is even more interesting is how you were able to review his medical records from home. When I was in training we could not do that at all. And if there was something critical or time related you could have run back to the hospital to fix it, or even order a CPOE instantly. Wow, I wish I was starting all over again

  11. Why was the patient in the hospital??
    Would the Cleveland Clinic ever admit it erred?
    you said: “This patient had few clinical symptoms and appeared the healthiest amongst all the patients I managed the entire month.”