Winning the Doctor Lottery

A poignant piece recently appeared in the journal Health Affairs and was rapidly devoured on social media by the health policy community. The story is a harrowing first person account of a woman’s multiple interactions with doctors. The doctors in the story are either very good or very bad. One pediatrician turns the author and her sick son away on three consecutive days with colic, only to have a more careful partner sound the alarm and discover pyloric stenosis. The author then recounts the tale of her father’s death at age 42 due to a surgeon who operated for diverticulitis unnecessarily.

My family and I haven’t always won The Doctor Lottery. My father’s surgeon, for instance, had pushed him to have the bowel resection to “cure” him of diverticulitis, a disease in which the colon’s lining becomes inflamed. He stitched up my father’s intestines with a suture known to dissolve in patients who’ve been on steroids and hadn’t read my father’s chart to see that his internist had recently had him on cortisone. Nor did he look at the list of medications my father had carefully written down on his patient intake forms. When the sutures dissolved, my father, who had a bleeding disorder, went into shock. His abdomen was distended and hard.

My mother asked the nurse to page the surgeon. “My husband is in so much pain!” she said. The surgeon, who was playing golf, told the nurse to tell my mother, “Pain after surgery is normal.” By the time my father developed a fever, and peritonitis, it was too late. He died of a heart attack.

It’s a moving anecdote with a tragic ending that has the requisite story elements – arrogant uncaring doctor ignoring patient and family concerns while on the golf course – that policy folks use to argue for remaking the current health care system into a more patient-centric world. Unfortunately, medicine is hard, and while there are certainly errors that are avoidable, many are not. The best surgeon, the best system, and the best medical care are at times no match for the randomness of life. A certain percentage of patients will have an infection after an abdominal surgery despite every current safeguard that is known. The vast majority of patients with abdominal pain and distention after surgery do not routinely need to be reoperated on. Deciding who to reoperate on is often challenging. Is a good surgeon one who takes every patient who has abdominal pain and distention back to the operating room? Is it feasible to have an attending surgeon on hand to evaluate every complaint of abdominal pain? Should we ban all surgeons from playing golf for 48 hours after they operate?

None of these questions have answers that don’t involve tradeoffs in the real world. In policy world, however, solutions are magical constructs that don’t involve robbing Peter to pay Paul. In this fantasy world Peter and Paul find a leprechaun with a pot of gold at the end of a rainbow. As a result, the solutions proposed to ‘ending the doctor lottery’ involves fostering strong patient doctor relationships that align incentives based on number of patients seen rather than the value of care delivered. Apparently, what promises to save us is a large order of payment models based on value, with teams composed of generous helpings of social workers, behavior health experts, and cute puppies.

It is with this noble intent that our physician overlord masters in the Center for Medicare and Medicaid services , at the bidding of the public and congress, have applied themselves to the small task of assigning value to physicians. There are many prongs to this worthy desire to measure the nation’s doctors, but a particularly sharp prong advanced by the Agency for Healthcare Research and Quality is the patient-centered Consumer Assessment of Healthcare Providers and Systems (CAHPS). The CAHPS tool is actually a standardized survey that has been in use since 1997 to measure and report on the experiences of consumers with the health care services they come into contact with. More recently, a sister to the CAHPS tool was born so that physicians in office settings may be measured by their patients – this was named the Clinician and Group – CAHPS survey (CG-CAHPS). The stated goal of this tool is to publicly report survey results to allow patients to choose good doctors.

The assessments are performed by practices and health care systems. Patients receive surveys that seek to get to the heart of what everyone wants in a physician – Does your physician listen carefully? Did your physician spend enough time with you? It appears that practices have some latitude in how the question is asked, what questions are asked, as well as how to interpret the results.

Value-based care sounds good, and enjoys widespread support among every non-clinician that seems to matter in the world of health care policy. Physicians seem generally apathetic, though overconfident about the coming valuations – after all, it’s always the other guy that sucks. Not surprisingly, the worst physicians have the least insight into their own limitations, and suffer the most from delusions of grandeur. While physicians may be excessively poor at grading themselves, the physician community that loves gossip about as much as the Real Housewives of Atlanta are much less forgiving. Yes, that’s right – there is general widespread agreement among physicians of the worst among us.

So, imagine my surprise when one of the good guys that I worked and trained with called to tell me that he had the lowest CG-CAHPS scores in his group and he may need ‘remediation.’ Value-based care takes on a whole new dimension when you’re the one that carries the 21st century version of the scarlet letter.

Unpacking the genesis of a bad CG-CAHPS score is an exercise in revealing the idiocy that results from the many good intentions in healthcare. In this particular case, the hospital sends out a survey to patients that have come into contact with its physicians. Those who respond to the survey select answers that range from ‘always’ to ‘most of the time’ to ‘never’. Only ‘always’ counts towards a ‘Top Box’ score, and this Top Box score is then compared to a national average to generate the provider’s percentile. For instance. if 9 out of 10 patients checked off ‘always’ to ‘Did Provider listen carefully to you?’ your Top Box score is 90% – but if the national or health system average for that category is much higher – that score may still put you in the 50th percentile.

The problems with all of this are legion. Of the 1500 unique patients who this physician saw in the prior year, 120 patients took the time to respond to the survey. I’m always surprised that anyone fills out any surveys – I fill out one every 2 years. Regardless, of the 120 patients who responded, six chose not to select the top box, resulting in this physician being labeled a problem in need of remediation. Beyond the problem of generalizing from 120 patients who have an unnatural affinity for filling out surveys in the mail, one wonders if there is more to a physician’s worth than her ability to communicate? As a medical student, I recall a surly surgeon who minced few words in his communication with patients, but was technically brilliant. Many a grateful patient or family was indebted to him for a life saved, but I recall a smattering of patients put off by an approach that had little time for the worried well.

In the name of transparency, CMS plans to publicly share this quality information via an online physician compare tool to allow patients to finally win the doctor lottery, and perhaps more importantly, tie reimbursement to value.

Health systems nervous about decreasing reimbursements related to their bad physicians need not worry because riding furiously to their rescue are health care consultants, who for a pretty penny, promise a smooth transition to this new world. These words from the Studer-Huron health care consultancy appear designed to allay the health system executive’s fears:

“Plenty of evidence shows that patient experience and clinical quality are two sides of the same coin. You already want to provide the best possible care. And now that Clinician and Group Consumer Assessment of Healthcare Providers and Systems is here, there’s a new reason to focus on patient perception: CG CAHPS will impact ACOs, PQRSs, PCMHs, and many other programs, and survey results will link to payments in 2015.”

These same consultants lined up not long ago to help hospitals achieve pay for performance metrics. It surprises no practicing physician that pay for performance metrics and value based payments as currently designed were an abject failure. While there are some like Ashish Jha (Harvard School of Public Health) who have noticed and publicly called out the failure of value-based payment, the answer disappointingly appears to be ever better patient-centered metrics. The latest idea that relates to my scant enthusiasm for basing value on patient surveys, unfortunately, comes from no other than Dr. Jha, who wrote recently in JAMA on a proposal to query Medicare patients 30 to 60 days after discharge on the quality of care they received, and tie up to 10% of a hospitals reimbursement to these scores. I can almost feel the frisson of excitement travel through the offices of the Studer-Huron group at this latest opportunity to manage patient perception and save the day.

I fear a noble profession has lost the plot when it chooses to measure value based on patient satisfaction simply because it is the easiest and most politically correct metric to measure. It seems that the vision of measuring value is what’s important – it matters not that the value quantified by these wonderful tools is the health care policy equivalent of fake news. What matters is that surveys measure something that can be quantified, regressed, risk-adjusted and published. The truth may be shrouded in darkness, but falsehoods found where the light happens to shine now comes to masquerade as the truth.

When it comes to one’s health, the desire for an assurance of quality is an understandable one. We are supposed to assure quality by making medical school admission a privilege reserved for those who have demonstrated intellectual vigor, board certifications that test competency, and continuing education to attempt to demonstrate maintenance of competency. Unfortunately, we live in a time where acceptance into medical school relates more to virtue signaling than intellectual horsepower, and board certification is a mechanism to siphon dollars from physicians to take tests that have little to do with the practice of medicine, and certainly don’t weed out the bad.

The medical profession has done itself few favors by having a remarkably anemic approach to ferreting out physicians who fall egregiously below professional standards. There are no perfect solutions, but I would suggest with much bias that having a healthy pool of primary care physicians not employed and beholden to health systems are vital to improving the chances patients have to win the doctor lottery. I understand the public desire for guarantees when it comes to those we trust with our lives. The only thing the current approach guarantees is the health of the bank accounts of health care consultants, but protecting patients from bad doctors? What a joke.

Anish Koka is a cardiologist in private practice in Philadelphia. Most of the opinions he has aren’t put on surveys, but can be found on twitter @anish_koka

Categories: Uncategorized

9 replies »

  1. Thanks Anish. CMS’ Physician Compare (CP) intends to use CG-CAHPS as one measure of customer service quality, I believe. Other Q+P measures will be added over time…or at least that was the plan last time I checked in. CP has underwhelmed to date, from a clinician and consumer/patient perspective. With the change of administrations and the priority HHS Sec Price puts on the doc-patient relationship, we could well see modifications in PC. But that will take rule-making since much of what PC is supposed to do is mandated by the Affordable Care Act. Unless of course Congress eventually kills PC by killing the ACA. Notably, the current repeal and replace bill being debated in Congress does not touch PC or the many other non-insurance ACA programs. Republicans said they’d address all that down the road, if and when they get a bill on the insurance marketplaces through, something that’s still uncertain.

  2. “Winning the Doctor Lottery?” as a title that would NOT normally generate thoughtful and precise comments. Given that, I commend my colleagues, so far, for the above commentary. To the final paragraph of the POST, I add only one observation. Our nation’s healthcare reform strategy lacks a means to establish within each community, its equitable health care for both ‘Basic’ as well as ‘Complex’ HEALTH needs.
    We have solved the scientific mandate for Complex HEALTH needs without solving the humanitarian mandate for Basic HEALTH needs. Every other advanced/developed (UN definitions) nation has a strategy to assure the equitable availability of Primary Healthcare within each community. The USA does not. The Principles for Managing a Commons (as in our nation’s ‘out of control’ healthcare spending) would require a locally managed and nationally promoted process to offer equitably available Primary Healthcare, community by community. The concepts of Social Capital and Community Resiliency apply.
    I am increasingly aware that the social determinants of HEALTH for each citizen are a far more important as a cause of unstable HEALTH than we realize. As a result, it is impossible for the standard hospital system/enterprise to assume ultimate responsibility for “mitigating” these issues. The scientific and humanitarian mandates for each person’s Stable HEALTH typically generates a level of cognitive dissonance that prevents progress. Unfortunately, adapting to the current Paradigm Shift within our nation’s healthcare has come to a halt.
    Remember that our nation’s maternal mortality ratio continues to annually worsen. Also, remember that this potentially affects the HEALTH of one out of every two citizens. Using known data from 2001-06, the state by state poverty level has a highly negative Pearson Correlation with each state’s maternal mortality ratio (WHO definition) with a p of <0.05 . Note also, that there is NO state by state data available since the 2001-2006 data set. See Obstetrics & Gynecology, Sept 2016 for the most recently reported and accurate national data for 2000 – 2014. The point of all this, the usual means to correct a major HEALTH problem has achieved nothing. For best 10 of the 51 advanced/developed nations, they have an average maternal mortality ration of 3.8 per 100,000 live births. In 2004, our ratio was "23.8" ( for 2000 it was 18.8).
    To achieve enhanced Primary Healthcare that is offered equitably to each citizen within each community will require strategies not currently within our nation's conversation. This will be true whether we end up with a single payer or not. The sooner we manage the humanitarian mandate for HEALTH the less onerous the rationing will be.

  3. I would go to a surgeon who had the lowest CRPs (highly sensitive c-reactive proteins) in his patients three weeks after the surgery.

  4. The human factor in physicians brings out the qualities we want, compassion, caring, empathy. It also brings out the qualities we don’t like, arrogance, conceit, anger. We could therefore turn to Watson as our physician, but then where would that get us?
    As long as humans are physicians, I think it would be very tough to consistently get perfect ratings on all these “measures” we are documenting. But then, that makes it easier for the payers to justify lower payments doesn’t it?

  5. Thanks Steve. I implicitly assumed that being really good at surgery implies good risk-adjusted outcomes vs. peers. I wonder how many hospitals will push back against surgeons who intimidate the nurses, especially when they are high RVU generators. I guess a lot depends on how the hospital leadership balances a desire for a collegial and collaborative culture vs. maximizing revenue and profit.

  6. Define “jerk.”

    Now compare your definition with your patient’s.

    Herein lies the problem.

  7. Satisfaction scores are most useful as a marketing tool. You really want to know if your patients are happy so that they will come back. Really! This is actually an important number we want to track.

    As a measure of quality? Not so much. I am sure we all have stories along this line. My favorite is my OB floor. Whatever group we had in charge of rating our floor said that my guys rated in the 99th percentile for quality, but only 20th percentile for patient satisfaction. (Which I pretty much knew as we had a bunch of cranky older guys who hated being up at night, but did good work.) I was asked by an administrator what I thought I should do about this. I told him time would take care of it. Some of the older guys have retired, quality is in the 80-90 percentile range. Satisfaction is 50 ish. Much better. Now if only I can get the last of the older guys to go maybe I can get our satisfaction scores to 99th percentile and our quality to 20th.


  8. “For surgeons, by contrast, I can tolerate a lot of arrogance if he/she is really good at surgery. ”

    You really shouldn’t. What you want is a good outcome. The surgeon who is technically brilliant but a jerk (this is actually pretty rare) often has outcomes brought down because of the inability to manage care post-op. We have certainly seen this and have had to let a couple of strong RVU generators go because they intimidated ICU nurses to the point where they would rather let the patient linger through the night rather than call the surgeon at home.

  9. As a patient, I’m pretty skeptical of patient satisfaction surveys and scores. I generally think good healthcare and good teaching for that matter fall into the category of I know it when I see it or experience it but I can’t necessarily reduce it to contract language or prove it in court.

    When it comes to primary care, I think too many patients evaluate doctors on the three A’s – affability, availability and, last and least, ability. Personally, when we’re talking about primary care or specialties like cardiology where my relationship with the doctor is long and ongoing, I’m primary interested in good diagnostic skills and good communication skills. Hopefully, personal chemistry is at least acceptable and, hopefully, good and I can get appointments when I need them on a reasonably timely basis.

    For surgeons, by contrast, I can tolerate a lot of arrogance if he/she is really good at surgery. Our interaction will, most likely, be a one-time event with perhaps a follow-up visit or two to make sure everything is healing properly and no infections have developed.

    If hospitals want to measure patient satisfaction, perhaps they should concentrate on what many companies call their net promoter score. That is, were you sufficiently satisfied with your care experience to recommend this doctor to a family member, relative or friend? If not, why not?