Reed Abelson and Gardiner Harris, the authors of the June 4th New York Times article critical of the Dartmouth Atlas and research, have acknowledged Elliott Fisher and my concerns and clarified the record in their posting on the New York Times webpage. They originally claimed that we failed to price adjust any of the Atlas measures. They now acknowledge that we do, but they’re hard to find on the Atlas website, a point we concede. They originally claimed that quality measures were not available on the Atlas website. They now acknowledge that quality measures are on the website, but they don’t like them. We agree quality measures can be better – the type of research we do is always open to improvement — and Dr. Fisher has recently co-chaired an NQF committee with precisely this goal. (See our more detailed response.)
But the primary purpose of this posting is to respond to the attack by Mr. Harris on the professional ethics of the Dartmouth researchers. The key issue seems to be whether the research in two landmark 2003 Annals of Internal Medicine articles (here and here) were misrepresented by the Dartmouth researchers. In his posting Mr. Harris asserts:
In an aside, when was the last time you saw researchers so profoundly mischaracterize their own work? How is it possible that they could claim their annals pieces concluded something when they didn’t? I can’t remember ever seeing that happen.
We are disappointed by this accusation. We can understand Mr. Harris’s frustrations in understanding the research, as it is often nuanced and tricky to follow. This lack of understanding is illustrated by their recent New York Times posting, where they state:
In statistical terms, [the Dartmouth researchers’] claim is referred to as a negative correlation between spending and health outcomes, which means that when spending goes up, the health of patients goes down.
They have confused the idea of a correlation (high spending hospitals on average do slightly worse on quality and outcomes) with causation (if a hospital spends more money, outcomes for those patients will get worse).
The more fundamental point, however, is their claim that we misrepresenting the two 2003 Annals of Internal Medicine studies written by Dr. Fisher and others. Ms. Abelson and Mr. Harris state that
The Dartmouth work has long been cited as proving that regions and hospitals that spend less on health care provide better care than regions and hospitals that spend more…. As the article noted, [Dr. Fisher] asked in Congressional testimony last year, “Why are access and quality worse in high-spending regions?”
Now we come to their smoking gun(s):
Those [Annals] studies did conclude that there was no association between higher spending and better health, but they did not show any link between higher spending and worse health.
One of the paper’s arguments was summarized in the abstract this way: “Neither quality of care nor access to care appear to be better for Medicare enrollees in higher-spending regions.”
The second paper’s summary: “Medicare enrollees in higher-spending regions receive more care than those in lower-spending regions but do not have better health outcomes or satisfaction with care.”
Ms. Abelson and Mr. Harris are correct to note that the studies conclusively rule out the null hypothesis that more spending is associated with better outcomes – the major point of the paper. But anyone who reads both articles will come away with more than that — a finding that outcome measures are worse on average in high-cost regions. We did a quick tabulation of the large number of outcomes measures in the article – a total of 42 different measures (these are reported in more detail in our background paper). Of the total, 23 showed significantly worse outcomes in high-spending regions, 14 showed no significant effects, and just 5 showed significant positive effects in high-spending regions.
In other words, if one were to construct an index of quality, it would show nearly 5 significantly negative measures of quality of care in the high cost regions for every one positive measure.
So when Abelson and Harris claim that Fisher and others are overstating the results of the papers, they are wrong. Perhaps this simply reflects a lack of experience in reading and interpreting scientific papers. But that is no excuse to be making unfounded accusations against us.
We have also been disappointed at the adversarial nature of the process.
It began in February with Mr. Harris announcing at the beginning of an interview that he was going to “take down” the Dartmouth Atlas (as documented in Maggie Mahar’s blog). And it is ending (we hope) with Mr. Harris’s posting on The Health Care Blog questioning our ethical standards. This saddens us because of the missed opportunity to improve the dialogue in Washington and elsewhere about the strengths and limitations of research on regional variation.