Uncategorized

Commentology: Times Reporters Respond

New York Times health policy reporter Gardiner Harris responded to THCB founder and publisher Matthew Holt’s comments on the recent series of reports he has authored with business writer Reed Abelson questioning the science behind the Dartmouth Atlas.  Gardiner had this to say in defense of his newspaper’s investigation:

The main point of Reed’s and my pieces about the Dartmouth work is that the data are simply not good enough to guide spending decisions in the government’s $484 billion Medicare program. If the Dartmouth researchers had acknowledged this point, our story would not have been all that interesting. But they cannot bring themselves to do this, and in fact they have repeatedly exaggerated and mischaracterized their own work in public settings to suggest it can be prescriptive.

An ancillary point was to warn those on capitol hill, the administration and journalists to be wary of those highly popular maps from the Atlas. You have scoffed that it’s a small thing that the Dartmouth researchers fail to adjust their online data for price and illness. But misunderstandings about this are widespread. That landmark piece by Dr. Gawande that you cited used the Atlas’s unadjusted data. Dozens of stories in newspapers and magazines around the country have used the unadjusted data to criticize health institutions. Even David Cutler, among the top health economists in the country, was unaware that the atlas offered largely unadjusted data.

Accuracy may seem a small point to you. It is not to us.

Our Friday piece also pointed out that Dr. Elliott Fisher and Mr. Jon Skinner claimed that their 2003 Annals pieces had found a negative correlation between spending and outcomes. In fact, the pieces found no correlation between spending and outcomes. This is not a small distinction. If there’s a negative correlation, cuts in spending will actually improve health. If no correlation has been found, then cuts become far harder and perhaps more painful. We cannot go into reforms of our healthcare system believing that the work will be easy. But that is what the Dartmouth researchers have suggested, and this siren song has had an enormous impact on Capitol Hill.

In an aside, when was the last time you saw researchers so profoundly mischaracterize their own work? How is it possible that they could claim their annals pieces concluded something when they didn’t? I can’t remember ever seeing that happen.

–Gardiner Harris

Livongo’s Post Ad Banner 728*90

19
Leave a Reply

19 Comment threads
0 Thread replies
0 Followers
 
Most reacted comment
Hottest comment thread
9 Comment authors
InfoMarkMargalit Gur-ArieDr. SteveMariaEPI_Mark Recent comment authors
newest oldest most voted
Nate
Guest
Nate

” But it is convenient to tie the causality to the association reflected in the Atlas to spending variation – to support an economic policy argument for resource redistribution and reduction in healthcare.” problem is politicians and journalist are beating us over the head with this proven facts, like AGW science is being misused for political gain. People concerned with the truth have no choice but to attack the study becuase the study is being used to attack the truth. If the authors now claim their study can’t support the claims being made with it they should have stood up… Read more »

InfoMark
Guest
InfoMark

Of course Dr. Steve and Margalit have it correctly. The Atlas points out some interesting associations with spending variations that have unanticipated outcomes. The possible causal links here (SES variations, population illness severity differences, practice quality differences, healthy/healing environmental differences, etc.)are not explored by the Atlas. But it is convenient to tie the causality to the association reflected in the Atlas to spending variation – to support an economic policy argument for resource redistribution and reduction in healthcare. It is politically easier to argue this than to build policy responses to some of the realistically more likely causal realities, and… Read more »

Dr. Steve
Guest
Dr. Steve

Way to go Margalit. Of course, it’s either fish or fowl. Either the Atlas researchers are saying that the two move independently and have nothing to do with each other (which is what their research seems to show) or that increased spending is linked to poorer outcomes (which is what they claim in public). Once they go for the latter, they then have to argue that it’s causal or who cares about any of their research. Of course most experts think that in fact spending and outcomes are both driven by a third ‘confounder’. Most health researchers fret about one… Read more »

Margalit Gur-Arie
Guest

So after reading the new response, I keep going back to this one sentence: “They [NYT reporters] have confused the idea of a correlation (high spending hospitals tend to do poorly on most measures of quality and outcomes) with causation (if a hospital spends more money, outcomes for those patients will get worse).” The correlation example cited is number of storks and birth rates. So if the Dartmouth folks are saying that spending is only correlated with outcomes, but there is no cause and effect, then spending and outcomes have nothing to do with each other. There may be a… Read more »

Margalit Gur-Arie
Guest

Dartmouth researchers respond to the NYT response:
http://www.dartmouthatlas.org/downloads/press/NYT_Redux_final%5B1%5D.pdf

Dr. Steve
Guest
Dr. Steve

I have been wading through this tiff and have concluded that those that side with dartmouth are clinging to two very problematic syllogisms. 1) The tone of the NYT article was mean, so Dartmouth must be beyond reproach. 2) The health care system is in shambles, and Dartmouth says it is so, their researchers must be beyond reproach (and by extension, the NYT must be really saying the system is perfect and that can’t be right). I have put a little more thought into this and decided that it is safe to ignore the tone and the larger context of… Read more »

Maria
Guest
Maria

Nate doesn’t seem to understand that bias is often in the eye of the beholder. People won’t complain when they read something that agrees with their own views, but when it’s the opposite situation, they’re quick to scream “Bias!” I guess it means their own biases are showing. Everyone has a bias. Simply by virtue of your gender, where you grew up, what your parents were like, your socioeconomic class and how much education you received, you have biases. It’s neither good nor bad; it’s just how it is. The key is in recognizing where your biases lie and trying… Read more »

EPI_Mark
Guest
EPI_Mark

Like any ecological study the Dartmouth Atlas, and derivatives, are great as a generators of hypotheses, but a poor substitute for the hard work of healthcare efficacy and quality assessment (including access, effectiveness, etc.). In this the Times reporters are spot-on. It is somewhat human (or academic) nature to maximize (hype) the value of your research, and the Dartmouth group probably aren’t more guilty here than most academics. But this country is in extremely bad shape if (because) its academics are driving health policy decisions!

Nate
Guest
Nate

” Glaring errors like that kinda throw sand in the gears of your credibility.”
Really Rick you gauge creditbility on grammer and spelling? Anything written in English by a non english speaker should be ignored then? Pretty ignorant perspective in my opinion.

Rick
Guest
Rick

Nate is clearly no journalist. The word you’re looking for, Nate, is “biased” not “bias.” Glaring errors like that kinda throw sand in the gears of your credibility. And to Bill Jones, MD: fine, let’s put your money where your mouth is. All the Dartmouth folk are doing is taking an available data set and applying scientific method to it. If you think their method is wrong, then you must have a pretty good idea of what a good method would look like. Using the same data set, describe for us a broad outline of a good way to test… Read more »

Peter
Guest
Peter

Yes, we all know how UN-bias Nate is at assessing opinions, articles, facts.

propensity
Guest
propensity

Agree with Gardiner and Reed. They need to hold the officials accountable.

Nate
Guest
Nate

“should not have a preconceived agenda.” maybe I just have bad luck but when I am in Cleveland the Plain Dealer is overtly bias. When I travel through the airports the LAT and NYT are terribly bais. AP and Reuters are caught in bias all the time, they don’t even pretend to be journalist any more. The only non bias papers I ever recall reading are the army times and some rural small town papers. “research/science should not have a preconceived agenda.” I agree it should not but most of it does. This article was far from as bad as… Read more »

MD as HELL
Guest
MD as HELL

Data is irrelavant when a political issue is in play.

rbar
Guest
rbar

Nate, in risk of stating the obvious: -no, newspaper articles (unless you refer to opinion journalism) should not have a preconceived agenda. They may often have one, but at the very least the auhor(s) tries to hide it. -no, research/science should not have a preconceived agenda. You may be referring to generation of a scientific hypothesis, which is subsequently supported or refuted by the data (for instance, results of an experiment, or data exploration). But it does happen, especially in research sponsered by institutions that have an interest in a specific outcome – one important example being research sponsered by… Read more »