In Defense of Small Data

In Defense of Small Data

4
SHARE

flying cadeuciiI read with interest a recent editorial that opined on the poor evidence for screening in cancer trials. The evidence was judged poor because apparently no screening trial has demonstrated a clear reduction in all-cause mortality, only disease-specific mortality.  One example discussed in the analysis reviews the data for colon cancer screening and notes that, while there were a statistically significant lower number of deaths related to colon cancer in the screened group, the total mortality in the two groups was no different.  The authors posit that the study is either underpowered for total mortality or that the screened patients may have more deaths due to the ‘downstream effects’ of screening.  The provocative conclusion by many a tweet and retweet is that cancer screening has not been shown to save lives. Apparently the path to progress in medicine now must be paved by studies with millions of patients.  I understand the desire for more and more data, but I see danger in the sanctimonious protestations of those who can only find truth within the confines of a million-person, randomized control trial.  This approach ignores the history of advances in clinical medicine, most of which live far outside of the boundaries of million-strong randomized clinical trials.

The history of interventional cardiology is particularly instructive. The first catheterization of the human heart was performed by a surgical resident named Werner Forsmann in 1929 – on himself. He thought it was a good idea to catheterize his own left antecubital vein, pass a ureteral tube into his right atrium, and then walk to the radiology department to document his accomplishment.  Forsman was fired for his troubles.  His work was rejected by the scientific establishment of the time and he languished for two decades – at times having difficulty finding gainful employment, and ultimately giving up research- until 1956, when he was finally recognized for his felicitous discovery.  The technique did not find widespread use and was mostly the provenance of physiology labs until a German physician named Andreas Grunzig decided to make an attempt at opening chronically narrowed coronary arteries using balloons and catheters.

The first coronary angioplasty was performed by Grunzig September 16, 1977 using a home-made catheter on a 38 year old with disabling angina.  The procedure was remarkably successful.  The patient remained free of angina and had no residual stenosis on a repeat catheterization at the 10 year anniversary of the seminal event. Cynics would say that Grunzig ushered in an era that saw the rapid rise in unnecessary coronary interventions, and they would be right.  But it was techniques developed by people like Grunzig who made it possible for a Seattle cardiologist named Marcus Dewood to take patients in the throes of a heart attack and inject contrast into the coronary tree to show that heart attacks were a result of blood clots that formed in arteries that fed the heart.  Soon after, cardiologists began blowing up balloons to open up thrombosed arteries in patients having heart attacks and primary angioplasty – arguably the single biggest leap forward in the treatment of cardiovascular disease – was born.

The epidemiology of cardiovascular disease, as a result, has seen dramatic change over the last half century.  Interestingly, the rate of acute myocardial infarctions has been fairly constant over time, but cardiovascular mortality has dropped significantly.

While the decline in cardiovascular mortality is at times attributed to the success of primary prevention and medical therapy, the data for primary prevention in this arena is relatively weak.
That does not mean that tobacco cessation and aspirin do not work, it’s just hard to show what exactly the size of these effects are.  A study in the New England Journal tried to adjudicate the number of deaths prevented by specific risk factors using assumptions that would kindly be called shaky.  Excerpted below is the example given in the paper.
Screen Shot 2016-03-08 at 8.06.18 PM
Screen Shot 2016-03-08 at 8.06.36 PM

The paper assumes from a metanalysis that a 20mmHg drop in blood pressure will reduce deaths by 50%. This is not a statement that can be made broadly for a variety of reasons. The starting blood pressure matters, it would be wrong to suggest all deaths prevented were cardiovascular deaths, and it is quite possible that aggressive blood pressure control in certain populations (diabetics) resulted in poorer outcomes.  The faulty assumptions are rescued, of course, with the use of an exponential function, because Euler’s number fixes everything.  This same paper also attributed a reduction in deaths to interventions (revascularization for chronic stable angina) that have never been shown to reduce mortality!

I realize it has become dangerous to use one’s clinical experience to inform one’s views.  While I have no quarrel with evidence, the reality is that the longer I practice, the more I realize that clinical scenarios rarely fit even the best designed clinical trials.  In light of the underwhelming evidence with regards to primary prevention and cardiovascular mortality, my thoughts on the impressive reduction in cardiovascular mortality are driven by a group of patients that I cannot forget.  These were patients who had large ‘missed’ myocardial infarctions.  I can still vividly remember racing to place a central line in a middle-aged Cambodian man who had arrived in the CCU more than 24 hours after the onset of chest pain.  He had suffered a proximal LAD infarct  – an occlusion in an artery that fed a very large part of his heart.  As I rapidly added medications in an attempt to support a failing heart, a cardiac marker was called from the lab – a cardiac troponin that was over 400.  Cardiac troponins are released into the blood stream as cardiac muscle dies.  The higher the troponin, the more cardiac muscle that has died.  The vast majority of troponins that I get called for are under 1.  A troponin of 400 was inconceivable to me at the time.  My patient didn’t have a chance – he was dead within 12 hours.

I had other patients with extensive heart attacks that could not be revascularized – they all did poorly, driving it home to to me in a way that reading about it never could – that the time to revascularization was the single most important metric that predicted a good outcome.  So when I look at the graphs that show a marked drop off in cardiovascular mortality that accelerated in 1980, I do not believe that it was coincidence that this also marked the beginning of the era of coronary intervention for heart attacks. Again, this is not to suggest primary prevention does not matter, rather there is little that clinically has seemed to have as direct of a clinical impact on patient outcomes than revascularization to limit the size of an infarct.
I wonder how Forsmann and Grunzig would have survived in the current climate that looks down on any clinical data that has fewer than a million patients.  Grunzig demonstrated results of coronary angioplasty in his first four patients at the American Heart Association meeting in 1977.  He was interrupted by applause, and was so stunned he was unsure if he could proceed. I suspect today he would have been rewarded by denigrating tweets from the ‘elites’ among us asking for more data.
These apostates worshiping at the temple of data see themselves as gladiators championing the interests of patients.  They are right and wrong.  The current messy path medicine forges ahead on involves hurting patients.  Grunzig used primitive catheters he made in his kitchen and patients in those early years had complications that sometimes required emergent bypass, and at other times, resulted in death.  But here’s the thing, I don’t think Grunzig did anything wrong – he simply did the best he could with the tools and the knowledge available to him.  Without Grunzig, there would have been no DeWood, and no progress to a world of interventional cardiologists on call to open up an occluded artery within 90 minutes of a heart attack.While well intentioned, the purists that seek to use big data to protect us flirt with therapeutic nihilism.  Forcing all progress through the funnel of a million strong clinical trial is a bar too high, a bar that protects us from medicine itself.  No one benefits from that.
Anish Koka is a cardiologist based in Philadelphia.

Leave a Reply

4 Comments on "In Defense of Small Data"


Member
Mar 9, 2016

This is a terrific piece, Anish. As you point out, the system wants predictability, not innovation, hence the emphasis on large numbers. Sadly, trading one for the other will gets us neither…

Member
Mar 10, 2016

Thanks Michael. We are our own worst enemies. Those that make/advise policy are physicians who seek control, with the misguided idea that forcing us all to practice the same way, and do research the same way is the promised land. Obviously I couldn’t disagree more.

Member
Mar 9, 2016

Nice line: “I realize it has become dangerous to use one’s clinical experience to inform one’s views.”

via TweetBot

Member
Mar 10, 2016

Right, the ‘ounce of evidence’ folks give clinical experience no heft. Which is why they can be replaced by Watson. :). Never mind the evidence is many times of the soft hypothesis generating variety that I excerpted above..