By SAURABH JHA, MD

To the two certainties of life, death and taxes, add another two: mammograms and controversy surrounding mammograms.

The Canadian National Breast Screening Study (CNBSS) has reported results of its long term follow-up in the BMJ: no survival benefit of screening mammograms.

To paraphrase Yogi Berra “it’s mammography all over again.”

Is the science settled then?

No.

Before I wade further it’s important to understand what is implied by “settling the science.”

Einstein said “no amount of experimentation can prove me right; a single experiment can prove me wrong.”In physical sciences a theory need only be disproven once for it to be cast aside. Heliocentricity cannot coexist with Ptolemy’s universe. The statement “all swans are white” is disproven by a single black swan.

What do we do with the studies that showed survival benefit of screening mammograms? Why does the CNBSS not close the debate over mammograms, like Galileo did with celestial egocentricity?

The simple and simplistic answer is because there are powerful advocacy groups, special interests; the pink-industrial complex who have a vested interest in undermining the science.

But that lends to conspiratorial thinking. Special interests cannot undermine Maxwell’s equations or Faraday’s laws just because they do not like them.

The testability of Maxwell’s equations is inherently different from verifying that screening mammograms increase life expectancy. We must acknowledge two types of science; the former, physical science, a hard science; the latter, a hybrid of biology and epidemiology, soft science.

Soft science is a misnomer. There is nothing soft about performing a randomized controlled trial (RCT), the methodological gold standard; in ensuring factors that falsely augment or attenuate impact of screening mammograms are evenly distributed, data reliably collected, cause of death accurately recorded and correctly inferred. But the human factor and all its inevitable foibles are unavoidable in soft sciences.

Despite the rigor required of an RCT it doesn’t withstand scrutiny as well as Galileo’s method of investigating the relative speed of a heavy and light object thrown from the top of the Leaning Tower of Pisa. You can imprison Galileo if you don’t like his results, but you can’t rationally argue about the methodology that delivered the results.

But you can rationally argue and rationally disagree about the methodology of an RCT.

Was randomization truly achieved even if genuinely attempted? Daniel Kopans, an academic breast imager, seems to think not.

Detractors say that the patients had physical examination before randomization, violating a fundamental principle of RCT and raising questions about unconscious bias during randomization. Meaning that the trial coordinators were more likely to send a patient who had a palpable lump and palpable nodes to the screening arm.

The screening mammogram arm of the study did have excess advanced cancers at enrollment (19 in the screening arm versus 5 in the control).

Is this expedient use of statistical purity or a genuine concern? That depends on your point of view; yes your opinion.

Could that make a difference? Of course.

Is the difference significant? That depends on your definition of “significant”, which again depends on your opinion.

If you advocate screening mammograms, you might amplify this deficiency. If you are indifferent you might ignore it. Crucially, there is no accepted and acceptable science on how one should approach limitations of RCTs that ensures that the critique has the consistency of a measuring device such as a Geiger counter. Subjectivity is unconquerable.

Kopans is an indefatigable advocate of mammography. Bias, you say. Case closed.

Except it is not. That Kopans has raised an objection, and I doubt he would highlight weakness of an RCT that showed survival benefit of screening mammograms, tells us not only about Kopans but also the RCT.

It takes two to bias.

Then there is the mammogram. A homogeneous product like a 30 cm ruler or semi-lunar protractor? Far from it.

Quality of images vary. Quality of interpretation varies. That’s variation in variation. Quadratic variation, a lot of variation. An untrained eye interpreting a bad study could miss many more cancers than a trained eye interpreting high quality study. Furthermore, technology evolves. 1980 was a different country.

Detractors say that the quality of the mammograms and their interpretation in the CNBSS left a lot to be desired. Is that true? Possibly.

Is that significant?

Truth is that I don’t know. You would have to randomize patients to low quality mammograms with poor interpreters and high quality mammograms with good interpreters. Good luck with getting that approved by institutional review board.

Despite our desire to rationally plan healthcare through “science”, the methodology of soft sciences is inherently too weak to overcome ideologies that are inherently too strong which are present across the political spectrum. Soft science cannot deliver Plato’s Republic; not without a lot of “rational” elders arguing.

You can’t have an opinion over Maxwell’s equations. Test them if you don’t believe them. You can have an opinion over RCTs. If the results are negative and you don’t like them criticize the randomization. If they are positive and you don’t like them question their external validity. If you like the results retweet them day and night.

But the study does tell us something. That the survival benefit from screening mammograms can’t be tremendous. Were it so, the shortcomings of the trial, plausibly and probably real, would not be paraded. The intensity of quibble is usually inversely proportional to the significance of what is being quibbled about.

But mammograms are the third rail. USPSTF proposes and government disposes. After their controversial guidelines from 2009, the HHS secretary reassured women that they must “keep doing what you have been doing for years”. Or just ignore those pesky number-crunchers.

USPSTF dared raise age of screening mammograms to 50. If they declare mammography futile, which I don’t think they ever will, I dare the HHS secretary to enforce the recommendations. In fact, I dare the Canadian government to stop screening mammograms based on the long term results of their own trial.

Mammography is one of the most frequent causes of litigation. This technique that purportedly doesn’t save lives implicates lots of radiologists for failing to save lives. Explain that.

Can the BMJ study be presented in defense of a missed cancer?

An epidemiologist in the courtroom for the defense: “Well even if radiologist had picked up that subtle density on your mammogram that has turned out to be cancer chances are that you would have been overdiagnosed and it is the therapy anyway to which you should be thankful”.

The plaintiff attorney would eat the epidemiologist alive! How dare the young woman flanked by a loving husband and two adorable children be considered a mere statistic?

External validity of RCTs doesn’t cross court room doors. If you want to know vox populi look no further than a jury’s verdict. Society does not want women to die prematurely from breast cancer.

Strong science has little chance against popular will. And soft science? Not a hope in hell.

Saurabh Jha, MD (@RogueRad) is an Assistant Professor of Radiology at the University of Pennsylvania. His scholarly interests include the value of imaging and dealing with uncertainty in clinical decision making. Jha views most problems in medicine as problems of imperfect information. He trained in the UK and migrated to USA for more predictable weather and a larger yard.