To the two certainties of life, death and taxes, add another two: mammograms and controversy surrounding mammograms.
The Canadian National Breast Screening Study (CNBSS) has reported results of its long term follow-up in the BMJ: no survival benefit of screening mammograms.
To paraphrase Yogi Berra “it’s mammography all over again.”
Is the science settled then?
Before I wade further it’s important to understand what is implied by “settling the science.”
Einstein said “no amount of experimentation can prove me right; a single experiment can prove me wrong.”In physical sciences a theory need only be disproven once for it to be cast aside. Heliocentricity cannot coexist with Ptolemy’s universe. The statement “all swans are white” is disproven by a single black swan.
What do we do with the studies that showed survival benefit of screening mammograms? Why does the CNBSS not close the debate over mammograms, like Galileo did with celestial egocentricity?
The simple and simplistic answer is because there are powerful advocacy groups, special interests; the pink-industrial complex who have a vested interest in undermining the science.
But that lends to conspiratorial thinking. Special interests cannot undermine Maxwell’s equations or Faraday’s laws just because they do not like them.
The testability of Maxwell’s equations is inherently different from verifying that screening mammograms increase life expectancy. We must acknowledge two types of science; the former, physical science, a hard science; the latter, a hybrid of biology and epidemiology, soft science.
Soft science is a misnomer. There is nothing soft about performing a randomized controlled trial (RCT), the methodological gold standard; in ensuring factors that falsely augment or attenuate impact of screening mammograms are evenly distributed, data reliably collected, cause of death accurately recorded and correctly inferred. But the human factor and all its inevitable foibles are unavoidable in soft sciences.
Despite the rigor required of an RCT it doesn’t withstand scrutiny as well as Galileo’s method of investigating the relative speed of a heavy and light object thrown from the top of the Leaning Tower of Pisa. You can imprison Galileo if you don’t like his results, but you can’t rationally argue about the methodology that delivered the results.
But you can rationally argue and rationally disagree about the methodology of an RCT.
Was randomization truly achieved even if genuinely attempted? Daniel Kopans, an academic breast imager, seems to think not.
Detractors say that the patients had physical examination before randomization, violating a fundamental principle of RCT and raising questions about unconscious bias during randomization. Meaning that the trial coordinators were more likely to send a patient who had a palpable lump and palpable nodes to the screening arm.
The screening mammogram arm of the study did have excess advanced cancers at enrollment (19 in the screening arm versus 5 in the control).
Is this expedient use of statistical purity or a genuine concern? That depends on your point of view; yes your opinion.
Could that make a difference? Of course.
Is the difference significant? That depends on your definition of “significant”, which again depends on your opinion.
If you advocate screening mammograms, you might amplify this deficiency. If you are indifferent you might ignore it. Crucially, there is no accepted and acceptable science on how one should approach limitations of RCTs that ensures that the critique has the consistency of a measuring device such as a Geiger counter. Subjectivity is unconquerable.
Kopans is an indefatigable advocate of mammography. Bias, you say. Case closed.
Except it is not. That Kopans has raised an objection, and I doubt he would highlight weakness of an RCT that showed survival benefit of screening mammograms, tells us not only about Kopans but also the RCT.
It takes two to bias.
Then there is the mammogram. A homogeneous product like a 30 cm ruler or semi-lunar protractor? Far from it.
Quality of images vary. Quality of interpretation varies. That’s variation in variation. Quadratic variation, a lot of variation. An untrained eye interpreting a bad study could miss many more cancers than a trained eye interpreting high quality study. Furthermore, technology evolves. 1980 was a different country.
Detractors say that the quality of the mammograms and their interpretation in the CNBSS left a lot to be desired. Is that true? Possibly.
Is that significant?
Truth is that I don’t know. You would have to randomize patients to low quality mammograms with poor interpreters and high quality mammograms with good interpreters. Good luck with getting that approved by institutional review board.
Despite our desire to rationally plan healthcare through “science”, the methodology of soft sciences is inherently too weak to overcome ideologies that are inherently too strong which are present across the political spectrum. Soft science cannot deliver Plato’s Republic; not without a lot of “rational” elders arguing.
You can’t have an opinion over Maxwell’s equations. Test them if you don’t believe them. You can have an opinion over RCTs. If the results are negative and you don’t like them criticize the randomization. If they are positive and you don’t like them question their external validity. If you like the results retweet them day and night.
But the study does tell us something. That the survival benefit from screening mammograms can’t be tremendous. Were it so, the shortcomings of the trial, plausibly and probably real, would not be paraded. The intensity of quibble is usually inversely proportional to the significance of what is being quibbled about.
But mammograms are the third rail. USPSTF proposes and government disposes. After their controversial guidelines from 2009, the HHS secretary reassured women that they must “keep doing what you have been doing for years”. Or just ignore those pesky number-crunchers.
USPSTF dared raise age of screening mammograms to 50. If they declare mammography futile, which I don’t think they ever will, I dare the HHS secretary to enforce the recommendations. In fact, I dare the Canadian government to stop screening mammograms based on the long term results of their own trial.
Mammography is one of the most frequent causes of litigation. This technique that purportedly doesn’t save lives implicates lots of radiologists for failing to save lives. Explain that.
Can the BMJ study be presented in defense of a missed cancer?
An epidemiologist in the courtroom for the defense: “Well even if radiologist had picked up that subtle density on your mammogram that has turned out to be cancer chances are that you would have been overdiagnosed and it is the therapy anyway to which you should be thankful”.
The plaintiff attorney would eat the epidemiologist alive! How dare the young woman flanked by a loving husband and two adorable children be considered a mere statistic?
External validity of RCTs doesn’t cross court room doors. If you want to know vox populi look no further than a jury’s verdict. Society does not want women to die prematurely from breast cancer.
Strong science has little chance against popular will. And soft science? Not a hope in hell.
Saurabh Jha, MD (@RogueRad) is an Assistant Professor of Radiology at the University of Pennsylvania. His scholarly interests include the value of imaging and dealing with uncertainty in clinical decision making. Jha views most problems in medicine as problems of imperfect information. He trained in the UK and migrated to USA for more predictable weather and a larger yard.
I was just seeking this info for some time. After six hours of continuous Googleing, finally I got it in your site. I wonder what is the lack of Google strategy that don’t rank this type of informative web sites in top of the list. Normally the top web sites are full of garbage.
Replica loui vuitton purses outlet http://e222.bugsfestival.it
Dr. Jha, great op-ed.
The observer effect:
In the methods of the study: “Participants then had a physical (clinical) breast examination and were taught breast self examination by trained nurses”
The issue for me was what was the question under study?
Assuming all x-ray studies are equal the question then becomes is mammography no better than a good self breast exam and physician exam in that age group? (…and all other biases raised.)
Did the experiment make the participants and physicians more careful about breast examinations? Are all self breast exams and physician exams equal?
Another thing to consider:
Breast CA is a constellation of diseases, and the ultimate outcomes depend on the type of tumor, tumor markers, age and health of the patient, etc, etc.
If a woman has an aggressive tumor, even if caught earlier, it may make very little difference in the outcome, and converse for a minimally aggressive malignancy.
It is very hard for lay people to understand the nuances of medical conditions, and all of the factors that play in to the final outcome of the disease.
With mammography it isn’t that the science isn’t “settled” -it is (read “The Mammogram Myth” by Rolf Hefti)… but the disinformation and propaganda campaigns by the mammogram industry against sound but inconvenient anti-mammogram facts isn’t.
The widespread confusion among women about mammography, or the controversy about it, is the outcome of a continuous concentrated effort of the mammogram advocates, the pink ribbon movement, and their defenders, to ridicule, overlook, deny, obscure, or hide the real but inconvenient evidence.
Your wife is absolutely correct. Mammography is such an emotionally charged minefield with so much CYA (cover your a*#) that the accuracy of the test is compromised.
But that mostly accounts for the false positives, not the false negatives.
Nice discussion, you play out all sides well. But I think the gist of what you said is here – “the survival benefit from screening mammograms can’t be tremendous.”
That I believe is the message women and clinicians need to get from this and other RCTs of mammograms to date. Benefit, perhaps, but not as large as everyone has been led to believe.
Thanks for this post.
The problem is that mammography depends on the interpretation of the radiologist (no disrespect intended, I am married to one).
In a country where there is better control over liablility, and the populace recognizes the fallability of physicians as nonmalicilous in intent, I believe the rate of false positives would be less and one could get more reasonable information from the mammogram.
In the U. S., we have propogated both the fear of breast cancer and the promise of the mammogram as almost a panacea in most patient’s minds. My wife actually had a patient disbelieving she had a malignancy because “I’ve gotten a mammogram every year!”. I’m sure the patient’s next visit was to a lawyer because obviously someone must have missed something before.
This subject is so politically charged in the respect of Women’s Health, that I’m not sure we could ever come to a reasonable conclusion about the utility of the mammogram.
Oh, and also my wife did a mammogram on a fragile 90 year-old a couple of weeks ago.
Great story Legacy and very instructive. It’s a bit like those who pay more for flight insurance than insurance that covers everything (including flights).
Good article about difficult issue.
Mammography and Breast Cancer have long existed in a parallel universe that doesn’t obey the laws of other universes. Rationality about mammography is in short supply – on both sides.
Anecdote: Back in the early 90’s I had the experience of interpreting the mammogram of a woman who had been diagnosed with metastatic adenocarcinoma from an unknown primary. I looked at all the images carefully and told her the study was negative.
She was there with her daughter and her husband. She hugged me and her husband and daughter both shook my hand. Almost everyone was happy.
1) Mammography has a known false negative rate. Meaning that she actually could have breast cancer – although that is unlikely – and it wouldn’t show on the mammogram.
2) Breast cancer has a better prognosis than metastatic carcinoma from an unknown primary! In fact it would have been better if I had found a breast cancer! She would more likely have been cured!
And so it goes. In the meantime I will recommend to my wife that she continue to get her yearly mammo (she is over 50). And when my daughters reach the appropriate age I will recommend to them that they get mammograms as well.
But I will educate them to have low expectations for what mammography can actually deliver. And I hope that they never contribute to or become part of any organization with a pink ribbon. Organizations that live on the hysteria surrounding breast cancer.
Good points. In the absence of overwhelmingly clear science, the usual method will be used to convince people: fear. Patients will fear cancer. Doctors will fear lawsuits. Politicians will fear reprisal. No one will “dare” budge on the issue. This is why prevention is the new unbridled growth industry of medicine. This is why futile care will continue to be subsidized at all costs. This is how to make unbelievable profit based on little more than a belief system, using the infirmed as a shield. Great system.
Thanks for reading. The people will decide health policy. Preventive medicine is the new big thing.
Science will be incidental. Occasionally it will be cherry picked to justify prior positions.
Once patients are paying with their own money….it begins to change: patients suddenly start to ask if the additional expenditure is worth it….some will even start to ask doctors if there have been randomized control trials and if not, why the doc still recommends the procedure/intervention (shocked doc!). We have seen this as high deductible plans (each dollar not spent is the patient’s dollar) linked to health savings account have gained market share over the past 5 years…….it will accelerate as many ACA/Obamacare plans have extremely high deductibles.