The Difficult Science, Part II

“Despite their great explanatory powers these laws [such as gravity] do not describe reality. Instead, fundamental laws describe highly idealized objects in models.”

— Nancy Cartwright, “Do the Laws of Physics State the Facts?”

In Part I the limitations of science in helping us make wise choices and decisions about our health were examined.

Because of an inherent difficulty in establishing causation, absolute certainty is unattainable even in science. Medical knowledge follows Karl Popper’s theory of science because the right answer, whether about what causes ulcers or if you should take hormone replacement therapy, keeps changing with the publication of new studies. And most depressingly of all, a respected expert on evidence-based medicine concludes, “The majority of published studies are likely to be wrong.”

Part I ended with some suggestions that seemed to imply that savvy patients should enroll in a graduate level statistics class and understand the subtleties of observational studies, meta analysis, and randomized controlled clinical trials. Being an informed health care consumer is evidently difficult indeed.

Part II explores how we all have to change if we are to live wisely in a time of rapid transformation of the American healthcare system that everyone agrees needs to decrease per-capita cost and increase quality.


When I talk to physicians about pay for performance programs, I am always asked why should doctors be responsible for patient behavior that they cannot control. Even if we were able to have health care access for all and eliminate every error in medicine, we would only account for 10% of whether an individual stays healthy. Environment and genetics account for about 35%, but the remaining 55% of whether one stays well depends on behavior (exercise, smoking, diet) and social support systems (families, communities, places of worship).

Patients do need to change; more care is not always better care; more expensive care can be unproven and dangerous to your health; understanding the trade-offs involved in all medical decisions is imperative; becoming an empowered or e-Patient makes sense to me because nobody cares as much about your health as you do; having more financial skin in the game makes sense to me because now those that pay (government and employers) are not represented when doctors and patients make decisions.

But, I think it is ludicrous and unreasonable to think that patients have to make all these changes on their own. They cannot possibly do it alone. And I think it is important to remember patients come in all kinds and shapes. Not everybody wants to be an empowered patient. One bioethicist in 1990 said, “We must render a patient’s responsibility to the physician unacceptable, and we must insist that patients take primary responsibility for making decisions related to their health care.” This bioethicist doesn’t understand patients; he doesn’t understand illness, and he certainly doesn’t understand the health care delivery system.

I have written elsewhere at length about the ideal doctor patient relationship, but let me insert one paragraph from that blog post here:

“Schneider in The Practice of Autonomy finds that some patients may want to reasonably give up their right to make their own medical decisions because they feel less competent than their physicians, because they are too exhausted, depressed, irritable, and confused by their illness to think straight, and because they want to be manipulated into a course of action they desire but still resist. He describes patients’ desires as ‘complex, ambiguous, and ambivalent.’ A patient quoted in this important book says, ‘I needed the doctors to take control so I could use all my energy for recovering.’”

My friend and colleague e-Patient Dave who survived Stage IV kidney cancer weighed in on January 9, 2009 in his blog where he wrote, “As someone whose butt was saved by excellent medical care, I find it unimaginable to consider doctors ‘incapable of determining what will benefit’ me. What, like I was going to think up high-dosage Interleukin-2 on my own?”

Patients do need to become wiser and savvier consumers of health care, but it would help if others did a better job of supporting them in this endeavor.


Patients would be better served if the media did a superb job of putting new medical “breakthroughs” in proper context. Former US Senator David Durenberger has recently emphasized the importance of a responsible news media in an era of health care transformation:

“At no time in our history have we been more dependent on good reporting about things beyond our scope or our control than we are today. The ability of unreliable or biased information and its reporting to distort public opinion and destroy public confidence in policy-makers has made it well nigh impossible for elected officials to deliver the hard news we need to hear. Or the good news that is possible from appropriate behavior change.”

The Reader’s Digest played an important role in informing Americans of the link between smoking and lung cancer in 1952 and in explaining why switching to filter cigarettes did not protect against cancer in 1957.

However, the media today often falls woefully short of doing the superb job that we as patients and citizens need. The editors of The New England Journal of Medicine in 1994 wrote, “The problem is not in the research but in the way it is interpreted for the public…. An association between two events is not the same as a cause and effect.” They emphasized the importance of reporters articulating the limitations of any one single scientific study.

Health journalist Gary Schwitzer has established Health News Review where a panel of more than a dozen experts grade health news stories for their accuracy, balance, and completeness. Using a zero to five star rating system, this website is an important resource for anyone trying to become a more informed citizen or consumer. This project has been honored with the Mirror Award and the Knight-Batten Award for Innovation in Journalism. The problem is that many of the stories in our newspapers and on our television newscasts fall far short of a 5-star rating.

The Science Literacy Project is another resource trying to improve the level of reporting, and they point out that journalists need to learn “enough about how studies are designed and conducted to be able to tell your listeners how solid the research really is…The gold standard is a randomized, double-blind, controlled trial. Other kinds of studies…can provide intriguing hints but not firm evidence.”

Scott Maier, a veteran news reporter and university journalism professor, explains that the entire reporting culture works against the kind of health reporting that is needed, “We want to look at the positive aspects of medical breakthroughs, we want stories that pay off with some dividend. If you want the story to have its fullest impact, you’re more likely to exaggerate what the expert says than you are to question it.”

We need a health care media that is skeptical and questions medical studies so that the public understands the limitations of any one “breakthrough.”


With due respect to the editors of The New England Journal of Medicine, there are problems with research culture, and investigators need to change as much as journalists and patients.

A study documenting 788 retracted papers from 2000 to 2010 in the Journal of Medical Ethics concludes, “American scientists are significantly more prone to engage in data fabrication or falsification than scientists from other countries.” American scientists were lead authors on 169 papers retracted for serious errors and 84 retracted for outright fraud. It is important to recognize that retractions for errors can be part of the normal process of scientific discovery and confirmation by replication, and retractions for any reason are rare. However, cases of outright fraud like Dr. Scott Reuben, the Massachusetts anesthesiologist who had 21 papers retracted, undermine the public’s confidence in medicine because his papers changed the way millions of patients were treated for postoperative pain. Ivan Oransky, MD and Adam Marcus provide a valuable resource for us all with their Retraction Watch blog that keeps track of this phenomenon.

The publish or perish culture found at American universities may be compromising research objectivity and integrity, according to Danielle Fanelli, author of an analysis of 1,300 academic papers published in the United States. ProPublica’s discovery that more than a dozen Stanford medical school faculty were paid speakers for pharmaceutical companies also lend support to the need for change in the research culture. Stanford had been applauded for its tough conflict of interest policy that prohibited such presentations, but enforcement and accountability appear to be lacking. The University of Pennsylvania, the University of Pittsburgh, and the University of Colorado Denver are also looking into similar situations.

David H. Freedman’s book Wrong: Why Experts Keep Failing Us – And How to Know When Not to Trust Them does an admirable job of explaining how difficult it is to get research right and how the culture does not always help maintain objectivity and integrity. “The beliefs of researchers are shaped by ‘all of the vanities, vested interests, hunches, experiences, politics, careerism, grantsmanship tactics, competing cadres of collaborators, imperfections, and backgrounds of the scientists investigating problems at any time.’” Since research is a human activity, bias is rampant and unavoidable. Freedman details how researchers measure what doesn’t matter, mismeasure, toss out inconvenient data, keep reanalyzing data using different statistical models until they discover an association, don’t publish negative findings, and fail as referees in peer review to find obvious flaws in research papers. Tomaso Poggio, a tenured computer scientist as MIT states, “There’s much more competition for tenure in academia now than there was twenty years ago. It’s almost a little sick.” UCLA cancer researcher Jeffrey H. Miller tells Freedman, “The way science works is, when you end up backing a theory, you can’t afford to be wrong or your grant will suffer.”

Jonah Lehrer also points out how hard it is for researchers to find the truth. “It’s hard because reality is complicated, shaped by a surreal excess of variables. But it’s also hard because scientists aren’t robots: the act of observation is simultaneously an act of interpretation.” When Lehrer wrote an article on scientific replication and the decline effect, he was criticized by some in the scientific community for giving aid and comfort to those who deny climate change or evolution. Lehrer described the frustration of biologist Michael Jennions who looked at hundreds of papers and 44 meta-analyses and discovered a consistent decline effect over time. “This is a very sensitive issue for scientists. You know, we’re supposed to be dealing with hard facts, the stuff that’s supposed to stand the test of time. But when you see these trends you become a little more skeptical of things.” Lehrer emphasizes that the decline effect makes it imperative that we consider each single scientific study in context.

While the randomized, double blind clinical trial is the evidence-based medicine gold standard of today, they are so expensive and time-consuming that enough of them will never be finished to serve the needs of Medicare or patients with various conditions. Computer simulation may provide an alternative to the clinical trial that is better than the status quo. Dr. David Eddy compared the computer simulation approach to the clinical trials approach by trying to predict the outcomes of the seven-year Collaborative Atorvastatin Diabetes Study (Cards). Of the four principal findings of the Cards trial, Eddy’s Archimedes computer model predicted two correctly, a third within the margin of error, and a fourth just below the margin of error. Eddy estimates that the computer simulation took a few months and cost 200th the cost of the Cards trial ().

Having served on the tenure review committee at the University of Iowa Carver College of Medicine and having also been on the medical school faculty at UCSF, Allegheny University of the Health Sciences, and Michigan State College of Human Medicine, I can personally attest that the culture of medical research needs to be improved. Researchers need to follow the rules, be open to new approaches, and be transparent about the limits of science.


Physicians need to adjust to tremendous change in the practice of medicine due to the Internet, the wide-spread adoption of electronic medical records, e-Patients, the emergence of patient social media sites, the increase of patient-directed research, the movement to allow nurses and physician assistants to practice to the top of their licenses, the increased federal funding for comparative effectiveness research, the demand for more accountability, and even the use of avatars and video games for health and wellness.

Physician discontent appears to be a growing problem both for patients and for providers, and the above-described changes are playing a role in increasing this lack of professional satisfaction.

I propose Dr. Lewis Blowers, a general surgeon, and Dr. Robert Parker, a pediatrician, as role models. When John E. Wennberg shared the variations in local tonsillectomy rates in Vermont with the State Medical Society, Dr. Blowers and Parker instituted a second opinion process, which lowered the chance of a child in Morrisville undergoing a tonsillectomy from 60% to less than 10%. They did not challenge the data; they did not react defensively; they did not try to defend a lucrative way of treating sore throats; they did what was in the best interest of their patients, even though it required them to change their style of practice.

In order to be professionally fulfilled and undiscouraged, physicians need to develop the humility, courage, and existential strength required to view all of these disruptive changes as potential ways to better take care of their patients. Accepting and admitting that medical science does not have all the answers is part of developing humility.

This need for professional humility is nothing new; when Dr. Robert Lovett of Children’s Hospital in Boston told Franklin Delano Roosevelt there was no medical treatment for his polio induced poor muscle control, Roosevelt developed his own exercise program. Roosevelt even bought and managed a spa in Warm Springs, Georgia where he and others could exercise in a warm swimming pool. When the American Orthopedic Association refused to let Roosevelt speak in 1926, he crashed the meeting and negotiated an agreement to evaluate his patient-developed Warm Springs program. Talk about an empowered patient.

The movie The King’s Speech movingly depicts the success of an uncredentialed, failed Australian actor in treating the King of England’s speech problem, and it also documents the failure of the knighted royal medical doctors’ approach.

I would hope that physicians would become the leading advocates of shared decision making. This is the one approach that offers us the ability to both decrease per-capita cost and increase quality. “A recent Cochrane review of randomized clinical trials comparing shared decision making supported by decision aids to obtaining informal consent through usual care showed an average 24% decline in demand for a wide range of elective surgeries and tests.” One estimate suggests that such an approach could save Medicare $4 billion a year. The Foundation for Informed Decision Making is a valuable resource for such decision aids that need to be much more widely adopted in the United States.


Patients, the media, researchers, and clinicians all have to change if we are to live wisely in a time of enormous change and transformation for the American health care system. We must be willing to accept the limitations of science and be open to shared decision making that recognizes that there are trade-offs in any decision made in this uncertain and unpredictable place we call reality.

Part III will describe how medical schools, payers, and employers need to change (coming soon).

Kent Bottles, MD, is past-Vice President and Chief Medical Officer of Iowa Health System (a $2 billion
health care organization with 23 hospitals). He was responsible for the day-to-day operations of a large education and research organization in Michigan prior to his work with in Iowa with IHS. Kent posts frequently at his new blog, Kent Bottles Private Views.

Livongo’s Post Ad Banner 728*90

Leave a Reply

19 Comment threads
0 Thread replies
Most reacted comment
Hottest comment thread
14 Comment authors
DavidMark ReidBillBarry Carolpcp Recent comment authors
newest oldest most voted

Thanks for a well written opinion piece. It is great to see that it has generated debate. Although it is hard to follow the point pcp is trying to make is it to have an argument or provide a constructive informed blog? Maybe pcp did not read part 1. If we sit in a perfect world of research and theory we ignore all the variables available and that our patients/clients can come in with different variables to those who are measured even in the best gold standard research as they were excluded as participants. The accumulated evidence remains more reliable… Read more »

Mark Reid
Mark Reid

This is a wonderful article. Fortunately, despite all the complexities in our medical system, a single doctor presented with a sick patient can usually make a decent decision about what to do next. One of the goals of our educational system is to make it so ’round about the same things will get done in Maine as in Nevada. We generally succeed at that goal. Brand new treatments are inherently risky and they help in all cases unless they hurt you. We never really know until a few years from now. Despite all the details, if you get shot or… Read more »


“studies that show that primary care physicians do not always utilize the best aids”
No, actually the study said:
“underutilize potentially more effective behavioral or psychological treatments”
If you’re really arguing for aggressive use of “potentially effective treatment” rather than evidence-based medicine, I guess we’ll have to agree to disagree.

Kent Bottles

PCP is missing the important point that everyone involved in health care must change and that we should utilize the best available science and shared decision making aids so that patients and physicians together can support each other to make our nation more healthy. Of course each individual must take responsibility for their own wellness, but there are several studies that show that primary care physicians do not always utilize the best aids (http://ow.ly/3wbhU), that some methods like motivational interviewing work best for smoking cessation (http://ow.ly/3wbIC), and that physician input is important to patient behavioral change (http://ow.ly/3wb42, http://ow.ly/3wbeh) . Although… Read more »


Kudos to Dr. Bottles for reinforcing the need for all of us (clinicians, patients and patient support systems) to do a much better job of educating ourselves and making the appropriate interpretations of scientific evidence regarding medical conditions and treatment options. His observation is that the flow of information and rate of change in health care makes it challenging to keep up with emergent developments in a timely manner. The need for this ongoing “education” often runs counter to the natural tendency to resist change and go with the status quo despite indications that it is not always in the… Read more »


“Like much in life, incentives matter.”
Exactly. And what docs can do for issues like these, unfortunately, matters very little. But it sounds like Dr. Bottles feels like it’s reasonable to cut my pay if my patients skip church and make other “unacceptable” life-style decisions.

Barry Carol
Barry Carol

While I agree that personal behavior is probably the most important factor influencing health status, I think many people engage in unhealthy behavior such as smoking, drinking too much and eating tasty but unhealthy foods to relive stress and mitigate the many aggravations in life. While observers from afar might attribute such behavior to a lack of discipline, I suggest that it’s not that simple. Moreover, for those with lower socioeconomic status, they often cannot access healthy foods at reasonable prices or afford them even if they could. That all said, much higher taxes on cigarettes did drive smoking rates… Read more »


Maybe I’m not understanding the original post. Seems like you’re changing your argument.
You start off discussing concerns about P4P, and acknowledge that health is only about 10% based on medical intervention and is primarily determined by life style choices, such as church membership.
You then say that patients are unable to make significant behaviour changes on their own (I should be referring my patients to churches?).
I didn’t see anything about interpreting scientific research, decisions about testing, etc.

Kent Bottles

I cannot possibly agree with pcp that people should not be supported by a media that does a good job in reporting the real meaning of scientific studies or that researchers should not follow rules against conflict of interest or that researchers should not try as hard as possible to account for their inevitable biases or that doctors should not use decision aids to help patients better understand the trade-offs involved in making difficult decisions about testing or treatment. To say that patients should be left totally on their own to be responsible for their decisions in these complex areas… Read more »


“the remaining 55% of whether one stays well depends on behavior (exercise, smoking, diet) and social support systems (families, communities, places of worship) . . .
But, I think it is ludicrous and unreasonable to think that patients have to make all these changes on their own. They cannot possibly do it alone.”
Couldn’t disagree more. These are exactly the sort of changes that need to be made by oneself. Becoming empowered or an e-patient has nothing to do with success in these areas, and, unfortunately, medical intervention is also a dismal failure here.

Gregg Masters

Seriously Kent! I nominate you for the ‘keepin’ em honest surveillance context award’. Incredible piece with, again, fabulous links and references. I am now a subscriber of of ‘Retraction Watch’.
Blown away!


“all of us need to do a better job of realizing that there is a big difference between association and causation”
Copy that. Hat size correlates highly with shoe size. But big heads don’t cause big feet.
I would also add that measures of association (that may or may not indicate causality) extend well beyond the simple first-order sums-of-squares-partitions-cross-products linear correlations (e.g. Pearson-R) that everyone learns (and then often misuses).

Kent Bottles

http://ow.ly/3vTBq This link to Wire’s weirdest indicators of medical risk is a good example of the difference between association and causation. I really do think all of us need to do a better job of realizing that there is a big difference between association and causation.


@R. Phillip Melch, MD
That’s about as incoherent, pointless quip as I’ve heard. Would you care to connect whatever ephemeral dots may be implicit?

Michelle W

Thanks for the informative and thought-provoking article, as well as the link to Health News Review. The points about media coverage are well made, especially the part about correlation versus causation. I think that’s a very difficult concept for people to understand. Often reading the whole article will reveal a much smaller linkage between two variables than the headline promotes. There is, of course, a danger on the opposite extreme of being so skeptical that nothing gets covered or read. News sources have limited space, after all, and have to choose which stories to push: treat all studies with extreme… Read more »