Uncategorized

The Difficult Science, Part II

“Despite their great explanatory powers these laws [such as gravity] do not describe reality. Instead, fundamental laws describe highly idealized objects in models.”

— Nancy Cartwright, “Do the Laws of Physics State the Facts?”

In Part I the limitations of science in helping us make wise choices and decisions about our health were examined.

Because of an inherent difficulty in establishing causation, absolute certainty is unattainable even in science. Medical knowledge follows Karl Popper’s theory of science because the right answer, whether about what causes ulcers or if you should take hormone replacement therapy, keeps changing with the publication of new studies. And most depressingly of all, a respected expert on evidence-based medicine concludes, “The majority of published studies are likely to be wrong.”

Part I ended with some suggestions that seemed to imply that savvy patients should enroll in a graduate level statistics class and understand the subtleties of observational studies, meta analysis, and randomized controlled clinical trials. Being an informed health care consumer is evidently difficult indeed.

Part II explores how we all have to change if we are to live wisely in a time of rapid transformation of the American healthcare system that everyone agrees needs to decrease per-capita cost and increase quality.

PATIENTS

When I talk to physicians about pay for performance programs, I am always asked why should doctors be responsible for patient behavior that they cannot control. Even if we were able to have health care access for all and eliminate every error in medicine, we would only account for 10% of whether an individual stays healthy. Environment and genetics account for about 35%, but the remaining 55% of whether one stays well depends on behavior (exercise, smoking, diet) and social support systems (families, communities, places of worship).

Patients do need to change; more care is not always better care; more expensive care can be unproven and dangerous to your health; understanding the trade-offs involved in all medical decisions is imperative; becoming an empowered or e-Patient makes sense to me because nobody cares as much about your health as you do; having more financial skin in the game makes sense to me because now those that pay (government and employers) are not represented when doctors and patients make decisions.

But, I think it is ludicrous and unreasonable to think that patients have to make all these changes on their own. They cannot possibly do it alone. And I think it is important to remember patients come in all kinds and shapes. Not everybody wants to be an empowered patient. One bioethicist in 1990 said, “We must render a patient’s responsibility to the physician unacceptable, and we must insist that patients take primary responsibility for making decisions related to their health care.” This bioethicist doesn’t understand patients; he doesn’t understand illness, and he certainly doesn’t understand the health care delivery system.

I have written elsewhere at length about the ideal doctor patient relationship, but let me insert one paragraph from that blog post here:

“Schneider in The Practice of Autonomy finds that some patients may want to reasonably give up their right to make their own medical decisions because they feel less competent than their physicians, because they are too exhausted, depressed, irritable, and confused by their illness to think straight, and because they want to be manipulated into a course of action they desire but still resist. He describes patients’ desires as ‘complex, ambiguous, and ambivalent.’ A patient quoted in this important book says, ‘I needed the doctors to take control so I could use all my energy for recovering.’”

My friend and colleague e-Patient Dave who survived Stage IV kidney cancer weighed in on January 9, 2009 in his blog where he wrote, “As someone whose butt was saved by excellent medical care, I find it unimaginable to consider doctors ‘incapable of determining what will benefit’ me. What, like I was going to think up high-dosage Interleukin-2 on my own?”

Patients do need to become wiser and savvier consumers of health care, but it would help if others did a better job of supporting them in this endeavor.

THE MEDIA

Patients would be better served if the media did a superb job of putting new medical “breakthroughs” in proper context. Former US Senator David Durenberger has recently emphasized the importance of a responsible news media in an era of health care transformation:

“At no time in our history have we been more dependent on good reporting about things beyond our scope or our control than we are today. The ability of unreliable or biased information and its reporting to distort public opinion and destroy public confidence in policy-makers has made it well nigh impossible for elected officials to deliver the hard news we need to hear. Or the good news that is possible from appropriate behavior change.”

The Reader’s Digest played an important role in informing Americans of the link between smoking and lung cancer in 1952 and in explaining why switching to filter cigarettes did not protect against cancer in 1957.

However, the media today often falls woefully short of doing the superb job that we as patients and citizens need. The editors of The New England Journal of Medicine in 1994 wrote, “The problem is not in the research but in the way it is interpreted for the public…. An association between two events is not the same as a cause and effect.” They emphasized the importance of reporters articulating the limitations of any one single scientific study.

Health journalist Gary Schwitzer has established Health News Review where a panel of more than a dozen experts grade health news stories for their accuracy, balance, and completeness. Using a zero to five star rating system, this website is an important resource for anyone trying to become a more informed citizen or consumer. This project has been honored with the Mirror Award and the Knight-Batten Award for Innovation in Journalism. The problem is that many of the stories in our newspapers and on our television newscasts fall far short of a 5-star rating.

The Science Literacy Project is another resource trying to improve the level of reporting, and they point out that journalists need to learn “enough about how studies are designed and conducted to be able to tell your listeners how solid the research really is…The gold standard is a randomized, double-blind, controlled trial. Other kinds of studies…can provide intriguing hints but not firm evidence.”

Scott Maier, a veteran news reporter and university journalism professor, explains that the entire reporting culture works against the kind of health reporting that is needed, “We want to look at the positive aspects of medical breakthroughs, we want stories that pay off with some dividend. If you want the story to have its fullest impact, you’re more likely to exaggerate what the expert says than you are to question it.”

We need a health care media that is skeptical and questions medical studies so that the public understands the limitations of any one “breakthrough.”

RESEARCHERS

With due respect to the editors of The New England Journal of Medicine, there are problems with research culture, and investigators need to change as much as journalists and patients.

A study documenting 788 retracted papers from 2000 to 2010 in the Journal of Medical Ethics concludes, “American scientists are significantly more prone to engage in data fabrication or falsification than scientists from other countries.” American scientists were lead authors on 169 papers retracted for serious errors and 84 retracted for outright fraud. It is important to recognize that retractions for errors can be part of the normal process of scientific discovery and confirmation by replication, and retractions for any reason are rare. However, cases of outright fraud like Dr. Scott Reuben, the Massachusetts anesthesiologist who had 21 papers retracted, undermine the public’s confidence in medicine because his papers changed the way millions of patients were treated for postoperative pain. Ivan Oransky, MD and Adam Marcus provide a valuable resource for us all with their Retraction Watch blog that keeps track of this phenomenon.

The publish or perish culture found at American universities may be compromising research objectivity and integrity, according to Danielle Fanelli, author of an analysis of 1,300 academic papers published in the United States. ProPublica’s discovery that more than a dozen Stanford medical school faculty were paid speakers for pharmaceutical companies also lend support to the need for change in the research culture. Stanford had been applauded for its tough conflict of interest policy that prohibited such presentations, but enforcement and accountability appear to be lacking. The University of Pennsylvania, the University of Pittsburgh, and the University of Colorado Denver are also looking into similar situations.

David H. Freedman’s book Wrong: Why Experts Keep Failing Us – And How to Know When Not to Trust Them does an admirable job of explaining how difficult it is to get research right and how the culture does not always help maintain objectivity and integrity. “The beliefs of researchers are shaped by ‘all of the vanities, vested interests, hunches, experiences, politics, careerism, grantsmanship tactics, competing cadres of collaborators, imperfections, and backgrounds of the scientists investigating problems at any time.’” Since research is a human activity, bias is rampant and unavoidable. Freedman details how researchers measure what doesn’t matter, mismeasure, toss out inconvenient data, keep reanalyzing data using different statistical models until they discover an association, don’t publish negative findings, and fail as referees in peer review to find obvious flaws in research papers. Tomaso Poggio, a tenured computer scientist as MIT states, “There’s much more competition for tenure in academia now than there was twenty years ago. It’s almost a little sick.” UCLA cancer researcher Jeffrey H. Miller tells Freedman, “The way science works is, when you end up backing a theory, you can’t afford to be wrong or your grant will suffer.”

Jonah Lehrer also points out how hard it is for researchers to find the truth. “It’s hard because reality is complicated, shaped by a surreal excess of variables. But it’s also hard because scientists aren’t robots: the act of observation is simultaneously an act of interpretation.” When Lehrer wrote an article on scientific replication and the decline effect, he was criticized by some in the scientific community for giving aid and comfort to those who deny climate change or evolution. Lehrer described the frustration of biologist Michael Jennions who looked at hundreds of papers and 44 meta-analyses and discovered a consistent decline effect over time. “This is a very sensitive issue for scientists. You know, we’re supposed to be dealing with hard facts, the stuff that’s supposed to stand the test of time. But when you see these trends you become a little more skeptical of things.” Lehrer emphasizes that the decline effect makes it imperative that we consider each single scientific study in context.

While the randomized, double blind clinical trial is the evidence-based medicine gold standard of today, they are so expensive and time-consuming that enough of them will never be finished to serve the needs of Medicare or patients with various conditions. Computer simulation may provide an alternative to the clinical trial that is better than the status quo. Dr. David Eddy compared the computer simulation approach to the clinical trials approach by trying to predict the outcomes of the seven-year Collaborative Atorvastatin Diabetes Study (Cards). Of the four principal findings of the Cards trial, Eddy’s Archimedes computer model predicted two correctly, a third within the margin of error, and a fourth just below the margin of error. Eddy estimates that the computer simulation took a few months and cost 200th the cost of the Cards trial ().

Having served on the tenure review committee at the University of Iowa Carver College of Medicine and having also been on the medical school faculty at UCSF, Allegheny University of the Health Sciences, and Michigan State College of Human Medicine, I can personally attest that the culture of medical research needs to be improved. Researchers need to follow the rules, be open to new approaches, and be transparent about the limits of science.

CLINICIANS

Physicians need to adjust to tremendous change in the practice of medicine due to the Internet, the wide-spread adoption of electronic medical records, e-Patients, the emergence of patient social media sites, the increase of patient-directed research, the movement to allow nurses and physician assistants to practice to the top of their licenses, the increased federal funding for comparative effectiveness research, the demand for more accountability, and even the use of avatars and video games for health and wellness.

Physician discontent appears to be a growing problem both for patients and for providers, and the above-described changes are playing a role in increasing this lack of professional satisfaction.

I propose Dr. Lewis Blowers, a general surgeon, and Dr. Robert Parker, a pediatrician, as role models. When John E. Wennberg shared the variations in local tonsillectomy rates in Vermont with the State Medical Society, Dr. Blowers and Parker instituted a second opinion process, which lowered the chance of a child in Morrisville undergoing a tonsillectomy from 60% to less than 10%. They did not challenge the data; they did not react defensively; they did not try to defend a lucrative way of treating sore throats; they did what was in the best interest of their patients, even though it required them to change their style of practice.

In order to be professionally fulfilled and undiscouraged, physicians need to develop the humility, courage, and existential strength required to view all of these disruptive changes as potential ways to better take care of their patients. Accepting and admitting that medical science does not have all the answers is part of developing humility.

This need for professional humility is nothing new; when Dr. Robert Lovett of Children’s Hospital in Boston told Franklin Delano Roosevelt there was no medical treatment for his polio induced poor muscle control, Roosevelt developed his own exercise program. Roosevelt even bought and managed a spa in Warm Springs, Georgia where he and others could exercise in a warm swimming pool. When the American Orthopedic Association refused to let Roosevelt speak in 1926, he crashed the meeting and negotiated an agreement to evaluate his patient-developed Warm Springs program. Talk about an empowered patient.

The movie The King’s Speech movingly depicts the success of an uncredentialed, failed Australian actor in treating the King of England’s speech problem, and it also documents the failure of the knighted royal medical doctors’ approach.

I would hope that physicians would become the leading advocates of shared decision making. This is the one approach that offers us the ability to both decrease per-capita cost and increase quality. “A recent Cochrane review of randomized clinical trials comparing shared decision making supported by decision aids to obtaining informal consent through usual care showed an average 24% decline in demand for a wide range of elective surgeries and tests.” One estimate suggests that such an approach could save Medicare $4 billion a year. The Foundation for Informed Decision Making is a valuable resource for such decision aids that need to be much more widely adopted in the United States.

CONCLUSION

Patients, the media, researchers, and clinicians all have to change if we are to live wisely in a time of enormous change and transformation for the American health care system. We must be willing to accept the limitations of science and be open to shared decision making that recognizes that there are trade-offs in any decision made in this uncertain and unpredictable place we call reality.

Part III will describe how medical schools, payers, and employers need to change (coming soon).

Kent Bottles, MD, is past-Vice President and Chief Medical Officer of Iowa Health System (a $2 billion
health care organization with 23 hospitals). He was responsible for the day-to-day operations of a large education and research organization in Michigan prior to his work with in Iowa with IHS. Kent posts frequently at his new blog, Kent Bottles Private Views.

19 replies »

  1. Thanks for a well written opinion piece.
    It is great to see that it has generated debate. Although it is hard to follow the point pcp is trying to make is it to have an argument or provide a constructive informed blog? Maybe pcp did not read part 1.
    If we sit in a perfect world of research and theory we ignore all the variables available and that our patients/clients can come in with different variables to those who are measured even in the best gold standard research as they were excluded as participants.
    The accumulated evidence remains more reliable than one opinion and leads to wonderful breakthroughs like penicillin that save lives. The bias inherant in research sadly leads to terrible outcomes like Viox and as our need for power and money accelerates in a competitive environment errors are more likely to occur
    Medicine is becoming more difficult to understand and even be up to date with as we are so busy generating research to underpin our practice. Yet many practices remain that are unproven.
    Surely it is why we need Cochrane and other systematic review sites to help us better understand of what evidenced is and how it can be biased underpinning all our educations and blogs like this to challenge our thoughts and opinions.
    Thanks
    David

  2. This is a wonderful article. Fortunately, despite all the complexities in our medical system, a single doctor presented with a sick patient can usually make a decent decision about what to do next. One of the goals of our educational system is to make it so ’round about the same things will get done in Maine as in Nevada. We generally succeed at that goal. Brand new treatments are inherently risky and they help in all cases unless they hurt you. We never really know until a few years from now.
    Despite all the details, if you get shot or run over or catch cancer, your chances of surviving are better today than they were in the 70’s. Most young people have no exposure to medical care at all and this has nothing to do with medical treatments and everything to do with immunizations, clean water and fluoride. Oh yeah and laws that protect the people from a poisoned environment, a dangerous workplace, and one another.
    A valuable experiment would be to bring a “Doctors without Borders” team to a major American city (like Detroit) and have them deliver the basic medical care they deliver for those patients. Likely we could do it simpler, cheaper, and at times even better than we do it now with all the insurance paperwork, litigation, and corporate welfare. While the practice of medicine has changed immeasurably in the past 40 years, the textbook is largely the same.

  3. “studies that show that primary care physicians do not always utilize the best aids”
    No, actually the study said:
    “underutilize potentially more effective behavioral or psychological treatments”
    If you’re really arguing for aggressive use of “potentially effective treatment” rather than evidence-based medicine, I guess we’ll have to agree to disagree.

  4. PCP is missing the important point that everyone involved in health care must change and that we should utilize the best available science and shared decision making aids so that patients and physicians together can support each other to make our nation more healthy. Of course each individual must take responsibility for their own wellness, but there are several studies that show that primary care physicians do not always utilize the best aids (http://ow.ly/3wbhU), that some methods like motivational interviewing work best for smoking cessation (http://ow.ly/3wbIC), and that physician input is important to patient behavioral change (http://ow.ly/3wb42, http://ow.ly/3wbeh) .
    Although physicians cannot control what their patients do, it is undeniable that physician advice does play a role in modifying unhealthy behaviors. It makes sense that physicians should not give up on this important task, no matter how difficult it can be in individual cases. It also makes sense to explore and evaluate what really does work best in a shared decision making relationship.

  5. Kudos to Dr. Bottles for reinforcing the need for all of us (clinicians, patients and patient support systems) to do a much better job of educating ourselves and making the appropriate interpretations of scientific evidence regarding medical conditions and treatment options.
    His observation is that the flow of information and rate of change in health care makes it challenging to keep up with emergent developments in a timely manner.
    The need for this ongoing “education” often runs counter to the natural tendency to resist change and go with the status quo despite indications that it is not always in the patient’s best interest.
    Clinicians who feel that this responsibility falls entirely in the patient’s lap are making an unrealistic assumption regarding health literacy.
    We as a society do a poor job of educating and incentivizing people to understand and act upon what is best for their overall health.
    Clincians, public health officials. the news media and other patient support systems (yes, even churches) have a responsibility to educate, encourage and support positive health & wellness behavior in patients.
    It’s a major issue and cannot be left solely to individual patients, who as Dr. Bottles notes often are psychologically impacted by their physical health issues.

  6. “Like much in life, incentives matter.”
    Exactly. And what docs can do for issues like these, unfortunately, matters very little. But it sounds like Dr. Bottles feels like it’s reasonable to cut my pay if my patients skip church and make other “unacceptable” life-style decisions.

  7. While I agree that personal behavior is probably the most important factor influencing health status, I think many people engage in unhealthy behavior such as smoking, drinking too much and eating tasty but unhealthy foods to relive stress and mitigate the many aggravations in life. While observers from afar might attribute such behavior to a lack of discipline, I suggest that it’s not that simple. Moreover, for those with lower socioeconomic status, they often cannot access healthy foods at reasonable prices or afford them even if they could. That all said, much higher taxes on cigarettes did drive smoking rates down by more than 50% in the U.S. over the last 40 years to about 20%, second lowest in the world after Canada. We should probably rethink how and to what extent we subsidize food and agriculture. Like much in life, incentives matter.

  8. Maybe I’m not understanding the original post. Seems like you’re changing your argument.
    You start off discussing concerns about P4P, and acknowledge that health is only about 10% based on medical intervention and is primarily determined by life style choices, such as church membership.
    You then say that patients are unable to make significant behaviour changes on their own (I should be referring my patients to churches?).
    I didn’t see anything about interpreting scientific research, decisions about testing, etc.

  9. I cannot possibly agree with pcp that people should not be supported by a media that does a good job in reporting the real meaning of scientific studies or that researchers should not follow rules against conflict of interest or that researchers should not try as hard as possible to account for their inevitable biases or that doctors should not use decision aids to help patients better understand the trade-offs involved in making difficult decisions about testing or treatment. To say that patients should be left totally on their own to be responsible for their decisions in these complex areas is to abdicate part of the responsibility of being a physician.

  10. “the remaining 55% of whether one stays well depends on behavior (exercise, smoking, diet) and social support systems (families, communities, places of worship) . . .
    But, I think it is ludicrous and unreasonable to think that patients have to make all these changes on their own. They cannot possibly do it alone.”
    Couldn’t disagree more. These are exactly the sort of changes that need to be made by oneself. Becoming empowered or an e-patient has nothing to do with success in these areas, and, unfortunately, medical intervention is also a dismal failure here.

  11. Seriously Kent! I nominate you for the ‘keepin’ em honest surveillance context award’. Incredible piece with, again, fabulous links and references. I am now a subscriber of of ‘Retraction Watch’.
    Blown away!

  12. “all of us need to do a better job of realizing that there is a big difference between association and causation”
    ___
    Copy that. Hat size correlates highly with shoe size. But big heads don’t cause big feet.
    I would also add that measures of association (that may or may not indicate causality) extend well beyond the simple first-order sums-of-squares-partitions-cross-products linear correlations (e.g. Pearson-R) that everyone learns (and then often misuses).

  13. http://ow.ly/3vTBq This link to Wire’s weirdest indicators of medical risk is a good example of the difference between association and causation. I really do think all of us need to do a better job of realizing that there is a big difference between association and causation.

  14. @R. Phillip Melch, MD
    That’s about as incoherent, pointless quip as I’ve heard. Would you care to connect whatever ephemeral dots may be implicit?

  15. Thanks for the informative and thought-provoking article, as well as the link to Health News Review. The points about media coverage are well made, especially the part about correlation versus causation. I think that’s a very difficult concept for people to understand. Often reading the whole article will reveal a much smaller linkage between two variables than the headline promotes. There is, of course, a danger on the opposite extreme of being so skeptical that nothing gets covered or read. News sources have limited space, after all, and have to choose which stories to push: treat all studies with extreme objectivitiy (in, say, the Vulcan manner), and you run the risk of readers or viewers simply ignoring the stories. Finding a balance between total news noise and ignorance is a tricky thing, and one I think all news producers (traditional and online alike) are striving to find.

  16. Kent:
    Responsibility for ones actions and decisions is so fragile in an economy that is such. Until we make it an expectation that is reimbursable (as in volume of child enrollments for Medicaid reimbursement measures) than we are stuck with what we have (until better decisions can be made).
    Social Media IS the consumers saving grace in educating the public for education sake. It is we (and WE ARE scientists too) who expose and discuss the matters that make headlines (if not only in our micro-sphere). Education = Power; Knowledgeable Decisions = Create Powerful Choices.
    Social Media by credentialed individuals (such as US) add to the choices that the consumer has to review. I personally try to keep it simple with usable chunks of information that IS user friendly. Social Media adds to the accountability that government must make (inevitably).
    All of this is said on the day I cannot find a user friendly article on a key health care issue facing Colorado (hmmm)…
    Of course, this is simply my humble opinion and I am just a nurse (oh and I get such a rise out of people when I say, “just a nurse”) that I could not resist.
    Keep fighting the good fight.
    Respectfully,
    Bonnie Kavanagh RN

  17. @BobbyG: As well the same can be said about lapdog generating global warming sky is falling fat old ex VPs from Tennessee.

  18. Great post.
    ___
    “accept the limitations of science”
    ___
    To the ascendant Tea Bagger Idiocracy and their infotainment media running dogs, the very fact that the scientific method necessarily assumes the possibility of error nullifies its legitimacy.

  19. Hmm, never really thought about the role of the media in health care needs, but, the media of late is so complicit in being manipulated or just being a plain lackey to serve their own self interests in being king makers, at least in politics as well read and heard, I wouldn’t trust them to change for the better on their own accord.
    Besides, we see how well the internet has taken over the role of the media. And not for the better. Face it, every one has an agenda until proven otherwise is sincerely for true altruism and concern for the public.
    And this site is no exception to my opinion.