Last week, a study in the New England Journal of Medicine called into question the effectiveness of surgical checklists for preventing harm.
Atul Gawande—one of the original researchers demonstrating the effectiveness of such checklists and author of a book on the subject—quickly wrote a rebuttal on the The Incidental Economist.
He writes, “I wish the Ontario study were better,” and I join him in that assessment, but want to take it a step further.
Gawande first criticizes the study for being underpowered. I had a hard time swallowing this argument given they looked at over 200,000 cases from 100 hospitals. I had to do the math. A quick calculation shows that given the rates of death in their sample, they only had about 40% power .
Then I became curious about Gawande’s original study. They achieved better than 80% power with just over 7,500 cases. How is this possible?!?
The most important thing I keep in mind when I think about statistical significance—other than the importance of clinical significance —is that not only does it depend on the sample size, but also the baseline prevalence and the magnitude of the difference you are looking for. In Gawande’s original study, the baseline prevalence of death was 1.5%.
This is substantially higher than the 0.7% in the Ontario study. When your baseline prevalence approaches the extremes (i.e.—0% or 50%) you have to pump up the sample size to achieve statistical significance.
So, Gawande’s study achieved adequate power because their baseline rate was higher and the difference they found was bigger. The Ontario study would have needed a little over twice as many cases to achieve 80% power.
This raises an important question: why didn’t the Ontario study look at more cases?
Continue reading “Why Bad Research Makes It into Good Medical Journals”
Filed Under: OP-ED, THCB
Tagged: Atul Gawande, Checklists, Josh Herigon, NEJM, Ontario Checklist Study
Mar 18, 2014
Don’t assume anything.
Assumptions can kill.
Assuming something regarding your own health care can cost you money, cause you pain, and yes, even kill you. Here’s my list of potentially harmful assumptions:
1. No news is good news
If you have a test done and don’t hear anything about the result, do not assume it is fine. This assumption kills people. I have too many patients with too much information flying at me every day for me to catch every important detail. Sometimes things are missed, but sometimes the results don’t come to our office. We have trained our patients to expect an email or letter with their results within a certain amount of time, so they sometimes call when the test results don’t come in. I tell them to do so in the clinical summary sheet I hand out at the end of each visit, but the assumption remains.
Continue reading “Zen and the Art of Not Thinking Magically”
Filed Under: Uncategorized
Tagged: Assumptions, Care coordination, Checklists, Choosing Wisely Campaign, doctor/ patient relationship, OP-ED, patient-doctor communication, personal health records, primary care, recommendations, Smart Medicine, standard care, The Insider's Guide To Health Care
Jun 25, 2012
Who doesn’t love a Top 10 list? Creating them is an art form. So when it was formally proposed by Dr. Brody in 2010 in the NEJM that each specialty create their own “Top 5 list” of unnecessary care, it seemed like a straightforward – if not downright provocative – suggestion.
“The Top Five list would consist of five diagnostic tests or treatments that are very commonly ordered by members of that specialty, that are among the most expensive services provided, and that have been shown by the currently available evidence not to provide any meaningful benefit to at least some major categories of patients for whom they are commonly ordered. In short, the Top Five list would be a prescription for how, within that specialty, the most money could be saved most quickly without depriving any patient of meaningful medical benefit,” he wrote.
And yet, thus far the only groups that have seemed to have taken him up on the suggestion have been the primary care specialties of Internal Medicine, Family Medicine and Pediatrics – notably amongst the least compensated fields in health care.
This is a great start, but c’mon guys, where are the rest of you? Dr. Brody wrote you a “prescription.” We have a term for your behavior: “noncompliance.”
Not to say that there hasn’t been some progress. The ABIM Foundation has indeed put together an impressive list of organizations participating in their “Choosing Wisely” campaign. They also have begun to be instrumental in funding projects towards this goal. Costs of Care has highlighted far-reaching areas of non-value-based care, including a recent thoughtful essay about robotic surgery. We must now consolidate on these small gains and move this forward across all specialties in medicine.
Continue reading “The Letterman Approach to Cost Awareness”
Filed Under: Uncategorized
Tagged: ABIM, Checklists, Choosing Wisely, cost awareness, Costs, UCSF
Apr 6, 2012
Far too many patients are harmed rather than helped from their interactions with the health care system. While reducing this harm has proven to be devilishly difficult, we have found that checklists help. Checklists help to reduce ambiguity about what to do, to prioritize what is most important, and to clarify the behaviors that are most helpful.
The use of checklists helped to reduce central-line associated bloodstream infections at The Johns Hopkins Hospital, in hospitals throughout Michigan, and now across the United States. Clinicians have begun to develop, implement and evaluate checklists for a variety of other diagnoses and procedures.
Patients can also use checklists to defend themselves against the major causes of preventable harm. Here are a few you can use:
Health care-associated infections
- Ask about your hospital’s rates of central-line associated bloodstream infections in the intensive care unit. The best hospitals use the definitions provided by the Centers for Disease Control and Prevention and have rates less than one infection per 1,000 catheter days. A rate above three should cause concern.
- Whenever clinicians enter your room, ask if they have washed their hands. Request that visitors also wash their hands often. Washing can be with alcohol gel or soap and water.
- If you have any type of catheter, ask every day if that catheter can be removed.
Continue reading “A Safety Checklist for Patients”
Filed Under: THCB
Tagged: Checklists, Johns Hopkins, Patient Safety, Peter Pronovost, The Insider's Guide To Health Care
Mar 15, 2012
As the new year starts, I’m eager for a fresh start and working on improving myself both physically and emotionally. I’m also eager for the NFL playoffs and seeing how my favorite team, the New England Patriots, fares under the leadership of Coach Bill Belichick and quarterback Tom Brady. Doctors and health care can learn much from their examples.
Over the past decade, the New England Patriots have been dominant appearing in 40 percent of the Super Bowls played and winning 3 out of 4. Nothing prior to 2000, would have suggested this superior performance with playoff appearances only six times from 1985 to 2000 and two Super Bowl appearances, both losses. Their new head coach Bill Belichick hired in 2000 had a losing record in his prior stint at Cleveland. Their current quarterback Tom Brady was drafted in the second to last round.
Continue reading “What Doctors Can Learn from the New England Patriots”
Filed Under: Physicians, THCB
Tagged: Atul Gawande, Checklists, hospital staff, improving medial care, New England Patriots
Jan 9, 2012
From the start of the patient safety movement, the field of commercial aviation has been our true north, and rightly so. God willing, 2011 will go down tomorrow as yet another year in which none of the 10 million trips flown by US commercial airlines ended in a fatal crash. In the galaxy of so-called “high reliability organizations,” none shines as brightly as aviation.
How do the airlines achieve this miraculous record? The answer: a mix of dazzling technology, highly trained personnel, widespread standardization, rigorous use of checklists, strict work-hours regulations, and well functioning systems designed to help the cockpit crew and the industry learn from errors and near misses.
In healthcare, we’ve made some progress in replicating these practices. Thousands of caregivers have been schooled in aviation-style crew resource management, learning to communicate more clearly in crises and tamp down overly steep hierarchies. Many have also gone through simulation training. The use of checklists is increasingly popular. Some hospitals have standardized their ORs and hospital rooms, and new technologies are beginning to catch some errors before they happen. While no one would claim that healthcare is even close to aviation in its approach to (or results in) safety, an optimist can envision a day when it might be.
The tragic story of Air France flight 447 teaches us that that even ultra-safe industries are still capable of breathtaking errors, and that the work of learning from mistakes and near misses is never done.
Continue reading “The Crash of Air France 447: Lessons for Patient Safety”
Filed Under: THCB, The Vault
Tagged: Air France 447, Bob Wachter, Checklists, Patient Safety
Jan 2, 2012
If you have been at your nursing job for a while, you’ve probably almost forgotten.
Forgotten what it was like to come in to the healthcare system you now work for and realize there are hundreds of new protocols for you to learn and adhere by as a nurse. After years of routine, you now go about your day as if you actually have some choice in the way you give care.
At one point you probably did. I was not around during this age of nursing. The age when we had autonomy. Freedom to practice. Freedom to be innovative.
Today, I am somewhat saddened by the current state of the nursing profession. Don’t get me wrong: I LOVE what I do. I am so thankful for the opportunities set before me.
But whatever happened to “nursing judgment”? Or “nursing decision”?
I can’t tell you how much recently I’ve heard the phrase, “It is hospital policy that…” “You can’t do that, it is protocol that…”
I understand the need for protocols. They help us in the case that something goes wrong and the hospital gets sued. Did the nurse adhere to the protocol? If not, they will be subject to disciplinary action and take the fall. If something goes wrong and there is no protocol, the hospital can say in its defense: “There is now a protocol in place.”
Maybe a less cynical need for protocols: promote and regulate evidenced-based practice among nurses. Evidenced-based practice was developed for a reason: it brings good outcomes and protects the patients.
Even so, to me it seems we are being protocoled to extinction.
Continue reading “Nurses: Protocol -ed to Extinction?”
Filed Under: Uncategorized
Tagged: Checklists, Nurses, Nursing, Protocols, Sarah Beth Cowherd
Mar 10, 2011
Recently, I got to shake hands with and also have lunch with doctor-writer extraordinaire Atul Gawande! He was nearly everything I had made him out to be. He wore a snappy blue blazer, a jumble of ID tags, and round specs befitting a prominent Harvard academic doc … only he wore them in a manner that suggested a man of action. Am I gushing? Sorry. I’ll stop … except to say that he had the chicken salad on white, which sat largely untouched as he drove through the conversation.
One of the greatest storytellers in the history of health care, Atul has discovered something very important about the way we deliver complex procedures … a series of checklists that bring down costs and improve outcomes … and yet adoption of his findings is incredibly low! He is seriously considering raising money to fund—NOT to continue his research and writing—but to literally fund a team of Maoist-like activists to cajole the ranks of docs and hospitals into adopting these bloody checklists!! These checklists are real bluebirds. Nobody loses their job from the adoption of these things. Except for a few embarrassing ‘re-operations to fix terrible mistakes, nobody loses any money either. So what gives?
Atul told me the story of penicillin adoption … I was stunned at how long it took for this miracle drug to be mainstreamed. I remember from my OB-GYN days the number of docs who were still doing continuous fetal monitoring during labor, twenty years and five studies after it was shown to be counter-indicated … and episiotomies, and circumcisions (ouch)!
These are good people. I have met literally tens of thousands of docs and can count the truly questionable people on one hand. So what is it?
There is no market mechanism for the solution. That’s what.
If a payer came to me and gave me Atul’s checklist and said they’d pay even 5% more for a surgery done according to his checklist, I’d build it into an EMR and flick it in within a week! It’s a no-brainer for me and it’s good money for the doc! Ya know what that would be an example of? That would be MEANINGFUL use of an EMR.
God Bless you, Atul. I’m in for a donation … but not for the Maoists. You go find the truth and we’ll go make a market for it.
PS … Atul signed my copy of his book!
Jonathan Bush co-founded athenahealth, a leading provider of internet-based business services to physicians since 1997. Prior to joining athenahealth, he served as an EMT for the City of New Orleans, was trained as a medic in the U.S. Army, and worked as a management consultant with Booz Allen & Hamilton. He obtained a Bachelor of Arts in the College of Social Studies from Wesleyan University and an M.B.A. from Harvard Business School.
Filed Under: Uncategorized
Tagged: Atul Gawande, Checklists
Aug 6, 2010