A short piece in The Health Care Blog reveals (albeit unintentionally) why so many outside of healthcare think the medical establishment still doesn’t get it.
The post, written by a general internist and residency program director, asked why an increasing number of internal medicine doctors are failing their internal medicine board exams. The pass rate has reportedly declined over the last several years from 90% to 84%. (Disclosure: I passed this required test about a decade ago.)
His differential included two possibilities:
(1) The test is getting harder – The testing agency said this wasn’t the case.
(2) Millennials lack the study habits of their elders, and have become great “looker-upers.” – The author suggested this was a key factor, and several commentators enthusiastically agreed.
The basic thesis here that in the Days of Giants, doctors worked harder, learned more, and were better. Nowadays, doctors are relatively complacent, less invested, less informed, and are generally worse – which is what’s reflected on the board exams.
Let me suggest a third possibility – perhaps today’s doctors are providing better care to patients than their predecessors were a generation ago. Maybe today’s doctors have figured out that in our information age, your ability to regurgitate information is less important than your ability to access data and intelligently process it. Maybe what makes you a truly effective doctor isn’t your ability to assert dominance by the sheer number of facts you’ve amassed, but rather how well you are able to lead a care team, and ensure each patient receives the best care possible.
In other words, what if the problem isn’t the doctors, who are appropriately adapting, but rather the tests (and the medical establishment), which may not be?
I’ve previously expressed concern that healthcare workers in general are subject to a series of expensive exams that are viewed as assessments of, and then proxies for quality, yet which are rarely correlated with any meaningful quality measure. They remain an important source of money and status for the societies (guilds) that develop and administer them. To the extent that we attach value to certification measures uncorrelated with quality, we are enablers, part of the problem.
I’ve not always seen eye to eye with technologist and investor Vinod Khosla, yet he’s absolutely right when he points to the rapidly accumulating amounts of medical information available in the world, and the absurdity of expecting any individual person to truly master it.
The doctor of the future should have the humility to understand her own limitations, and know how to use technology productively to find what she needs. She should be comfortable enough with technology so that it’s used in service of patient care, and not in place of it. She must also recognize when to distrust technology, and perhaps most importantly, know when to set it aside.
If board scores suggest young physicians aren’t learning the currently required material, it may not be test-takers who are failing.
David Shaywitz is co-founder of the Center for Assessment Technology and Continuous Health (CATCH) in Boston. He is a strategist at a biopharmaceutical company in South San Francisco. This post originally appeared in his regular Forbes column.
Categories: Uncategorized
I cannot agree more Tom. I too found out today that I failed and our study practices were very similar. That test was complete bullshit and it’s just a way for the guilds to make money. Bullshit.
I just received my exam results, i failed. I not only completed a great residency at a university but have NEVER failed a test in my life. I studied my ass of for 7 months, went thru MKSAP twice and read all the books not to mention other sources. Not only was the test complete bullshit, its literally a mental game. I learned that if you have other stuff going on in life do not take this test, reschedule it, because now im considering leaving medicine and never coming back. My patients btw love me and i have been nominated as physician of the month several times. well looks like 1 test could take that all away and probably just did
l
I’d say it goes beyond residency reform and has to be medical education reform to its core, specifically for clinical rotations.
This article interested me, but I’m not really sure if it really addressed the issue that I personally see plaguing our medical education in today’s world. In this world that technology is rapidly advancing and more and more information is at the fingertips of the physician, every physician is being forced to look information up. The largest issue I see in medical education is in the teaching. Many teaching physicians feel the pressure of time and efficiency to see their patients (20 minutes in an out patient setting is indeed generous). Physicians are expected to run on time, garner the history from the patient, examine them, and then formulate the assessment and plan and inform this to the patient in terms they can understand so that they are informed. Oh and document all of this on the medical record. All in 15 minutes. This is an extremely difficult task and generally causes the physician to run behind for the day. Add in the patient phone calls and prescription requests throughout the day and this is a fairly packed day. Now add in the education that physician is supposed to bestow upon a rotating medical student or resident. If he’s already stressed for time, where does he cut corners to make up for this time? And, at the end of the day, the teaching physician wishes to go home to their family. So commonly heard by rotating students and residents is “You’ll get plenty of this as a resident/when you’re out in practice.”
In my mind the injustice in our education system begins early. The third and Fourth years in medical school is spent doing rotations. And I know from personal experience, these students, who have minimal clinical experience, are given time constraints on seeing patients. So many times they are only given 15-20 minutes or are often interrupted while seeing the patient so that the overseeing physician can remain efficient and not fall too far behind. To top it off, many students and residents on rotations are only shadowing and observing. Which in my mind does provide them with some education, but there is no greater learning tool than to figure things out on their own. So if you ask me the reason more and more young doctors are failing their boards, I would have to say it is from the weak knowledge foundation that they have obtained during the education on rotations. I am truly fearful for the future of medicine because of this fact. I think we really have a generation of physicians coming through that will not have any ability to effectively see patients without having to look everything up and physicians that will downright be dangerous when they are first practicing medicine because they will be learning from their mistakes. Mistakes that they should’ve been able to make as students or residents but were unable to because they were only shadowing or not given any time to truly develop a good algorithm during patient interaction. I blame the educational system for failing our students needs and watering down medicine. In fact, I actually have rather original ideas in how to fix this problem and some portions of the health care costs or burdens in ways that have never really been thought of before, but that is a much longer train of thought.
I think it is up to young physicians how they learn their required material, if they don’t they can fail.
Excellent. It is a new world, is it not? The Internet has literally changed everything, to include medicine.
One possible medical future is examined in the Eric Topol, M.D. book, The Creative Destruction of Medicine.” A great read.
Many other possibilities exist. More important is to recognize that PROFITS are up with retesting and the “curve” is adjusted by a board of 12 individuals “in it for the glory and money”-i.e. NO freedom from corruption or conflicts of interests. The ABMS is more a “guild mechanism” to restrict numbers of “eligible physicians” based on an arbitrary test. The original “qualified ot serve as a consultant in” has now been degraded to a MOC certificate indicating compliance with a non-validated, non-government private corporate program only. The ABMS has NO impact in the world EXCEPT the USA where we have the longevity of Ghana and infant mortality is terrible.
We are not failing young physicians. The system of using HIT care record devices is sapping their cognitive power to learn clinical medicine and clinical judgment.
HIT care record devices and their CPOE and CDS modules represent a new disease that requires painstaking time devoted to understanding its idiosyncrasies and whimsical flaws and error creating capacities.
For those of you commenting on and writing these blogs, you ought to disclose whether you are an interested watcher or an active user of these toxic instruments of care.
If you have ever used them, you will quickly understand why the trainees have lost the capacity to effectively manage patients’ care and pass the boards.
Yes!
Can someone share any data that correlates being “board certified” with clinical competence/decision-making and better patient outcomes?
What do most patients want and need from their physicians? Generalists vs. specialists? Is it something you can test? What I assume when I see someone who has passed such a credentialing exam is that they are a problem solver. Perhaps there’s a difference between being able to memorize and being able to problem solve. For example, you’re more likely to do well on legal profession tests (LSAT and bar exam) if you can read and comprehend complex material quickly; memorization less of an issue. What I want in any advisor (medical, legal, financial, or anyone else) is someone who can sort through a lot of information and give a sound opinion on what my best options are, whether it’s related to a diagnosis, a prognosis or a treatment. I hope being “board certified” is at least one criteria that will point me toward someone who can do that.
I agree wholeheartedly about your uncertainty with “Board” testing. I passed my college boards, MCATS, internal med and specialty boards with high grades on the first attempt. I’m convinced that the ability to pass the boards doesn’t mean you are a good doc. It means that you can memorize like a demon, retain the memorized info for a short period of time, use the memorized data to take the test and then forget it.
I’d love to see data that looks at the value of these tests as a predictor of quality. I don’t think there is any – and there should be hard data!!!
4th hypothesis:
Residents and interns are squeezed by the hospital system and work demands that they aren’t having the time to learn what they are supposed to.
Residency reform. That’s what this blog should be all about.
You may be right, but do you have any evidence to support your claims? Most practicing physicians I know, even older ones, are grateful for the information these devices provide at their fingertips.
Judgment is an essential ingredient in effective care. Judgment requires creative energy and cognitive power and is not available on these eperimental HIT devices. Fact is that the HIT devices are impediments to the needs of the would be effective clinician, and dumb down the user, which is why I am excited about the failure rate in the Boards. QED
Perhaps, Dr. Horwitz, but do you really want to trust your memory in those cases where you think you have something memorized? I would be interested to know what things you think you have memorized but actually do not.
I agree with this posting. Let me restate some things I replied in response to Dr. Shumann’s post:
This development of falling board scores is a concern, but before we ascribe blame to health IT, Gen Y/millenial laziness, or anything else, can we see some data supporting the assignment of blame? Let’s not let this finding be a Rorschach test for everything each of us does not like about medicine, healthcare, or society.
Similar to Dr. Shaywitz, I have a more fundamental question: Is an exam like the ABIM the best indicator of how a physician best applies his or her knowledge to the optimal care of patients? In other words, when it comes to applying knowledge to real patients, does it matter whether the knowledge resides in the physician’s head or is easily accessible on a computer? Do we want performance on this test to be the standard by which we judge the competence of physicians? I thought we were getting away from that approach.
Instead, why don’t we test physicians by some sort of vignette process, whereby they are presented with a case, armed with their brain, their computer, and anything else they want to use? Then we can see who orders the right tests, prescribes the right treatment, etc.?
Having just (phew!) passed my first recert exam, I read this post and the related one with interest. I think both viewpoints are important. I agree with Dr. Shaywitz that for complicated hospital inpatients, where you have a lot of time to spend per patient, it is more important that physicians have the skills to look up the latest treatments, differential diagnoses, etc than necessarily to try to memorize a whole ton of facts that change on a regular basis. On the other hand, though, I do most of my clinical work in the outpatient primary care setting. At 20 min per patient (generously), I really need a pretty comprehensive and accurate fund of knowledge that I can access without doing a lot of real-time looking up. Sure, yesterday I saw a patient I thought might have Cushing’s and had to spend a couple minutes looking up the latest outpatient diagnostic algorithm. But I can’t do that for every patient, and at a minimum I need to know enough to know what to look up. So I think it’s reasonable to test me on a fund of knowledge I should be expected to use on a daily basis, and to test my general pattern recognition for common or deadly complaints. Similarly, I write questions on biostatistics and epidemiology for the USMLE exams. This is a fund of knowledge I feel strongly that doctors need to know cold in order to appropriately interpret all the other stuff they look up. Which is just to say, I think there is still a reasonable role for test-taking though modification of focus and types of questions is important.
I think this critique raises an interesting question about whether our whole educational paradigm is now outdated. Most education, whether k-12, college, or graduate school, measures ability through high-pressure test-taking where speed and memory are the only skills being rewarded. Those, incidentally, are also the skills that computers have rendered obsolete.
Hasn’t medicine always required “looker-upper” skills? There is NO WAY that a physician can hold all the literature on one disease in their heads, much less the spectrum of conditions that an internist will treat in the course of a year.
Why not teach medical students and intern/residents how to look for/evaluate clinical literature that’s relevant to the case before them, rather than force them to cram ahead of a board exam? That guarantees that they won’t retain much past the actual exam, making the cranial stuff-fest useless for actual medical practice.
THANK YOU. I find that a really distinctive generation gap is developing where anyone younger is getting slammed for doing things differently. But, from my/our perspective, how have the methods of a previous generation worked? Our generation isn’t seeing the results from those methods working for the needs of today’s—our— world.
Additionally, if there is information at your fingertips, wouldn’t the knowledge of how to use that information be more valuable than just memorizing all of it? There is even more data now than there was before. How and who can accurately determine what is actually relevant?
You are correct.
Also, I like your post. Well done.
I suspect both John Schumann and I enthusiastically agree re: goals of technology…
John: http://www.theatlantic.com/health/archive/2012/06/are-computers-getting-between-you-and-your-doctor/258147/
Me: http://www.theatlantic.com/health/archive/2012/10/humanism-in-digital-health-do-we-have-to-sacrifice-personal-connections-as-we-improve-efficiency/264325/
I agree with the author’s point, in that retaining a lot of facts is probably no longer as important as it used to be, and the boards are becoming outmoded.
I’d be surprised, however, if today’s young doctors are providing better care, mainly because from what I read of today’s teaching ward, the focus on largely on the iPatient rather than the patient. So it seems that today’s doctors aren’t getting time to sit, connect, and collaborate with patients.
Technology when it works well is great, but in practice it often takes a lot of time to sit at the computer and fill things out. Also, there is now more information than ever to be collected from the digital files. Plus, computer interfaces are often badly designed.
All this makes it hard for doctors to find the time, mental energy, and emotional energy to do just what Shaywitz says they should do: connect with the patient, and connect with the care team.
What we need: tech tools that are better designed, more usable, and don’t suck up a lot of our time or mental energy. Then we need a little more time to connect with patients and colleagues, so we can address the human parts of medicine that tech will never replace.
Doctors have become decerebrate due to their use and reliance on Poorly designed and poorly usable HIT devices.