Yesterday at the faculty meeting, we learned that the first year residents in anesthesia will now have to take AND PASS a written exam at the end of their first year. They will have a certain number of tries and if a resident can’t pass it by the third try they’re either out of the program or held back in some way. Now, it used to be when I was a baby resident that the first year residents took the certification exam that the third years took, and it was graded on a curve based on year. You didn’t have to pass it or get a certain grade; it was sort of a reality check, to see how you were doing. I don’t know who’s brilliant idea this new test was, other than the people who administer and charge for the test. It might be a solution in search of a problem, I have no idea.
Here’s the thing. Testing freaks residents out. They have been taking high-stakes tests their whole entire lives. In high school they had to get As and score a 1400 on the SAT. In college they still had to get As, but also had to ace the MCAT. In med school the tests might have been pass/fail but USMLE Steps 1 and 2, both of which are taken during med school, certainly weren’t. Results of those had bearing on what residency you got into. The result of all this standardized testing is that every resident has PTSD about tests, and every resident has had years to figure out how he or she can most quickly cram in the amount of information necessary to do well on the test. Residents are masters of this. There is absolutely no reason to read the textbook, which is likely 8 years out of date anyway, when you can go straight to the review books and practice exams online. Especially if the threat of expulsion or repetition, both of which are disasters on multiple foreign and domestic fronts, is held over their heads.
Cramming for a test is not learning. Let me make that perfectly clear. You forget it the minute you walk out of the exam room. Learning facts and passing multiple choice exams is not learning. It is not necessary to UNDERSTAND a subject to do well on it on tests. Prime example: me in calculus. I had no idea what I was doing or what it meant but I knew if I memorized how to run the steps I could always get the answer right. It was hilarious in a really sad way that I didn’t appreciate at the time. Med students and residents are not lazy learners. Given the right incentive they want to understand the material. But they also know two very important facts: 1. Most of the facts are not necessary to clinical functioning. A lot of the questions asked are about things you never actually use in practice. Everybody knows the dweeb who aces the exam but is completely hopeless in the clinic. Conversely you can have a great clinician who somehow managed to make it this far without knowing the partition coefficient of Sevoflurane. 2. The most important thing is always what the people in power think of you. The system is rigged so that passing the exams substitutes for more in depth and complicated evaluations. A great exam result is likely to get you off the hook for practical failings.
This new test will likely result in all first year anesthesia residents freaking out. They will haul around review books, new ones that have generated a lot of money for the people who write those things. They will look sidelong at their friends to see who is studying more than who or who knows more esoteric facts. They will stay up late and yawn in the OR. Study groups will form. Marriages will falter. Children will go unfed. And once the exam is over, nothing will have changed. Their skills will be the same. They will have been learning facts in place of learning how to give anesthesia. Awesome.
Shirie Leng, MD is a practicing anesthesiologist at Beth Israel Deaconess Medical Center in Boston. She blogs regularly at medicine for real, where this post originally appeared.
Excellent commentary about increasing use of testing — as a substitute for meaningful evaluation of the skills that matter. In the best of all possible worlds, wherever that is, one would hope to see thoughtful use of BOTH.
Looking back from retirement, it seems to me that I always preferred to refer patients to physicians that knew the limits of their knowledge. They were adept at filling in the details when more information was needed AND willing to ask for help whenever that was more appropriate. Of course the ability to communicate with both peers and patients (often now considered clients or consumers, but I will not deal with that shift here) was essential to the best use of their knowledge and wisdom.
It is always best to temper our enthusiasm for new ways to measure things (including skills) with thoughtful, fresh consideration of what may be lost by the proposed change, and how the change may affect our longer term goals.
Maybe a better way to test physicians in training would be to give them tests where they could have access to their usual information resources? That would be a much better indicator of how well they could apply information to solve real clinical problems.