The United States Medical Licensing Examination (USMLE) Step
1, a test co-sponsored by the Federation of State Medical Boards (FSMB) and the
National Board of Medical Examiners (NBME), has been the exam that people love
to hate. For many years, blogs, Twitter feeds, and opinion pieces have been
accumulating urging the presidents of the FSMB/NBME to stop reporting a 3-digit
score and instead report a pass/fail score. This animosity towards the Step 1
exam originates from the reality that medical schools have increasingly focused
their curriculum on teaching what the Step 1 wants you to learn – medical
trivia that almost always has no bearing on how to approach a clinical problem.
This “Step 1 Madness” is unhealthy. The reasons for its
existence are many: residency and fellowship programs allow it to exist by
idolizing higher scores, some believe it is a metric that can predict future
quality of care, board pass rates, etc. And some are naïve enough to think that
what is tested on the Step 1 is actually useful medical knowledge! It may be
due to a combination of the above that the Step 1 has found itself in such a
peculiar spot. However, the emphasis on the Step 1 score means that medical
students’ fate is being determined by a single test. Nobody wants their fate to
be so unmalleable.
“YOUR LIKELIHOOD OF SECURING RESIDENCY TRAINING DEPENDS ON MANY FACTORS – INCLUDING THE NUMBER OF RESIDENCY PROGRAMS YOU APPLY TO.”
So begins the introduction to Apply Smart: Data to Consider When Applying to Residency – a informational campaign from the Association of American Medical Colleges (AAMC) designed to help medical students “anchor [their] initial thinking about the optimal number of applications.”
In the era of Application Fever – where the mean number of applications submitted by graduating U.S. medical students is now up to 60 – some data-driven guidance on how many applications to submit would be welcome, right?
And yet, the more I review the AAMC’s Apply Smart campaign, the more I think that it provides little useful data – and the information it does provide is likely to encourage students to submit even more applications.
This topic will be covered in two parts. In the first, I’ll explore the Apply Smart analyses and air my grievances against their logic and data presentation. In the second, I’ll suggest what the AAMC should do to provide more useful information to students.
Introduction to Apply Smart
The AAMC unveiled Apply Smart for Residency several years ago. The website includes lots of information for students, but the piece de resistance are the analyses and graphics that relate the number of applications submitted to the likelihood of successfully entering a residency program.
Recently, I was on The Accad and Koka Report to share my opinions on USMLE Step 1 scoring policy. (If you’re interested, you can listen to the episode on the show website or iTunes.)
Most of the topics we discussed were ones I’ve already dissected on this site. But there was an interesting moment in the show, right around the 37:30 mark, that raises an important point that is worthy of further analysis.
ANISH: There’s also the fact that nobody is twisting the arms of program directors to use [USMLE Step 1] scores, correct? Even in an era when you had clinical grades reported, there’s still seems to be value that PDs attach to these scores. . . There’s no regulatory agency that’s forcing PDs to do that. So if PDs want to use, you know, a number on a test to determine who should best make up their class, why are you against that?
BRYAN: I’m not necessarily against that if you make that as a reasoned decision. I would challenge a few things about it, though. I guess the first question is, what do you think is on USMLE Step 1 that is meaningful?
ANISH: Well – um – yeah…
BRYAN: What do you think is on that test that makes it a meaningful metric?
ANISH: I – I don’t- I don’t think that – I don’t know that memorizing… I don’t even remember what was on the USMLE. Was the Krebs Cycle on the USMLE Step 1?
I highlight this snippet not to pick on Anish – who was a gracious host, and despite our back-and-forth on Twitter, we actually agreed much more than we disagreed. And as a practicing clinician who is 15 years removed from the exam, I’m not surprised in the least that he doesn’t recall exactly what was on the test.
I highlight this exchange because it illuminates one of the central truths in the #USMLEPassFail debate, and that is this:
Physicians who took Step 1 more than 5 years ago honestly don’t have a clue about what is tested on the exam.
That’s not because the content has changed. It’s because the memories of minutiae fade over time, leaving behind the false memory of a test that was more useful than it really was.
United States medical education system is heralded as one among the top in the
world for medical training. Given the strict standards of education, multiple
licensing boards, and continuous oversight by governing bodies, getting a placement
to train in the US is extremely competitive. In 2017 alone, nearly 7000+ non-US citizens
(commonly referred to as “foreign medical graduates”) applied to compete with 24,000+
US citizens for American residency spots to pursue specialty training. The
reasons for this competitiveness are simple. The vast majority of medical institutions
in the US boast a comprehensive curriculum that entails basic sciences,
clinical principles, practical and hands-on didactics, and enriched exposure to
the clinical aspects of patient care. This training produces astute clinicians
that are capable of resolving the most complex diagnoses while providing comprehensive
it is high time to recognize that being a shrewd clinician is no longer a
sufficient product for the demands of the healthcare market today. That is to
say, the scope of medicine today for a physician has gone far beyond resolving
complex medical problems, but demands a higher understanding of multidisciplinary
skillsets, most important of which are finance and legal theory. In these
aspects, the US medical education system direly underprepares physicians, and
thus, requires a thorough reevaluation.
art of medicine, as much as it was originally developed to be purely about the
betterment of patient health, has become yet another siloed service industry.
Simply put, patients are customers, and physicians are increasingly held
accountable for the financial metrics and revenue their work produces. Compensation
models are increasingly favoring productivity based payment methods, such
as the relative value unit (RVU) system, and are moving away from the
traditional, salaried physician. This has resulted in increased pressure on
physicians to become more efficient with their workload and patient docket,
while managing the often turbulent and contradictory interests of insurance,
patients, and hospital administration.
Burnout is one of the biggest problems physicians face today. We believe that addressing it early — in medical school — through coaching gives physicians the tools they need to maintain balance and meaning in their personal and professional lives.
We say that after reading comments from participants in our coaching program, “A Whole New Doctor,” developed at Georgetown University School of Medicine. This program, born almost by chance, provides executive coaching and leadership training to medical students, who are exactly the right audience for it.
Medical students tend to begin their education as optimistic 20-somethings, eager to learn and eager to see patients. After spending one or two years on the academic study of medicine, they move to the wards where they observe the hidden curriculum — a set of norms, values, and behaviors conveyed in implicit and explicit ways in the clinical learning environment.
In the hospital, convenience and expediency, deference to specialists, and factual knowledge tend to replace the holistic and patient-centered care that is lauded during the preclinical years. This new culture nudges some students to the brink of burnout and depression. Some consider suicide.
I’m going to show you the Match rate and mean Step 1 score for three groups of residency applicants. These are real data, compiled from the National Resident Matching Program’s (NRMP) Charting Outcomes in the Match reports.
U.S. Allopathic Seniors: 92% match rate; Step 1 232.3
U.S. Osteopathic Seniors: 83% match rate; Step 1 225.8
International Medical Graduates, or IMGs (both U.S. and non-U.S. citizen: 53% match rate; Step 1 223.6
Now. What do you conclude when you look at these numbers?
In the debate over the U.S. Medical Licensing Examination’s (USMLE) score reporting policy, there’s one objection that comes up time and time again: that graduates from less-prestigious medical schools (especially IMGs) need a scored USMLE Step 1 to compete in the match with applicants from “top tier” medical schools.
In fact, this concern was recently expressed by the president of the National Board of Medical Examiners (NBME) in an article inAcademic Medicine (quoted here, with my emphasis added).
“Students and U.S. medical graduates (USMGs) from elite medical schools may feel that their school’s reputation assures their successful competition in the residency application process, and thus may perceive no benefit from USMLE scores. However, USMGs from the newest medical schools or schools that do not rank highly across various indices may feel that they cannot rely upon their school’s reputation, and have expressed concern in various settings that they could be disadvantaged if forced to compete without a quantitative Step 1 score. This concern may apply even more for graduates of international medical schools (IMGs) that are lesser known, regardless of any quality indicator.”
The funny thing is, when I look at the data above, I’m not sure why we would conclude that IMGs are gaining advantage from a scored Step 1. In fact, we might conclude just the opposite – that a scored Step 1 is a key reason why IMGs have a lower match rate.
Jessica DaMassa’s European tour continues. This week she’s at the #WebIT conference in Sofia, Bulgaria (no, I couldn’t find it on a map either!) and the #HealthIn2Point00 takeovers continue! This time the guest is pioneering British surgeon Shafi Ahmed, who has lots to say about medical education, the future of digital hospitals, what he’s up to in Bolivia and how cool #WebITHealth will be–Matthew Holt
Medical education is dynamic and constantly adapting to the needs of society. With new technological advances, scientific discoveries, and healthcare policies arising each day, the amount of information medical students are required to learn increases exponentially. Many describe the early years of medical education as a vicious cycle of cramming and forgetting with block exams, shelf exams, and board exams. Long-term retention is rarely rewarded and the integration across topics is limited. On the contrary, medicine IS a life-long learning process that is heavily dependent on the ability to attain, integrate, and apply data.
Unfortunately, time is limited, and as a result, cramming often prevails as the method of choice for many students. As medical students, we constantly find ourselves re-learning large amounts of information time and time again, always preparing for the next exam or hurdle, rather than thinking years down the line when we will be taking care of patients. This is very inefficient.
In June, Duke medical students wrote an article entitled “Want to enhance medical education? Use Spaced Repetition”. This article proposed a strategy that revolves around the cognitive technique known as spaced repetition. Spaced repetition takes advantage of time and reinforces one’s knowledge the moment before one forgets it. This technique involves reviewing material according to a schedule determined by a temporal relationship known as the “spacing effect”.
One of the most compelling medical stories in the country is unfolding within the sprawling landscape of inland Southern California. The story centers on the University of California, Riverside School of Medicine where G. Richard Olds, MD, the school’s dean, is taking on one of the uber challenges in health care today: How to get doctors into areas significantly underserved by health care professionals.
The UC-Riverside School of Medicine is in its infancy having welcomed its first class of 50 students just last year. But it has embarked on an innovative program fueled by a passion not only to get doctors into geographic areas where they are most urgently needed, but also to make sure these physicians practice specialties most in demand. “There are 18 new medical schools in the United States and the vast majority are just like existing medical schools,” says Dean Olds. “We are substantially different than most other new schools. We are designed around a unique mission – to try and address the health workforce needs of inland Southern California. We need to train health care professionals who come from backgrounds and communities they will be taking care of.”