I’m going to show you the Match rate and mean Step 1 score for three groups of residency applicants. These are real data, compiled from the National Resident Matching Program’s (NRMP) Charting Outcomes in the Match reports.
U.S. Allopathic Seniors: 92% match rate; Step 1 232.3
U.S. Osteopathic Seniors: 83% match rate; Step 1 225.8
International Medical Graduates, or IMGs (both U.S. and non-U.S. citizen: 53% match rate; Step 1 223.6
Now. What do you conclude when you look at these numbers?
In the debate over the U.S. Medical Licensing Examination’s (USMLE) score reporting policy, there’s one objection that comes up time and time again: that graduates from less-prestigious medical schools (especially IMGs) need a scored USMLE Step 1 to compete in the match with applicants from “top tier” medical schools.
In fact, this concern was recently expressed by the president of the National Board of Medical Examiners (NBME) in an article inAcademic Medicine (quoted here, with my emphasis added).
“Students and U.S. medical graduates (USMGs) from elite medical schools may feel that their school’s reputation assures their successful competition in the residency application process, and thus may perceive no benefit from USMLE scores. However, USMGs from the newest medical schools or schools that do not rank highly across various indices may feel that they cannot rely upon their school’s reputation, and have expressed concern in various settings that they could be disadvantaged if forced to compete without a quantitative Step 1 score. This concern may apply even more for graduates of international medical schools (IMGs) that are lesser known, regardless of any quality indicator.”
The funny thing is, when I look at the data above, I’m not sure why we would conclude that IMGs are gaining advantage from a scored Step 1. In fact, we might conclude just the opposite – that a scored Step 1 is a key reason why IMGs have a lower match rate.
Jessica DaMassa’s European tour continues. This week she’s at the #WebIT conference in Sofia, Bulgaria (no, I couldn’t find it on a map either!) and the #HealthIn2Point00 takeovers continue! This time the guest is pioneering British surgeon Shafi Ahmed, who has lots to say about medical education, the future of digital hospitals, what he’s up to in Bolivia and how cool #WebITHealth will be–Matthew Holt
Medical education is dynamic and constantly adapting to the needs of society. With new technological advances, scientific discoveries, and healthcare policies arising each day, the amount of information medical students are required to learn increases exponentially. Many describe the early years of medical education as a vicious cycle of cramming and forgetting with block exams, shelf exams, and board exams. Long-term retention is rarely rewarded and the integration across topics is limited. On the contrary, medicine IS a life-long learning process that is heavily dependent on the ability to attain, integrate, and apply data.
Unfortunately, time is limited, and as a result, cramming often prevails as the method of choice for many students. As medical students, we constantly find ourselves re-learning large amounts of information time and time again, always preparing for the next exam or hurdle, rather than thinking years down the line when we will be taking care of patients. This is very inefficient.
In June, Duke medical students wrote an article entitled “Want to enhance medical education? Use Spaced Repetition”. This article proposed a strategy that revolves around the cognitive technique known as spaced repetition. Spaced repetition takes advantage of time and reinforces one’s knowledge the moment before one forgets it. This technique involves reviewing material according to a schedule determined by a temporal relationship known as the “spacing effect”.
One of the most compelling medical stories in the country is unfolding within the sprawling landscape of inland Southern California. The story centers on the University of California, Riverside School of Medicine where G. Richard Olds, MD, the school’s dean, is taking on one of the uber challenges in health care today: How to get doctors into areas significantly underserved by health care professionals.
The UC-Riverside School of Medicine is in its infancy having welcomed its first class of 50 students just last year. But it has embarked on an innovative program fueled by a passion not only to get doctors into geographic areas where they are most urgently needed, but also to make sure these physicians practice specialties most in demand. “There are 18 new medical schools in the United States and the vast majority are just like existing medical schools,” says Dean Olds. “We are substantially different than most other new schools. We are designed around a unique mission – to try and address the health workforce needs of inland Southern California. We need to train health care professionals who come from backgrounds and communities they will be taking care of.”
Last year, about 43 million people around the globe were injured from the hospital care that was intended to help them; as a result, many died and millions suffered long-term disability. These seem like dramatic numbers – could they possibly be true?
If anything, they are almost surely an underestimate. These findings come from a paper we published last year funded and done in collaboration with the World Health Organization. We focused on a select group of “adverse events” and used conservative assumptions to model not only how often they occur, but also with what consequence to patients around the world.
Our WHO-funded study doesn’t stand alone; others have estimated that harm from unsafe medical care is far greater than previously thought. A paper published last year in the Journal of Patient Safety estimated that medical errors might be the third leading cause of deaths among Americans, after heart disease and cancer.
While I find that number hard to believe, what is undoubtedly true is this: adverse events – injuries that happen due to medical care – are a major cause of morbidity and mortality, and these problems are global. In every country where people have looked (U.S., Canada, Australia, England, nations of the Middle East, Latin America, etc.), the story is the same.
Patient safety is a big problem – a major source of suffering, disability, and death for the world’s population.The problem of inadequate health care, the global nature of this challenging problem, and the common set of causes that underlie it, motivated us to put together PH555X.
It’s a HarvardX online MOOC (Massive Open Online Course) with a simple focus: health care quality and safety with a global perspective.
Earlier this month Shiv and Ryan published a piece in the Annals of Internal Medicine, entitled What Can Medical Education Learn from Facebook and Netflix? We chose the title because, as medical students, we realized the tools our classmates are using to socialize and watch TV use more sophisticated algorithms than the tools we use to learn medicine.
What if the same mechanisms that Facebook and Netflix use—such as machine learning-based recommender systems, crowdsourcing, and intuitive interfaces—could transform how we educate our health care professionals?
For example, just as Amazon recommends products based on other items that customers have bought, we believe that supplementary resources such as questions, videos, images, mnemonics, references, and even real-life patient cases could be automatically recommended based on what students and professionals are learning in the classroom or seeing in the clinic.
That is one of the premises behind Osmosis, the flagship educational platform of Knowledge Diffusion, Shiv’s and Ryan’s startup. Osmosis uses data analytics and machine learning to deliver the best medical content to those trying to learn it, as efficiently as possible for the learner.
Since its launch in August, Osmosis has delivered over two million questions to more than 10,000 medical students around the world using a novel push notification system that syncs to student curricular schedules.
Osmosis is aggregating medical school curricula and extracurricular resources as well as generating a tremendous amount of data on student performance. The program uses adaptive algorithms and an intuitive interface to provide the best, most useful customized content to those trying to learn.
In the 2012 National Residency Match Program Survey, which is sent out to residency program directors around the country by the NRMP, the factor that was ranked highest with regards to criteria considered for receiving an interview—higher than honors in clinical clerkships, higher than extracurricular experiences or AOA election, and even higher than evidence of professionalism, interpersonal skills, and humanistic qualities—was the USMLE Step 1 score.
When considering where to rank an interviewed applicant, the Step 1 score took a backseat to some of the aforementioned criteria that are perhaps more telling of what kind of person the interviewee is, although it was still one of the highest considered criteria for ranking applicants as well.
When a single exam is given this level of importance in determining a future physician’s most critical period in career development—their residency—we have to look carefully at our system.
Two points of consideration come to mind. First, is it wise to weigh a test score so heavily? Many students and faculty could easily point out that student performance on exams by no means always reflects their clinical acumen and social skills when seeing patients.
Medicine is, after all, an art far more than a science.
Nonetheless, it would be foolish to assume that scores have no worth—a high score on an exam, particularly a behemoth such as the USMLE Step 1, points out many qualities in an individual: hard work, persistence, discipline, and frankly, an understanding of textbook medicine.
And thus, we are left somewhere in the middle—perhaps we should weigh scores less than we do, but when you have to sort through thousands of applications, the only standardized metric to quickly compare is, in the end, a number somewhere between 192 and 300.
Why should I be in the same room with these people?
That’s one of the many smart questions participants posed at a Stanford Medical School meeting I attended last weekend. If I had been daydreaming (I’d never do that), I might have thought the question was for me. You see, the participants were a handpicked set of national medical education experts, folks nominally from the status quo medical-education-industrial complex—the very thing we’re trying to change.
You might think that they embodied that dreaded status quo. I’m happy to report they did not—not even close. I’m also relieved to tell you that the question (in spite of my paranoia) wasn’t for me. Instead, it was one of many challenges these thoughtful, passionate teachers tossed at each other.
“Why are we in the room?” was a challenge to each other. Why and when should teachers be in the same room with the learners?
When you think about it, that’s actually a central question if you’re attempting to use online education to flip the medical education experience. It’s also a brave one if you’re a teacher: justify the time you spend with your students.
The blog and comments point, we think, to a confusing set of principles being considered, perhaps, out of context?
Those comments range from: ACOs will lead to better figuring out what is best (impossible) – to mismatched information regarding a specific clinical case (reasonable). What is striking is that we have medical students worrying about costs of care.
Instead, shouldn’t we be teaching them to understand the value of information for decision-making? Shouldn’t we be teaching them the concepts of co-dependent testing leading to all tests being less useful than we think?
Shouldn’t we be teaching students the concepts of decision-analysis, and thresholds, and patient’s being involved in the decisions? Shouldn’t we be teaching that it is better to know than to think we know? Shouldn’t we be doing studies rather than scratching at the “tragedy of the commons” (so many physicians feasting on the grassy fields of a sick patient)?