I was going to write a lengthy post, but then my medical education spirit animal Bryan Carmody did and said most of what I have time to say at the moment better than I would have anyway. Take the time to read it: USMLE Pass/Fail: A Brave New Day. He’s created an impressive collection of excellent writing about acronym fiends like the AAMC, NBME, and NRMP, and this is no exception.
This is a Brave New World. It will be a period of change, and it may very well be a rough transition. I know in this post-fact world we live in that people are cynical and want to cling to an objective measure. The system was flawed but in many ways predictable, and that comfort goes a long way. Students know what to expect. For those who put the time in and succeed, doors can be reliably opened.
Everyone’s concerns about shifting pressure to Step 2 CK, replacement by other likely useless metrics, elite schools, etc etc are all valid. A better future isn’t guaranteed, and Step 2 CK will certainly be the new default if it’s allowed to be (though even that would be an improvement since it’s a better test; I suspect it will become pass/fail within a few years as well).
But Step 1 is not good measure.
It doesn’t measure what matters, creates false distinctions across similar applicants, and may even select for some negative qualities. It’s turned medical school into an overpriced correspondence course and forced students to waste all of their energies spending more and more time mastering less and less useful material. None of our knowledge assessments from the MCAT to the USMLE to any of the board certification exams actually do what they are designed to do. And that’s a huge problem when we’ve absolved ourselves of meaningful holistic and true performative review and instead let a bunch of basically anonymous Angoff panels decide what it is a doctor is and does.
Arthur Jones of Proctor & Gamble once remarked that “all organizations are perfectly designed to get the results they get.” Well our system–from medical admission to MOC–is designed to get unsustainable negative results.
Schools and residency programs have a couple of years to figure out a more meaningful way to evaluate students and select residents. Pass/Fail Step 1 is a lit fire under everyone’s tushes. It’s not a solution; it just opens a door.
2 Comments
Agreed.
Not much else to add, you’ve summarised it well. All Step 1 selects for… is people who take tests with the content of step 1 well. That’s it. Any fantasy that it actually makes you a better doctor in my opinion are rubbish, and the students defending that notion likely don’t understand as they haven’t had enough time (well basically any, because they’re students) as actual doctors.