A Virtual Step 2 CS: More convenient, equally if not more useless

The NBME announced a new virtual USMLE Step CS would replace the pre-pandemic in-person exam that’s topped the list of medical student headaches for recent generations. While light on details, it sounds like it’s going to be a bunch of telehealth visits that will do away with the pretense that the exam evaluates students’ ability to perform or interpret a real physical exam.

This week they released a brief and extremely scripted sounding podcast. Details are still forthcoming, but a couple of highlights:

David Johnson, “Chief assessment officer” with the Federation of State Medical Boards (FSMB):

The community feedback that we received reinforced some of the already understood challenges with the past format, things we were aware of. For instance, the limited number of test sites…the cost of the exam, and some perceptions of [an] artificial nature of the standardized patient interaction and the patient note, as well as admittedly the limited feedback that was provided to examinees and scoring. Those were the things that came up that frankly, we were not surprised to hear about.

Bit of an underplay here.

Conspicuously missing: I guarantee that the most common feedback they received was that the exam should be canceled forever. Clinical skills assessment was previously and could easily be performed by medical schools, which all conduct standardized patient assessments anyway. If the LCME accreditation can’t confirm that schools are able to perform this core function, then what’s the point of students spending all these years and all this money going to medical school.

Dr. Chris Feddock, Executive Director of the NBME’s “Clinical Skills Evaluation Collaboration”:

The rework of the prior exam using a computer-based platform…will enable the USMLE to meet its mission, which is serving the state medical boards.

I do like how honest this statement is. Make no mistake, the mission of the USMLE is to serve the state medical boards. Not the public. Not doctors. Not science. Not to ensure anything measurable or meaningful. These box-checking endeavors are in service to bureaucracy, not to produce good outcomes.

They go on to say that there was “broad agreement among stakeholder groups” that Step CS is important. It’s worth noting that “stakeholder groups” in this context refers to the incestuous tangle of mostly highly-profitable non-profit medical organizations like the NBME, ECFMG, and the FSMB. The many students and doctors who are actually involved in the education/training process or the practice of medicine are part of the “community.”

The community, of course, has no recourse, and these “self”-regulatory bodies have no oversight. They have no need or desire to prove, for example, that their tests do anything close to what they’re designed to do other than be standardized and consistent year-to-year (at best, construct validity without predictive validity).

They have never, and probably will never, prove that Step CS, in its current or future form, does anything other than transfer student loan money from students to the NBME.