What is quality care, and how do you define it?
I suspect for most people that quality is like pornography in the classic Supreme Court sense—you know it when you see it. But quality is almost assuredly not viewed that way when zoomed out to look at care delivery in a broad, collective sense. Instead, it’s often reduced to meeting a handful of predetermined outcome or compliance metrics, like pneumonia readmission rates or those markers of a job-well-done as defined in the MIPS program.
The reality is that authoritative, top-down central planning in something as variable and complicated as healthcare is extremely challenging, even if well-intentioned. As Goodhart’s law says, “When a measure becomes a target, it ceases to be a good measure.”
In the real world, I would argue a quality radiology report is one that is accurate in its interpretation and clear in its communication. But without universal peer review, double reading, or an AI overlord monitoring everyone’s output, there is no way to actually assess real quality at scale. You can’t even tell from the report if someone is correct in what they’re saying without looking at the pictures. Even “objectively” assessing whether they are clear or helpful in just their written communication requires either a human reviewer or AI to grade language and organization by some sort of mutually agreed-on rubric. It’s simply not feasible without a significant change to how healthcare is practiced.
And so, we resort to proxy metrics—like whether the appropriate follow-up recommendation for a handful of incidental findings was made. The irony, of course, is that many of these quality metrics are a combination of consensus guidelines and healthcare gamesmanship developed by non-impartial participants with no proof they reflect or are even associated with meaningful quality at all.
We should all want the quality of radiology reporting to improve, both in accuracy and in clarity. Many of these problems have been intractable because potential solutions are not scalable with current tools and current manpower—which is why soon you’ll be hearing about AI for everything, because AI solves the scaling problem, and even imperfect tools over the coming years will rapidly eclipse our current methods like cursory peer review.
Everyone would rather have automated incidental finding tracking than what most of us are still using for MIPS compliance. Right now, it’s still easy to get dinged and lose real money because you or your colleagues omitted some BS footer bloat about the source of your follow-up recommendations for pulmonary nodules too often. Increased quality without increased effort is hard to complain about.
But even just imagine you have a cheap LLM-derived tool that catches sidedness errors (e.g. right abnormality in the findings and left in the impression) or missing clarity words like forgetting the word “No” in the impression (or, hey, even just the phrase “correlate clinically”). This already exists: it’s trivial, requires zero pixel-based AI, (and—I know—is rapidly becoming table stakes for updated dictation software), but widespread adoption would likely have a more meaningful impact on real report quality than most of the box checking we do currently. A company could easily create a wrapper for one of several current commercial products and sell it tomorrow for those of us stuck on legacy systems. It might even be purchased and run by third parties (hospitals, payors, Covera Health, whatever) to decide which groups have “better” radiologists.
But now, take it that one step further. We’ve all gotten phone calls from clinicians asking us to translate a colleague’s confusing report. Would a bad “clarity score” get some radiologists to start dictating comprehensible reports?
It’s not hard to leap from an obviously good idea (catching dictation slips) to more dramatic oversight (official grammar police).
Changes to information processing and development costs mean the gap between notion and execution is narrowing. As scalable solutions proliferate, the question then becomes: who will be the radiology quality police, and who is going to pay for it?