DASH Commentary
by Dr. Ricky Hu, UBC Radiology Resident
“Human physicians will continue to play a pivotal role in healthcare for the next 10-20 years.”
A recent article in Radiology Business with comments from Dr. Mitchell Katz, CEO of NYC Health and Hospitals suggested AI could replace a "great deal of radiologists” if only we can get through regulatory approval. This claim would result in an extensive paradigm shift in our clinical and technological workflow and raises alarms, not only regarding job security but also the appropriateness of introducing AI into our workplace.
Aside from my clinical training, I have worked as an AI research engineer for 10 years and I still strongly believe human physicians will continue to play a pivotal role in healthcare for the next 10-20 years. There is no question that the capability of new AI architectures is rapidly progressing, from convolutional neural networks (CNNs), which have achieved excellent accuracy in optical image classification since their breakthrough performance in the ImageNet challenge (a large-scale competition using millions of labeled images) to generative AI with large language models like chatGPT. Many health systems have now tested pilot AI tools from pulmonary embolism detection, stroke identification, and 3D image reconstruction.
However, there is an important distinction between AI that produces results and language that appears convincing versus AI that is actually accurate. I believe that the usage of AI is no different than any other tool where maximum patient safety is the priority and there are several limitations we should be cautious of in its implementation.
Major limitations to keep in mind include the following:
1. Generally speaking, AI is not capable of true logic in a similar way humans are.
The phrase I have used is that humans are excellent at bottom-up thinking, using baseline clinical knowledge to deduce a disease process. AI, by contrast, operates through top-down thinking, recognizing when a conclusion is mathematically correlated to data and applying that correlation to new cases.
For example, a neural network is a multi-layered set of intermediate nodes that feed forward a weighted output to subsequent layers, with weights updated with more training data and essentially optimizing complex correlations between input to output. The transformer architecture, used in GPT, employs a self-attention structure to predict the next token (or letter in its output) based on prior input. In other words, complex AI models perform highly detailed correlations with hundreds of millions of parameters, as opposed to humans who produce logical deduction from foundational concepts.
A Real-Case Example
A recent critical case we saw on call provides a motivating example highlighting the importance of clinical intuition. A patient with abdominal pain received a CT scan of their abdomen with difficult visualization of the bowels, a frequent challenge due to how tightly our organs are spaced, varying from person to person. There were small gas locules and fluid in the abdomen that appeared external to the bowel. Zooming in closely, there was slightly more haziness surrounding one small bowel wall, but it was next to the gallbladder, liver, arteries, and veins, and was not clearly isolated (in a way that, for instance, an obvious gallstone was). The haziness, or "fat stranding" could be a sign of inflammation (from infection, trauma, fluid overload or one of dozens of possible causes) or a completely benign finding (from overlapping bowel, patient movement, temporary distention of that bowel from passage of food contents). After review of the physical exam, lab values, discussion with emergency physician, and review of prior images, the final diagnosis was called as pneumoperitoneum from a perforated ulcer in the small bowel, a surgical emergency requiring immediate lifesaving operative intervention. This critical diagnosis required multiple stages of logical reasoning, including piecing together clinical history, tracing the boundaries of every organ, contextualizing the imaging patterns, and ruling out hundreds of possible diseases. There is currently no AI tool that can perform this task but this was a diagnosis and reasoning that a trainee is expected to understand. Currently, the majority of AI papers have been on developing narrow, focused problems, such as recognizing pneumonia on x-ray, rather than multi-purpose AI (i.e. Agentic AI) for tasks like reading a full CT of the chest or abdomen to identify if any abnormality exists (an exponentially more difficult problem requiring understanding how surrounding structures affect each other). However, even 2D-based AI has not become widely implemented to support any argument for replacing physician ability at this time.
2. Complex prediction models require enormous amounts of training data.
In statistics, the classical rule of thumb is "one-in-ten", meaning at least 10 samples are required per variable for that event (i.e. if a model has 10 variables to predict one disease, a minimum of 100 samples are needed). Medical imaging data is even more complex. A single CT scan slice that is 512x512 pixels would have 262,144 pixels or "variables" (though not all independent of one another, so the one-in-ten rule is relaxed). However, a CT scan has multiple slices going down the length of the body, generating up to hundreds of still 512x512 images, multiplying the sample size required. In addition, confounders such as patient's anatomical structure, additional medical conditions, demographics, and scanner models, all add more variables. To put into perspective, CNNs for single optical images such as recognizing flowers or for self-driving cars, simpler tasks not requiring 10+ years of medical training, were trained on millions of images.
The critical limitation is that we do not have thousands, let alone millions, of CT scans for each of the hundreds of diseases that a radiologist is trained to identify. Humans can deduce how their organs are expected to look from baseline knowledge of geometry, anatomy, and physiology, whereas AI relies on correlation from seeing similar cases previously.
There are ways AI is trying to overcome this with techniques such as transfer learning or foundational models where for cases without enough data, the AI "borrows" a trained model from another similar problem as starting point for optimization. However, this is still relying on the correlation between the two problems being similar rather than using fundamental logical steps.
3. Implementing new digital infrastructure is a lengthy and challenging process.
Particularly in healthcare , there are extremely laborious regulatory assessment barriers before technology is approved that often take years, partly from the risk involved in introducing technologies that influence critical decisions. There is plentiful initial low-hanging fruit that would increase the efficiency of clinical practice, such as an LLM that that summarizes the body of a report to create a conclusion for the ordering provider, registration software that can link regions across prior images to compare changes (a notoriously difficult task that has not been addressed), or a database crawler that can retrieve patient's medical history to provide context for imaging findings. These tasks do not require complex logic but can take up hours of clinician’s time each week and are far simpler than reading medical images, but have not been widely implemented. In BC and most of Canada, we still lack the capability to automatically retrieve patient data and relevant images from across institutions, requiring several minutes of manual work for each patient case.
4. There are ethical challenges with bias and inequity.
From classical statistics, one of the strongest tools to address bias is by using diverse and plentiful data. A new challenge with modern AI models is they can identify subtle patterns in data, including unintended biases, that humans may have not been looking for. A 2022 study showed that AI was able to identify race, even when it was not trained to do so. In other words, AI can discriminate in hidden ways we aren't able to immediately detect, which opens concerns for inequitable treatment of patients especially if AI is unsupervised. Concerningly, generative AI has no restrictions on what tasks the user commands (exponentially requiring more data to train) but have been tuned to produce linguistically compelling outputs which may not necessarily be accurate. This is ripe for confirmation bias, particularly when generative AI hallucinates false references and provides cursory logic which may be persuasive to a non-technical expert (or even a physician from another specialty). There are countless anecdotes of an AI providing validation or a correct diagnosis that a physician missed in retrospect, but this does not mean AI outperforms physicians. There are far more scenarios where an AI provides an incorrect diagnosis that would cause adverse outcomes. This stresses the importance of statistical best practices, multiple trials to test with rigorous analysis of errors such as false positives/negatives, and that conclusions should not be made from anecdotal evidence, pilot studies, or trials without diverse data.
Conclusion
If an AI tool's scope is not physician-replacement, but rather physician-aid, then a human can act as a checkpoint to use discretion powered by clinical knowledge. These tools are rapidly being introduced, from AI scribes which assist in notetaking to AI risk stratification systems that intake lab values to predict outcomes, such as the Kidney Failure Risk Equation. The common link between current successful AI tools is that they are firstly narrow in scope and focus on one task, reducing confounding variables from trying to generalize requiring additional data and focusing on one task. Secondly, the successful AI tools are paired with human quality control, as when clinical scenarios are not straightforward, a human can use discretion to disregard the correlative conclusion from AI and utilize foundational knowledge to provide sensible clinical decision making.
In conclusion, if used within the appropriate scope, there is certainly promise for AI to improve patient care. Medical imaging volumes have been increasing in Canada, and addressing availability of tests is paramount for timely patient care. It certainly would be beneficial to start with AI that can perform laborious simple tasks to increase efficiency such as proofreading, summarizing reports, and even screening for abnormal regions to flag for review.
But for now, to achieve correct decision making and safe patient care, this will ultimately require the logical reasoning of a human brain.