EMBRACING HUMAN-AI DISAGREEMENT IN MEDICINE
Giorgia Pozzi, TU Delft – Postdoctoral researcher Ethics & Epistemology of AI
Karin Jongsma, UMC Utrecht – Associate Professor Bioethics
(Based on the Medical Humanities, Uehiro Oxford Institute, and OxStai roundtable When the Doctor and the AI Disagree: Beyond Opposition , 30 May 2025, with
Dominic Wilkinson, University of Oxford – Professor of Medical Ethics
Bartek Papiez , University of Oxford – Associate Professor of Medical Image Analysis and Machine Learning)
In a modern hospital, a complex case arises. A 68-year-old patient presents with symptoms that could indicate either a severe cardiac condition or a less urgent gastrointestinal issue. The attending physician, drawing on years of clinical experience and nuanced contextual judgment, suspects the latter and recommends outpatient follow-up. However, the hospital’s AI diagnostic support system flags a high probability of a cardiac event and advises immediate admission and further cardiac testing.
This disagreement between the human clinician and the AI system poses a new kind of challenge—one that will become increasingly common as artificial intelligence becomes more embedded in clinical workflows. While disagreements among human professionals have long been part of medical practice, the growing role of AI raises pressing philosophical and ethical questions about how such conflicts should be interpreted and dealt with.
In the academic literature about human-AI disagreement is often assumed that disagreement is a problem that need to be overcome. The existing approaches, while directing in opposing directions of how such disagreements should be resolved, have in common that they search for ways to resolve disagreements by determining who has epistemic authority: the idea is that either the AI systems or the human physician is better positioned to determine what should be done based on their knowledge, training, or experience. So understood, one of both is supposed to be necessarily right, and the other wrong. While these approaches might lead to faster decision-making, they risk providing a binary and overly simplified picture of medical practice that cannot account for the entanglement of different factors leading to disagreement.
The assumption that disagreement in human-AI decision-making can be settled through epistemic authority thus deserves further scrutiny. Not all disagreements are problematic, nor can or should they be resolved by referring to accurate or superior knowledge. Furthermore, medical decisions are rarely about facts alone; they often involve values, uncertainties, and competing interpretations of risk, benefit, and responsibility. Hence, accuracy is only one relevant aspect among many others that need to be properly contextualized.
This raises several open questions. First, what criteria should determine which disagreements need to be resolved? Some disagreements may be harmless or even productive, spurring reflection or innovation. Others, especially those involving patient treatment, will demand careful consideration to avoid harm. But how do we distinguish between them?
Second, how should we account for the complexity of medical decision-making itself? Decisions often involve multiple aspects: scientific data, clinical judgment, ethical principles, patient values, and systemic constraints. A decision about whether to admit a patient to intensive care, for example, might involve not just prognosis and consent, but also bed availability, patient expectations, and relevant protocols. Any framework for resolving human-AI disagreement must be sensitive to this multifaceted reality.
Third, how should disagreements be practically translated? What about the patient’s role in this triad—should they be informed of the disagreement, and how should their preferences be factored into further decision-making? Expert agreement largely seems to be desirable because it is likely to augment patient trust. However, medical decision-making is unavoidably uncertain, thus it seems more suitable to constructively deal with emerging disagreements rather than trying to find one-size-fits-all solutions to be applied across the board. These might not do justice to the specificities of a patient’s situation and their right to actively shape medical discourse.
We consider human-AI disagreement in medicine not merely a procedural or practical obstacle—but as a window into the deep epistemic and moral structures of healthcare. Disagreement between human and AI systems challenges long-standing assumptions about expertise, authority, and trust. It invites a re-examination of how decisions should be made under uncertainty and in the light of assumptions about the limits of human clinicians and AI systems.
Find out more about the Medical Humanities Research Hub.
