The use of decision support tools and automation in medical practice is not new. Early tools employed simple regression models to reduce heterogeneity and offer more structured care. In cardiology, initial examples of decision support tools included the Framingham risk scores for primary prevention risk stratification and automated sliding scale dose adjustments for heparin or warfarin. Since then, automation processes have become more sophisticated and are now widespread in many routine aspects of medical practice. In cardiology, these include automated preliminary interpretations of electrocardiography (ECG) tracings as well as automated measurements and image analysis in echocardiography, nuclear medicine, and cardiac magnetic resonance imaging. Other automated clinical decision support systems help reduce prescribing errors by alerting clinicians to drug-drug interactions or incorrect dosages, for example. With recent advances in artificial intelligence (AI), expectations for the future of such tools have grown dramatically. Yet, the potential negative implications of these technologies have received considerably less attention.

Clinical decision support systems can improve medical decision-making and patient outcomes. However, these systems are not flawless and may produce incorrect outputs. Most studies assessing their performance rely on conventional medical metrics such as sensitivity, specificity, accuracy, and area under the receiver operating characteristic curve. Yet, it is far more important to evaluate the real-world impact of these tools, particularly in light of how clinicians interact with them. This is especially relevant given that the European Union Artificial Intelligence Act mandates human oversight for high-risk AI systems. The issue becomes even more critical with the widespread use of generative AI chatbots, as the current generation of complex AI tools has led to a growing trend of overreliance on imperfect automation.

[…]

Daniel Kahneman reinforced and popularized the idea of two distinct modes of thinking. System 1 relies on pattern recognition, detecting simple relationships, correlating stereotypes, and associating ideas in a search for coherence. It is fast, intuitive, automatic, and requires little or no effort or energy. It cannot be turned off and continuously monitors the environment, driven by a survival need rooted in our ancestral past. System 2, on the other hand, demands attention, compares objects based on various attributes, and makes deliberate choices among options. It evaluates scenarios using logical and statistical reasoning. Because it requires attention, its activity is interrupted when that attention is diverted. Therefore, tasks that require effort interfere with one another (for instance, making a turn while driving and performing a complex calculation simultaneously is impossible). However, System 2 is slow, deliberate, effortful, and orderly. The two systems are interconnected: System 1 effortlessly generates impressions and feelings that serve as the main source for the explicit beliefs and deliberate choices made by System 2. In daily life, we often make decisions based on heuristics—mental shortcuts closely linked to System 1—used to save time and energy. While extremely efficient, this faster way of thinking opens the door to cognitive biases, which are systematic tendencies that distort logical reasoning and lead to flawed decisions.

The causes of error in diagnostic medicine can be categorized into two main groups: perceptual errors and interpretive errors. Perceptual errors occur when an abnormality is present on the image but goes undetected. These may be related to image acquisition, post-processing, or equipment quality. Interpretive errors arise when the abnormality is seen but its significance is misunderstood. In such cases, cognitive biases may be involved and are estimated to account for up to 74% of diagnostic errors.^– Approximately 30 types of cognitive biases have been described in literature. In diagnostic imaging, the most relevant include anchoring, confirmation bias, availability bias, attribution bias, framing effect, and satisfaction of search.^-,

[…]