Search

In this paper, we study human–AI collaboration protocols, a design-oriented construct aimed at establishing and evaluating how humans and AI can collaborate in cognitive tasks. We applied this construct in two user studies involving 12 specialist radiologists (the knee MRI study) and 44 ECG readers of varying expertise (the ECG study), who evaluated 240 and 20 cases, respectively, in different collaboration configurations. We confirm the utility of AI support but find that XAI can be associated with a “white-box paradox”, producing a null or detrimental effect. We also find that the order of presentation matters: AI-first protocols are associated with higher diagnostic accuracy than human-first protocols, and with higher accuracy than both humans and AI alone. Our findings identify the best conditions for AI to augment human diagnostic skills, rather than trigger dysfunctional responses and cognitive biases that can undermine decision effectiveness.

Patient-reported outcome Measures (PROMs) are validated questionnaires or self-report instruments of the perception of patients about their own health status in response to a medical intervention. In the era of patient-centred care, consideration about the effectiveness of a medical treatment should be grounded not only on the physicians' assessment but also on the increase in PROM scores that is perceived as relevant by the patients, namely the minimum clinically important differences (MCID). In this study, we collected data from the IRCCS Galeazzi Orthopaedic Institute (IOG) of Milan, Italy, to develop a preoperative machine learning model predicting the non-achievement of the MCID threshold in specific PROM scores 6 months after spine surgery, namely the Oswestry Disability Index (ODI) and the Physical Score of the Short Form 36 (SF-36). At IOG, nearly 39% of spinal surgeries do not achieve a minimum clinically important improvement and, of these, 22% are associated with negative outcomes. This is mainly due to the fact that IOG is a tertiary healthcare facility that receives the most critical cases from a vast territory (practically from all over the country) and it is known that spinal deformities and other related problems are difficult to solve, especially in an aging population. In this view, it is important to early identify those patients who will likely not benefit from treatment and, therefore, will not reach the MCID. This would help avoid overdiagnosis, reduce overuse and the related costs for unnecessary treatment, as well as optimise the allocation of resources and support more appropriate choices in shared decision-making.

In this paper, we contribute to the deconstruction of the concept of accuracy with respect to machine learning systems that are used in human decision making, and specifically in medicine. We argue that, by taking a socio-technical stance, it is necessary to move from the idea that these systems are “agents that can err”, to the idea that these are just tools by which humans can interpret new cases in light of the technologically-mediated interpretation of past cases, like if they were wearing a pair of tinted glasses. In this new narrative, accuracy is a meaningless construct, while it is important that beholders can “believe in their eyes” (or spectacles), and therefore trust the tool enough to make sensible decisions.

Background: In Italy, one of the country most affected by the COVID-19 pandemic, the first autoch-thonous case appeared in Lombardy on February 20th, 2020. One month later, the number of COVID-19 patients in Lombardy exceeded 17000 and about 3500 had died. Because of this rapid increase in infected people scientists wonder whether SARS-CoV-2 was already highly circulating in Lombardy before such date. Plasma levels of aspartate aminotransferase (AST) and lactate dehydrogenase (LDH) were shown to be highly increased in COVID-19 patients. Monitoring their levels in Emergency Room patients during the months preceding February 20th, 2020, might shade light on the prevalence of the disease in the pre-COVID-19 period. Methods: We retrospectively analyzed the AST and LDH levels from more than 30.000 patients admitted to the San Raffaele Hospital Emergency Room (ER) between September 2019 and May 2020 as well as between September 2018 and May 2019. The number of patients diagnosed with respiratory tract diseases were also analyzed. Results: Data showed that the ER averaged AST and LDH levels are highly sensitive to the presence of COVID-19 patients. During, the months preceding February 20th, 2020, AST and LDH levels, as well as the number of patients diagnosed with respiratory tract diseases were similar to their 2019 counterparts. Conclusions: No significant evidence showing that a large number of COVID-19 patients were admitted to the San Raffaele Hospital ER before February 20th, 2020, was found. Thus, the virus was likely circulating, within the Hospital catchment area, either in low amounts or through asymptomatic individuals. Because of the high LDH and AST levels’ variations induced by COVID-19, routine blood tests might be exploited as a surveillance indicator for a possible second wave.

In this paper we propose that Artificially Intelligent (AI) systems in medicine are the new drugs. Through study and comparison of the existing corporate pharmaceutical models, we advocate a new approach in how the performance of AI solutions (both for diagnostic and prognostic aims) is measured and promoted, on how the impact of these solutions is assessed and evaluated in terms of both outcome and safety (about which the new term technovigilance has been introduced), and how education and a better awareness of the impact of these systems on heath care by multiple stakeholders should be promoted. Any drug must be evaluated in terms of efficacy and safety, but also in terms of real-world effectiveness, efficiency and cost-effectiveness, that is in terms of its role as a component of a wider and more complex socio-technical system where it must fit a broader process of inquiry and care, which involves patients, care givers and many other stakeholders. For this reason, we also propose, as a priority, changes in the current wide-spread attitudes that evaluate machine performance against humans, and instead to adopt a collective knowledge approach. In this light, the leading metaphor informing the public debate on the role of AI in medicine should not be that of cognitive augmentation, with its risks of feeding unrealistic expectations and restoring the harmful image of the medical hero, but rather of a collaboration-oriented adjunct in more redundant and tighter teams of competent decision makers. Not only could this lead to a more efficient and automated health care, but we argue to a better collective endeavour aimed at the well-being of patients as well as at the satisfaction of physicians.

Surgical skills training plays an important role in medical education. In recent years, substantial progress has been made in the development of simulation programs and tools for the training and assessment of a trainee’s performance. However, these devices have generated controversy about their validity for arthroscopic surgical training, and the bridge between technological development and educational needs has not yet been clearly established. From an educational point of view, a key feature for a well-designed training program is that the learning objectives should be explicitly defined (Biggs 2003). The aim of this chapter is to address the learning objectives in simulation training, and subsequently, we focus on the number of procedures that are required to become competent.

We present a model which allows to define in an uniform way information quality dimensions related to heterogeneous types of information, such as structured data managed in data bases, semi-structured and unstructured texts and images. We first define a set of concepts that allow to represent several basic characteristics of such heterogeneous types of information. Then, we introduce a general categorization of quality dimensions and sul>-(liinensions, which are then specialized to structured data, semi- and unstructured texts and images. In so doing, we provide, to our knowledge, a first attempt to unify the information quality issue for heterogeneous information types.

Made available in DSpace on 2022-04-28T19:54:48Z (GMT). No. of bitstreams: 0 Previous issue date: 1998-01-01 The fate of three pesticides (vinclozolin, dimethoate, cyproconazole) in plums, from field treatment to the drying process, was studied. Only vinclozolin showed measurable residue concentrations at harvest, while dimethoate was completely degraded after two weeks and cyproconazole was present at negligible levels just after treatment. During the drying process into prunes, the residues were not reduced during the fruit washing stage, but the drying stage led to complete elimination of vinclozolin residues. Dipartimento di Tossicologia Università di Cagliari, viale Diaz 182, 09126 Cagliari Depto. de Quim. Orgânica UNESP, Araraquara, SP Ctro. Regionale Agrario Sperimentale, viale Trieste 111, Cagliari Depto. de Quim. Orgânica UNESP, Araraquara, SP

BackgroundThe COVID-19 pandemia due to the SARS-CoV-2 coronavirus, in its first 4 months since its outbreak, has to date reached more than 200 countries worldwide with more than 2 million confirmed cases (probably a much higher number of infected), and almost 200,000 deaths. Amplification of viral RNA by (real time) reverse transcription polymerase chain reaction (rRT-PCR) is the current gold standard test for confirmation of infection, although it presents known shortcomings: long turnaround times (3-4 hours to generate results), potential shortage of reagents, false-negative rates as large as 15-20%, the need for certified laboratories, expensive equipment and trained personnel. Thus there is a need for alternative, faster, less expensive and more accessible tests.Material and methodsWe developed two machine learning classification models using hematochemical values from routine blood exams (namely: white blood cells counts, and the platelets, CRP, AST, ALT, GGT, ALP, LDH plasma levels) drawn from 279 patients who, after being admitted to the San Raffaele Hospital (Milan, Italy) emergency-room with COVID-19 symptoms, were screened with the rRT-PCR test performed on respiratory tract specimens. Of these patients, 177 resulted positive, whereas 102 received a negative response.ResultsWe have developed two machine learning models, to discriminate between patients who are either positive or negative to the SARS-CoV-2: their accuracy ranges between 82% and 86%, and sensitivity between 92% e 95%, so comparably well with respect to the gold standard. We also developed an interpretable Decision Tree model as a simple decision aid for clinician interpreting blood tests (even off-line) for COVID-19 suspect cases.DiscussionThis study demonstrated the feasibility and clinical soundness of using blood tests analysis and machine learning as an alternative to rRT-PCR for identifying COVID-19 positive patients. This is especially useful in those countries, like developing ones, suffering from shortages of rRT-PCR reagents and specialized laboratories. We made available a Web-based tool for clinical reference and evaluation1.

Background Healthcare-associated infections (HAIs) are the most frequent adverse events in healthcare and a global public health concern. Surveillance is the foundation for effective HAIs prevention and control. Manual surveillance is labor intensive, costly and lacks standardization. Artificial Intelligence (AI) and machine learning (ML) might support the development of HAI surveillance algorithms aimed at understanding HAIs risk factors, improve patient risk stratification, identification of transmission pathways, timely or real-time detection. Scant evidence is available on AI and ML implementation in the field of HAIs and no clear patterns emerges on its impact. Methods We conducted a systematic review following the PRISMA guidelines to systematically retrieve, quantitatively pool and critically appraise the available evidence on the development, implementation, performance and impact of ML-based HAIs detection models. Results Of 3445 identified citations, 27 studies were included in the review, the majority published in the US (n = 15, 55.6%) and on surgical site infections (SSI, n = 8, 29.6%). Only 1 randomized controlled trial was included. Within included studies, 17 (63%) ML approaches were classified as predictive and 10 (37%) as retrospective. Most of the studies compared ML algorithms’ performance with non-ML logistic regression statistical algorithms, 18.5% compared different ML models’ performance, 11.1% assessed ML algorithms’ performance in comparison with clinical diagnosis scores, 11.1% with standard or automated surveillance models. Overall, there is moderate evidence that ML-based models perform equal or better as compared to non-ML approaches and that they reach relatively high-performance standards. However, heterogeneity amongst the studies is very high and did not dissipate significantly in subgroup analyses, by type of infection or type of outcome. Discussion Available evidence mainly focuses on the development and testing of HAIs detection and prediction models, while their adoption and impact for research, healthcare quality improvement, or national surveillance purposes is still far from being explored.

Objective of the work is the development of prognostic machine learning models that predict qualitative and quantitative measures of postnatal growth in very low birth weight preterm infants. Observational retrospective data about 964 infants at risk are retrieved from “Fondazione Monza e Brianza per il bambino e la mamma“’s electronic medical record. Both prenatal (gestational, socioeconomic, etc.) and perinatal (nutritional, respiratory assistance, drug prescription and daily growth) data up to a week after birth are the features included. Model’s performances are compared to previous literature and human performance, showing a substantial improvement (in e.g., best regression MAE=0.49, best classification AUC=0.94).

We undertook a user study to evaluate whether the perceived utility of some common open data sets for family doctors would increase if they are rendered in interactive heat maps. We also investigated whether Parallel Coordinates (PC) are perceived as a convenient diagram to summarize multiple patients data; and whether making PCs interactive would increase their informativity. We interviewed 29 expert family doctors through a questionnaire to find out that: interactive maps make health datasets be perceived as more useful, especially in regard to registers on exposure to carcinogens. PCs were found to be informative visualizations but making them interactive increases their perceived informativity. In light of this evidence, designing for better interactivity is worthy of further efforts in Human-Data Interaction research.

Total hip (THA) and total knee (TKA) arthroplasty procedures have steadily increased over the past few decades, and their use is expected to grow further, mainly due to an increasing number of elderly patients. Cost-containment strategies, supporting a rapid recovery with a positive functional outcomes, high patient satisfaction, and enhanced patient reported outcomes, are needed. A Fast Track surgical procedure (FT) is a coordinated perioperative approach aimed at expediting early mobilization and recovery following surgery and, accordingly, shortening the length of hospital stay (LOS), convalescence and costs. In this view, rapid rehabilitation surgery optimizes traditional rehabilitation methods by integrating evidence-based practices into the procedure. The aim of the present study was to compare the effectiveness of Fast Track versus Care-as-Usual surgical procedures and pathways (including rehabilitation) on a mid-term patient-reported outcome (PROs), the SF12 (with regard both to Physical and Mental Scores), 3 months after hip or knee replacement surgery, with the use of Propensity score-matching (PSM) analysis to address the issue of the comparability of the groups in a non-randomized study. We were interested in the evaluation of the entire pathways, including the postoperative rehabilitation stage, therefore, we only used early home discharge as a surrogate to differentiate between the Fast Track and Care-as-Usual rehabilitation pathways. Our study shows that the entire Fast Track pathway, which includes the post-operative rehabilitation stage, has a significantly positive impact on physical health-related status (SF12 Physical Scores), as perceived by patients 3 months after hip or knee replacement surgery, as opposed to the standardized program, both in terms of the PROs score and the relative improvements observed, as compared with the minimum clinically important difference. This result encourages additional research into the effects of Fast Track rehabilitation on the entire process of care for patients undergoing hip or knee arthroplasty, focusing only on patient-reported outcomes.

In this paper, we present a fundamental framework for defining different types of explanations of AI systems and the criteria for evaluating their quality. Starting from a structural view of how explanations can be constructed, i.e., in terms of an explanandum (what needs to be explained), multiple explanantia (explanations, clues, or parts of information that explain), and a relationship linking explanandum and explanantia, we propose an explanandum-based typology and point to other possible typologies based on how explanantia are presented and how they relate to explanandia. We also highlight two broad and complementary perspectives for defining possible quality criteria for assessing explainability: epistemological and psychological (cognitive). These definition attempts aim to support the three main functions that we believe should attract the interest and further research of XAI scholars: clear inventories, clear verification criteria, and clear validation methods.

The rise of personalized medicine and its remarkable advancements have revealed new requirements for the availability of appropriate medical decision-making models. Computer science is an area that plays an essential role in the field of personalized medicine, where one of the goals is to provide algorithms and tools to extrapolate knowledge and improve the decision-support process. The minimum clinically important difference (MCID) is the smallest change in PROM scores that patients perceive as meaningful. Treatment that does not achieve the minimum level of improvement is considered inappropriate as well as a potential waste of resources. Using the MCID threshold to identify patients who fail to achieve the minimum change in PROM that results in a meaningful outcome may aid in pre-surgical shared decision-making. The decision tree algorithm is a method for extracting valuable information and providing further meaningful information to the domain expert that supports the decision-making. In the present study, different tools based on machine learning were developed. On the one hand, we compared three XGBoost models to predict the non-achievement of the MCID at six months post-operation in the SF-12 physical score. The prediction score threshold was set to 0.75 to provide three decision-making areas on the basis of the high confidence (HC) intervals; the minority class was re-balanced by weighting the positive class to penalize the loss function (XGBoost cost-sensitive), oversampling the minority class (XGBoost with SMOTE), and re-sampling the negative class (XGBoost with undersampling). On the other hand, we modeled the data through a decision tree (assessment tree), based on different complexity levels, to identify the hidden pattern and to provide a new way to understand possible relationships between the gathered features and the several outcomes. The results showed that all the proposed models were effective as binary classifiers, as they showed moderate predictive performance both regarding the minority or positive class (i.e., our targeted patients, those who will not benefit from surgery) and the negative class. The decision tree visualization can be exploited during the patient assessment status to better understand if those patients will benefit or not from the medical intervention. Both of these tools can come in handy for increasing knowledge about the patient’s psychophysical state and for creating an increasingly specialized assessment of the individual patient.

In this paper, we consider some key characteristics that relational AI should exhibit to enable decision hybrid agencies that include subject-matter experts and their AI-enabled decision aids, especially when these latter ones have been developed by following a machine learning approach. We will hint at the design requirements of guaranteeing that AI tools are: open, multiple, continuous, cautious, vague, analogical and, most importantly, adjunct with respect to decision making practices. We will argue that especially adjunction is an important condition to design for. Adjunction entails the design and evaluation of human-AI interaction protocols aimed at improving AI usability, that is decision effectiveness and efficiency, while also guaranteeing user satisfaction and human and social sustainability, as well as mitigating the risk of automation bias, technology over-reliance and user deskilling. These high-level aims are compatible with the tenets of a relational approach to the design of AI tools to support decision making and collaborative practices. This is the extended version of the article published in the proceedings of the first conference on Hybrid Human Artificial Intelligence (HHAI'22) Amsterdam, the Netherlands, 13-17 June, 2022. To cite this article, please use the following format: Cabitza, F., & Natali, C. (2022). Open, multiple, adjunct. Decision support at the time of relational AI. In HHAI2022: Augmenting Human Intellect (pp. 243-245). IOS Press. https://doi.org/10.3233/FAIA220204, {"references":["De Michelis G. Aperto, molteplice, continuo: gli artefatti alla fine del Novecento. Zanichelli, Milano; 1998.","Zenisek J, Holzinger F, Affenzeller M. Machine learning based concept drift detection for predictive maintenance. Computers & Industrial Engineering. 2019;137:106031.","Lu D, Tao C, Chen J, Li F, Guo F, Carin L. Reconsidering generative objectives for counterfactual reasoning. Advances in Neural Information Processing Systems. 2020;33:21539-53.","Goddard K, Roudsari A, Wyatt JC. Automation bias: a systematic review of frequency, effect mediators, and mitigators. Journal of the American Medical Informatics Association. 2012;19(1):121-7.","Gabora L. Reframing convergent and divergent thought for the 21st century. arXiv preprint arXiv:181104512. 2018.","Wu J, Quian X, Wang MY. Advances in generative design. Computer-Aided Design. 2019;116:102733","Chac ́on JC, Nimi HM, Kloss B, Kenta O. Towards the Development of AI Based Generative Design Tools and Applications. In: International Conference on Design, Learning, and Innovation. Springer; 2020. p. 63-73.","Tschandl P, Rinner C, Apalla Z, Argenziano G, Codella N, Halpern A, et al. Human–computer collabo- ration for skin cancer recognition. Nature Medicine. 2020;26(8):1229-34.","Rundo L, Pirrone R, Vitabile S, Sala E, Gambino O. Recent advances of HCI in decision-making tasks for optimized clinical workflows and precision medicine. Journal of biomedical informatics. 2020;108:103479.","Verma S, Dickerson J, Hines K. Counterfactual explanations for machine learning: A review. arXiv preprint arXiv:201010596. 2020.","Campagner A, Cabitza F, Ciucci D. Three–way classification: Ambiguity and abstention in machine learning. In: International Joint Conference on Rough Sets. Springer; 2019. p. 280-94.","Vovk V, Gammerman A, Shafer G. Algorithmic learning in a random world. Springer Science & Business Media; 2005.","Campagner A, Cabitza F, Berjano P, Ciucci D. Three-way decision and conformal prediction: Isomor- phisms, differences and theoretical properties of cautious learning approaches. Information Sciences. 2021;579:347-67.","Keane M. Analogical mechanisms. Artificial Intelligence Review. 1988;2(4):229-51.","Baselli G, Codari M, Sardanelli F. Opening the black box of machine learning in radiology: can the proximity of annotated cases be a way? European Radiology Experimental. 2020;4(1):1-7.","Cornelissen NAJ, van Eerdt RJM, Schraffenberger HK, Haselager WFG. Reflection machines: increas- ing meaningful human control over Decision Support Systems. Ethics and Information Technology. 2022;19(24).","Cabitza F, Campagner A, Sconfienza LM. Studying human-AI collaboration protocols: the case of the Kasparov's law in radiological double reading. Health Information Science and Systems. 2021;9(1):1- 20.","Kasparov G, Greengard M. Deep Thinking: Where Machine Intelligence Ends and Human Creativity Begins. Millennium Series. John Murray Press; 2017. Available from: https://books.google.it/ books?id=ffYZDQAAQBAJ.","Cabitza F, Campagner A, Simone C. The need to move away from agential-AI: Empirical investigations, useful concepts and open issues. International Journal of Human-Computer Studies. 2021;155:102696.","Tsamados A, Aggarwal N, Cowls J, Morley J, Roberts H, Taddeo M, et al. The ethics of algorithms: key problems and solutions. AI & SOCIETY. 2021:1-16.","Grote T, Berens P. On the ethics of algorithmic decision-making in healthcare. Journal of medical ethics. 2020;46(3):205-11.","Cabitza F, Rasoini R, Gensini GF. Unintended consequences of machine learning in medicine. Jama. 2017;318(6):517-8.","Bond RR, Novotny T, Andrsova I, Koc L, Sisakova M, Finlay D, et al. Automation bias in medicine: The influence of automated diagnoses on interpreter accuracy and uncertainty when reading electrocar- diograms. Journal of electrocardiology. 2018;51(6):S6-S11.","Spitzer M. Outsourcing the mental? From knowledge-on-demand to Morbus Google. Trends in Neuro- science and Education. 2016;5(1):34-9.","Carr NG. The Shallows: What the Internet is Doing to Our Brains. W.W. Norton; 2010. Available from: https://books.google.it/books?id=9-8jnjgYrgYC.","Farindon P. Cabitza, Federico. In: Pelillo M, Scantamburlo T, editors. Cobra AI: Exploring Some Unintended Consequences. MIT Press; 2021. p. 87-104.","Malone TW. Superminds: The Surprising Power of People and Computers Thinking Together. Little Brown; 2018. Available from: https://books.google.it/books?id=Qe0zDwAAQBAJ","Pierce J. Undesigning interaction. Interactions. 2014;21(4):36-9.","Cabitza F. Biases affecting human decision making in AI-supported second opinion settings. In: Inter- national Conference on Modeling Decisions for Artificial Intelligence. Springer; 2019. p. 283-94.","Boni M. The ethical dimension of human–artificial intelligence collaboration. European View. 2021;20(2):182-90.","Taddeo M, Floridi L. How AI can be a force for good. Science. 2018;361(6404):751-2.","Floridi L. Establishing the rules for building trustworthy AI. Nature Machine Intelligence. 2019;1(6):261-2.","Methnani L, Aler Tubella A, Dignum V, Theodorou A. Let Me Take Over: Variable Autonomy for Meaningful Human Control. Frontiers in Artificial Intelligence. 2021;4. Available from: https:// www.frontiersin.org/article/10.3389/frai.2021.737072.","The Council of Europe's Ad-Hoc Committee on AI /CAHAI). Towards Regulation of AI Systems. Council of Europe; 2020.","High Level Expert Grouèp on Artificial Intelligence (HLEG AI). thics Guidelines for Trustworthy Artificial Intelligence. European Commission; 2019.","Shahriari K, Shahriari M. IEEE standard review—Ethically aligned design: A vision for prioritizing hu- man wellbeing with artificial intelligence and autonomous systems. In: 2017 IEEE Canada International Humanitarian Technology Conference (IHTC). IEEE; 2017. p. 197-201","European Commission, Centre JR, Nativi S, De Nigris S. AI Watch, AI standardisation landscape state of play and link to the EC proposal for an AI regulatory framework. Publications Office; 2021","Smuha NA. The EU approach to ethics guidelines for trustworthy artificial intelligence. Computer Law Review International. 2019;20(4):97-106.","Meza Mart ́ınez MA, Nadj M, Maedche A. Towards an integrative theoretical framework of interactive machine learning systems. 2019.","Wiethof C, Bittner EA. Hybrid Intelligence–Combining the Human in the Loop with the Computer in the Loop: A Systematic Literature Review. 2021.","Xu W, Dainoff MJ, Ge L, Gao Z. From Human-Computer Interaction to Human-AI Interaction: New Challenges and Opportunities for Enabling Human-Centered AI. ArXiv. 2021;abs/2105.05424.","Holzinger A. Interactive machine learning for health informatics: when do we need the human-in-the- loop? Brain Informatics. 2016;3(2):119-31","Parasuraman R, Manzey DH. Complacency and bias in human use of automation: An attentional inte- gration. Human factors. 2010;52(3):381-410.","Shneiderman B. Human-centered artificial intelligence: three fresh ideas. AIS Transactions on Human- Computer Interaction. 2020;12(3):109-24.","Skitka LJ, Mosier K, Burdick MD. Accountability and automation bias. International Journal of Human- Computer Studies. 2000;52(4):701-17.","Sloman S, Fernbach P. The Knowledge Illusion: Why We Never Think Alone. Penguin; 2017.","Frischmann B, Selinger E. Re-engineering humanity. Cambridge University Press; 2018.","Scalera L, Gallina P, Gasparetto A, Seriani S. Anti-Hedonistic Machines. Int J Mech Control. 2017;18:9- 16.","Cabitza F, Campagner A, Ciucci D, Seveso A. Programmed inefficiencies in DSS-supported human de- cision making. In: International Conference on Modeling Decisions for Artificial Intelligence. Springer; 2019. p. 201-12.","Ohm P, Frankle J. Desirable inefficiency. Fla L Rev. 2018;70:777.","Tenner E. The Efficiency Paradox: What Big Data Can't Do. Knopf Doubleday Publishing Group; 2018. Available from: https://books.google.it/books?id=PgAtDwAAQBAJ.","Hildebrandt M. Privacy as protection of the incomputable self: From agnostic to agonistic machine learning. Theoretical Inquiries in Law. 2019;20(1):83-121.","Sadin ́E. L'intelligence artificielle, ou, L'enjeu du si`ecle: anatomie d'un antihumanisme radical. Collec- tion Pour en finir avec. L' ́Echapp ́ee; 2018. Available from: https://books.google.it/books?id= yJ1uvQEACAAJ.","Muller JZ. The Tyranny of Metrics. Princeton University Press; 2018. Available from: https:// books.google.it/books?id=J3GYDwAAQBAJ.","Crawford K. The Atlas of AI: Power, Politics, and the Planetary Costs of Artificial Intelligence. Yale University Press; 2021. Available from: https://books.google.it/books?id=KfodEAAAQBAJ.","Buc ̧inca Z, Malaya MB, Gajos KZ. To trust or to think: cognitive forcing functions can reduce overre- liance on AI in AI-assisted decision-making. Proceedings of the ACM on Human-Computer Interaction. 2021;5(CSCW1):1-21.","Christakis NA. Blueprint: The Evolutionary Origins of a Good Society. Little, Brown Spark; 2019.","Hildebrandt M. Algorithmic regulation and the rule of law. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences. 2018;376(2128):20170355.","Park JS, Barber R, Kirlik A, Karahalios K. A Slow Algorithm Improves Users' Assessments of the Algorithm's Accuracy. Proceedings of the ACM on Human-Computer Interaction. 2019;3(CSCW):1-15.","Holzinger AT, Muller H. Toward Human–AI Interfaces to Support Explainability and Causability in Medical AI. Computer. 2021;54(10):78-86.","Kaur S, Sharma R. Emotion AI: Integrating Emotional Intelligence with Artificial Intelligence in the Digital Workplace. In: Singh PK, Polkowski Z, Tanwar S, Pandey SK, Matei G, Pirvu D, editors. In- novations in Information and Communication Technologies (IICT-2020). Cham: Springer International Publishing; 2021. p. 337-43.","Jankuloski F, Bozinovski A, Pacovski V. Artificial Intelligence: Simulating Human Emotion and Sur- passing Human Intelligence. 2020.","Montemayor C, Halpern J, Fairweather A. In principle obstacles for empathic AI: why we can't replace human empathy in healthcare. Ai & Society. 2021:1-7.","Nass C, Steuer J, Tauber E, Reeder H. Anthropomorphism, agency, and ethopoeia: computers as social actors. In: INTERACT'93 and CHI'93 conference companion on Human factors in computing systems; 1993. p. 111-2.","Hamacher A, Bianchi-Berthouze N, Pipe AG, Eder K. Believing in BERT: Using expressive communi- cation to enhance trust and counteract operational error in physical Human-robot interaction. In: 2016 25th IEEE international symposium on robot and human interactive communication (RO-MAN). IEEE; 2016. p. 493-500.","Shneiderman B. Human-centered AI. Issues in Science and Technology. 2021;37(2):56-61.","Hildebrandt M. New animism in policing: re-animating the rule of law. The SAGE handbook of global policing. 2016:406-28.","Atkinson C, Brooks L. In the Age of the Humanchine. ICIS 2005 Proceedings. 2005:11.","IEEE. Ethically Aligned Design (Version 1). The IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems. 2016.","Turing AM. Computing Machinery and Intelligence. Mind. 1950;59(October):433-60.","Ghassemi M, Oakden-Rayner L, Beam AL. The false hope of current approaches to explainable artificial intelligence in health care. The Lancet Digital Health. 2021;3(11):e745-50.","Adadi A, Berrada M. Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI). IEEE Access. 2018;6:52138-60.","Cabitza F, Alderighi C, Rasoini R, Gensini GF. \"Handle with care\": about the potential unintended consequences of oracular artificial intelligence systems in medicine. Recenti progressi in medicina. 2017;108(10):397-401."]}

Although deep learning-based AI systems for diagnostic imaging tasks have virtually showed superhuman accuracy, their use in medical settings has been questioned due to their “black box”, not interpretable nature. To address this shortcoming, several methods have been proposed to make AI eXplainable (XAI), including Pixel Attribution Methods; however, it is still unclear whether these methods are actually effective in “opening” the black-box and improving diagnosis, particularly in tasks where pathological conditions are difficult to detect. In this study, we focus on the detection of thoraco-lumbar fractures from X-rays with the goal of assessing the impact of PAMs on diagnostic decision making by addressing two separate research questions: first, whether activation maps (as an instance of PAM) were perceived as useful in the aforementioned task; and, second, whether maps were also capable to reduce the diagnostic error rate. We show that, even though AMs were not considered significantly useful by physicians, the image readers found high value in the maps in relation to other perceptual dimensions (i.e., pertinency, coherence) and, most importantly, their accuracy significantly improved when given XAI support in a pilot study involving 7 doctors in the interpretation of a small, but carefully chosen, set of images.

Human Activity Recognition (HAR) has been studied extensively, yet current approaches are not capable of generalizing across different domains (i.e., subjects, devices, or datasets) with acceptable performance. This lack of generalization hinders the applicability of these models in real-world environments. As deep neural networks are becoming increasingly popular in recent work, there is a need for an explicit comparison between handcrafted and deep representations in Out-of-Distribution (OOD) settings. This paper compares both approaches in multiple domains using homogenized public datasets. First, we compare several metrics to validate three different OOD settings. In our main experiments, we then verify that even though deep learning initially outperforms models with handcrafted features, the situation is reversed as the distance from the training distribution increases. These findings support the hypothesis that handcrafted features may generalize better across specific domains.

The concept of calibration is key in the development and validation of Machine Learning models, especially in sensitive contexts such as the medical one. However, existing calibration metrics can be difficult to interpret and are affected by theoretical limitations. In this paper, we present a new metric, called GICI (Global Interpretable Calibration Index), which is characterized by being local and defined only in terms of simple geometrical primitives, which makes it both simpler to interpret, and more general than other commonly used metrics, as it can be used also in recalibration procedures. Also, compared to traditional metrics, the GICI allows for a more comprehensive evaluation, as it provides a three-level information: a bin-level local estimate, a global one, and an estimate of the extent confidence scores are either over- or under-confident with respect to actual error rate. We also report the results from experiments aimed at testing the above statements and giving insights about the practical utility of this metric also to improve discriminative accuracy.

In this article we propose a novel technique for the re-calibration of Machine Learning (ML) models. This technique is based on the computation of confidence intervals for the probability scores provided by any ML model. Compared to existing and commonly used calibration methods, the proposed approach has two important advantages: first, under weak assumptions it provides theoretical guarantees about calibration; second, this method does not require any further data other than the training set used for ML model development. We illustrate the effectiveness of the proposed approach on a benchmark dataset for COVID-19 diagnosis, by comparing the proposed method against commonly used re-calibration techniques.

Background and Objective Evaluation of AI-based decision support systems (AI-DSS) is of critical importance in practical applications, nonetheless common evaluation metrics fail to properly consider relevant and contextual information. In this article we discuss a novel utility metric, the weighted Utility (wU), for the evaluation of AI-DSS, which is based on the raters’ perceptions of their annotation hesitation and of the relevance of the training cases. Methods We discuss the relationship between the proposed metric and other previous proposals; and we describe the application of the proposed metric for both model evaluation and optimization, through three realistic case studies. Results We show that our metric generalizes the well-known Net Benefit, as well as other common error-based and utility-based metrics. Through the empirical studies, we show that our metric can provide a more flexible tool for the evaluation of AI models. We also show that, compared to other optimization metrics, model optimization based on the wU can provide significantly better performance (AUC 0.862 vs 0.895, p-value

Objectives The European Biological Variation Study (EuBIVAS), which includes 91 healthy volunteers from five European countries, estimated high-quality biological variation (BV) data for several measurands. Previous EuBIVAS papers reported no significant differences among laboratories/population; however, they were focused on specific set of measurands, without a comprehensive general look. The aim of this paper is to evaluate the homogeneity of EuBIVAS data considering multivariate information applying the Principal Component Analysis (PCA), a machine learning unsupervised algorithm. Methods The EuBIVAS data for 13 basic metabolic panel linked measurands (glucose, albumin, total protein, electrolytes, urea, total bilirubin, creatinine, phosphatase alkaline, aminotransferases), age, sex, menopause, body mass index (BMI), country, alcohol, smoking habits, and physical activity, have been used to generate three databases developed using the traditional univariate and the multivariate Elliptic Envelope approaches to detect outliers, and different missing-value imputations. Two matrix of data for each database, reporting both mean values, and “within-person BV” (CVP) values for any measurand/subject, were analyzed using PCA. Results A clear clustering between males and females mean values has been identified, where the menopausal females are closer to the males. Data interpretations for the three databases are similar. No significant differences for both mean and CVPs values, for countries, alcohol, smoking habits, BMI and physical activity, have been found. Conclusions The absence of meaningful differences among countries confirms the EuBIVAS sample homogeneity and that the obtained data are widely applicable to deliver APS. Our data suggest that the use of PCA and the multivariate approach may be used to detect outliers, although further studies are required.

The current gold standard for COVID-19 diagnosis, the rRT-PCR test, is hampered by long turnaround times, probable reagent shortages, high false-negative rates and high prices. As a result, machine learning (ML) methods have recently piqued interest, particularly when applied to digital imagery (X-rays and CT scans). In this review, the literature on ML-based diagnostic and prognostic studies grounded on hematochemical parameters has been considered. By doing so, a gap in the current literature was addressed concerning the application of machine learning to laboratory medicine. Sixty-eight articles have been included that were extracted from the Scopus and PubMed indexes. These studies were marked by a great deal of heterogeneity in terms of the examined laboratory test and clinical parameters, sample size, reference populations, ML algorithms, and validation approaches. The majority of research was found to be hampered by reporting and replicability issues: only four of the surveyed studies provided complete information on analytic procedures (units of measure, analyzing equipment), while 29 provided no information at all. Only 16 studies included independent external validation. In light of these findings, we discuss the importance of closer collaboration between data scientists and medical laboratory professionals in order to correctly characterise the relevant population, select the most appropriate statistical and analytical methods, ensure reproducibility, enable the proper interpretation of the results, and gain actual utility by using machine learning methods in clinical practice.

One of the next frontiers in medical research, particularly in orthopaedic surgery, is personalized treatment outcome prediction. In personalized medicine, treatment choices are adjusted for the patient based on the individual’s and their disease’s distinct features. A high-value and patient-centered health care system requires evaluating results that integrate the patient’s viewpoint. Patient-reported outcome measures (PROMs) are widely used to shed light on patients’ perceptions of their health status after an intervention by using validated questionnaires. The aim of this study is to examine whether meteorological or light (night vs. day) conditions affect PROM scores and hence indirectly affect health-related outcomes. We collected scores for PROMs from questionnaires completed by patients (N = 2326) who had undergone hip and knee interventions between June 2017 and May 2020 at the IRCCS Orthopaedic Institute Galeazzi (IOG), Milan, Italy. Nearest neighbour propensity score (PS) matching was applied to ensure the similarity of the groups tested under the different weather-related conditions. The exposure PS was derived through logistic regression. The data were analysed using statistical tests (Student’s t-test and Mann−Whitney U test). According to Cohen’s effect size, weather conditions may affect the scores for PROMs and, indirectly, health-related outcomes via influencing the relative humidity and weather-related conditions. The findings suggest avoiding PROMs’ collection in certain conditions if the odds of outcome-based underperformance are to be minimized. This would ensure a balance between costs for PROMs’ collection and data availability.

Medical errors have a huge impact on clinical practice in terms of economic and human costs. As a result, technology-based solutions, such as those grounded in artificial intelligence (AI) or collective intelligence (CI), have attracted increasing interest as a means of reducing error rates and their impacts. Previous studies have shown that a combination of individual opinions based on rules, weighting mechanisms, or other CI solutions could improve diagnostic accuracy with respect to individual doctors. We conducted a study to investigate the potential of this approach in cardiology and, more precisely, in electrocardiogram (ECG) reading. To achieve this aim, we designed and conducted an experiment involving medical students, recent graduates, and residents, who were asked to annotate a collection of 10 ECGs of various complexity and difficulty. For each ECG, we considered groups of increasing size (from three to 30 members) and applied three different CI protocols. In all cases, the results showed a statistically significant improvement (ranging from 9% to 88%) in terms of diagnostic accuracy when compared to the performance of individual readers, this difference held for not only large groups, but also smaller ones. In light of these results, we conclude that CI approaches can support the tasks mentioned above, and possibly other similar ones as well. We discuss the implications of applying CI solutions to clinical settings, such as cases of augmented ‘second opinions’ and decision-making.

Treatment and prevention of cardiovascular diseases often rely on Electrocardiogram (ECG) interpretation. Dependent on the physician's variability, ECG interpretation is subjective and prone to errors. Machine learning models are often developed and used to support doctors; however, their lack of interpretability stands as one of the main drawbacks of their widespread operation. This paper focuses on an Explainable Artificial Intelligence (XAI) solution to make heartbeat classification more explainable using several state-of-the-art model-agnostic methods. We introduce a high-level conceptual framework for explainable time series and propose an original method that adds temporal dependency between time samples using the time series' derivative. The results were validated in the MIT-BIH arrhythmia dataset: we performed a performance's analysis to evaluate whether the explanations fit the model's behaviour; and employed the 1-D Jaccard's index to compare the subsequences extracted from an interpretable model and the XAI methods used. Our results show that the use of the raw signal and its derivative includes temporal dependency between samples to promote classification explanation. A small but informative user study concludes this study to evaluate the potential of the visual explanations produced by our original method for being adopted in real-world clinical settings, either as diagnostic aids or training resource.

We propose a novel approach to human interaction with artificial intelligence systems (HAII), alternative to the mainstream dyadic one where humans and AI are seen as interacting agents. Through two quantitative experiments and two qualitative in-field case studies, we show that the mainstream HAII paradigm presents potentially harmful design shortcomings as it can trigger negative dynamics such as automation bias and prejudices. Our proposal, on the other hand, is grounded in the Computer-Supported Cooperative Work literature, in which AI can be conceived as a component of a Knowledge Artifact (KA). This consists of an ecosystem of knowledge creation tools whose goal is to support a Ba (after Nonaka), i.e. a collective of competent decision makers. We highlight the cooperative nature of decision making and the AI functionalities that a KA should embed. These include eXplainable AI solutions, aimed at facilitating appropriation, but also functionalities that enable reasoning in a collaborative setting. Finally, we discuss how moving intelligence and agency from individual agents to the human collective can help to mitigate the shortcomings of dyadic HAII (e.g., deskilling), re-distribute responsibility in critical tasks, and revisit the HAII research agenda to align it with the needs of increasingly wide, heterogeneous and complex teams.

In recent years, Machine Learning (ML) has attracted wide interest as aid for decision makers in complex domains, such as medicine. Although domain experts are typically aware of the intrinsic uncertainty around it, the issue of Ground Truth (GT) quality has scarcely been addressed in the ML literature. GT quality is regularly assumed to be adequate, regardless of the number and skills of raters involved in data annotation. These factors can, however, potentially have a severe negative impact on the reliability of ML models. In this article we study the influence of GT quality, in terms of number of raters, their expertise, and their agreement level, on the performance of ML models. We introduce the concept of reduction: computational procedures by which to produce single-target GT from multi-rater settings. We propose three reductions, based on three-way decision, possibility theory, and probability theory. We provide characterizations of these reductions from the perspective of learning theory and propose two ML algorithms. We report the result of experiments, on both real-world medical and synthetic datasets, showing that GT quality strongly impacts on the performance of ML models, and that the proposed algorithms can better handle this form of uncertainty compared with state-of-the-art approaches.

In this article we discuss a novel utility metrics for the evaluation of AI-based decision support systems, which is based on the users’ perceptions of the relevance of, and risks associated with, the validation cases. We discuss the relationship between the proposed metric and other previous proposals in the specialist literature; in particular, we show that our metric generalizes the well-known Net Benefit. More in general, we make the point for having utility as the prime dimension to optimize machine learning models in critical domains, like the medical one, and to evaluate their potential impact on real-world practices.

Background:The interest in Minimally Invasive Surgery (MIS) techniques has greatly increased in the recent years due to their significant advantages, both in terms of outcome improvement and cost reduction. Also in spine surgery, MIS is now applicable to several conditions and, above all, in low back pain (LBP) treatment. However, reliable and objective measures of invasiveness, necessary to compare different procedures, are still lacking. Methods:In this article we study the application of Machine Learning (ML) techniques to define an invasiveness score for LBP procedures based on biological markers and inflammatory profiles. In so doing, we can assess the invasiveness of surgical procedures. We also propose a predictive model for treatment planning based on the evaluation of invasiveness of surgical alternatives for specific patients, using their pre-surgery biomarkers. The data used in study was characterized by low sample size and high-dimensionality, thus we adopted a combination of feature selection, careful selection of ML models and conservative model selection choices in order to address these concerns. We also performed an external validation based on a statistically significantly different datasets in order to confirm the relevance of the findings. Results:We report the results of an experimental study on real-world data, for which we obtained promising results for both considered applications: we report an AUC of 0.87 for the task of invasiveness score definition, and an AUC of 0.76 for the invasiveness prediction task. The results obtained on the external validation were in agreement with the obtained results. Further, in both cases the performances were considered as excellent by the involved clinicians and the selected predictive features were biologically relevant and associated with invasiveness and biological impact in the relevant literature. Conclusion:Our results show that ML techniques could be effectively employed not only for diagnosis or prognosis, but also for treatment planning, a task of fundamental importance toward personalized and value-based healthcare. These results also show that ML approaches could be effectively used even in scenarios (e.g. pilot studies) where only small samples are available.

With increasing variety and sophistication of computer-based medical devices, and more diverse users and use environments, usability is essential, especially to ensure safety. Usability standards and guidelines play an important role. We reviewed several, focusing on the IEC 62366 and 60601 sets. It is plausible that these standards have reduced risks for patients, but we raise concerns regarding: (1) complex design trade-offs that are not addressed, (2) a focus on user interface design (e.g., making alarms audible) to the detriment of other human factors (e.g., ensuring users actually act upon alarms they hear), and (3) some definitions and scope restrictions that may create “blind spots”. We highlight potential related risks, e.g. that clear directives on “easier to understand” risks, though useful, may preclude mitigating other, more “difficult” ones; but ask to what extent these negative effects can be avoided by standard writers, given objective constraints. Our critique is motivated by current research and incident reports, and considers standards from other domains and countries. It is meant to highlight problems, relevant to designers, standards committees, and human factors researchers, and to trigger discussion about the potential and limits of standards.

Background We focus on the importance of interpreting the quality of the labeling used as the input of predictive models to understand the reliability of their output in support of human decision-making, especially in critical domains, such as medicine. Methods Accordingly, we propose a framework distinguishing the reference labeling (or Gold Standard) from the set of annotations from which it is usually derived (the Diamond Standard). We define a set of quality dimensions and related metrics: representativeness (are the available data representative of its reference population?); reliability (do the raters agree with each other in their ratings?); and accuracy (are the raters’ annotations a true representation?). The metrics for these dimensions are, respectively, the degree of correspondence, Ψ, the degree of weighted concordanceϱ, and the degree of fineness, Φ. We apply and evaluate these metrics in a diagnostic user study involving 13 radiologists. Results We evaluate Ψ against hypothesis-testing techniques, highlighting that our metrics can better evaluate distribution similarity in high-dimensional spaces. We discuss how Ψ could be used to assess the reliability of new predictions or for train-test selection. We report the value of ϱ for our case study and compare it with traditional reliability metrics, highlighting both their theoretical properties and the reasons that they differ. Then, we report the degree of fineness as an estimate of the accuracy of the collected annotations and discuss the relationship between this latter degree and the degree of weighted concordance, which we find to be moderately but significantly correlated. Finally, we discuss the implications of the proposed dimensions and metrics with respect to the context of Explainable Artificial Intelligence (XAI). Conclusion We propose different dimensions and related metrics to assess the quality of the datasets used to build predictive models and Medical Artificial Intelligence (MAI). We argue that the proposed metrics are feasible for application in real-world settings for the continuous development of trustable and interpretable MAI systems.

In this work, we provide an overview of contemporary perspectives of design that may challenge the traditional design of IT and socio-technical systems. Our starting metaphor is that of ‘wicked problems’, where the singularity, incompleteness and intrinsic uncertainty of real world settings foregrounds how the worldview that designers offer to practitioners may be optimal in theory but useless in practice. To go beyond traditional notions of design and designer, we intercepted insights coming from minoritarian voices in both theoretic and practice-based design fields. ‘De-design’ is a term we coined to encompass this wide spectrum of approaches that make more resilient and sustainable information artifact, de-emphasize design as a theoretical construct, and reconsider practice as the leading principle of digital innovation. This paper is a narrative review of voices in an extensive array of fields: from Information Systems to Human-Computer Interaction, from End-User Development to Critical Design, from Software Design to Design Studies. Our contribution retraces the motivational roots of de-design and tries to characterise de-design by filling relational gaps between disparate approaches and by bringing them back to IT and socio-technical design, to make digital artifacts sustainable in all of the new environmental, organisational and cultural spaces near to come.

In this work we introduce a framework, based on three-way decision (TWD) and the trisecting-acting-outcome model, to handle uncertainty in Machine Learning (ML). We distinguish between handling uncertainty affecting the input of ML models, when TWD is used to identify and properly take into account the uncertain instances; and handling the uncertainty lying in the output, where TWD is used to allow the ML model to abstain. We then present a narrative review of the state of the art of applications of TWD in regard to the different areas of concern identified by the framework, and in so doing, we will highlight both the points of strength of the three-way methodology, and the opportunities for further research.

Summary. Background. The Lombardy region, Italy, has been severely affected by COVID-19. During the epidemic peak, in March 2020, patients needing intensive care unit treatments were approximately 10% of those infected. This fraction decreased to approximately 2% in the second part of April, and to 0.4% at the beginning of July. COVID-19 is characterized by several biochemical abnormalities whose discrepancy from normal values was associated to the severity of the disease. The aim of this retrospective study was to compare the biochemical patterns of patients during and after the pandemic peak in order to verify whether later patients were experiencing a milder COVID-19 course, as anecdotally observed by several clinicians of the same Hospital. Material and Methods. The laboratory findings of two equivalent groups of 84 patients each, admitted at the emergency department of the San Raffaele Hospital (Milan, Italy), during March and April respectively, were analyzed and compared. Results. White blood cell, platelets, lymphocytes and lactate dehydrogenase showed a statistically significant improvement (i.e. closer or within the normal clinical range) in the April group compared to March. Creatinine, C-reactive protein, Calcium and liver enzymes, were also pointing in that direction, although the differences were not significant. Discussion. The laboratory findings analyzed in this study were consistent with a milder COVID-19 course in the April group. After excluding several hypotheses, we concluded that our observation was likely the consequence of the lockdown strategy enforcement, which, by imposing social distancing and the use of respiratory protective devices, reduced viral loads upon infection. (www.actabiomedica.it).

In this paper, we address the problem of assessing the social value of open data. While the number of open data initiatives increases and many data sets are currently available to lay people, common citizens and end users, a still limited number of studies specifically address how to improve open data understandability, their usability by common users and the measurability of their value in terms of concrete outcomes and benefits for the intended communities and the individual who appropriate those data, to make them more personal and hence more valuable. Our goal is to contribute to the success of open data initiatives by defining a methodology by which to assess their perceived social value. In this paper, we present the conceptual content of the methodology, that is its main concepts and logic structure, and discuss it by means of an empirical user study in which we applied it to real-life open data sets and involving a large sample of prospective consumers of those open data. In particular, we focus on the health care domain in order to improve the welfare of the citizens that need health care services and use the Web to look for relevant information to address those situated needs. Among our main findings, we discovered a clear preference for visual information formats by women with respect to men, and a clear preference for hospitals ranking by disease for senior people with respect to younger.