Добавил:
kiopkiopkiop18@yandex.ru t.me/Prokururor I Вовсе не секретарь, но почту проверяю Опубликованный материал нарушает ваши авторские права? Сообщите нам.
Вуз: Предмет: Файл:

Ординатура / Хирургия / Библиотека им академика М.И. Перельмана / Книга_117_библиотеки_им_акад_М_И_Перельмана

.pdf
Скачиваний:
2
Добавлен:
15.09.2026
Размер:
15 Мб
Скачать
☆
Explainable AI inDisease Diagnosis
105
Explanations Using PDP The PDP plots for various characteristics of breast cell nuclei are shown in Fig.10. The PDP plot of mean concavity shows that the effect on diagnosis prediction decreases as mean concavity increases until 0.18. A slight increase in the same plot is seen at 0.27, but after 0.29, the diagnosis prediction is not affected by the increase in mean concavity. Similarly, in the PDP plot of mean compactness, the effect of mean compactness on diagnosis prediction is increased with increasing value of mean compactness initially, then afterwards a decrease was observed with increase of mean compactness.
XAI is adequately effective in generating explanations for the black-box models; however, there exist some challenges and gaps which are essential to address. The following section discusses the pros and cons of the XAI methods and addresses the challenges in XAI.
6 Discussion
The paper discusses various XAI methods that are being used in different applica­tions of healthcare. LIME [25] is an approximation algorithm which provides expla­nations (local) for each instance. It can be applied with varied data formats including
Fig. 8 (a) Reasoning by LIME method for correct breast cancer prediction. (b) Reasoning by LIME method for wrong breast cancer prediction
106
P. Bedi et al.
texts [25], images (Case Study 1 in Sect. 5), and relational data [26]. However, it cannot generate generalised explanations (global) for a set of instances. There are XAI methods such as SHAP [26], PDP [17], AL [17], LD [17], and Saliency maps [28] that can be used to generate global explanations. SHAP and Saliency maps can be applied to varied data formats such as relational data [26] and images (Case Study 1 in Sect. 5) whereas PDP, AL, and LD apply to numerical and categorical values. Saliency maps are more suitable for high-dimensional data such as images. PDP, AL, and LD are compatible with fewer attributes for the regression or classi­cation problems. LIME, SHAP, Saliency maps, PDP, AL, and LD are model­independent methods and can be applied to any ML/DL model. There are XAI
Fig. 9 Summary plot describing the impact of the feature on model output
Explainable AI inDisease Diagnosis
107
methods which are model dependent such as GNNExplainer [27] which is specic to Graph Neural Networks, and Grad-CAM [23] which can explain CNN models.
The XAI methods show promising results in explaining ML/DL model- generated predictions. However, some limitations exist, such as the illusion of explanatory depth, lack of evaluation of generated recommendations, and subjective inferences in the explanations generated by XAI methods. These limitations are discussed briey in this section.
Lack of Evaluation Metric for XAI Numerous XAI methods are available to pro­vide explanations for the machine-generated predictions. The explanations can be local (explanation for an instance) or global (explanation for the whole dataset). However, in both explanations, different XAI methods can give contradictory results (regarding feature importance and feature values) [39]. It raises the issue of which model provides the correct explanations and on what foundation we should believe these explanations. There is no standard evaluation metric to estimate the correctness of XAI-based explanations. The lack of evaluation metrics for the XAI- generated explanations creates uncertainty about their accuracy.
The Illusion of Explanatory Depth The lack of evaluation metrics for XAI meth­ods sometimes gives rise to another challenge called the illusion of explanatory depth. Users accept the algorithmically generated explanations without validation and verication due to the illusion of explanatory depth. It causes overcondence in explanations and indifference to the present uncertainties. The illusion of explana-
Fig. 10 PDP plot for breast cancer predictions
108
P. Bedi et al.
tory depth can cause harm on a large scale for non-technical users since the testing of explanations generated by XAI algorithms relies on the verication of end users only. In [40], the authors experimented with observing the effect of the illusion of explanatory depth in non-technical XAI users. During the experiment, users were asked to write interpretations of the explanations generated by the SHAP XAI method. It was observed that the interpretations differ signicantly despite the users’ condence in their explanations.
Subjective Interpretation XAI aims to make ML/DL models explainable by pro­viding reasons for model-based predictions. These reasons/explanations should be understandable to every user irrespective of their domain expertise. However, XAI­generated explanations can be interpreted differently by individuals. This is mainly due to two factors: an individual’s expertise area and biases. The expertise of an individual dramatically affects their interpretation of the explanation. On the other hand, even if the end user is an expert, their critical thinking can inuence their interpretation of XAI explanations and may result in wrong decision-making [41].
Transparency and Trust The adoption of AI-based systems is limited due to a lack of trust in their predicted decisions. Researchers believe increasing transparency in AI-based decisions by including explanations can help enhance users’ trust in AI-based systems. However, the relationship between transparency and trust is complicated. Unquestionably, XAI methods increase transparency, but it does not always increase trust in AI-based applications as expected [42]. A behavioural experiment was conducted to observe the effect on human trust after including insights/explanations in an ML-based ‘text classication decision support’ tool [43]. Two types of insights, highlighting the decisive words in the input text and the condence score of the decision support tool, were provided to the users. An adverse effect on trust was seen, and it was observed predominantly with correct predic­tions. In conclusion, increased transparency may also raise mistrust, which is a sig­nicant barrier to societal acceptance of AI.
Natural Language Explanations Natural language is the standard and most pre­ferred method of communication among humans. End users of different elds, be they doctors from the medicinal eld, educators from the education eld, or farmers from agriculture, prefer recommendations in natural (text) language only [44]. However, visuals are the more prominent form of explanation in XAI because it is simpler to visualise generated data than to represent the same in textual form. Therefore, XAI researchers need to focus on creating explanations in the preferred (textual) form or generate hybrid explanations containing visuals and text that enhance explanations and remove the ambiguity in interpretations of explanations.
These problems with XAI demonstrate that there is signicant scope for improve­ment in XAI research.
Explainable AI inDisease Diagnosis
109
7 Conclusion
Explainable AI (XAI) recently gained attention when researchers started trying to generate explanations and reasons behind the black-box models’ predicted out­comes. In this chapter, we have studied and explored various XAI methods and existing taxonomies. A comprehensive all-inclusive taxonomy was proposed in the chapter while considering various XAI categories and criteria. The effective use of post-hoc methods of XAI, including Local Interpretable Model Agnostic Explanation (LIME), Shapley Additive Explanations (SHAP), Partial Dependence Plot (PDP), and Gradient Class Activation Map (Grad-CAM) in AI-based applications were dis­cussed. The working principles of these methods were analysed in depth across various domains, particularly in healthcare. This chapter examined different XAI applications for disease detection, including Grad-CAM in arrhythmia detection, PDP and accumulated proles in cancer prediction, LIME in early detection of fatal blood disease, and SHAP in Parkinson’s disease. Analysis of XAI revealed that it is portrayed in a favourable way as a tool for increasing transparency, but its limita­tions have not been made explicit. This chapter discussed signicant challenges in XAI, such as the lack of standard evaluation metrics, subjective interpretations, the illusion of explanatory depth, and the relation between transparency and trust in XAI.The discussion included the analysis and results of trials conducted in the lit­erature to study the impact of these challenges on decision-making.
References
1. Trocin C (2021) Responsible AI for digital health: a synthesis and a research agenda. Inf Syst Front 23(3):1–19. https://doi.org/10.1007/s10796- 021- 10146- 4
2. Srivastava G etal (2022) XAI for cybersecurity: state of the art, challenges, open issues future directions. Cyber Secur Appl 1(1):1–33. http://arxiv.org/abs/2206.03585
3. Mankodiya H, Jadav D, Gupta R, Tanwar S, Hong WC, Sharma R (2022) OD-XAI: explain­able AI-based semantic object detection for autonomous vehicles. Appl Sci 12(11):5310.
https://doi.org/10.3390/app12115310
4. Garouani M, Ahmad A, Bouneffa M, Hamlich M, Bourguin G, Lewandowski A (2022) Towards big industrial data mining through explainable automated machine learning. Int J Adv Manuf Technol 120(1–2):1169–1188. https://doi.org/10.1007/s00170- 022- 08761- 9
5. Barnard P, Marchetti N, DaSilva LA (2022) Robust network intrusion detection through explainable articial intelligence (XAI). IEEE Netw Lett 4(3):167–171. https://doi.
org/10.1109/lnet.2022.3186589
6. Conati C, Barral O, Putnam V, Rieger L (2021) Toward personalized XAI: a case study in intel­ligent tutoring systems. Artif Intell 298:103503. https://doi.org/10.1016/j.artint.2021.103503
7. Song Y, Kim K, Park S, Park SK, Park J (2023) Analysis of load-bearing capacity factors of textile-reinforced mortar using multilayer perceptron and explainable articial intelligence. Constr Build Mater 363:129560. https://doi.org/10.1016/j.conbuildmat.2022.129560
8. Humer C et al (2022) ChemInformatics Model Explorer (CIME): exploratory analy­sis of chemical model explanations. J Cheminform 14(1):1–14. https://doi.org/10.1186/
s13321- 022- 00600- z
110
9. Lakhani P, Sundaram B (2017) Deep learning at chest radiography: automated classication of pulmonary tuberculosis by using convolutional neural networks. Radiology 284(2):574–582.
https://doi.org/10.1148/radiol.2017162326
10. Szegedy C etal (2015) Going deeper with convolutions. In: Proceedings of the IEEE com­puter society conference on computer vision and pattern recognition, pp 1–12. https://doi.
org/10.1109/CVPR.2015.7298594
11. Krizhevsky A, Sutskever I, Hinton GE (2017) ImageNet classication with deep convolutional neural networks. Commun ACM 60(6):84. https://doi.org/10.1145/3065386
12. Yakar D, Ongena YP, Kwee TC, Haan M (2022) Do people favor articial intelligence over physicians? A survey among the general population and their view on articial intelligence in medicine. Value Heal 25(3):374–381. https://doi.org/10.1016/j.jval.2021.09.004
13. Wang Q, Huang K, Chandak P, Zitnik M, Gehlenborg N (2022) Extending the nested model for user-centric XAI: a design study on GNN-based drug repurposing. IEEE Trans Vis Comput Graph 29(1):1266–1276. https://doi.org/10.1109/TVCG.2022.3209435
14. El-Sappagh S, Alonso JM, Islam SMR, Sultan AM, Kwak KS (2021) A multilayer multimodal detection and prediction model based on explainable articial intelligence for Alzheimer’s disease. Sci Rep 11(1):1–26. https://doi.org/10.1038/s41598- 021- 82098- 3
15. Du Y, Antoniadi AM, McNestry C, McAuliffe FM, Mooney C (2022) The role of XAI in advice-taking from a clinical decision support system: a comparative user study of feature contribution-based and example-based explanations. Appl Sci 12(20):10323. https://doi.
org/10.3390/app122010323
16. Abir WH et al (2022) Explainable AI in diagnosing and anticipating leukemia using transfer learning method. Comput Intell Neurosci 2022(5140148):1–14. https://doi.
org/10.1155/2022/5140148
17. Ahmed ZU, Sun K, Shelly M, Mu L (2021) Explainable articial intelligence (XAI) for exploring spatial variability of lung and bronchus cancer (LBC) mortality rates in the contigu­ous USA.Sci Rep 11(1):1–15. https://doi.org/10.1038/s41598- 021- 03198- 8
18. Ribeiro MT, Singh S, Guestrin C (2016) ‘Why should I trust you?’ explaining the predictions of any classier. In: NAACL-HLT 2016 conference of the North American chapter of the Association for Computational Linguistics: human language technologies, proceedings of the demonstrations session, pp97–101. https://doi.org/10.18653/v1/n16- 3020
19. Adadi A, Berrada M (2018) Peeking inside the black-box: a survey on explainable articial intelligence (XAI). IEEE Access 6:52138–52160. https://doi.org/10.1109/
ACCESS.2018.2870052
20. Guleria P, Sood M (2022) Explainable AI and machine learning: performance evaluation and explainability of classiers on educational data mining inspired career counseling. Educ Inf Technol 28:1081–1116. https://doi.org/10.1007/s10639- 022- 11221- 2
21. Ivanovs M, Kadikis R, Ozols K (2021) Perturbation-based methods for explaining deep neural networks: a survey. Pattern Recogn Lett 150:228–2334. https://doi.org/10.1016/j.
patrec.2021.06.030
22. Fong R, Patrick M, Vedaldi A (2019) Understanding deep networks via extremal perturba­tions and smooth masks. In: Proc. IEEE Int. Conf. Comput. Vis., pp 2950–2958. https://doi.
org/10.1109/ICCV.2019.00304
23. Panwar H, Gupta PK, Siddiqui MK, Morales-Menendez R, Bhardwaj P, Singh V (2020) A deep learning and grad-CAM based color visualization approach for fast detection of COVID-19 cases using chest X-ray and CT-scan images. Chaos Solitons Fractals 140:110190. https://doi.
org/10.1016/j.chaos.2020.110190
24. Liu S, Li Z, Li T, Srikumar V, Pascucci V, Bremer PT (2019) NLIZE: a perturbation-driven visual interrogation tool for analyzing and interpreting natural language inference models. IEEE Trans Vis Comput Graph 25(1):651–660. https://doi.org/10.1109/TVCG.2018.2865230
25. Abdelwahab Y, Kholief M, Ahmed A, Sedky H (2022) Justifying Arabic text sentiment analy­sis using explainable AI (XAI): LASIK surgeries case study. Information 13(11):536. https://
doi.org/10.3390/info13110536
P. Bedi et al.
Explainable AI inDisease Diagnosis
26. Peng J et al (2021) An explainable articial intelligence framework for the deteriora­tion risk prediction of hepatitis patients. J Med Syst 45(5):45–61. https://doi.org/10.1007/
s10916- 021- 01736- 5
27. Ying R, Bourgeois D, You J, Zitnik M, Leskovec J (2019) GNNExplainer: generating explana­tions for graph neural networks. In: 33rd Conference on neural information processing systems (NeurIPS 2019), vol 32, pp1–12
28. Jo YY etal (2021) Detection and classication of arrhythmia using an explainable deep learn­ing model. J Electrocardiol 67:124–132. https://doi.org/10.1016/j.jelectrocard.2021.06.006
29. Lundberg SM, Lee SI (2017) A unied approach to interpreting model predictions. In: NIPS’17: proceedings of the 31st international conference on neural information processing systems, pp4768–4777
30. Aas K, Jullum M, Løland A (2021) Explaining individual predictions when features are depen­dent: more accurate approximations to Shapley values. Artif Intell 298:103502. https://doi.
org/10.1016/j.artint.2021.103502
31. Drancé M, Boudin M, Mougin F, Diallo G (2021) Neuro-symbolic XAI for computational drug repurposing. In: Proceedings of the 13th international joint conference on knowledge discovery, knowledge engineering and knowledge management, pp 220–225. https://doi.
org/10.5220/0010714100003064
32. Mellem MS, Kollada M, Tiller J, Lauritzen T (2021) Explainable AI enables clinical trial patient selection to retrospectively improve treatment effects in schizophrenia. BMC Med Inform Decis Mak 21(162):1–10. https://doi.org/10.1186/s12911- 021- 01510- 0
33. Liao W, Zou B, Zhao R, Chen Y, He Z, Zhou M (2020) Clinical interpretable deep learning model for glaucoma diagnosis. IEEE J Biomed Heal Informatics 24(5):1405–1412. https://doi.
org/10.1109/JBHI.2019.2949075
34. Zhang Z et al (2010) ORIGA-light: an online retinal fundus image database for glaucoma analysis and research. https://doi.org/10.1109/IEMBS.2010.5626137
35. Liu Y, Liu Z, Luo X, Zhao H (2022) Diagnosis of Parkinson’s disease based on SHAP value feature selection. Biocybern Biomed Eng 42(3):856–869. https://doi.org/10.1016/j.
bbe.2022.06.007
36. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architec­ture for computer vision. In: Proceedings of the IEEE computer society conference on com­puter vision and pattern recognition. https://doi.org/10.1109/CVPR.2016.308
37. Russakovsky O etal (2015) ImageNet large scale visual recognition challenge. Int J Comput Vis 115(3):211–252. https://doi.org/10.1007/s11263- 015- 0816- y
38. Ahsan MM etal (2023) Deep transfer learning approaches for Monkeypox disease diagnosis. Expert Syst Appl 216:119483. https://doi.org/10.1016/j.eswa.2022.119483
39. Collaris D, Vink LM, van Wijk JJ (2018) Instance-level explanations for fraud detection: a case study. pp28–33. http://arxiv.org/abs/1806.07129
40. Chromik M, Eiband M, Buchner F, Krüger A, Butz A (2021) I think I get your point, AI! The illusion of explanatory depth in explainable AI.In: IIUI ’21: 26th International conference on intelligent user interfaces, pp307–321. https://doi.org/10.1145/3397481.3450644
41. de Bruijn H, Warnier M, Janssen M (2022) The perils and pitfalls of explainable AI: strat­egies for explaining algorithmic decision-making. Gov Inf Q 39(2):101666. https://doi.
org/10.1016/j.giq.2021.101666
42. Zerilli J, Bhatt U, Weller A (2022) How transparency modulates trust in articial intelligence. Patterns 3(4):100455. https://doi.org/10.1016/j.patter.2022.100455
43. Schmidt P, Biessmann F, Teubner T (2020) Transparency and trust in articial intelligence systems. J Decis Syst 29(4):260–278. https://doi.org/10.1080/12460125.2020.1819094
44. Gale W, Oakden-Rayner L, Carneiro G, Bradley AP, Palmer LJ (2018) Producing radiologist­quality reports for interpretable articial intelligence. pp1–7. http://arxiv.org/abs/1806.00340
111
Explainable Articial Intelligence inDrug
Discovery
AbinV.Geevarghese
Abstract Creation of drugs using computers has been impacted by computational
intelligence. This growth continues to be fuelled by, amongst other factors, the widespread application of machine learning, and especially deep learning, through­out an array of academic regions, in addition to advances in computing software and hardware. Medical chemistry is beneting as a consequence of the early scepticism related to the use of AI in pharmaceutical discovery starting to dissipate. The use of deep learning has promise for the development of drugs because it enables advanced image comprehension, molecule function, and structure forecasting, including the computerised generation of new chemical compounds having particular character­istics. Overview of the present situation with AI in chemoinformatics. De novo molecular design, structure-based modelling, and chemical reaction prediction are among the topics covered in this chapter. To satisfy the demand for an innovative narrative of the computational language of molecular sciences, there’s an appetite for “explainable” methods for deep learning. The most signicant and explainable articial intelligence concepts are laid out in this chapter, which additionally expects potential uses and an array of unsolved challenges. I also hope that this chapter stimulates additional research on developing and implementation of methods of articial intelligence in drug discovery.
Keywords QSAR · Drug discovery · Articial intelligence · De novo drug design
A. V. Geevarghese (*) Department of Pharmacology, PSG College of Pharmacy, Coimbatore, Tamil Nadu, India e-mail: abin@psgpharma.ac.in
Ltd. 2024 R. Aluvalu et al. (eds.), Explainable AI in Health Informatics, Computational Intelligence Methods and Applications,
https://doi.org/10.1007/978-981-97-3705-5_6
113© The Author(s), under exclusive license to Springer Nature Singapore Pte
114
A. V. Geevarghese
1 Introduction
Computer-aided drug development has made extensive use of machine learning techniques [1–3]. The ability of articial neural networks with several hidden pro­cessing layers, or “deep learning” approaches, to use input data to automatically discover characteristics that might express non-linear input-output correlations has recently sparked renewed attention. These enhanced features of deep learning meth­odology improve existing articial intelligence techniques that rely on unique molecular descriptors [4, 5]. The application of deep learning has seen a relatively late rise in interest in the eld of medicinal drug discovery [6], which has already sparked an unparalleled boom of innovative modelling techniques and applica­tions [7–10].
The ever-evolving advancements in deep learning are currently assisting several elds of the chemical sciences [11–13]. In the past few years, several “articial intelligence” (AI) principles have been successfully used to computerised drug dis­covery [11, 12, 14]. The ability of advanced learning algorithms to simulate com­plex non-linear input-output relationships, perform pattern recognition, and extract features from low-level data representations is primarily responsible for this growth. Articial neural networks with several processing layers are what deep learning algorithms are made of. Some deep learning models have been shown to outperform well-known machine learning and quantitative structure-activity relationship (QSAR) methods for drug development [15–17]. Deep learning has further expanded the potential and utility of computer-assisted discovery, for example, in the elds of protein structure prediction, macromolecular target identication, molecule design, and chemical synthesis planning [18–21]. A model’s limited understanding fre­quently pays for the ability to capture complicated non-linear relationships between the input data (such as chemical structure representations) and the corresponding outputs (such as assay results). While efforts are currently made to clarify QSARs using computational insights and molecular descriptor analysis [22–27], models of deep neural networks are infamous for being inaccessible to the human mind right away [28]. The abundance of “rules of thumb” linking biological effects with physi­cochemical qualities, particularly in medicinal chemistry, highlights the readiness to sometimes trade precision in favour of models that better match human intuition [29, 30]. With the recent resurgence of neural networks in chemistry and medicine, automated analysis of medical and chemical information to extract and depict char­acteristics in a human-intelligible manner has been around since 1990s [31, 32].
There will be a greater demand for techniques that help in understanding and interpreting the models that underneath given the speed at which AI is being applied in drug development and related domains. Emphasis has been focused on explain­able AI (XAI) techniques in an attempt to reduce the uninterpretability of some machine learning models and to improve human reasoning and decision-making [33, 34]. Establishing the fundamental decision-making process transparent (“understandable”) avoiding making precise forecasts for the incorrect causes (the so-called clever Hans effect) [35], preventing unfair biases or unlawful injustice,
Explainable Articial Intelligence inDrug Discovery
115
and bridging gaps among the machine learning group and other disciplines of sci­ence all have the goals of offering useful explanations with the mathematical frame­works. Competent XAI may help scientists in overcoming “cognitive valleys” improving their comprehension and convictions on the process under investigation [36, 37].
Although there is some disagreement on the precise meaning of XAI, the author believes that a number of its characteristics are unquestionably favourable for drug design programmes [38]. Although there is some disagreement over the exact de­nition of XAI, the author is of the opinion that a number of its features are undoubt­edly advantageous for applications in drug development [39].
• Transparency—knowing how the system arrived at a particular conclusion.
• Justication—stating the merits of the model’s proposed solution.
• Informativeness—supplying fresh information to decision-makers.
• Calculating the degree of uncertainty in a prediction.
XAI-generated justications can usually be categorised into two distinct groups: global (which highlights the importance of the input variables in the model) and local (which focuses on individual predictions). Additionally, XAI might be model­dependent or model-agnostic, which inuences the possible application of each approach. There is no one-size-ts-all XAI technique in this architecture [40].
Future domain-specic challenges to AI-assisted medication development include the data structure that these approaches employ. In contrast to many other areas where deep learning has been shown to succeed, such as the processing of spoken languages and image identication, there is no naturally applicable, com­plete, “raw” molecular description. After all, molecules are models in and of them­selves as far as science is concerned. Therefore, using an “inductive” method that builds higher-order (like deep learning) models from lower-order ones (like molecu­lar representations or descriptors based on observable statements) is philosophically dubious [41]. The selection of the molecular “representation model” limits the amount of chemical information that can be retained, including pharmacophores, physicochemical properties, and functional groups, as well as the amount of infor­mation that can be explained and how well the resulting AI model performs.
Drug development is a difcult procedure. Inaccuracy, non-linearity, and seem­ingly random events set it apart from simple engineering. I must admit that I have a poor understanding of molecular pathology, and we are unable to create mathemati­cal models of pharmacological impact and accompanying reasons that are 100% accurate. Here, XAI holds forth the prospect of fostering human ingenuity and apti­tude for producing novel bioactive compounds with desired properties [42].
The foundation of generating novel medications is the investigation into whether pharmacological activity (or rather, “function”) may be deduced from the molecular structure and what components of the structure are signicant. The added difcul­ties and poorly communicated problems presented by multi-objective designing lead to molecular designs that all too frequently serve as compromise solutions. The practical strategy is to reduce the amount of syntheses and testing required to iden­tify and improve innovative hit and lead compounds, particularly when complex and