Ординатура / Хирургия / Библиотека им академика М.И. Перельмана / Книга_117_библиотеки_им_акад_М_И_Перельмана

The development of Articial Intelligence (AI) has advanced dramatically. AI includes Machine Learning (ML). ML’s primary function is to analyze data for recurring patterns or structures so that useful data models can be constructed. In the conventional sense, an algorithm is a limited set of rules for solving a problem. In contrast, modern data inputs and statistical analyses are trained with machine learn­ing [1] algorithms to generate a target value within a range. Articial intelligence has been crucial to the operations of many elds that have adapted to new informa­tion technology. Articial Intelligence (AI) stakeholders are increasingly demand­ing clarity as black-box Machine Learning (ML) models are utilized to make crucial assumptions in decision-making. Making and carrying out decisions that are not valid, acceptable, or easily interpretable in terms of their behavior [2] to pose a threat are discussed. In personalized medicine, for instance, professionals require much more information from the model than just a basic binary estimate to support their medical assessment [3]. People are typically unwilling to adopt procedures that are not easily understandable, tractable, or reliable. However, an improved comprehension of a model can lead to the modication of its defects. The aspect of understandability as a major design driver when developing an ML model can improve its implementation ability for the following reasons.

develop adversarial instabilities that could change the predictive model.

the output.

To avoid restricting the efciency of today’s AI systems, eXplainable AI (XAI) suggests developing a set of machine learning approaches while keeping a high order of learning performance in both accuracy and precision, which create more explainable models and by which humans can understand the model effectively.

While the earliest AI systems could be understood readily, opaque decision­making techniques like Deep Neural Networks (DNNs) have become more preva­lent in recent years. As a result of a set of powerful learning algorithms and their enormous computational power, Deep Learning (DL) models like DNNs have dem­onstrated empirical success. The latter space has hundreds of levels and millions of parameters, which qualies DNNs as sophisticated black-box models [4]. Transparency, or the quest for a clear knowledge of a model’s operating process, is the antithesis of black box [5].

AI encompasses several methods under its roof, such as ML, to predict outcomes. ML eventually achieves the objective of producing correct results by training the model. A cyberattack on the National Highway Authorities of India (NHAI) core server occurred in 2020 as a result of an inadequate cyber security system [9]. In such cases, eXplainable AI (XAI) can provide a proper explanation and identify efforts that should be made to avert future cyberattacks.

The chapter’s primary goal is to investigate the challenges and limitations of existing AI models in healthcare and the potential risks associated with the “black box”. Figure1 shows how research is spread out in different types of domains. In this context, it becomes evident that the healthcare domain gains more atten­tion in XAI.

Because the amount of structured and unstructured data is continuously growing, AI is an enabling paradigm revolutionizing the healthcare domain, and sophisticated wearable devices, such as tness devices, etc., may forecast the emergence of health issues in users by recording and interpreting their health data. XAI is the study that focuses on developing methods to explain the underlying assumptions and decision­making processes of AI systems. By studying black-box models and their decision­making processes, XAI helps healthcare providers verify the accuracy of the results produced by machine learning algorithms, which is critical in the medical industry. This explains the importance of AI in various healthcare domains and other elds.

When designing a deep learning system, developers don’t include a model that depicts how they think the problem’s causes interact with one another. Predictions made by deep learning algorithms, which may be trained on millions of data points, can be quite precise.

Nonetheless, deep learning algorithms can be mysterious despite their precision. The models used for classication by these systems can be incomprehensible to humans, even though their designers comprehend their architecture and the method by which the models are generated. The connections between input data and the nal classication may be indirect and fragile, even when methods are employed to determine the features for which a model assigns considerable weight when evaluat­ing a given example. Subtle changes to seemingly unrelated data can have a pro­found impact on how attributes are weighted. Furthermore, models might be built differently depending on the original conditions.

In domains where our understanding is lacking, we may use a recommendation system to prefer interpretability over prediction and diagnosis. When doctors give such recommendations to their patients, it can hurt them in the form of misdiag­noses or lead to unwanted testing. They may also encourage using ML systems for which they are not suitable, thinking that they are highly predictive models, but they are shown to be insufcient [10]. Table1 describes the literature on XAI applica­tions in different domains.