Добавил:
Sekretar
kiopkiopkiop18@yandex.ru
t.me/Prokururor I Вовсе не секретарь, но почту проверяю
Опубликованный материал нарушает ваши авторские права? Сообщите нам.
Вуз:
Предмет:
Файл:Ординатура / Хирургия / Библиотека им академика М.И. Перельмана / Книга_5525_Библиотеки_им_академика_М_И_Перельмана.pdf
Artificial Intelligence in Adaptive Radiation Therapy
innovation, thereby contributing significantly to societal advancements. Including
more diverse data from multiple institutions to train AI models also reduces biases
and enhances model generalizability. This collaborative approach not only enriches
the research landscape but also promotes the overall welfare of society.
Ensuring data security and privacy [28] is of utmost importance when it comes to
storing, transmitting, and sharing medical information. Particularly concerning
patients’ sensitive health data, such as names, addresses, and medical records,
stringent measures are in place to safeguard this information from any unauthorized
access, use, or disclosure. Compliance with regulations such as health insurance
portability and accountability act (HIPAA), national laws, and institutional policies
is mandatory in this regard. To maintain the confidentiality of patient data during
transmission, secure methods such as encrypted connections are employed, guaranteeing protection against interception or tampering. These measures serve to prevent
unauthorized access and data breaches of health records. However, the process of
data sharing is far from straightforward due to the intricate web of regulatory
requirements and the imperative need to safeguard health data privacy. Addressing
this challenge, a novel approach, federated learning [29], has gained traction in the
field of radiation oncology. This paradigm eliminates the necessity of data leaving
their originating institution. Instead, only the model derived from the data is
transferred. While this method partially resolves the issue, it does not always prove
as effective as direct data sharing, which continues to be a complex and evolving
endeavor in the realm of healthcare data management.
It is important to highlight that integrating certain standards into big data sharing
and storing, particularly in the healthcare sector, can be highly advantageous.
Standards such as DICOM, Health Level Seven (HL7), and the Fast Healthcare
Interoperability Resource (FHIR) not only enhance interoperability but also drive
cost efficiency, ensure compliance with regulations, and bolster data security within
the healthcare industry.
5.2.3 Data visualization
Data visualization [30] is the art of representing complex datasets using visual aids
such as charts, graphs, maps, and interactive dashboards. This visual representation
serves as a powerful tool, enabling users to explore, comprehend, and communicate
intricate data patterns effectively. When dealing with extensive datasets, visualization becomes especially crucial as it allows for quick exploration and identification of noteworthy patterns that might warrant further investigation. Additionally,
it aids in error detection by highlighting outliers, ensuring data accuracy and
reliability. Database tools integrated with visualization features offer users an
interactive and exploratory approach to analyzing data. For instance, in the context
of radiation oncology, visualizing OAR dose–volume histograms (DVHs) for an
entire patient cohort can reveal how treatment plans align with clinical goals.
Similarly, employing bar plots with dosimetric statistics grouped by outcome
variables can unveil potential correlations. Another key advantage of data visualization is its ability to facilitate information sharing. By presenting data visually, it
5-7
Artificial Intelligence in Adaptive Radiation Therapy
becomes easier to convey insights to diverse audiences, fostering a better understanding of the underlying information.
Several common visualization tools are widely used, including bar plots, histograms, line charts, and scatter plots. These tools provide different perspectives on the
data, allowing users to gain insights into various aspects of their datasets. Moreover,
advanced techniques such as tree maps, principal component analysis (PCA),
clustering algorithms, and t-distributed stochastic neighbor embedding (t-SNE)
[31] further enhance the visualization process. PCA, for example, reduces data
dimensions, simplifying complex datasets and aiding in visualization. Clustering
algorithms group similar data points, revealing inherent patterns, while t-SNE helps
visualize high-dimensional data in low-dimensional space, preserving data
relationships.
In summary, data visualization, coupled with sophisticated algorithms and tools,
not only aids in exploring and understanding large datasets but also serves as a
valuable tool for users to make informed decisions and communicate findings
effectively.
5.2.4 Knowledge creation and implementation
Knowledge creation involves acquiring insights through rigorous data analysis. This
process entails building models, identifying patterns, and devising efficient strategies
using data-driven approaches. Extensive validation and testing are essential components of this process. Once the tools are developed, they must be seamlessly
integrated into real clinical settings, ensuring practical implementation and continuous refinement.
The implementation of data-driven tools in healthcare presents numerous
challenges [32, 33]. First, these tools ideally should be interpretable, offering insights
into their decision-making processes to establish trust among healthcare professionals. Second, they need to be user-friendly and seamlessly integrated into existing
workflows or systems. Rigorous validation through clinical trials is also essential to
ensure their effectiveness and safety before they can be widely adopted.
Furthermore, continuous QA programs are essential to monitor their performance
over time. Ethical concerns, such as biases in algorithms and accountability in cases
of failure, must be thoroughly investigated. Additionally, the regulatory approval
process is required to ensure both efficacy and safety.
5.2.5 Data archiving and deletion
Effectively managing the continuous infl ux of data is essential, with archiving and
deletion playing pivotal roles in this process [34]. Archiving involves preserving data
for the long term, typically governed by a well-defined data archiving policy that
outlines the frequency and timing of these activities. This strategic approach not
only facilitates efficient data management but also ensures alignment with organizational data retention policies and regulatory standards. Archiving data, often
transitioned from active databases to cost-effective cold storage, becomes an
efficient tool for achieving compliance while mitigating storage costs. Retrieval
5-8
Artificial Intelligence in Adaptive Radiation Therapy
mechanisms are in place, allowing organizations to access archived data when
necessary. In sectors such as healthcare, archived data serve as a valuable resource
for clinical trials and medical research.
Data deletion is a crucial step in optimizing storage resources and streamlining
the management of expansive datasets. By eliminating outdated or inaccurate
information, organizations can maintain high data quality, ensuring the accuracy
and reliability of analytical processes. Additionally, the removal of unused or
outdated data plays a pivotal role in mitigating security risks, reducing the potential
surface area for breaches, and minimizing the impact in case of a security incident.
5.3 Big data analytics with AI
5.3.1 Data processing and integration
Data processing is a comprehensive procedure designed to transform raw data into a
format suitable for analysis. This intricate process encompasses various essential
stages. Data cleaning involves identifying and rectifying outliers, errors, inconsistencies, and inaccuracies. Data filtering allows the selection of specific data subsets
based on predefined inclusion or exclusion criteria. The transformative phase
involves several methods. Normalization, or standardization, scales numerical
values to a standard range, eliminating biases. Discretization or binning converts
continuous data into discrete variables, facilitating categorical analysis. Encoding
transforms categorical data into numeric values, ensuring compatibility with
analytical models. Data smoothing techniques, such as moving averages, reduce
noise, aiding in the identification of underlying trends. Additionally, data imputation could be used to replace missing data with values computed using statistical
methods or advanced machine learning techniques, ranging from basic mean or
median imputation to more sophisticated regression-based approaches.
In the advanced stages of processing, an analytical component becomes integral.
It aims to delve into patterns and relationships among data samples. Clustering, for
instance, unveils inherent patterns by grouping similar samples, while regression
forecasts relationships between variables, offering valuable insights for data transformation. Moreover, feature extraction methods such as PCA reveal essential
features contributing to dataset variance. This wealth of information provides
insights into processing and integrating the data for downstream modeling tasks.
In specific domains such as radiation oncology, additional processing steps are
required for handling various data sources. For text data, NLP techniques such as
tokenization, stemming, and sentiment analysis are applied to clinical notes, patient
histories, radiology reports, pathology reports, and EHRs. In imaging data,
processes such as image enhancement, registration, augmentation, and feature
extraction are implemented for the meticulous processing of medical images, such
as CT, MRI, and PET. These advanced processing steps not only ensure data
integrity and quality but also unlock deeper insights, particularly in complex fields
such as RO. Leveraging sophisticated methods becomes pivotal in extracting
meaningful information from diverse datasets as technology continues to evolve.
5-9
Artificial Intelligence in Adaptive Radiation Therapy
Data integration serves as a pivotal process, bringing together information from
diverse sources to enhance analysis, reporting, and decision-making. In specialized
fields such as radiation oncology, there is a growing emphasis on integrating patient
data from systems such as EHRs, ROIS, TPS, etc. By amalgamating patients’
medical histories, lab results, imaging data, clinical information, and treatment
plans, a comprehensive patient-specific view emerges, significantly supporting
clinical decision-making, particularly in the context of precision medicine.
Various methods can be employed for effective data integration. Matching and
linkage algorithms, alongside merge and join algorithms, play a crucial role in
identifying and linking data related to the same entity. Transformation and
clustering algorithms, as previously discussed, prove efficient when dealing with
data from diverse sources, contributing to a more unified and coherent dataset.
Additionally, the integration landscape benefits from the application of machine
learning algorithms such as decision trees, random forest (RF) [35], k-nearest
neighbors (kNN) [36], support vector machine (SVM) [37], and deep learning
methods.
These AI techniques not only assist in integrating data but also bring additional
advantages. They are adept at reducing dimensionality, extracting latent features,
and uncovering intricate patterns within a diverse set of input data. This dimensionality reduction and feature extraction contribute significantly to the seamless
integration of data from disparate sources, enhancing the overall efficacy of the
data integration process. As technology advances, AI techniques further refine the
ability to extract meaningful insights from complex datasets, making data integration an ever-evolving and powerful facet of modern data management.
5.3.2 AI modeling
AI modeling stands as a pivotal phase for unraveling inherent patterns in datasets
and crafting tools essential for informed clinical decision-making. The standard
approach involves constructing AI models and validating their performance on
independent datasets. During model development, a subset of data is often earmarked as a validation dataset. This subset becomes instrumental in refining the
training hyperparameters, such as tweaking the strength of penalization terms or
optimizing the choice of activation functions based on the model’s validation
performance.
The process of partitioning datasets into distinct sets involves various strategies.
Stratified sampling ensures an equitable distribution of target classes across subsets,
while time-based splitting arranges data chronologically. k-fold cross-validation
(CV) divides data into k folds, iteratively training the model on k – 1 folds and
testing on the remaining one. There are variations such as leave-one-out CV and
stratified CV. The nested cross-validation method employs an inner loop of CV to
pinpoint optimal model parameters and an outer loop to rigorously test the refined
model. In essence, the choice of dataset partitioning method should align with the
dataset’s characteristics and the nature (e.g. size) of the AI task at hand. These
techniques collectively contribute to the development of models that not only
5-10
Artificial Intelligence in Adaptive Radiation Therapy
Figure 5.4. Overview of four machine learning paradigms and the example algorithms. . Abbreviations: cML
= classical machine learning; DL = deep learning; SVM = support vector machine; RF = random forest;
MLP = multi-layer perceptron; CNN = convolutional neural network; FCN = fully convolutional network;
LSTM = long short-term memory; GRU = gated recurrent units; PCA = principal component analysis;
t-SNE = t-distributed stochastic neighbor embedding; AE= autoencoder; SAE = sparse autoencoders;
DAE = denoising autoencoders; VAE = variational autoencoders; GAN = generative adversarial networks.
models could be trained using all three learning paradigms (supervised, unsupervised, and semi-supervised).
*
These
unravel intricate associations within data but also generalize effectively to new and
unseen instances.
AI models encompass a wide array of categories, each tailored to the specific
nature of the datasets they handle. Figure 5.4 shows different paradigms in machine
learning along with the corresponding algorithms.
In supervised learning, which deals with labeled datasets, models are employed to
address classi fi cation or regression challenges. Classical machine learning methods
such as SVM, RF, k NN, and naive Bayes [38] are commonly utilized for these tasks.
Furthermore, advanced deep learning architectures such as multi-layer perceptron
(MLP), convolutional neural networks (CNNs) and their variants such as U-Net
[39], V-Net [40], and the fully convolutional network (FCN) [41] demonstrate
exceptional performance, particularly with tasks involving image inputs. Recurrent
neural networks (RNNs) and their variants, including gated recurrent units (GRU)
[42] and long short-term memory (LSTM) networks [43], play a pivotal role in
analyzing sequential data such as time series data, natural language, DNA
sequences, etc. However, RNNs struggle to capture long-term dependencies in
sequences due to the vanishing gradient problem. It is also challenging to parallelize
them due to the sequential processing manner. In contrast, transformer models [44]
can process entire sequences simultaneously thanks to parallel computing and the
self-attention mechanism.
Unsupervised learning deals with unlabeled datasets, focusing on tasks such as
clustering and dimensionality reduction. Techniques such as PCA, k-means clustering, and t-SNE are commonly employed for these purposes. Additionally,
5-11
Artificial Intelligence in Adaptive Radiation Therapy
sophisticated deep learning methods such as generative adversarial networks
(GANs) [45] and autoencoders, including variants such as denoising autoencoders
(DAEs) [46], sparse autoencoders (SAEs) [47], and variational autoencoders (VAEs)
[48], offer powerful tools for unsupervised learning tasks.
Semi-supervised learning emerges as a crucial field, particularly in scenarios
where labeled data are scarce or expensive to obtain. By leveraging both labeled and
unlabeled data, semi-supervised learning approaches effectively learn from labeled
examples while also capitalizing on the additional insights provided by unlabeled
data. This approach proves particularly valuable when resource-intensive labeling
processes pose constraints, enabling a more nuanced understanding of data patterns.
Various training strategies, including self-training, low-density separation [49], and
graph-based methods [50], contribute to the efficacy of semi-supervised learning
algorithms. Notably, certain architectures such as CNNs, RNNs, and transformers
exhibit versatility, allowing them to be applied across various learning paradigms,
including supervised, semi-supervised, and unsupervised learning.
Reinforcement learning (RL) represents a distinct paradigm in machine learning,
where intelligent agents learn to optimize their strategies by interacting with
environments to maximize rewards. RL methods, such as dynamic programming,
Monte Carlo tree search [51], Q-learning [52], and temporal difference learning,
exemplify this approach, offering powerful techniques for solving complex decisionmaking problems in dynamic environments.
Recently, there have been remarkable advancements in foundation models and
generative AI, signaling a pivotal breakthrough in the field of AI. A foundation
model [53] is a large-scale AI model pre-trained on vast unlabeled data across
various modalities such as text, images, audio, or video. It learns useful representations and patterns of data and can be adapted to perform a wide range of
downstream tasks. Foundation models exhibit exceptional flexibility, adapting to
supervised, semi-supervised, or unsupervised learning, depending on their training
methodology. They can be classified based on the type of input data modalities they
are designed to process. For example, large language models (LLMs) are text-based
foundation models primarily used for NLP tasks. Examples of LLMs include
generative pre-trained transformers (GPTs) [54] and bidirectional encoder representations from transformers (BERTs) [55]. Some LLMs are specifically tailored for
medical applications, such as pubMedBERT [56], BioMedLM [57], and
clinicalBERT [58]. Image-based models, consisting of self-distillation with no labels
(DINO) [59] and masked autoencoders (MAEs) [60], are trained on image datasets
and can be used for image recognition, object detection, and segmentation. Multimodal models are capable of handling multiple data types and are often trained to
align different modalities for tasks that benefit from cross-modal understanding.
Example models include contrastive language-image pretraining (CLIP) [61] and the
large language and vision assistant (LLaVA) [62]. Generative AI refers to AI
systems capable of generating new content akin to the data they were trained on. By
modeling the underlying data distribution, these systems produce novel samples
resembling the training data. Generative AI often builds upon foundation models.
For instance, an LLM could be initially trained to understand and represent text
5-12
Artificial Intelligence in Adaptive Radiation Therapy
data and be further fine-tuned or extended to perform generative tasks such as
human language generation.
5.4 The application of big data in radiation oncology
5.4.1 Medical image segmentation
The applications of big data in medical image segmentation have shown great
advancements in recent years [63]. In radiation oncology, contours of targets and
OARs are required to generate treatment plans. However, manual segmentation is
not only time-consuming but also prone to inter-observer variations. The slow
segmentation process also decreases the efficiency of the online adaptive workflow
and potentially affects the efficacy of adaptive plans. Therefore, it is critical to
develop fast automatic segmentation methods that can help relieve clinical burden,
standardize segmentation performance, and accelerate adaptive workflow.
Numerous pairs of images and contours in radiation oncology provide a rich
source for training AI-based segmentation models, which have achieved state-of-theart performance in medical image segmentation. Notably, the U-Net [39], introduced by Ronneberger et al in 2015, demonstrated the potential to generate accurate
segmentations from input image slices. U-Net consists of an encoder and a decoder
connected with skip connections. This encoder–decoder architecture has been
incorporated as the backbone of many advanced AI-based medical image segmentation models such as AnatomyNet [64] and nn-U-Net [65].
AI-based models have been trained to generate OAR segmentation based on singleor multi-modal images.As a CT scanis routinelyacquiredfor treatment planningin the
conventional image-guided radiotherapy workflow, numerous models have been
trained with pairs of CT images and OAR contours. For example, a WBNet proposed
by Chen et al was trained using 505 CT scans of the head and neck, thorax, abdomen,
and pelvis, and could accurately delineate 50 OARs [66]. Additionally, commercial AIbased segmentation tools such as Limbus Contour (Limbus AI, Canada) and Contour
ProtégéAI (MIM Software Inc, USA) have been integrated into many hospitals
worldwide for automatic OAR segmentation on CT scans [67, 68]. Many studies also
focused on OAR segmentation based on MRI [67, 69] or ultrasound images [70]dueto
growing interest in using other image modalities for treatment guidance. The HaN-Seg
dataset [71] contains both CT, MRI, and OAR contours of 56 head and neck cancer
patientsand could be trained to train a multi-modalsegmentationmodel. However,CT
andMRI scansof the same patientsarenot always available, which makesit challenging
to acquire sufficient training data. Valindria et al demonstrated the benefits of multimodel learning for multi-organ segmentation based on unpaired CT and MRI scans
acquired from different subjects [72].
The combination of big data and AI makes it possible to automatically generate
accurate tumor and target contours. Many studies have shown that AI models,
trained with pairs of clinical target volume (CTV) contours and CT scans, achieved
promising performance in delineating CTV for patients with head and neck cancer
[73], breast cancer [74], and cervical cancer [75]. Additionally, tumor segmentation
sometimes involves multi-modal images. MRI provides superior soft tissue contrast
5-13
Artificial Intelligence in Adaptive Radiation Therapy
compared with CT and has been increasingly used to delineate tumors in the brain,
prostate, and abdomen. For instance, multi-modal MR images and manually drawn
tumor contours from 180 glioblastoma patients were used to train a 3D multipath
DenseNet that achieved accurate glioblastoma tumor segmentation [76].
Additionally, a commercial AI tool uses both CT and MRI scans to accurately
detect and delineate metastatic brain tumors [77, 78]. On the other hand, PET
images provide a functional assessment of a tumor and have become the modality of
choice for delineating targets in patients with lung cancer, head and neck cancer, and
esophageal cancer [77]. Groendahl et al showed that CNNs trained with multimodal PET-CT images achieved better performance in delineating the gross tumor
volume of head and neck images compared to the model trained with PET or CT
images [79]. The modality-speci fi c segmentation network (MoSNet) based on PETCT images outperforms the state-of-the-art lung tumor segmentation models [80].
MedSAM, a foundation model trained with more than one million pairs of
medical images and contours, demonstrates significant potential to enable universal
medical image segmentation [81]. Public datasets for OAR segmentation include
multi-organ abdominal CT reference standard segmentations [82], segmentation of
thoracic organs-at-risk (SegTHOR) [83], and medical segmentation decathlon [84].
Public datasets for tumor segmentation include brain tumor segmentation (BraTS)
[85], liver tumor segmentation (LiTS) [86], and DeepLesion [87]. More public
datasets on image–contour pairs could be found in The Cancer Imaging Archive
(TCIA) [88] and The Cancer Genome Atlas (TCGA) [89].
5.4.2 Automatic treatment planning
Intensity modulated radiation therapy (IMRT) and volumetric modulated arc
therapy (VMAT) enable precise radiation delivery to tumors while sparing OARs.
However, the manual treatment planning process normally involves multiple rounds
of inverse optimization by trial and error to generate clinically acceptable plans.
This manual workflow is very time-consuming, requires significant human expertise,
and may not always yield optimal treatment plans. Online adaptive radiation
therapy, which allows better inter-fraction organ motion management compared
with non-adaptive treatment, has drawn significant clinical interest. However, this
requires decreasing planning time to a few minutes.
Automatictreatmentplanningcould be achievedby trainingAI modelsto predict3D
dose distribution,predict fluence maps,or automatethe hyperparameter tuningprocess
during inverse optimization. U-Net and its variants could be trained to predict clinical
dose distributions based on the input of CT and/or contours of targets and OARs.
Promising results have been achieved for cancer sites including the head and neck [90],
breast [91], and prostate [92]. The predicted dose distributions or the corresponding
DVHs could be fed into the dose optimization engine to generate deliverable plans.
However, as historical plans used for model training may not always be optimal, plans
generated using this method may be suboptimal. Instead of clinical dose distribution,
these models can also be trained to predict Pareto optimal dose distributions based on
5-14
Artificial Intelligence in Adaptive Radiation Therapy
the additional input of user-specified contourweights [93]orDVHs[94].This allowsthe
user to search for the optimal trade-off dose in real-time.
Fluence map prediction models could help skip the optimization step and
potentially speed up the automatic treatment planning process. These models can
be trained to take the AI-predicted dose distributions as their input and predict
fluence maps [95], predict fluence maps directly based on contours [96], or
simultaneously predict dose distribution and fluence maps based on CT and
contours [97]. On the other hand, RL agents could be trained to observe
intermediate plan DVHs and take action to adjust planning parameters for
generating the plan with the maximized quality score [98].
5.4.3 Treatment response prediction
Precision oncology represents a paradigm shift in cancer treatment, leveraging the
power of big data to tailor therapy to individual patients. This approach is grounded
in the understanding that cancer is not a singular disease but rather a highly
heterogeneous disease with unique genomic and phenotypic characteristics that vary
among individual patients [99]. AI models, trained on extensive datasets such as
patient demographics, genetic test results, medical images, and treatment plan dose
distribution, can predict how individual patients will respond to specific treatment
regimens. This could potentially allow us to improve patient outcomes and reduce
the toxicity and burden of unnecessary treatments.
Many efforts have been made to utilize large-scale omics data to predict radiation
therapy response. For example, genomic classifiers have been developed to stratify
local recurrence risk after radiation therapy in breast cancer patients and identify those
who would benefit from radiation therapy [100–102]. Radiomic features extracted
from medical images have been found to correlate with genomic biomarkers and
achieve promising performance in predicting treatment responses for patients with
lung cancer, rectal cancer, etc [103]. Moreover, the integration of multi-omics data
holds the potential to further enhance model prediction accuracy [104–106].
5.4.4 Quality assurance and patient safety
Ensuring QA and patient safety is a top priority in radiation oncology. The data
relevant to this domain come from a variety of sources, including QA procedures and
incident learning systems (ILSs) [107, 108]. Data generated from QA procedures
include machine-specific data and patient-specific data. Machine-specific data can be
from various categories, including dosimetric, mechanical, imaging, and respiratory
gating. These data are normally obtained by measurements or checks during machine
and TPS acceptance/commissioning/upgrading and routine daily/monthly/annual
QA. Patient-specific data may include patient-specific QA conducted using either an
array detector or electronic portal imaging device [109], treatment plan review [110],
and machine delivery log files [119]. The ILS within radiation oncology [111] serves as
a pivotal platform for healthcare professionals to both report and glean insights from
incidents and near-misses tied to cancer treatment. Through meticulous data
collection and analysis, ILS fosters a continuous drive toward enhancing quality
5-15
Artificial Intelligence in Adaptive Radiation Therapy
within radiation oncology practices. It empowers healthcare providers to discern
patterns, pinpoint root causes, and highlight areas ripe for improvement in patient
safety and treatment delivery. Radiation Oncology ILS (RO-ILS) [112], jointly
sponsored by ASTRO and AAPM, spearheads the establishment of a national
database tasked with housing incidents, near-misses, and unsafe conditions reported
across various institutions. Advocating for a collaborative approach, RO-ILS
encourages sharing and learning from events submitted nationwide. This initiative
promotes a culture of safety and collaboration within the radiation oncology
community, ultimately culminating in elevated patient care outcomes.
With a wealth of data stemming from both QA and ILS, AI emerges as a
powerful ally in several key areas. It can adeptly discern trends in QA task
performance, identifying outliers that warrant closer examination. Furthermore,
AI streamlines QA workflows by automating tasks such as plan review and imaging
QA, thereby enhancing both efficiency and precision. By leveraging predictive
analytics, AI forecasts which aspects are more prone to failure, allowing for
proactive intervention and resource allocation [113].
5.4.5 Clinical decision support
Big data and AI tools hold significant promise in aiding healthcare professionals in
decision-making processes by furnishing them with relevant, timely, and evidencebased information [114]. Clinical decision support systems, underpinned by big data
and AI, can amalgamate patient-specific data and medical knowledge to offer
recommendations for diagnosis, treatment, and patient care. In the realm of
medicine, there is a burgeoning interest in leveraging foundation models across
various data sources, including EHRs, clinical guidelines, medical literature,
medical imaging, biological sequences, and molecular profiles [115, 116].
EHRs encompass both structured data—such as billing codes, demographics, and
medications—and unstructured data, including medical notes, radiology reports, and
lab reports. Challenges include irregularities such as ambiguous jargon and nonstandard phrasal structure, necessitating domain expertise. Some example datasets
include the deidentified clinical acronym sense inventory (CASI) [117] which contains
snippets of clinical notes across specialties in four University of Minnesota-affiliated
hospitals. The MIMIC-III criticalcare databasecontainsapproximately 2 million notes
written between 2001 and 2012 in the ICU of Beth Israel Deaconess Medical Center
[118, 119]. Foundation models have demonstrated capabilities such as extracting drug
names from medical reports [120], responding to patient queries [121], summarizing
clinical text [122], and predicting clinical needs based on clinical notes [123].
The wealth of information found in the scientific literature, clinical guidelines,
and knowledge bases such as NCT guidelines and PubMed identifiers is invaluable
for clinical decision-making. However, manually extracting predictive information
from these sources can be time-consuming and requires extensive expertise. LLMs
offer significant promise in this regard. For instance, a few-shot prediction model
can leverage LLM representations, which encapsulate prior knowledge in scientific
literature, to predict drug pair synergy in rare tissues with limited data [124].
5-16