Добавил:
Sekretar
kiopkiopkiop18@yandex.ru
t.me/Prokururor I Вовсе не секретарь, но почту проверяю
Опубликованный материал нарушает ваши авторские права? Сообщите нам.
Вуз:
Предмет:
Файл:Ординатура / Хирургия / Библиотека им академика М.И. Перельмана / Книга_5525_Библиотеки_им_академика_М_И_Перельмана.pdf
Artificial Intelligence in Adaptive Radiation Therapy
10.1.1 Overview of chapter content
This chapter focuses on cutting-edge applications of deep learning applicable to AIassisted dose prediction and re-planning in ART. To establish a baseline for
understanding the advantages of deep learning methods, we begin with a brief
mention of traditional knowledge-based techniques that rely on rule-based algorithms and classical machine learning. Then, we explore the landscape of deep
learning-based dose prediction, including convolutional neural networks (CNNs)
that have shown remarkable promise in fast volumetric dose prediction.
The discussion will also cover challenges inherent to using modern AI solutions in
the prediction of clinically acceptable dose volumes, including the need for standardized datasets and evaluation metrics, the impact of data quality, model
interpretability, and the potential gaps between the predicted dose distributions
and a truly optimal and personalized result for a patient.
Following the sections centered on dose prediction, we will discuss exciting
possibilities for integrating AI in the re-planning stage of ART workflows,
emphasizing the role of AI in enhancing planning efficiency. The chapter will
conclude with a forward-looking perspective on the next steps in AI-assisted dose
prediction and re-planning, highlighting potential avenues for innovation in this
rapidly evolving field.
10.2 The landscape of AI-assisted dose prediction
AI-assisted dose prediction holds immense potential for streamlining radiotherapy
treatment planning by shortening the time needed for a planner and a physician to
iteratively arrive at a high-quality plan. Indeed, the aim of dose prediction is to
generate a dose estimation that closely mimics the desired plan dose, i.e. what a
skilled planning team would produce manually. As shown in figure 10.2, the
Figure 10.2. Hypothetical workflow showing two pathways for using AI-assisted dose prediction to generate
the machine parameters (or instructions) needed to deliver the intended dose. After dose prediction, the
resulting dose information can help guide planners in their search for a high-quality treatment or serve as
inputs to an automatic-planning platform, where, for instance, it can help define optimization objectives.
10-3
Artificial Intelligence in Adaptive Radiation Therapy
predicted dose can then serve as a valuable tool for guiding both physician directives
and plan generation, ultimately reducing the time required for subsequent steps in
the planning workflow.
In recent years, the landscape of dose prediction research has undergone a
significant shift from traditional knowledge-based approaches, including those using
classical machine learning, to deep learning methods [3, 12, 14–16]. This section
navigates this shift by first giving a brief overview of traditional dose prediction
techniques followed by a look at deep learning methods, focusing on characteristics
of this technology relevant to dose prediction for ART.
10.2.1 Traditional machine learning for dose prediction
Before the widespread use of traditional machine learning techniques, atlas-based
methods and statistical models represented the most common knowledge-based
strategies for developing high-quality radiation treatment plans with optimal dose
distributions [12, 14, 17]. These rule-based approaches leveraged existing highquality clinical data to correlate characteristics—such as CT-derived geometric
features—of new and previously treated patients. From the correlations, the systems
could then recommend possible dosimetric outcomes for a new patient such as dose–
volume metrics and dose–volume histogram (DVH) curves.
Early efforts using machine learning often followed a similar philosophy, relying on
hand-engineered features as inputs. Nevertheless, these knowledge-based tools leveraged the advantages of machine learning methods, such as the ability to readily model
complex non-linear relationships, to achieve superior performance. Today, some
commercial tools such as RapidPlan (Varian Medical Systems, Palo Alto, CA, USA),
continue to rely on classical machine learning techniques such as support vector
machines, random forests, and shallow artificial neural networks trained to predict
achievable DVH curves or voxel-wise dose values for new patients [12, 13, 18]. In the
case of RapidPlan, the resulting DVH curves can help define optimization objectives
that guide the properties of the resulting treatment plan.
Knowledge-based methods, both rule-based or those using classical machine
learning, have served to demonstrate the potential of dose prediction and automatic
planning in radiotherapy, with some groups integrating them in end-to-end planning
pipelines [19]. Commonly reported benefits from the adoption of knowledge-based
methods include enhancements in plan quality, improved planning consistency, and
reductions in planning times, sometimes exceeding 50% [12, 20–22].
Despite their well-documented advantages, these traditional methods have some
limitations. For instance, they rely on a limited set of predefined hand-engineered
features, which may fail to capture enough complexity in the data to ensure
accuracy, in particular for patients with atypical geometries. Furthermore, many
of these methods estimate DVH curves, which lack spatial information and suffer
from non-uniqueness, as different volumetric dose distributions can produce
identical DVH curves for an organ. Finally, the reported planning times with these
tools might exceed what is demanded by online and real-time ART.
10-4
Artificial Intelligence in Adaptive Radiation Therapy
10.2.2 Deep learning-based dose prediction
Deep learning methods have emerged as powerful tools for predicting volumetric
dose distributions in radiation therapy planning [2, 12, 14, 17 ]. The core strength of
deep learning lies in its ability to automatically discover intricate mappings between
input data (e.g. patient anatomy) and the intended output (e.g. dose distributions)
[23]. This ability is particularly important when working with complex datasets such
as those used in radiation therapy. Moreover, deep learning benefits from the highly
optimized and freely available frameworks used for the creation and deployment of
models [24, 25]. In combination with powerful graphical processing units (GPUs),
models built with these tools can output predictions for entire 3D dose volumes in
seconds [2], a speed beneficial to online and real-time ART.
In the last four years, the research momentum behind dose prediction has proved
both steady and strong, with about 20–30 scholarly articles published annually
including 19 on deep learning-based methods in 2020 alone. While the majority of
published works have focused on the prostate [26–34], a common site for proof-ofconcept studies, and the head and neck [35–48], considered a highly challenging site
to convincingly demonstrate a model’s capability, researchers have also investigated
other sites, including the lungs [49, 50] cervix [38, 51, 52], and breast [53–55],
showcasing the versatility of deep learning methods.
Given the breadth of the research in AI-assisted dose prediction, it is challenging—
and outside of our scope—to fully capture the diversity of the existing ideas and
implementations. Some of this diversity was captured by a single initiative for the
advancement of dose prediction techniques, the OpenKBP Grand Challenge hosted
by the American Association of Physicists in Medicine (AAPM) [56]. This competition, the first of its kind, not only saw a strong international involvement, including
195 participants from 28 countries, but also resulted in head-to-head comparisons of
28 unique prediction methods all working with the same data and evaluated equally.
The top-performing models, all based on deep learning, highlighted the potential for
deep learning to revolutionize dose prediction. The challenge also underscored the
importance of model comparison using standardized datasets and evaluation metrics,
which enables researchers to objectively evaluate the performance of different dose
prediction methods and identify areas for improvement. This collaborative approach
to model development and validation is crucial for advancing the field of AI-assisted
dose prediction and ultimately improving patient care in radiation therapy.
The versatility of deep learning for dose prediction is demonstrated in figure 10.3,
which shows results for example test patients (i.e. left out of the training data),
including a head and neck patient treated with volumetric modulated arc therapy
(VMAT), a breast cancer patient treated with VMAT, a cervix case treated with
VMAT, a head and neck patient treated with scanning proton beams, and a prostate
case treated with scanning proton beams. In all cases, predictions were made with
models trained using the architecture described in Gronberg et al, a variation of the
U-Net architecture [57] that ranked second in the OpenKBP Grand Challenge
competition [56, 58
]. Thus, this figure highlights how a single robust architecture can
accurately handle multiple sites and modalities.
10-5
Artificial Intelligence in Adaptive Radiation Therapy
Figure 10.3. The left panel shows axial slices comparing the ground truth (GT) dose to that predicted by a
deep learning model (DL) for volumetric modulated arc therapy (VMAT) and intensity-modulated proton
therapy (IMPT) cases. The same architecture was used for all cases shown. The right column displays the
corresponding dose–volume histograms of each patient comparing the dose predicted (dashed) and ground
truth (solid) curves. STV = scanning target volume; PTV = planning target volume; CTV = clinical target
volume.
While many of the successful deep learning-based dose prediction methods could
benefit ART pipelines, for instance, by being used in the fashion illustrated by
figure 10.2, only a couple have explicitly targeted ART. Recently, a method called
intentional deep overfit learning (IDOL) was proposed for deep learning strategies
targeting ART [42, 59, 60]. For dose prediction, the use of IDOL to generate
10-6
Artificial Intelligence in Adaptive Radiation Therapy
patient-specific models resulted in significant improvements compared to results
from training the same architectures in a more conventional, population-based style
[42, 60]. The work of these researchers marks an attempt to predict a truly
personalized dose distribution. This is a promising direction, since leveraging the
speed of deep learning with methods that enhance the personalization of the
predictions can unlock the true potential of AI-assisted dose prediction for ART.
10.2.2.1 Deep learning architectures used in dose prediction
U-Net [57, 61] has emerged as the most widely adopted deep learning architecture
for dose prediction tasks. Nguyen et al [62] and Kearney et al [27] were among the
pioneering researchers to demonstrate the effectiveness of U-Nets in this domain.
Since then, numerous variations of U-Net have been explored for dose prediction
across various anatomical sites, each offering unique advantages and trade-offs.
Figure 10.4 illustrates a generalized U-Net model, similar to those employed in
dose prediction. The architecture consists of four resolution levels, represented by
three sets of gray blocks with varying sizes—linked by ‘skip connections’—and a
central gray block representing the ‘bottleneck’ or lowest resolution level.
These blocks, referred to here as convolutional blocks, typically apply a sequence
of operations involving convolutions, normalization, and activation functions.
Figure 10.4. Schematic representation of U-Net architecture variations, composed of modular elements such
as ‘convolutional’ blocks that operate on data by applying convolution, activation, and normalization
operations. Downsampling, up-sampling, and skip connections, both classic and attention gated, are also
illustrated as additional modules. Arrows indicate the direction of data flow, with dashed lines indicating an
optional path used when a secondary deep learning architecture is available.
10-7
Artificial Intelligence in Adaptive Radiation Therapy
The model’ s encoder path (left side) progressively reduces the spatial resolution of
the feature maps through downsampling operations, such as max pooling or strided
convolutions. Conversely, the decoder path (right side) gradually recovers the spatial
resolution using up-sampling techniques, such as nearest-neighbor interpolation or
transpose convolutions.
Skip connections play a crucial role in U-Net architectures, allowing information
to flow directly from the encoder to the decoder at corresponding resolution levels.
These connections can be implemented through simple concatenation operations or,
for example, using a more sophisticated attention-gated mechanisms [63–65].
Attention-gated skip connections enable the model to selectively focus on relevant
features from the encoder, with the intention of enhancing the accuracy of the
predicted outputs.
The components of convolutional blocks can significantly impact the model’s
performance and computational efficiency. Sources of variation in block types and
operations used in dose prediction models include:
1. ResNet-like blocks [27, 66]: These blocks incorporate skip connections, as
proposed in the ResNet architecture [67], allowing for deeper networks and
improved gradient flow.
2. Dense blocks [45]: Inspired by the DenseNet architecture [68], these blocks
utilize a dense set of concatenation operations to propagate the outputs of
convolutional layers, effectively reusing features and increasing the cumulative number of forward-propagated features without increasing the number
of trainable parameters.
3. Dilated convolutions [58, 69]: These convolutions expand the receptive field of
the model without sacrificing resolution in the feature maps, enabling the
capture of broader contextual information [70, 71].
4. Activation and normalization functions: Researchers have experimented with
various activation functions, such as replacing the commonly used rectified
linear unit (ReLU) [72] with Mish [73], and normalization techniques, such
as substituting batch normalization [74] with group normalization [75], to
enhance model performance and stability [47].
In some implementations, the outputs of U-Net serve as inputs to a secondary
model, as indicated in figure 10.4. When the secondary model acts as a discriminator, the resulting architecture becomes trainable with an adversarial scheme. For
example, the discriminator can learn to identify outputs deviating from the
distribution of high-quality clinical dose volumes, providing an additional quality
check akin to human oversight. For this reason, such architectures, which fall in the
category of generative adversarial networks (or GANs), have been proposed to
improve the accuracy and realism of dose predictions [30, 63, 76,
77].
Alternatively, a pre-trained CNN network can help extract features from both the
clinical and predicted dose, which can be compared in the loss function to introduce
additional penalties [47]. The extracted features are based on the trained—and thus
task specific—filters of the pre-trained model, e.g. a ResNet 3D trained for video
10-8
Artificial Intelligence in Adaptive Radiation Therapy
classification [78], and should produce matching results after operating on the
clinical and predicted dose volumes if both were identical.
The third possibility illustrated in figure 10.4 involves using a second U-Net in a
cascaded manner to further refine the predictions from the first model [46 ]. While
computationally and resource intensive, the cascaded U-Net approach demonstrated superior performance in the OpenKBP Grand Challenge [56]. This technique
has also resulted in superior performance for segmentation tasks [79].
The search for an optimal model design for dose prediction remains an empirical
process, requiring extensive experimentation and domain expertise. The field of deep
learning is highly dynamic, with new and increasingly capable methods being
proposed frequently. This rapid progress is reflected in the diverse and complex
landscape of architectural designs for dose prediction, which can prove challenging
for new practitioners to navigate. Fortunately, well-designed, robust architectures,
such as the top performers from the OpenKBP Grand Challenge, serve as excellent
starting points for researchers entering the field. These proven models demonstrate
remarkable versatility, accurately predicting dose distributions across different
anatomical sites and even treatment modalities with minimal modifications. As
exemplified by the results in figure 10.3, these architectures showcase the promise of
deep learning in dose prediction and provide a solid foundation for further
advancements in the field.
As research in this field continues to advance, we can expect further refinements
and innovations in deep learning architectures tailored for dose prediction. These
advancements will likely focus on improving prediction accuracy, computational
efficiency, and adaptability to various clinical scenarios, ultimately enhancing the
quality and efficiency of radiotherapy treatment planning.
10.2.2.2 Training and evaluation strategies
Just as there are numerous variations in architectures, the methodology for training
and evaluating dose prediction models often differs between studies. One such
difference involves dividing the data into random patches (or subregions) as opposed
to using full volumes for the inputs. The patch-based approach can reduce the
resource requirements while augmenting the effective number of inputs to the model
[45]. On the other hand, using full volumes may better capture global contextual
information and help a model uncover the underlying physics governing the dose
deposition [66].
Input channels play a crucial role in enabling the model to learn an effective
mapping between inputs and outputs. Most studies use a CT scan along with a set of
contours marking the position of relevant OARs, thus providing anatomical context
to the model. Additionally, an input channel with prescription information is often
included, typically constructed using the target volumes to communicate the
maximum prescribed dose at each voxel in the final dose distribution.
Researchers have also explored incorporating additional inputs to improve
prediction accuracy. Some authors have investigated using distance information,
e.g. the distance between OARs and the surface of target volumes, similar to the
inputs of some traditional knowledge-based techniques [80, 81]. Furthermore, beam
10-9
Artificial Intelligence in Adaptive Radiation Therapy
geometry information has been used to enhance the accuracy of model predictions,
particularly when the training data reflects the use of heterogeneous beam arrangements [49, 76, 82, 83]. This information can be provided as a contour, such as that of
an OAR, with nonzero values assigned to voxels with a high probability of receiving
dose due to their proximity to the beam path. Alternatively, a fast dose calculation
method can be used to produce an initial guess of an unmodulated dose distribution
that communicates the desired beam arrangement [49].
The choice of loss function is critical for the success of a dose prediction model
[84]. Most applications of dose prediction employ a mean squared error (MSE) or
mean absolute error (MAE). In some cases, these popular choices are combined with
terms that apply weighted penalties in the predictions inside regions of interest [58]
or regularization techniques. Some researchers have also investigated the benefits of
loss functions incorporating terms derived from DVH metrics [52] or using
approximations of the DVH curves to impart domain-specific knowledge to the
training [30, 66, 85]. Other types of loss functions, such as adversarial loss functions,
have also been explored in the literature [30, 38, 63, 84].
An understanding of the generalizability and limitations of dose prediction
methods can help us learn how to best translate t hem into clinical settings. To
address this, some authors have explored training techniques that investigate the
generalizability of models. For instance, studies have examined how pre-trained
models performed with data from different anatomical sites [86] an d other
institutions [80]. This type of work is important, as the ability to use models
across different sites and institutions could facilitate the adoption of dose
prediction.
To ensure the reliability and robustness of dose prediction models, thorough
evaluation using appropriate metrics and validation strategies is essential.
Commonly reported metrics include mean absolute error (or dose score), errors
in the mean and maximum dose received in regions of interest, errors in the
radiation dose delivered to a specific percentage of the volume of relevant
structures, errors in the conformity index, gamma passing rate evaluations, and
the Dice similarity coefficient to quantify the overlap between isodose surfaces of
the predicted and actual dose. In addit ion to these, Babier et al prop osed the DVH
score, which quantifies the average error in some relevant DVH metrics for both
OARs and target volumes [56]. Although the majority of studies include one or
more of these metrics, a standardized strategy to eval uate models has yet to be
defined.
10.2.3 Challenges in AI-assisted dose prediction
A potential limitation of AI-assisted dose prediction involves the use of data from
past plans to train models expected to perform well in today’s clinics. This could
present serious issues especially when past plans do not reflect state-of-the-art
practices. Thus, as treatment techniques evolve over time, models will likely require
periodic revisions to ensure that their performance aligns with the latest standards.
Some techniques in AI, including transfer learning and online learning [87
10-10
, 88], can
Artificial Intelligence in Adaptive Radiation Therapy
help alleviate or overcome circumstances when sudden changes in standards makes a
model unreliable.
Another challenge lies in clearly evaluating the quality—in a clinical sense—of
the predicted dose [89]. Many dose prediction models are trained on diverse datasets
produced by several planners and, thus, varying in quality. The output of models
trained on such data may produce suboptimal results representing the average
quality of the training data. This hypothesis should be tested with clinically relevant
methods for quantifying the quality of dose distributions.
Furthermore, directly comparing the outputs from dose prediction models with
delivered dose volumes, as it is often done, may not adequately convey the clinical
utility of the predicted dose distributions. A more informative approach could be to
first use the predicted dose to generate a deliverable dose, e.g. through inverse
optimization, and then compare how well each deliverable dose volume—predicted
and clinical—satisfy clinical directives [39].
The success of deep learning models for dose prediction in clinical settings
remains mostly unexplored, with some studies indicating that tools that perform
successfully during initial testing might not achieve the same degree of success when
deployed in the clinic [90]. To reduce risks and ensure the safe deployment of AIassisted tools in the clinic, systems designed to identify potential errors and quantify
uncertainty are essential. Nguyen et al proposed methods to quantify the uncertainty
of predictions, providing a feedback mechanism that can reveal to users when and
where a model lacks confidence [43]. Such techniques can enhance the interpretability of results, a known challenge in the adoption of AI tools.
Users of AI-assisted tools also run the risk of becoming over-reliant on deep
learning algorithms, which may reduce the amount of quality control checks
performed and potentially lower treatment quality. This is an observed consequence
of the use of automation called automation bias [91]. The establishment of clear
guidelines for integrating AI-assisted dose prediction into clinical workflows could
help reduce this risk.
The field of AI-assisted dose prediction can also benefit from the development of
several standardized datasets and clearly defined evaluation metrics to quantify
model effectiveness. The performance of deep learning models is generally proportional to the amount of training data available. In the context of dose prediction,
limited data size and high variability in the training data both negatively impact
performance. On the other hand, a lack of variability, even for large data volumes,
can lead to biased dose prediction tools that produce errors when applied to patients
with properties outside the training data distribution. Collaboration between
institutions to share knowledge and data is also essential to mitigate problems
such as bias and to leverage the data-driven performance of deep learning. However,
such collaborations present significant challenges for the medical community, as
they can impose resource and time requirements that are difficult for busy clinics to
meet. As new technologies become available, tools to facilitate data sharing and
their use in data-driven technologies will likely lead to significant improvements in
AI-assisted dose prediction.
10-11
Artificial Intelligence in Adaptive Radiation Therapy
10.3 Re-planning workflows powered by AI
In ART, when the evaluation of a scheduled plan indicates suboptimal quality, such
as demonstrating a high risk for loss in target coverage or unnecessary dose to
OARs, re-planning is triggered. Re-planning aims to generate a new plan that
enhances both target coverage and normal tissue sparing compared to the scheduled
plan, as illustrated in figure 10.5, for a hypothetical online ART pipeline. However,
re-planning comes with an undesirable consequence: the potential to significantly
increase the overall complexity of the radiotherapy treatment [1, 3, 4, 92, 93]. This
increased complexity poses a particular challenge in online ART, where the time
window for re-planning is extremely limited, ideally in the order of a few minutes.
Under these stringent time constraints, substantial human involvement in plan
generation may be limited or even prohibited. Therefore, rapid re-planning
techniques are not merely bene fi cial; they are a technological necessity for online
ART [92, 94]. Notably, the development of such techniques not only addresses the
challenges of online ART but also has the potential to benefit other forms of
adaptive radiotherapy, such as offline ART, by streamlining the re-planning process
and reducing the overall workload in a busy clinic.
AI, particularly deep learning, has the potential to accelerate or automate several
steps essential for efficient and effective re-planning, including accurate segmentation, dose prediction, plan optimization, and quality assurance. Thus, integrating
deep learning techniques into re-planning pipelines could significantly enhance both
online and offline ART workflows. Currently, commercially available tools for
ART, such as Varian’s Ethos system, are leveraging deep learning and classical
machine learning to meet the challenges of re-planning in online ART [94, 95]. These
systems are actively contributing to the growing body of evidence supporting the
benefits of automation and AI in radiation therapy. In this section, we explore four
exciting applications of deep learning in radiation therapy and their potential roles
in streamlining the re-planning process for both online and offline ART.
Figure 10.5. Flowchart illustrating the decision-making process in a hypothetical online ART pipeline during
an intermediate fraction. When new images (e.g. CBCT) are acquired, the process first determines if adaptation
is triggered based on observed anatomical changes. If triggered, a deep learning-based segmentation for the
new images begins, and the results are subsequently evaluated. The output of the segmentation step helps in
the estimation of the expected dose to the patient if the previous plan was followed. Once the daily dose under
the scheduled plan is determined and evaluated, two outcomes become possible: delivering the scheduled plan
if it is deemed acceptable or starting re-planning to generate a new plan that satisfies clinical constraints for
target coverage and organs-at-risk sparing.
10-12