Добавил:
Sekretar
kiopkiopkiop18@yandex.ru
t.me/Prokururor I Вовсе не секретарь, но почту проверяю
Опубликованный материал нарушает ваши авторские права? Сообщите нам.
Вуз:
Предмет:
Файл:Ординатура / Хирургия / Библиотека им академика М.И. Перельмана / Книга_5525_Библиотеки_им_академика_М_И_Перельмана.pdf
Artificial Intelligence in Adaptive Radiation Therapy
mimics human visual perception by considering similar features rather than solely
intensities, was implemented in three different versions of increasing complexity: on
a single convolutional layer, on multiple layers with uniform weights, and on
multiple layers with different weights assigning greater importance to layers yielding
lower MAE [28].
8.2.3 Generative adversarial networks
A generative adversarial network (GAN) comprises a generative network and a
discriminative network that undergo simultaneous training. The generative network
is tasked with producing synthetic images, while the discriminative network learns to
distinguish between real and synthetic images. The overarching training objective of
a GAN is to enable the generative network to generate synthetic images of utmost
realism, deceiving the discriminator. In tandem, the discriminative network strives
to accurately classify images as either real or synthetic. The training process involves
adversarial competition between these networks until equilibrium is achieved. In a
production setting, the trained generative network is applied to generate synthetic
images for new inputs.
Similar to autoencoders, GANs have found application in early publications on
medical image synthesis using deep learning. In a study by Nie et al a fully
convolutional autoencoder, devoid of fully connected layers, served as the generative
network, while a standard AE was employed for the discriminative network [29].
Both networks utilized a binary cross-entropy loss function. Notably, the discriminative network’s loss aimed to minimize the difference between assigned labels and
ground truth conventionally. In contrast, the generative network’s loss was
formulated to maximize the error of the discriminative network by minimizing the
disparity between the labels assigned by the discriminative network and an incorrect
label. Given that the study employed a patch-to-patch training approach, limiting
the contextual information available in training samples, an auto-context model was
introduced. This model integrates low-level and contextual information derived
from low-level appearance features to refine the synthesis results.
Various iterations of GANs have been explored, each tailored to specific tasks.
Emami et al introduced the conditional GAN (cGAN) in their work on CT synthesis
from MR [30]. In contrast to the traditional unconditional GAN, both the
generative and discriminative networks of the cGAN are exposed to input images,
such as MR images in CT synthesis from MR. This design involves conditioning the
loss function of the discriminator on the input images, proving to be particularly
effective for image-to-image translation tasks [31].
Liang et al incorporated CycleGAN into their study on synthetic CT generation
based on cone beam computed tomography (CBCT) [32]. The CycleGAN features
two generators: a CBCT/CT generator and a CT/CBCT generator, alongside two
discriminators: a real CT/synthetic CT discriminator and a real CBCT/synthetic
CBCT discriminator. The first cycle involves transforming the input CBCT into a
synthesized CT using the CBCT/CT generator, and then regenerating a cycle CBCT
from the synthetic CT using the CT/CBCT generator. This cycle CBCT is compared
8-5
Artificial Intelligence in Adaptive Radiation Therapy
to the original input CBCT, generating CBCT cycle consistency loss.
Simultaneously, a real CT-synthetic CT discriminator distinguishes between the
real and synthetic CT to produce CT adversarial loss, akin to a standard GAN. To
enforce a one-to-one mapping between CT and CBCT, a second cycle transformation from CT to CBCT is performed, mirroring the first cycle but swapping
the roles of CBCT and CT. CycleGAN introduces an innovative approach by
including an inverse mapping network through cycle consistency loss. This addition
enhances network performance, in particular in scenarios where exact matching
image pairs in training sets are unavailable. CycleGAN exhibits a robust tolerance
for misalignment in paired training datasets, a critical advantage in inter-modality
synthesis where acquiring precisely matched image pairs is challenging. Many
studies employ registration to pair training images, preserving quantitative pixel
values and minimizing baseline geometric discrepancies [33]. This approach enables
the network to concentrate on mapping details and accelerates training while
addressing the inherent difficulties associated with achieving exact image pair
matches.
Diverse architectures of feature extraction blocks have demonstrated efficacy
across various applications. Several studies have highlighted the effectiveness of
autoencoders with residual blocks, particularly in tasks involving image transformation where source and target images exhibit significant similarity, such as the
transition between CT and CBCT. Given the visual resemblance but quantitative
differences between these image pairs, residual blocks, comprising a residual
connection coupled with multiple hidden layers, have been incorporated into the
network to discern and learn the distinctions within these pairs. In this configuration,
an input traverses these hidden layers via the residual connection. Consequently, the
hidden layers work to minimize a residual image between the source and the ground
truth target images, aiming to reduce noise and artifacts. This approach stands in
contrast to standard autoencoder blocks where a feed-forward summation is used.
The residual connection in the blocks effectively enforces the minimization of
differences, contributing to the refinement of image synthesis [33–37].
On the other hand, dense blocks adopt a different strategy by concatenating
outputs from preceding layers rather than utilizing feed-forward summation. This
design choice enables the capture of multi-frequency information, encompassing
both high and low-frequency details. Dense blocks prove particularly advantageous
in scenarios of inter-modality image synthesis, such as MR-to-CT and PET-to-CT
[19, 38–42]. By capturing a broader spectrum of information, dense blocks enhance
the representation of the mapping from the source image modality to the target
image modality, resulting in more comprehensive and accurate synthesis outcomes.
In the realm of GANs, autoencoders and their variants are frequently employed
for both generative and discriminative networks. Notably, Emami et al opted for a
ResNet architecture in their generative network [30]. They modified the architecture
by eliminating fully connected layers and introducing two transposed convolutional
layers following residual blocks, effectively leveraging deconvolution. Meanwhile,
Kim et al innovatively combined the U-Net architecture with a residual training
scheme in their generative network [43]. Olberg et al proposed a deep spatial
8-6
Artificial Intelligence in Adaptive Radiation Therapy
pyramid convolutional framework, incorporating an atrous spatial pyramid pooling
(ASPP) module within a U-Net architecture [44]. This module performs atrous
convolution at multiple rates concurrently, allowing for the exploitation of multiscale features to characterize a single pixel. The encoder in this framework captures
rich multi-scale contextual information, enhancing its ability for image translation.
In contrast to the complexity often found in the generator, the discriminator is
commonly implemented in a more straightforward fashion. Liu et al introduced a
common example comprising a few down-sampling convolutional layers, followed
by a sigmoid activation layer to binarize the output [41].
GANs and their variants incorporate adversarial loss functions alongside image
quality and accuracy loss functions embedded within architectures such as U-Net.
The adversarial term, distinct from the reconstruction term that ensures image
intensity accuracy, gauges the correctness of the discriminator’s decision regarding
real or synthetic images. Commonly used loss functions include binary cross-entropy
or similar sigmoid cross-entropy, as well as negative log-likelihood functions
outlined in the original GAN publication within computer vision. However, training
challenges may arise, including divergence due to vanishing gradients and mode
collapse when the discriminator is optimized for a fixed generator [45]. To mitigate
these issues, Emami et al introduced the use of least-square loss, which has
demonstrated greater stability during training and yields higher quality results
[30]. Another alternative is the Wasserstein distance loss function, known for its
smoother gradient flow and faster convergence. In GANs, providing true or false
labels from the discriminator to the generator may not be sufficient for improvement
and can lead to numerical instability due to vanishing or exploding gradients. To
address this, Ouyang et al employed a feature-matching technique. This involves
specifying a new objective function where the generator aims to synthesize images
that match the expected values of features on intermediate layers of the discriminator, rather than directly maximizing the final output of the discriminator [46].
8.2.4 Denoising diffusion probabilistic model
The denoising diffusion probabilistic model (DDPM) is emerging as one of the most
promising deep generative models, showcasing impressive capabilities in various
tasks such as image generation, superresolution, and image inpainting. This model
operates in two stages: a forward stage that progressively introduces noise, and a
reverse stage aimed at denoising and reconstructing the original sample incrementally. The forward stage comprises multiple small steps, where the image undergoes
slight corruption by Gaussian noise. In the reverse stage, a trained neural network is
employed to estimate the noise at each reverse diffusion step.
In contrast to GANs, DDPM exhibits greater stability during training, displaying
reduced susceptibility to mode collapse and diminished sensitivity to hyperparameters [47]. However, given the relative novelty of DDPM, as of the writing of this
chapter, no publications in peer-reviewed journals have been identified. The current
insights are drawn from pioneering studies available as preprints, providing a
8-7
Artificial Intelligence in Adaptive Radiation Therapy
preliminary glimpse into the capabilities and characteristics of DDPM in comparison to other generative models.
Lyu et al proposed diffusion and score-matching models for conversion between
MRI and CT images.[48] In their study, they explored four distinct sampling
strategies, one of which involved the use of the DDPM. The authors demonstrated
that the CT images generated through their proposed method exhibited favorable
results when compared to those produced by conventional convolutional neural
network and GAN models.
Pan et al introduced an innovative approach in the form of an MRI-to-CT
transformer-based DDPM (MC-DDPM) [49]. This model utilizes diffusion processes in conjunction with a shifted-window transformer network to generate
synthetic CT images from MRI data. Specifically, a shifted-window transformer
V-Net (Swin-V-Net) is employed in the reverse process to denoise noisy CT images
conditioned on MRI input, resulting in the generation of high-quality, noise-free CT
images. The proposed MC-DDPM demonstrated statistically significant improvements across various metrics for both brain and prostate sites when compared to
competing GAN-based networks. However, it is worth noting that the MC-DDPM
is not without limitations. One notable drawback is its heavy computational burden,
leading to longer inference times in comparison to GAN-based methods. Despite
this drawback, the model’s superior performance in terms of image quality metrics
underscores its potential in addressing the MRI-to-CT synthesis task, offering a
valuable alternative to existing approaches. Additionally, the feasibility of conditional DDPM in generating synthetic CT from CBCT has also been demonstrated
by Peng et al and Fu et al, further expanding the applicability of this innovative
approach [50, 51].
8.3 Synthetic CT from CBCT
CBCT and CT image reconstruction share fundamental physics principles related to
x-ray attenuation and back projection. However, their practical implementation in
terms of acquisition and reconstruction, as well as their clinical applications, varies
significantly. Consequently, in the context of this review, they are treated as two
distinct imaging modalities.
CBCT has found extensive application in image-guided radiation therapy
(IGRT), primarily for assessing patient set-up errors and inter-fractional motion.
This is achieved by comparing the displacement of anatomical landmarks relative to
the treatment planning CT images. As adaptive radiation therapy techniques gain
prevalence, more sophisticated applications of CBCT are being explored. These
include challenging tasks such as daily dose estimation and automated contouring,
facilitated by deformable image registration (DIR) with CT imaging acquired
during the simulation process. The evolving role of CBCT in these advanced
applications reflects its increasing significance in improving the precision and
adaptability of radiation therapy procedures.
In contrast to CT scanners that employ fan-shaped x-ray beams with multi-slice
detectors, CBCT utilizes a cone-shaped x-ray beam directed onto a flat panel
8-8
Artificial Intelligence in Adaptive Radiation Therapy
detector. While the flat panel detector provides high spatial resolution and extensive
coverage along the z-axis, it is susceptible to increased scatter signal due to x-ray
scatter generated throughout the entire body volume reaching the detector. The
presence of scatter signals gives rise to pronounced streaking and cupping artifacts in
CBCT images, contributing to substantial quantitative CT errors. These errors pose
challenges in the calibration of the CBCT Hounsfield unit (HU) to electron density,
particularly when utilizing CBCT images for dose calculation. Furthermore, the
compromised image contrast and bone suppression can introduce significant errors
in DIR for the propagation of contours from planning CT to CBCT. The diminished
image quality of CBCT thus limits its utility in advanced quantitative applications
within the realm of radiation therapy.
8.3.1 Noise and artifact reduction
Deep learning-based methods, as listed in table 8.1, have been proposed to address
and enhance CBCT HU values in comparison to CT, leveraging the advantages
offered by image translation techniques. CBCT images are reconstructed from
numerous 2D projections captured from various angles. In certain studies, neural
Table 8.1. Summary of studies on CBCT-based synthetic CT for image quality improvement. (Adapted from
[1]. CC BY 4.0.)
Projection
or image
Network
U-Net Image Pelvis: 20, 5-fold cross
AE Image Lung: 15 training/5 testing PSNR (dB):8.823 Xie et al [53]
U-Net Image Head and neck: 30 training/
CycleGAN Image Brain: 24, leave-one-out
U-Net Projection 1800 projections in training
U-Net Image Head and neck: 40 training/
CycleGAN Image Pelvis: 16 training/4 testing Mean error (HU):
domain
Site, and # of patients in
training/testing
validation
7 validation/7 testing
Pelvis: 6 training/5 testing
Pelvis: 20, leave-one-out
(simulation)/200
validation (simulation)/
360 testing (phantom)
15 testing
Key findings in
synthetic CBCT
quality Author, year
PSNR (dB): 50.9 Kida et al [52]
MAE (HU): 18.98
(head and neck)
42.40 (pelvis)
MAE (HU): 13.0
± 2.2 (brain)
16.1 ± 4.5 (pelvis)
MAE (HU): 17.9
± 5.7
MAE (HU): 49.28 Yuan et al [55]
(2, 14)
Chen et al [25]
Harms et al [33]
Nomura et al [54]
Kida et al [56]
8-9
Artificial Intelligence in Adaptive Radiation Therapy
networks have been applied in the projection domain, focusing on the enhancement
of 2D projection images to improve the overall quality before volume reconstruction. The refined projection images are then employed in the reconstruction process
to generate CBCT image volumes with improved quality. Alternatively, some
methods operate in the image domain, directly taking the reconstructed CBCT
image volumes as input and producing synthetic CT images with enhanced quality
as output.
Projection-domain methods offer advantages when dealing with a larger number
of training 2D projection images (typically >300) compared to image-domain
methods, where the number of training image slices is generally fewer (<100) for
each scan. Moreover, projection-domain methods can be more effective in addressing the unpredictable cupping and streaking artifacts caused by scatter in CBCT
images, as these artifacts are less predictable than those in projection images. Neural
networks find it easier to learn from projection images due to the reduced variability
in per-patient artifactual features compared to the image domain. In the image
domain, where the variability in artifactual features is greater, models are typically
not trained on non-anthropomorphic phantoms because the learned features may
not be applicable across different patient image sets.
Nomura et al demonstrated that features characterizing scatter distribution in
anthropomorphic phantom projections can be successfully learned from nonanthropomorphic phantom projections in the projection domain [54]. This success
is attributed to the neural network effectively capturing the inherent relationship
between scatter distribution and objective thickness in the projection domain. In
contrast, the relationship between scatter artifact and objective appearance is more
intricate in the image domain, making it challenging for neural networks to learn
and generalize easily.
8.3.2 Online dose calculation
Synthetic CTs have shown substantial improvements over original CBCTs in terms
of dosimetric accuracy, bringing them closer to planning CT for photon dose
calculations. Some reviewed literature is summarized in table 8.2. The feasibility of
synthetic CTs in volumetric modulated arc therapy (VMAT) planning has been
investigated across various body sites, assessing select dose–volume histogram
(DVH) metrics, dose, and/or gamma differences. As shown in figure 8.1, it is
demonstrated that significant local dosimetric errors occur in regions with severe
artifacts in original CBCTs. Synthetic CTs effectively mitigate these artifacts and the
associated dosimetric errors [37].
However, it is worth noting that achieving acceptable dosimetric accuracy with
synthetic CT in proton planning is more challenging compared to photon planning.
This is primarily due to the presence of range shifts, which can be as substantial as
5 mm. Managing these range shifts poses a significant challenge in proton planning
with synthetic CTs, highlighting the importance of carefully addressing such
complexities for accurate dose calculations in proton therapy [57–59].
8-10
Artificial Intelligence in Adaptive Radiation Therapy
Table 8.2. Summary of studies on CBCT-based synthetic CT for dose calculation in radiation therapy.
(Adapted from [1]. CC BY 4.0.)
Projection
Network
or image
domain
Site, and # of patients in
training/testing Key findings in dosimetry Author, year
U-Net Projection Pelvis: 15 training/7 testing/
8 evaluation
CycleGAN Image Pelvis: 18 training/7
validation/8 testing
U-Net Image Pelvis: 27 training/7
validation/8 testing
U-Net Image Head and neck: 50 training/
10 validation/10 testing
CycleGAN Image Head and neck: 81 training/
9 validation/20 testing
U-Net Image Head and neck: 33, 3-fold
cross validation
CycleGAN Image Pancreas: 30. leave-one-out DVH metrics difference < 1 Gy Liu et al [37]
Passing rate for 2% dose differ-
ence: 100% for photon plan,
15%–81% for proton plan
Passing rate for 2% dose differ-
ence: 100% for photon plan,
71%–86% for proton plan
Passing rate for 2% dose differ-
ence: > 99.5% for photon
plan,
> 80% for proton plan
Average DVH metrics
difference: 0.2% ± 0.6%
Gamma passing rate at (1%,
1 mm): 96.26 ± 3.59%
Gamma passing rate at (2%,
2 mm): 93.75%–99.75%
(proton)
Hansen et al [58]
Kurz et al [57]
Landry et al [59]
Li et al [60]
Liang et al [32]
Adrian et al [61]
8.3.3 Online image segmentation
The segmentation of targets and organs at risk (OARs) on daily imaging is a crucial
yet time-consuming step in the adaptive re-planning process. Manual delineation
can significantly prolong the entire process, making it imperative to explore more
efficient methods. CBCT-based auto organ delineation has been investigated,
involving contour propagation through rigid or deformable registration between
daily CBCT and planning CT. However, the accuracy of contour propagation is
highly dependent on the quality of registration, often requiring manual editing and
verification. While semi-automated methods have been explored to speed up OAR
contouring, their improvement is limited, prompting the need for faster organ
delineation methods for CBCT-based plan adaptation.
Deep learning-based segmentation algorithms have shown promising performance in CT contouring for various anatomical sites. Nevertheless, the inferior image
quality of CBCT compared to CT poses challenges in directly applying existing
convolutional neural network-based algorithms on CBCT images. Dai et al proposed a two-in-one deep learning model that first utilizes a cycleGAN network to
8-11
Artificial Intelligence in Adaptive Radiation Therapy
Figure 8.1. Dose comparison between CT- and CBCT-, as well as between CT- and sCT-based plans.
CBCT = cone-beam CT; sCT = synthetic CT. (Reproduced from [
Sons. Copyright 2020 American Association of Physicists in Medicine.)
37] with permission from John Wiley &
convert CBCT to synthetic CT [62]. Subsequently, a mask scoring regional convolutional neural network is applied to the synthetic CT to obtain organ contours. The
enhanced image quality of synthetic CT from CBCT is expected to facilitate the
image segmentation task. Evaluated on pancreas cancer patients, the proposed
method has demonstrated significant improvements across all metrics for the
majority of selected organs compared to direct segmentation on CBCT.
8.4 Synthetic CT from MRI
The current standard for radiation therapy planning involves the sequential use of
both MRI and CT imaging modalities on patients. This approach is driven by the
complementary strengths of each modality, with MR images offering superior softtissue contrast crucial for delineating tumors and OARs [63], while CT images
provide electron density maps essential for accurate dose calculations and serve as
reference images for pre-treatment positioning. The delineation of tumor and OAR
contours typically begins with MR images and is then transferred to CT images
through image registration, facilitating treatment planning and dose assessment.
However, the dual-modality approach incurs additional costs and time for
patients and introduces systematic positioning errors of up to 2 mm during the
CT-MRI image registration process [64–66]. Furthermore, the CT scan contributes a
8-12
Artificial Intelligence in Adaptive Radiation Therapy
non-negligible ionization dose to patients, particularly those necessitating resimulation [67]. Consequently, there is a compelling need to explore alternatives,
such as a treatment planning workflow solely reliant on MRI, to address these
challenges. The advent of MR-linac technology further encourages the exclusive use
of MRI in radiotherapy [68, 69]. Despite the potential advantages, it is important to
note that MR cannot directly substitute CT in current radiotherapy workflows. This
limitation arises from the fact that MR images derive signal from hydrogen nuclei,
precluding the direct provision of material attenuation coefficients necessary for
electron density calibration and subsequent dose calculations.
The preference for replacing CT with MR is extending into current PET imaging
practices. Traditionally, CT is frequently combined with PET, allowing both
imaging examinations to be conducted sequentially on the same table. CT images
play a crucial role in this set-up, as they are utilized to generate a 511 keV linear
attenuation coefficient map. This map, derived through a piece-wise linear scaling
algorithm [70, 71], is then employed to correct PET images for attenuated
annihilation photons within the patient’s body, ensuring a satisfactory level of
image quality. The integration of MR with PET has emerged as a promising
alternative to the established PET/CT systems. MR offers notable advantages as
mentioned above. However, akin to the challenges faced in radiation therapy, MR
images cannot directly provide the 511 keV attenuation coefficients required for the
attenuation correction process in PET imaging. Consequently, the solution lies in
the incorporation of MR-to-CT image synthesis within PET/MR systems to enable
accurate photon attenuation correction. This innovative approach capitalizes on the
strengths of MR imaging while addressing the specific needs of PET imaging for
enhanced diagnostic capabilities.
8.4.1 Synthetic image accuracy
An overview of studies focused on synthesizing CT from MR images for radiation
therapy and PET attenuation correction is listed in tables 8.3 and 8.4, respectively.
In the context of CT synthesis applications for radiation therapy, the mean absolute
error (MAE) emerges as the predominant and well-defined metric used to assess
image quality. Nearly every study in this domain reported the image quality of its
synthetic CT using MAE as a key evaluation criterion. On the other hand, when it
comes to synthetic CT in PET attenuation correction, the assessment of synthetic CT
quality is more commonly conducted indirectly. Instead of directly evaluating the
synthetic CT itself, studies tend to assess the quality of PET attenuation correction.
This suggests that, in this application, the focus is often on the impact of synthetic
CT on PET image quality and attenuation correction accuracy. In instances where
studies present multiple variants of methods, it is noted that the selected method for
inclusion in the tables is based on achieving the best MAE for radiation therapy or
the best PET quality for PET attenuation correction. This approach allows for a
concise representation of the most effective variants in terms of image quality
metrics for each respective application.
8-13
Key findings in
dosimetry
Artificial Intelligence in Adaptive Radiation Therapy
dose in PTV < 1.01%
N/A
Dose difference < 1.6%
Mead dose difference –
0.03% ± 0.05%
overall,
90% of prescription
–0.07% ± 0.22% in >
dose volume
N/A
Key findings in image
quality
Site, and # of patients in training/
testing
MAE (HU): 92.5 ± 13.9 N/A
Brain: 16
Pelvis: 22
(brain)
MAE (HU): 85.4 ± 9.24
42.4 ± 5.1 (pelvis)
Brain: 16, leave-one-out
Pelvis: 22, leave-one-out
(prostate)
MAE (HU): 65 ± 10
cervical cancer), 32 (prostate)
56 ± 5 (rectum)
training/59 (rest) testing
59 ± 6 (cervix)
MAE (HU): 40.5 ± 5.4
(2D)
37.6 ± 5.1 (3D)
Table 8.3. Summary of studies on MR-based synthetic CT for radiation therapy. (Adapted from [1]. CC BY 4.0.)
Han [16] U-Net 1.5 T T1w without contrast Brain: 18, 6-fold cross validation MAE (HU): 84.8 ± 17.3 N/A*
Author, year Network MR parameters
Nie et al [29] GAN N/A
Xiang et al [87]AE T1w
Dinkla et al [75] AE 1.5 T T1w Brain: 52, 2-fold cross validation MAE (HU): 67 ± 11 Dose difference < 1%
Arabi et al [76] U-Net 3 T T2w Pelvis: 39, 4-fold cross validation MAE (HU): 32.7 ± 7.9 Dose difference < 1%
Chen et al [73] U-Net 3 T T2w Pelvis: 36 training/15 testing MAE (HU): 29.96 ± 4.87 Dose difference of max
8-14
Emami et al [30] GAN 1 T post-gadolinium T1w Brain: 15, 5-fold cross validation MAE (HU): 89.3 ± 10.3 N/A
Maspero et al [88] GAN Dixon in-phase, fat and water Pelvis: 91 (59 prostate + 18 rectal + 14
Dinkla et al [72] U-Net 3 T in-phase Dixon T2w Head and neck: 22 training/12 testing MAE (HU): 75 ± 9
Fu et al [21] U-Net 1.5 T T1w without contrast Pelvis: 20, 5-fold cross validation