Добавил:
Sekretar
kiopkiopkiop18@yandex.ru
t.me/Prokururor I Вовсе не секретарь, но почту проверяю
Опубликованный материал нарушает ваши авторские права? Сообщите нам.
Вуз:
Предмет:
Файл:Ординатура / Хирургия / Библиотека им академика М.И. Перельмана / Книга_5525_Библиотеки_им_академика_М_И_Перельмана.pdf
Artificial Intelligence in Adaptive Radiation Therapy
[24] Zhao X, Stanley D N, Cardenas C E, Harms J and Popple R A 2023 Do we need patient-
specific QA for adaptively generated plans? Retrospective evaluation of delivered online
adaptive treatment plans on Varian Ethos J. Appl. Clin. Med. Phys.
[25] Xu Y, Xia W, Ren W, Ma M, Men K and Dai J 2024 Is it necessary to perform
measurement-based patient-specific quality assurance for online adaptive radiotherapy with
Elekta Unity MR-linac? J. Appl. Clin. Med. Phys.
[26] Cai B, Green O L, Kashani R, Rodriguez V L, Mutic S and Yang D S 2018 A practical
implementation of physics quality assurance for photon adaptive radiotherapy Z. Med.
Phys.
28 211–23
[27] Rangaraj D et al 2013 Catching errors with patient-specific pretreatment machine log file
analysis Pract. Radiat. Oncol.
[28] Qian J et al 2010 Dose reconstruction for volumetric modulated arc therapy (VMAT) using
cone-beam CT and dynamic log files Phys. Med. Biol.
[29] Teke T, Bergman A M, Kwa W, Gill B, Duzenli C and Popescu I A 2010 Monte Carlo
based, patient-specific RapidArc QA using linac log files Med. Phys.
[30] Cheng B et al 2023 Development and clinical application of a GPU-based Monte Carlo
dose verification module and software for 1.5 T MR-linac Med. Phys.
[31] Lin J et al 2024 ART2Dose: a comprehensive dose verification platform for online adaptive
radiotherapy Med. Phys.
[32] Wang Y, Mazur T, Park J, Yang D, Mutic S and Li H 2017 Development of a fast Monte
Carlo dose calculation system for online adaptive radiation therapy quality assurance Med.
Phys.
44 3130–1
[33] Bahrami A, Karimian A and Arabi H 2021 Comparison of different deep learning
architectures for synthetic CT generation from MR images Phys. Med.
[34] Han X 2017 MR-based synthetic CT generation using a deep convolutional neural network
method Med. Phys.
[35] Hsu S-H, Cao Y, Huang K, Feng M and Balter J M 2013 Investigation of a method for
generating synthetic CT models from MRI scans of the head and neck for radiation therapy
Phys. Med. Biol.
[36] Lei Y et al 2019 MRI-only based synthetic CT generation using dense cycle consistent
generative adversarial networks Med. Phys.
[37] Liang X et al 2023 Bony structure enhanced synthetic CT generation using Dixon sequences
for pelvis MR-only radiotherapy Med. Phys.
[38] Chen L, Liang X, Shen C, Nguyen D, Jiang S and Wang J 2021 Synthetic CT generation
from CBCT images via unsupervised deep learning Phys. Med. Biol.
[39] Gao L et al 2021 Generating synthetic CT from low-dose cone-beam CT by using
generative adversarial networks for adaptive radiotherapy Radiat. Oncol.
[40] Liu Y et al 2020 CBCT-based synthetic CT generation using deep-attention cycleGAN for
pancreatic adaptive radiotherapy Med. Phys.
[41] Liang X et al 2019 Generating synthesized computed tomography (CT) from cone-beam
computed tomography (CBCT) using CycleGAN for adaptive radiation therapy Phys.
Med. Biol.
[42] Jonsson J, Nyholm T and Söderkvist K 2019 The rationale for MR-only treatment
planning for external radiotherapy Clin. Transla. Radiat. Oncol.
64 125002
44 1408–19
58 8419
3 80–90
51 18–30
25 e14175
55 3597–610
46 3565–81
50 7368–82
47 2472–83
24 e13876
37 116–23
50 3172–83
90 99–107
66 115019
16 202
18 60–5
7-14
Artificial Intelligence in Adaptive Radiation Therapy
[43] Bird D, Henry A M, Sebag-Montefiore D, Buckley D L, Al-Qaisieh B and Speight R 2019
A systematic review of the clinical implementation of pelvic magnetic resonance imaging-only
planning for external beam radiation therapy Int. J. Radiat. Oncolo. Biol. Phys.
[44] Chen L et al 2023 AI empowered diagnostic MRI based simulation-omitted hippocampal-
sparing whole brain radiation therapy on Ethos Annual Meeting of the American
Association of Physicists in. Medicine (Houston, TX, 23–27 July)
[45] Chen Z, King W, Pearcey R, Kerba M and Mackillop W J 2008 The relationship between
waiting time for radiotherapy and clinical outcomes: a systematic review of the literature
Radiother. Oncol.
[46] Nikolov S et al 2018 Deep learning to achieve clinically applicable segmentation of head
and neck anatomy for radiotherapy arXiv:
[47] Nikolov S et al 2021 Clinically applicable segmentation of head and neck anatomy for
radiotherapy: deep learning algorithm development and validation study J. Med. Internet
Res.
23 e26151
[48] Tong N, Gou S, Yang S, Ruan D and Sheng K 2018 Fully automatic multi-organ
segmentation for head and neck cancer radiotherapy using shape representation model
constrained fully convolutional neural networks Med. Phys.
[49] Vrtovec T, Močnik D, Strojan P, Pernuš F and Ibragimov B 2020 Auto-segmentation of
organs at risk for head and neck radiotherapy planning: from atlas-based to deep learning
methods Med. Phys.
[50] Cardenas C E, Yang J, Anderson B M, Court L E and Brock K B 2019 Advances in auto-
segmentation Semin. Radiat. Oncol.
[51] van der Veen J, Gulyban A, Willems S, Maes F and Nuyts S 2021 Interobserver variability
in organ at risk delineation in head and neck cancer Radiat. Oncol.
[52] Gonzalez Y, Shen C, Jung H and Jia X 2019 A human behavior-driven deep-learning
approach for automatic sigmoid segmentation Int. J. Radiat. Oncol. Biol. Phys.
[53] Gonzalez Y et al 2021 Semi-automatic sigmoid colon segmentation in CT for radiation
therapy treatment planning via an iterative 2.5-D deep learning approach Med. Image Anal.
68 101896
[54] Ibrahim S S and Ravi G 2023 Deep learning based brain tumour classification based on
recursive sigmoid neural network based on multi-scale neural segmentation Int. J. Recent
Innov. Trends Comput. Commun.
[55] Balagopal A et al 2021 A deep learning-based framework for segmenting invisible clinical
target volumes with estimated uncertainties for post-operative prostate cancer radiotherapy
Med. Image Anal.
[56] Bi N et al 2019 Deep learning improved clinical target volume contouring quality and
efficiency for postoperative radiation therapy in non-small cell lung cancer Front. Oncol.
1192
[57] Elguindi S et al 2019 Deep learning-based auto-segmentation of targets and organs-at-risk
for magnetic resonance imaging only planning of prostate radiotherapy Phys. Imaging
Radiat. Oncol.
[58] Jin D et al 2021 DeepTarget: gross tumor and clinical target volume segmentation in
esophageal cancer radiotherapy Med. Image Anal.
[59] Liu Z et al 2020 Development and validation of a deep learning algorithm for auto-
delineation of clinical target volume and organs at risk in cervical cancer radiotherapy
Radiother. Oncol.
87 3–16
1809.04430
45 4558–67
47 e929–50
29 185–97
16 120
11 77–86
72 102101
12 80–6
68 101909
153 172–9
105 479–92
105 S93–4
9
7-15
Artificial Intelligence in Adaptive Radiation Therapy
[60] Ma C Y et al 2022 Deep learning-based auto-segmentation of clinical target volumes for
radiotherapy treatment of cervical cancer J. Appl. Clin. Med. Phys.
[61] Men K et al 2018 Fully automatic and robust segmentation of the clinical target volume for
radiotherapy of breast cancer using big data and deep learning Phys. Med.
[62] Nori H, King N, McKinney S M, Carignan D and Horvitz E 2023 Capabilities of GPT-4
on medical challenge problems arXiv:
[63] Thirunavukarasu A J, Ting D S J, Elangovan K, Gutierrez L, Tan T F and Ting D S W
2023 Large language models in medicine Nat. Med.
[64] Li Z et al 2022 Patient-specific daily updated deep learning auto-segmentation for MRI-
guided adaptive radiotherapy Radiother. Oncol.
[65] Elmahdy M S, Ahuja T, Heide U A V D and Staring M 2020 Patient-specific finetuning of
deep learning models for adaptive radiotherapy in prostate CT 2020 IEEE 17th Int. Symp.
on Biomedical Imaging (ISBI) (3–7 April 2020) pp 577–80
[66] Ding J, Zhang Y, Amjad A, Xu J, Thill D and Li X A 2022 Automatic contour refinement
for deep learning auto-segmentation of complex organs in MRI-guided adaptive radiation
therapy Adv. Radiat. Oncol.
[67] Kim N, Chun J, Chang J S, Lee C G, Keum K C and Kim J S 2021 Feasibility of continual
deep learning-based segmentation for personalized adaptive radiation therapy in head and
neck area Cancers
[68] Ma L et al 2022 Registration-guided deep learning image segmentation for cone beam CT-
based online adaptive radiotherapy Med. Phys.
[69] Chen X, Men K, Li Y, Yi J and Dai J 2019 A feasibility study on an automated method to
generate patient-specific dose distributions for radiotherapy using deep learning Med. Phys.
46 56–64
[70] Nguyen D et al 2019 A feasibility study for predicting optimal radiation therapy dose
distributions of prostate cancer patients from patient anatomy using deep learning Sci. Rep.
9 1076
[71] Fan J, Wang J, Chen Z, Hu C, Zhang Z and Hu W 2019 Automatic treatment planning
based on three-dimensional dose distribution predicted from deep learning technique Med.
Phys.
46 370–81
[72] Liu Z et al 2019 A deep learning method for prediction of three-dimensional dose
distribution of helical tomotherapy Med. Phys.
[73] Kearney V, Chan J W, Haaf S, Descovich M and Solberg T D 2018 DoseNet: a volumetric
dose prediction algorithm using 3D fully-convolutional neural networks Phys. Med. Biol.
63 235022
[74] Götz T I, Schmidkonz C, Chen S, Al-Baddai S, Kuwert T and Lang E W 2020 A deep
learning approach to radiation dose estimation Phys. Med. Biol.
[75] Bakx N, Bluemink H, Hagelaar E, van der Sangen M, Theuws J and Hurkmans C 2021
Development and evaluation of radiotherapy deep learning dose prediction models for
breast cancer Phys. Imaging Radiat. Oncol.
[76] Kandalan R N et al 2020 Dose prediction with deep learning for prostate cancer radiation
therapy: model adaptation to different treatment planning practices Radiother. Oncol.
228–35
[77] Ma J et al 2021 A feasibility study on deep learning-based individualized 3D dose
distribution prediction Med. Phys.
13 702
7 100968
2303.13375
29 1930–40
177 222–30
49 5304–16
46 1972–83
17 65–70
48 4438–47
23 e13470
50 13–9
65 035007
153
7-16
Artificial Intelligence in Adaptive Radiation Therapy
[78] Barragán-Montero A M et al 2021 Deep learning dose prediction for IMRT of esophageal
cancer: the effect of data quality and quantity on model performance Phys. Med.
[79] Chen X et al 2021 DVHnet: a deep learning-based prediction of patient-specific dose
volume histograms for radiotherapy planning Med. Phys.
[80] Nguyen D et al 2018 Three-dimensional radiotherapy dose prediction on head and neck
cancer patients with a hierarchically densely connected U-Net deep learning architecture
arXiv:
1805.10397
[81] Barragán-Montero A M et al 2019 Three-dimensional dose prediction for lung IMRT
patients with deep neural networks: robust learning from heterogeneous beam configurations Med. Phys.
[82] Maniscalco A, Liang X, Lin M-H, Jiang S and Nguyen D 2023 Intentional deep overfit
learning for patient-specific dose predictions in adaptive radiotherapy Med. Phys.
63
[83] Maniscalco A, Liang X, Lin M-H, Jiang S and Nguyen D 2023 Single patient learning for
adaptive radiotherapy dose prediction Med. Phys.
[84] Gao Y, Shen C, Gonzalez Y and Jia X 2022 Modeling physician’s preference in treatment
plan approval of stereotactic body radiation therapy of prostate cancer Phys. Med. Biol.
115012
[85] Gonzalez Y, Shen C, Albuquerque K and Jia X 2020 Deep-learning based prediction of
physicians intention for high-dose-rate brachytherapy with tandem-and-ovoids applicator
Med. Phys. 47 E454–4
[86] Das I J, Cheng C-W, Chopra K L, Mitra R K, Srivastava S P and Glatstein E 2008 Intensity-
modulated radiation therapy dose prescription, recording, and delivery: patterns of variability
among institutions and treatment planning systems J. Natl Cancer Inst.
[87] Nelms B E et al 2012 Variation in external beam treatment plan quality: an inter-
institutional study of planners and planning systems Pract. Radiat. Oncol.
[88] Moore K L, Brame R S, Low D A and Mutic S 2011 Experience-based quality control of
clinical intensity-modulated radiotherapy planning Int. J. Radiat. Oncol. Biol. Phys.
545–51
[89] Ohri N, Shen X, Dicker A P, Doyle L A, Harrison A S and Showalter T N 2013
Radiotherapy protocol deviations and clinical outcomes: a meta-analysis of cooperative
group clinical trials JNCI: J. Natl Cancer Inst.
[90] Li X et al 2020 Automatic IMRT planning via static field fluence prediction (AIP-SFFP): a
deep learning algorithm for real-time prostate treatment planning Phys. Med. Biol.
175014
[91] Lee H et al 2019 Fluence-map generation for prostate intensity-modulated radiotherapy
planning using a deep-neural-network Sci. Rep.
[92] Ma L, Chen M, Gu X and Lu W 2020 Deep learning-based inverse mapping for fluence
map prediction Phys. Med. Biol.
[93] Vandewinckele L, Willems S, Lambrecht M, Berkovic P, Maes F and Crijns W 2022
Treatment plan prediction for lung IMRT using deep learning based fluence map
generation Physica Med.
[94] Wang W et al 2021 Deep learning-based fluence map prediction for pancreas stereotactic
body radiation therapy with simultaneous integrated boost Adv. Radiat. Oncol.
[95] Wang W et al 2020 Fluence map prediction using deep learning models—direct plan
generation for pancreas stereotactic body radiation therapy Front. Artif. Intell.
46 3679–91
105 387–93
9 15671
65 235035
99 44–54
48 2705–13
50 7324–37
100 300–7
83 52–63
50 5354–
67
2 296–305
81
65
6 100672
3 68
7-17
Artificial Intelligence in Adaptive Radiation Therapy
[96] Yuan Z et al 2022 Accelerate treatment planning process using deep learning generated
fluence maps for cervical cancer radiation therapy Med. Phys.
[97] Mnih V et al 2015 Human-level control through deep reinforcement learning Nature 518
529
[98] Silver D et al 2017 Mastering the game of Go without human knowledge Nature 550 354
[99] Silver D et al 2016 Mastering the game of Go with deep neural networks and tree search
Nature
529 484
[100] Hrinivich W T and Lee J 2020 Artificial intelligence-based radiotherapy machine parameter
optimization using reinforcement learning Med. Phys.
[101] Liu Y et al 2022 Automatic inverse treatment planning of Gamma Knife radiosurgery via
deep reinforcement learning Med. Phys.
[102] Shen C, Chen L, Gonzalez Y and Jia X 2021 Improving efficiency of training a virtual
treatment planner network via knowledge-guided deep reinforcement learning for intelli-
gent automatic treatment planning of radiotherapy Med. Phys.
[103] Shen C, Chen L and Jia X 2021 A hierarchical deep reinforcement learning framework for
intelligent automatic treatment planning of prostate cancer intensity modulated radiation
therapy Phys. Med. Biol.
[104] Shen C et al 2019 Intelligent inverse treatment planning via deep reinforcement learning, a
proof-of-principle study in high dose-rate brachytherapy for cervical cancer Phys. Med.
Biol.
64 115013
[105] Shen C et al 2020 Operating a treatment planning system using a deep-reinforcement
learning-based virtual treatment planner for prostate cancer intensity-modulated radiation
therapy treatment planning Med. Phys.
[106] Sprouts D, Gao Y, Wang C, Jia X, Shen C and Chi Y 2022 The development of a deep
reinforcement learning network for dose–volume-constrained treatment planning in pros-
tate cancer intensity modulated radiotherapy Biomed. Phys. Eng. Express
[107] Zhang J et al 2021 An interpretable planning bot for pancreas stereotactic body radiation
therapy Int. J. Radiat. Oncol. Biol. Phys.
[108] Gao Y, Shen C, Jia X and Park Y K 2023 Implementation and evaluation of an intelligent
automatic treatment planning robot for prostate cancer stereotactic body radiation therapy
Radiother. Oncol.
[109] Valdes G, Scheuermann R, Hung C Y, Olszanski A, Bellerive M and Solberg T D 2016 A
mathematical framework for virtual IMRT QA using machine learning Med. Phys.
[110] Valdes G, Chan M F, Lim S B, Scheuermann R, Deasy J O and Solberg T D 2017 IMRT QA
using machine learning: a multi-institutional validation J. Appl. Clin. Med. Phys.
[111] Li J et al 2019 Machine learning for patient-specific quality assurance of VMAT: prediction
and classification accuracy Int. J. Radiat. Oncol. Biol. Phys.
[112] Tomori S et al 2018 A deep learning-based prediction model for gamma evaluation in
patient-specific quality assurance Med. Phys.
[113] Tomori S et al 2021 Systematic method for a deep learning-based prediction model for
gamma evaluation in patient-specific quality assurance of volumetric modulated arc
therapy Med. Phys.
[114] Du W, Cho S H, Zhang X, Hoffman K E and Kudchadker R J 2014 Quantification of
beam complexity in intensity-modulated radiation therapy treatment plans Med. Phys.
021716
66 134002
184 109685
48 1003–18
49 2877–89
47 2329–36
109 1076–85
45 4055–65
47 6140–50
49 2631–41
48 1909–20
8 045008
43 4323
18 279–84
105 893–902
41
7-18
Artificial Intelligence in Adaptive Radiation Therapy
[115] Hirashima H et al 2020 Improvement of prediction and classification performance for
gamma passing rate by using plan complexity and dosiomics features Radiother. Oncol.
250–7
[116] Lee C C et al 2021 Applying artificial intelligence to longitudinal imaging analysis of
vestibular schwannoma following radiosurgery Sci Rep.
[117] Kong F M et al 2017 Effect of midtreatment PET/CT-adapted radiation therapy with
concurrent chemotherapy in patients with locally advanced non-small-cell lung cancer: a
phase 2 clinical trial JAMA Oncol.
3 1358–65
11 3106
153
7-19
IOP Publishing
Artificial Intelligence in Adaptive Radiation Therapy
Yi Wang and X. Sharon Qi
Chapter 8
Imaging, imaging processing, and synthetic
computed tomography
Tonghe Wang and Xiaofeng Yang
Computed tomography (CT) image synthesis from cone-beam computer tomography
(CBCT) and magnetic resonance imaging (MRI) has been explored for various
applications within the adaptive radiation therapy workflow. Synthesized CT images
have demonstrated feasibility for quantitative tasks such as dose calculation, image
segmentation, registration, and PET attenuation correction. These images offer
improved quality over CBCT images and provide complementary information to
MR images. Deep learning techniques, ranging from convolutional networks to
generative models, have been extensively applied in these studies, offering significant
advantages in performance compared to traditional image processing methods. This
chapter reviews the deep learning methods employed in CT synthesis and examines
their potential applications in adaptive radiation therapy.
This chapter is adapted from ‘A review on medical imaging synthesis using deep
learning and its clinical applications’ by Wang et al [1], used under a CC BY 4.0 license.
8.1 Introduction
Synthesizing CT images from alternative imaging modalities constitutes a pioneering avenue in medical image synthesis, representing a focal point of extensive
research efforts within the field. Building upon its initial success, numerous
applications dedicated to the synthesis between diverse imaging modalities have
garnered active attention. The primary clinical motivation behind CT synthesis lies
in mitigating the exposure of patients to ionizing radiation, a factor associated with
potential side effects [2]. Additionally, the synthesis of CT images holds promise for
various clinic-oriented advantages, including cost reduction in hardware and
maintenance, as well as enhanced patient throughput.
This chapter concentrates on the synthesis of CT images, acknowledging that
current studies reveal synthetic CT results that still exhibit noticeable disparities
from authentic CT scans. This disparity presently precludes direct diagnostic
doi:10.1088/978-0-7503-6119-4ch8 8-1 ª IOP Publishing Ltd 2025. All rights,
including for text and data mining (TDM), artificial intelligence (AI) training, and similar technologies, are reserved.
Artificial Intelligence in Adaptive Radiation Therapy
application. Nevertheless, a wealth of research underscores the feasibility of
synthetic CT for non- or indirect-diagnostic purposes, such as its utility in treatment
planning for radiation therapy and PET attenuation correction.
8.2 Synthetic CT: deep learning methods
8.2.1 Conventional methods
The absence of a direct correspondence between magnetic resonance (MR) voxel
intensity and CT Hounsfield unit (HU) values gives rise to significant disparities in
image appearance and contrast, rendering intensity-based calibration methods
impractical. Notably, CT depicts air as dark and bone as bright, while MR portrays
both as dark. As a result, conventional calibration methods face challenges in
aligning these distinct characteristics. Existing approaches in the literature either
segment MR images into material-specific groups and assign corresponding CT HU
numbers, [3–8] or register MR images with an atlas possessing known CT HU values
[9–11].
The technique of CT number bulk-assignment can be traced back to Lee et al in
2003 [3]. They manually delineated the entire bone in the pelvic region on MR
images and assigned a bone value, designating the remaining region as water.
Building upon this, Jonsson et al extended a similar methodology to other
anatomical sites [4]. Keereman et al introduced the use of ultrashort echo time
(UTE) sequences as a replacement for conventional magnetic resonance imaging
(MRI) sequences [12]. The UTE sequence allows the derivation of an R2 map, which
represents bone with high values and soft tissue with low values. Subsequently, a
straightforward thresholding method is applied to the R2 map to assign piece-wise
constant attenuation coefficient values for air, soft tissue, and bone. In a similar vein,
Catana et al proposed a dual-echo UTE approach and devised associated image
processing procedures to generate a map suitable for thresholding [13]. With UTE,
Johansson et al developed a Gaussian mixture regression model to link the
intensities in MRs (two dual-echo UTE with different flip angles and one T2w
image) to CT images [7]. These advancements highlight the efforts to refine and
enhance the bulk-assignment of CT numbers, particularly in the context of utilizing
alternative imaging sequences and methodologies for improved accuracy and
efficiency.
On the other hand, atlas-based registration methods have been introduced as an
alternative approach. For instance, Kops and Herzog devised a method where a
common attenuation template was created from ten normal volunteers and spatially
normalized to the SPM2 standard brain shape [14]. Individual MR images were
subsequently registered with this template to obtain an attenuation map. It is
important to note that this method was originally developed for brain imaging and
necessitates a reliable and locally precise inter-subject registration, as mentioned by
the authors. Addressing the challenges posed by whole-body images characterized
by high inter-subject variability, Hofmann et al proposed a novel approach that
combines pattern recognition and atlas registration [15]. This hybrid method
8-2
Artificial Intelligence in Adaptive Radiation Therapy
effectively captures the global variation in anatomy, making it more suitable for the
complexities associated with whole-body imaging applications.
These methodologies rely heavily on the efficacy of segmentation and registration
techniques, which proves to be particularly challenging due to the ambiguous air/
bone boundary and substantial inter-patient variation. The complexities involved in
distinguishing these elements hinder the reliability and accuracy of these calibration
methods, emphasizing the need for innovative solutions in addressing the inherent
differences between MR and CT imaging modalities.
8.2.2 U-Net
In one of the pioneering studies utilizing deep learning for CT synthesis, Han
employed an autoencoder to synthesize CT images from MR images, adopting and
modifying a U-Net architecture [16]. The U-Net model in Han’s study comprised an
encoding and a decoding part. The encoder extracted hierarchical features from an
MR image input using convolutional, batch normalization, rectified linear unit
(ReLU), and pooling layers. Meanwhile, the mirrored decoder replaced pooling
layers with deconvolution layers, transforming the features and reconstructing the
predicted CT images from low to high-resolution levels. Short-cut connections were
introduced between the two parts on multiple layers. These short-cuts facilitated the
concatenation of early layers with late layers, allowing late layers to learn simple
features captured in early layers. In Han’s study, these short-cuts enabled highresolution features from the encoding part to be used as extra inputs in the decoding
part. Moreover, the original autoencoder design included fully connected ‘hidden
layers’, which connect every neuron in the previous layer to every neuron in the next.
However, these fully connected layers, crucial for image classification tasks, were
found to be less relevant for dense pixel-wise prediction. Therefore, Han’s model
eliminated fully connected layers, significantly reducing the number of parameters.
The study trained the model using pairs of MR and CT 2D slices, with a training
process minimizing a mean absolute error (MAE) loss function between the
predictions and ground truth. The use of an L1-norm loss function such as MAE
contributes to improved robustness to noise, artifacts, and misalignment among the
training images. Han’s work represents a significant advancement in the application
of deep learning to CT synthesis.
Most studies employing the U-Net architecture have generally adhered to the
outlined structure, yet there have been numerous proposed variants and improvements. For instance, in comparison to Han’s model, Jang et al and Liu et al applied
a similar encoder and decoder model without the inclusion of skip connections [17,
18]. Instead of utilizing CT images directly as ground truth in their MR-based CT
synthesis studies, they employed discretized maps from CTs, categorizing three
materials and framing CT synthesis as a segmentation problem. The final layer of
the decoder incorporated a multi-class softmax classifier, assigning probabilities to
each material class within each voxel (e.g. 0.5 for bone, 0.3 for air, and 0.1 for soft
tissue).
8-3
Artificial Intelligence in Adaptive Radiation Therapy
An additional noteworthy feature introduced by Jang et al is the inclusion of a
fully connected conditional random field, considering neighboring voxels during
label predictions [18]. This provided complementary information to the base
classifier, which focused on single voxels. In this application, the conditional random
field supplied 3D context to 2D image slices, establishing pairwise potentials
between all pairs of voxels by utilizing the model’s output and the original 3D
volume when predicting voxel labels.
A landmark advancement in U-Net architecture occurred when Dong et al
identified that the information carried in the long skip connection from the encoding
path often contained high-frequency and irrelevant components from noisy input
images [19]. To address this, they introduced a self-attention strategy that utilized
feature maps extracted from the coarse-scale early in the encoder module to identify
the most relevant emerging features. These features were assigned attention scores,
enabling the elimination of noise before concatenation. Alternatively, Hwang et al
adopted a strategy that employed skip connections only in deeper layers, offering an
alternative approach to handling noise and enhancing the efficiency of information
flow within the network [20].
The selection of building blocks within the encoding and decoding modules has
been a subject of exploration. Fu et al made several enhancements based on Han’s
architecture [21]. Notably, they replaced batch normalization layers, where normalization is applied across image subsets of the original sample to expedite convergence, with instance normalization layers. The latter performs normalization at
the level of image channels, contributing to further performance improvements,
particularly when training with a small batch size. Additionally, in the decoder, the
unpooling layers, responsible for up-sampling and reversing the pooling layers in the
encoder, were substituted with deconvolutional layers. These deconvolutional layers
produce dense feature maps, and the skip connections were replaced with residual
short-cuts inspired by ResNet. This alteration aims to conserve computational
memory more efficiently. Neppl et al opted to replace the ReLU layer with a
generalized parametric ReLU (PReLU) to adaptively adjust the activation function
[22]. In a similar vein, Torrado-Carvajal et al introduced a dropout layer before the
first transposed convolution in the decoder to mitigate overfitting concerns [23].
Various loss functions have been explored in the studies reviewed. In addition to
the commonly used L1-norm and L2-norms, which enforce voxel-wise similarity, the
total loss function often incorporates other functions describing different image
properties. For instance, Leynes et al employed a total loss function that was a
combination of MAE loss, gradient difference loss, and Laplacian difference loss
[24]. The latter two components aimed to enhance image sharpness. Similarly, Chen
et al combined MAE loss with structure dissimilarity loss to promote wholestructure-wise similarity [25]. To prevent overfitting, L2-regularization has been
integrated into the loss function in some studies [26, 27]. Kazemifar et al introduced
mutual information, widely employed in loss functions for image registration, into
their loss function. They demonstrated its advantages over MAE loss in better
compensating for misalignment between CT and MR images. Another innovative
addition is the perceptual loss introduced by Largent et al. This loss function, which
8-4