Добавил:
Sekretar
kiopkiopkiop18@yandex.ru
t.me/Prokururor I Вовсе не секретарь, но почту проверяю
Опубликованный материал нарушает ваши авторские права? Сообщите нам.
Вуз:
Предмет:
Файл:Ординатура / Хирургия / Библиотека им академика М.И. Перельмана / Книга_3592_Библиотеки_им_академика_М_И_Перельмана
.pdf
Vascular and Intravascular Imaging Trends, Analysis, and Challenges, Volume 1
https://t.me/medicina_free
transformation is often parameterized by B-splines and thin plate splines (TPS) since
they ensure a smooth deformation.
In most applications of endovascular IGIs that involve the treatment of
pathologies in cardiac [6] and cerebral vasculatures [7], and on vasculatures in the
abdomen (liver [8]), and others, it is sufficient to use rigid transformation of the 3D
image. The reason is that some vasculatures can be considered rigid, for instance,
cerebral and even cardiac, if image acquisition is gated to electrocardiography
signals. For vasculatures in the abdomen (e.g. liver and kidneys) a non-rigid
transformation may need to be used, however, the rigid-body transformation is
still required to initialize the posing of the 3D angiographic image.
10.3.3 Dimensional correspondence
The comparison of vessels and other vascular structures between the 3D volume and
the 2D image can occur in 3D space or on the 2D imaging plane. If the comparison
occurs on the 2D imaging plane, the intensities in the 3D volume, or other features,
have to be rendered or transformed into the 2D imaging plane by forward
projection. If the comparison occurs in the 3D space then intensities or other
features of the structures in 2D images are backprojected along rays towards the
x-ray source (figure 10.2). Backprojection is an important component of image
reconstruction methods, which require multiple 2D views to extract high quality 3D
image information. In IGIs two simultaneous 2D views acquired by biplane C-arm
imaging systems are sometimes used, e.g. during embolization of arteriovenous
malformations. The two mutually inclined 2D views can easily be used to
reconstruct point features such as vessel centerlines in 3D to be matched to the
vessel centerlines extracted from 3D volume to perform 3D–2D registration.
The intensity of 3D volumes can be projected to 2D by volume rendering methods
for visualization and to achieve dimensional correspondence. Digitally reconstructed radiography (DRR) [9] is a simulation of volume projection by a ray
casting operation. A DRR projection is formed by an integral function of voxel
intensities encountered along each ray in a cone of rays, which all emanate from the
x-ray source, pass through the volume and eventually hit the pixels on 2D image
plane, as depicted in figure 10.4.
Different integral functions can determine the cast intensity values of pixels on the
2D DRR image. Composite DRRs process all the voxel intensities which the ray
passes along its path. Each voxel is assigned an opacity and a color value based on
the tissue type and those values are integrated along the ray to determine the final
intensity value in the pixel. Different assignments for opacity and color maps are
used in applications of DRR-based visualization [9]. Two simpler methods for
volume visualization are minimum intensity projection (MinIP) and maximum
intensity projection (MIP), which assign to the pixel either the minimum or
maximum intensity value, respectively, as encountered along the ray path.
Generating the DRR or similar projections of the raw 3D volume is computationally demanding and may represent a bottleneck for a clinical application despite
speed improvements in recent GPU implementations [10]. Two parameters mainly
10-6
Vascular and Intravascular Imaging Trends, Analysis, and Challenges, Volume 1
https://t.me/medicina_free
Figure 10.4. Digitally reconstructed radiograph (DRR) based on a 3D pre-interventional image, and the
impact of 3D image sampling and DRR ray step length on the quality of the obtained DRR image.
affect the performance of a DRR projection: (i) 3D image sampling and (ii) ray step
size. To reduce the computational demand the sampling of 3D volume can be
reduced and the ray step length increased, however, this may have an adverse affect
on the DRR quality (figure 10.4) and the 3D–2D registration. Namely, tuning these
parameters for speed may drastically reduce the complexity of 3D vessel tree by poor
visualization of smaller vessels and may therefore further ill-condition the process of
matching 3D and 2D image information.
The second method for volume rendering is the projection of geometric primitives
such as edges and vertices by 3D perspective projection. Geometric structures or a
10-7
Vascular and Intravascular Imaging Trends, Analysis, and Challenges, Volume 1
https://t.me/medicina_free
polygonal mesh formed from the surface or boundaries of structures of interest are
rendered by wireframe methods. Texture mapping, shading and lighting operations
can be applied on the surfaces of geometric structures. This type of projection is
generally faster than ray casting methods, however, it requires accurate segmentation or feature detection and may result in the loss of important information
encoded in the original intensity values.
10.3.4 Number of views
The number of simultaneous intra-interventional x-ray views is determined by the
type of the C-arm employed during the intervention: one view is available on the
monoplane, while two are available on the biplanar C-arm systems. Registration
with two 2D views provides the advantage of better resolution of depth ambiguities
and possible occlusions of the structures of interest and, in general, improves the
convergence of 3D–2D registration to globally optimal solutions. Biplanar views are
more commonly used in certain clinical applications such as embolization of arterial
venous malformation (AVM), or in applications that require the use of deformable
registration and/or employ backprojection and reconstruction-based approaches to
achieve dimensional correspondence. However, most IGIs are carried out under a
single C-arm view, while the C-arm is only rotated occasionally in order to
disambiguate overlapping structures. The main reason for using a single x-ray
view is to limit the potentially hazardous radiation dose delivered to the patient
during IGI.
10.3.5 Registration basis
The registration basis determines the type of information extracted from both
modalities and employed to quantitatively assess their similarity for the purpose of
3D–2D registration. When a 3D volume is acquired by the C-arm just before the
intervention, the alignment of the 3D volume to intra-interventional 2D images may
be established by C-arm calibration. Hence, the registration basis may be calibration-
based. Another registration basis relies on stereotactic frames or external markers and
is referred to as extrinsic. The anatomical information present in the 3D and 2D
images is an intrinsic registration basis and may be further categorized as intensity-,
feature- and gradient-based [11].
10.4 Review of registration bases
Methods for 3D–2D image registration in IGI were reviewed and categorized into
extrinsic and intrinsic by Markelj et al [11]. The basis of registration can also be the
coordinate frames of imaging devices obtained through the device calibration. The
extrinsic methods employ artificial objects introduced manually into the imaged
scene and generally require the acquisition of at least two mutually inclined x-ray
projections [12, 13]. Conversely, the intrinsic methods base registration on the
anatomical information contained in 3D and 2D images and seem readily applicable
for registration of 3D to a single view 2D image. The intrinsic methods can be
further categorized according to registration basis as intensity-, feature- and
10-8
Vascular and Intravascular Imaging Trends, Analysis, and Challenges, Volume 1
https://t.me/medicina_free
gradient-based [11]. These methods also enable a higher level of automation and
seemingly easier integration into the clinical workflow of IGIs. General advantages
and disadvantages of the registration bases are summarized in table 10.1.
10.4.1 Calibration-based methods
Calibration-based 3D–2D registration is possible if the C-arm is used to acquire
both 3D and 2D images [14–16]. Such registration can be very accurate (0.2 mm)
Table 10.1. Summary of advantages and disadvantages of registration bases.
Registration basis Advantages Disadvantages
Calibration-based
Extrinsic
Intensity-based
Feature-based
• Simple usage
• Very fast execution
• Accurate registration
• Fast execution
• Accurate registration
• Based on 3D and 2D
anatomical information
• No 3D or 2D image
preprocessing needed
• Based on 3D and 2D
anatomical information
• Fast execution
• Possible if 3D and 2D images
acquired with the same imaging
system (e.g. C-arm).
• Patient movement invalidates registration, then a new 3D–2D image
pair has to be acquired.
• Artificial objects need to be affixed
to the patient, which may be timeconsuming, complex or even too
invasive.
• The artificial objects may occlude
structures of interest.
• Computationally expensive.
• Close-to-optimal initial registration
required.
• Unreliable out-of-plane translation
estimation.
• Accurate and reliable extraction
and matching of visual features
are difficult.
• Sensitive to image modality and
variations in anatomy and image
quality.
Gradient-based
• Based on 3D and 2D
anatomical information
• Fast execution
• Robust against anatomy
and image quality variation
10-9
• Close-to-optimal initial registration
required.
• Unreliable out-of-plane translation
estimation.
Vascular and Intravascular Imaging Trends, Analysis, and Challenges, Volume 1
https://t.me/medicina_free
and fast (a few seconds) [16], however, the main disadvantage is that even small
patient movement during IGI may cause large registration errors. To recover the
registration a new acquisition of the 3D image may be required, however, this
increases the radiation dose and contrast agent usage and is time-consuming,
therefore, it may be difficult to justify its use for the purpose of registration.
10.4.2 Extrinsic methods
The extrinsic methods base registration on matching artificial objects such as
catheters, markers or other radio-opaque objects in the imaged scene. The
advantage is that these objects are easy to detect in the images, however, the
disadvantage is that they need to be firmly affixed to the patient. While some
fixations may be too invasive, others may be too time-consuming or even overly
complex to perform during IGI. Navab et al [17] used multi-modality markers
visible on C-arm fluoroscopy with an optical tracking system to detect and correct
for patient or C-arm motion, while Hamming et al [18] used similar markers for
automatic initial image-to-world co-registration. Varnavas et al [12] used a radioopaque ruler to determine in-plane translations of the 3D image by reconstructing
and aligning a virtual fiducial marker. The marker was obtained based on manually
selected corresponding locations on the radio-opaque ruler visible on two 2D
projections. Truong et al [13] extracted a catheter visible in a 3D image and
performed a global rigid-body fit to minimize the distance between the medial lines
of the catheter and of the corresponding vessel, reconstructed in 3D through manual
selection of medial points on two (biplane) x-rays. Otake et al [19] estimate the
relative pose between x-ray images and the 3D anatomical structure using an inimage robust tracking fiducial and then register CTA with multiple x-rays by
intensity-based mutual information and gradient information 3D–2D image similarity measures, which are among the intrinsic methods reviewed next.
10.4.3 Intensity-based methods
The intensity-based methods establish the dimensional correspondence between 3D
and 2D images by using DRR, MIP or a similar 3D image visualization technique
that best models the actual projection of pre-operative 3D image into 2D detector
plane. Registration is then based on matching the projected 3D image to the intraoperative x-ray image(s) by a similarity measure such as mutual information [20],
gradient information [19], gradient difference [21], pattern intensity [22], gradient
correlation [23] or sum of squared differences [24]. Dong et al [25] introduced a
similarity measure based on the distance between coef ficients of orthogonal Zernike
moment decompositions of the DRR and the 2D image. Flach et al [26] perform
deformable registration of CT to low dose flouroscopic raw data by optimizing, in
an alternating manner, the sum of squared distances as data fidelity term and a fl
based diffusion as regularizer to obtain smooth deformation. They compare the
3D–2D registration results to 3D–3D registration between CT and low dose
tomographic flouroscopy. Unfortunately, most of the intensity-based image similarity measures have poor sensitivity with respect to translation of 3D images along
uid-
10-10
Vascular and Intravascular Imaging Trends, Analysis, and Challenges, Volume 1
https://t.me/medicina_free
the direction of projection (i.e. out-of-plane translation) and require close initialization due to the presence of local minima in the similarity measure. Furthermore,
generating the projections of the raw 3D image is computationally demanding and
may represent a bottleneck for a clinical application despite speed improvements in
recent GPU implementations [10].
10.4.4 Feature-based methods
In the intrinsic feature-based category of 3D–2D registration, image features such as
vessel centerlines, binary vessel masks, points or certain geometric primitives are
extracted from the images and utilized for registration. Therefore, methods such as
vessel segmentation and centerline extraction need to be performed on 3D or 2D
images in order to compute the similarity between the images.
Metz et al [27] aligned coronary centerlines of CTA and x-ray images using a
similarity measure composed of the distance transform of a projected vessel tree
model and a fuzzy segmentation of x-rays. Their method also accounts for heart beat
and respiratory motion by synchronizing heart phases. Rivest-Henault et al [28]
minimized the distances between the centerlines of the projected CTA and x-ray
images using different optimization algorithms for translation, rigid and affine
transformations. At the second stage they perform non-rigid alignment on biplane
x-rays using distances between centerlines with epipolar constraint as the image term
and displacement, smoothness and myocardium constraints as the internal energy
terms. Turgeon et al [29] registered binarized projections of segmented 3D coronary
arteries with segmented x-ray angiography images using entropy correlation
coefficient similarity on both single and biplane angiograms. Ruijters et al [30]
devised a similarity measure based on the distance transform of segmented CTA
projections and enhanced x-ray images in the form of a vesselness map. Groher et al
[31] employed a graph-based similarity specifically developed for registration of liver
vasculatures, which matches a segmented 3D vessel tree model to the enhanced
intra-operative image and simultaneously derives a segmentation of the 2D image.
This approach has been advanced to perform non-rigid alignment [8]. In a later
work, Groher et al [2] performed a deformable registration of the 3D segmented
model to the enhanced x-ray image by a cost function consisting of an external term
that minimizes the distance between the projected centerlines and locations with
high values in the enhanced image, and the internal term enforcing vessel length
preservation.
More recently, Baka et al [32] employed training of a population of CTA
coronary models by measuring landmark coordinates on cardiac surfaces and
estimating the motion. The obtained statistical motion model of CTA was registered
to the x-ray sequence by minimizing distances and orientation differences between
projected 3D vessel points and extracted 2D centerlines across all frames. Temporal
alignment between the CTA and x-ray was modeled by a piecewise linear function
and respiratory motion was constructed by quadratic interpolation of poses in the
first, central and last frames of the sequence. Metz et al [6] proposed a 3D+t/2D+t
(t: time) registration method that uses a patient specific dynamic coronary model
10-11
Vascular and Intravascular Imaging Trends, Analysis, and Challenges, Volume 1
https://t.me/medicina_free
derived from the CTA scan obtained by a vessel centerline extraction and
subsequent motion estimation. The model is aligned to the 2D+t x-ray sequence
by time varying rigid transformations, which takes breathing motion (which is also
rigid) and the temporal relation between CTA and x-ray time points into account.
The cost function at any time point is measured as average vessel centerline distance
between the CTA and the most probable x-ray centerlines. Baka et al [33] construct
probability distributions of moving and scene point sets of vessel centerlines in the
form of Gaussian mixture models (GMM) and use a similarity metric that minimizes
the difference between Gaussian mixtures of both point sets for a 3D–2D registration. The Jacobian matrix of the cost function was analytically computed to perform
gradient descent optimization. Later, orientations were added to the point sets to
create 4D GMM distributions. Kim et al [34] extract vessel centerlines of 3D and 2D
vasculature and perform deformable registration by using a thin plate spline based
robust point matching algorithm, which alternates between the estimation of the
correspondence and the transformation of the two point sets. They report that
outliers in both point sets are well handled by the method. Aksoy et al [35] decouple
the estimation of rotation and translation before matching segmented CTA to
segmented x-ray vessels. Rotation is recovered by matching a set of rotated DRR
templates of CTA to the binary x-ray vessels using scale and shift invariant similarity
measures computed from magnitudes of the Fourier transformation of the images.
In the second step, the 3D translation is recovered in the spatial domain by
minimizing the distance and maximizing the overlap ratio between the 3D vessel
model and the 2D vessels.
Some authors [36, 37] used reconstruction of 3D images from several x-ray images
acquired from different viewpoints in order to perform registration to the preoperative volume with 3D–3D registration methods. However, the 2D x-ray images
need to be acquired simultaneously to obtain a reconstruction without motion
induced artifacts. Furthermore, the quality of the reconstructed model depends on
the number of views. Serradel et al [38] used a generative model for CTA from
synthetic samples and simultaneously reconstructed the 3D structure of a non-rigid
coronary tree by estimating point correspondences between a single view x-ray
image and a reference 3D shape. Features are nodes generated in 3D and points of
interest that are extracted by the vesselness fi lter in the x-ray [39]. The cost function
minimizes the reprojection error by alternating matches of corresponding features
and perturbing non-rigid parameters stored as a principal component analysis
(PCA) model. Nodes are matched as an optimal assignment problem via minimization of Mahalanobis distance between the two points and the distance between
their orientations.
Because the raw 3D, 2D, or both 3D and 2D intensity information is reduced to a
small set of features, the feature-based methods are usually fast. However, the
accuracy and robustness of these methods are heavily dependent on the quality of
segmentations and extracted features, which must be devised for each image
modality and target anatomy of interest. As varying conditions are usually
encountered during IGI, these methods may be unreliable, or may require caseby-case tuning, and thus seem less suitable for practical use.
10-12
Vascular and Intravascular Imaging Trends, Analysis, and Challenges, Volume 1
https://t.me/medicina_free
10.4.5 Gradient-based methods
In gradient-based methods [ 40–42] a small subset of high-magnitude gradients (the
edges of structures of interest) can be corresponded between 3D and 2D images
through projection [40], backprojection [41, 43] or reconstruction [42]. The advantage of these methods is that the process of corresponding subsets of 3D and 2D
gradients, or gradient distributions encoded in gradient covariance matrices [43], is
more efficient than generating projection images such as DRRs or MIPs.
Furthermore, segmentation or feature extraction that may otherwise be modalityor anatomy-dependent is generally not required to extract the gradient features.
Hybrid feature- and gradient-based methods have also emerged, for example, to
register 3D and 2D cerebral angiograms. Mitrović et al [44] performed a model-toimage 3D–2D registration by matching geometric primitives of the 3D vessels such
as centerlines, radii and their principal local orientations to high-magnitude intensity
gradients of biplane x-rays. The aforementioned methods, however, were mainly
used in registration of 3D to biplane [44] or even multiplane x-ray images [41, 42].
When used for 3D to monoplane x-ray registration, the similarity measures
employed in these methods exhibit poor sensitivity to out-of-plane translations,
similarly to those used in the intensity-based methods.
10.5 Review of transformation estimation approaches
The registration basis, consisting of the image similarity measure and, possibly,
regularizer, defines a cost function, which needs to be optimized, i.e. minimized or
maximized, to estimate the transformation of either the 3D volume or the 2D
detector. A standard approach is to use iterative optimization techniques. Another
approach is by stratified optimization, which divides the transformation parameters
into subsets and estimates the parameters in each subset in a sequential order. There
are also regression-based approaches that employ multiple training datasets with
known optimal registration parameters to train a regression model, which is then
used to estimate the registration parameters for a new pair of 3D and 2D images.
General advantages and disadvantages of the three transformation estimation
approaches are summarized in table 10.2.
10.5.1 Iterative methods
Iterative optimization methods search for the optimal solution by minimizing or
maximizing a cost function consisting of the similarity measure between both
modalities and, in particular, deformable registration with an additional regularizer
of the transformation. Typically, the choice of the cost function affects the choice of
optimization methods.
In gradient-based optimizers, the gradients and/or Hessian of the cost function
are computed to determine the step or trust region size and step direction. If the cost
function is linear or quadratic in the vicinity of the initial alignment conditions then
gradient-based optimizers such as Newton, quasi-Newton and conjugate gradients
will exhibit fast convergence to the global solution. Commonly used gradient-based
10-13
Vascular and Intravascular Imaging Trends, Analysis, and Challenges, Volume 1
https://t.me/medicina_free
Table 10.2. Summary of advantages and disadvantages of transformation estimation approaches.
Estimation
approach Advantages Disadvantages
Iterative
Stratified
Regression-based
• Accurate
• Easy to implement
• Accurate
• Faster than iterative
• Less affected by initial
alignment
• Fast and fixed time
• Not affected by local optima
• Not affected by initial
alignment
• Variable convergence times
• Possible convergence to local
optima
• Affected by initial alignment
• Requires complex optimization
strategy for each subset
• Applicable for simple transformations (e.g. rigid)
• Less accurate
• Success depends on trained data
• Depends on accurate extraction of
features
optimizers in the literature are Broyden–Fletcher–Goldfarb–Shanno (BFGS), Polak
Ribiere and steepest gradient descent [2, 26, 27, 33]. If the cost function cannot be
negative as in distance-based similarities, it can be formulated as a regression
equation, then iterative nonlinear least squares methods such as Gauss–Newton and
Levenberg–Marquart are used to find the local solution [33]. Point cloud registration
methods such as iterative closest point can be solved by a least squares method,
which minimizes the squared distances between two points sets. If the energy
function is a well formulated Euler–Lagrange equation, consisting of internal and
external energy terms as in the case of deformable registration, gradient descent and
nonlinear conjugate gradient methods are frequently preferred [2, 26, 28].
In cases where analytical derivative computations are infeasible or computations
of finite differences is expensive or cannot approximate gradients accurately,
derivative-free optimization methods are employed. Some prominent examples are
best neighbor, Powell, Nelder–Mead (Down Hill Simplex) and BOBYQA. Powell
[6, 30] and best neighbor are the simplest algorithms, inspecting the cost function
with a step in all predetermined search directions and then taking the best one. They
are, however, prone to local extrema. Nelder–Mead [28, 29, 31, 45] evaluates the
function value at each vertex of a polytope and the one with the worst value is
replaced by another one using expansion, reflection or contraction operation.
BOBYQA uses a quadratic approximation of the cost function by interpolation in
a trust region and replaces an interpolation point in each iteration. Nelder–Mead
and BOBYQA are popular choices for non-convex functions since they are less
sensitive to local extrema than gradient-based optimizers. Henault et al [28] report
that the Nelder–Mead algorithm performs best in global alignment of vessel
centerlines for rigid and affine transformations. Nelder–Mead is the most frequently
10-14
Vascular and Intravascular Imaging Trends, Analysis, and Challenges, Volume 1
https://t.me/medicina_free
used derivative-free deterministic optimizer in 3D–2D angiography registration
literature.
The greatest difficulty that the gradient-based optimizers have is convergence to
local extrema. If the cost function is not convex around the initial alignment
conditions then they will converge to a local extrema. Several authors have tried
global optimization strategies to avoid the local extrema in cases of high or arbitrary
initial alignment offsets. Multi-resolution approaches start with low resolution
images and progressively proceed to higher resolutions after convergence in each
resolution [19]. Stochastic techniques generate random samples of search space and
a local optimizer is started at those positions. Stochastic methods are more suitable
for highly nonlinear functions with abundant local extrema. Some examples used are
random search, simulated annealing, evolutionary algorithms and sequential Monte
Carlo random sampling [28, 30, 46, 47]. Random search is the simplest stochastic
method that samples new positions in a given radius of the current position.
Simulated annealing algorithm generates a new point randomly by a probability
distribution with a scale proportional to the current temperature, which determines
the distance of the new point from the current one. The algorithm accepts all new
points that lower the value of the cost function, but also, with a certain probability,
points that raise the value of the cost function. The algorithm systematically lowers
the temperature, storing the best point found so far. By accepting points that raise
the value of the cost function, the algorithm avoids being trapped in local minima,
and is able to explore globally for more possible solutions. The covariance matrix
adaptation evolution strategy (CMA-ES) [19] is an evolutionary algorithm where
new candidate solutions are sampled from a multivariate normal distribution. The
mean of the distribution is updated such that the likelihood of previously successful
candidate solutions is maximized. Mutation is performed by adding a random
vector, a perturbation with zero mean. Pairwise dependences between the variables
in the distribution are represented by a covariance matrix. The covariance matrix is
updated by the covariance matrix adaptation (CMA) method that maximizes the
likelihood of previously successful search steps. CMA-ES has very high probability
of convergence to a global solution for a large class of functions regardless of initial
conditions. Florin et al [47] report that a sequential Monte Carlo sampling and
condensation technique performs better than random search and gradient-based
methods for global optimization. Sequential Monte Carlo sampling draws samples
from a posterior probability distribution function whose weights are updated
according to the principle of importance density [48]. Otake et al [10] use a multistart strategy for searching the global optimum. In a global multi-start method, the
search space is partitioned to multiple subspaces and separate CMA-ES optimizers
are run in a parallel fashion.
Discrete optimization techniques discretize the parameter space into subintervals.
Recent Markov random field (MRF) formulations of a registration energy functional usually include a unary potential that represents the data term, and a pairwise
potential between labels of discretized parameter space that represents the regularization term. An MRF-based energy function is formulated as the sum of all
potentials. Any similarity measure can be approximated by MRF models [49]. The
10-15
Соседние файлы в папке Библиотека им академика М.И. Перельмана