Добавил:
kiopkiopkiop18@yandex.ru t.me/Prokururor I Вовсе не секретарь, но почту проверяю Опубликованный материал нарушает ваши авторские права? Сообщите нам.
Вуз: Предмет: Файл:

Ординатура / Хирургия / Библиотека им академика М.И. Перельмана / Книга_3592_Библиотеки_им_академика_М_И_Перельмана

.pdf
Скачиваний:
0
Добавлен:
29.08.2026
Размер:
89 Мб
Скачать
IOP Publishing
https://t.me/medicina_free
Vascular and Intravascular Imaging Trends, Analysis, and
Challenges, Volume 1
Stent applications
Petia Radeva and Jasjit S Suri
Chapter 7
Blind inpainting and outlier detection using
logarithmic transformation and total variation
Manya V Afonso and J Miguel Sanches
In this work, we address the problem of image reconstruction with missing pixels or outliers, when the locations of the corrupted pixels are not known and when the noise is multiplicative, which is the algebraic model for ultrasound and synthetic aperture radar imaging. A logarithmic transformation is applied to convert the multiplication between the image, the binary mask and the Rayleigh distributed speckle noise into an additive problem. The image and mask terms are then estimated iteratively with total variation regularization applied on the image, and
regularization on the mask term, which imposes sparseness on the support set of
0
the missing pixels. The resulting alternating minimization scheme simultaneously estimates the image and mask, in the same iterative process. Experimental results show that the proposed method can deal with a larger fraction of missing pixels than two-phase methods, which rst estimate the mask and then reconstruct the image. Furthermore, it was experimentally observed that applying the algorithm on radio frequency (RF) images of the carotid artery or intravascular ultrasound images led to an outlier map which clearly delineates the lumen, and can therefore be applicable for segmentation.
7.1 Introduction
Faulty imaging sensors or bit errors during transmission can cause some pixels in an image to be lost or corrupted by impulse noise [14, 38]. In the case of missing pixel values, the corrupted pixels are assumed to have a value equal to zero, and the problem of estimating the complete image is called the inpainting problem [11, 42, 53]. When the image is corrupted by impulse noise, the corrupted pixels have values different from zero, which are drawn from a binary distribution (salt-and-pepper noise), or from a uniform distribution (random valued impulse noise).
doi:10.1088/2053-2563/ab01fach7 7-1 ª IOP Publishing Ltd 2019
Vascular and Intravascular Imaging Trends, Analysis, and Challenges, Volume 1
https://t.me/medicina_free
Ultrasound images and those from some other coherent imaging modalities are corrupted by a type of noise called speckle noise, which results from interference patterns. Although it is a random process, some works argue that it contains useful information about the microstructure of the tissue and the properties of the incident pressure eld [43, 49]. It has been shown that the observed ultrasound RF image can be well approximated as an element-wise multiplication between the image pixels which represent the morphological structure of the organ being imaged, and the speckle eld [43]. The statistics of the speckle noise in ultrasound RF images has been assumed to follow the Rayleigh distribution [1, 30, 58]. Other non-Rayleigh distributions such as the gamma distribution [55, 57] and Rayleigh mixture models [52] have also been considered in the literature.
Despeckling methods seek to reduce the speckle noise and make the morpho­logical structures more clearly visible. Denoising methods developed for this modality take into account the multiplicative algebraic relation and the Rayleigh statistics of the noise [4, 13, 50, 64]. Some methods characterize the spiky components of the speckle as outliers which do not t in the statistical distribution assumed. In [43], an outlier shrinkage step is applied on the log-transformed image to eliminate these outliers. An adaptive window method to compute local statistics and to discard local extrema and replace them by average values based on the local statistics on the B-mode image was proposed in [56]. It was reported in [65] that outliers in ultrasound images resulted from tissue shifting and made image registra­tion difcult.
Image reconstruction and inpainting methods for multiplicative noise generally require the pixel locations of the outliers to be known and therefore cannot be applied to estimate the pixel values at the locations of the outliers. In [5], a new method for blind image inpainting was proposed, which estimates the values of pixels which are missing or corrupted with impulse noise, when their locations are unknown. This method was also extended to non-additive and non-Gaussian noises. In this chapter, we review this method for the case of Rayleigh multiplicative speckle noise, and show that applying it on an ultrasound image can be used as a segmentation method.
The image to be estimated has n pixels and is represented as a vector, say in lexicographic ordering,
n
. Let
be the number of observed pixels or
<mn
pixels free from impulse noise. The loss of pixels or observing a partial set of m pixels out of n can be represented as an element-wise multiplication of the image with a binary mask in which all but m pixels are zero. In our representation, this observation process is represented as a multiplication of the vector n × n identity matrix
missing pixels set to zero.
nm
)
For multiplicative noise, the mapping from
with the respective diagonal elements corresponding to the
to the partially observed image
with a size
given by
is
ηyAx(),
S
7-2
(7.1)
η
η
1
1
1
1
Vascular and Intravascular Imaging Trends, Analysis, and Challenges, Volume 1
https://t.me/medicina_free
where the speckle noise term
is Rayleigh or Gamma distributed, and the
S
multiplication is element-wise.
When the image is corrupted by impulse noise, those pixels of the zeros on the diagonal of the observation matrix to the noise eld
Our goal is to estimate the image estimate the matrix
,
I
ηη=·+−yAx I A()( ).
G
n
from the partial observations
which indicates which pixels are outliers.
have a value that corresponds
I
corresponding to
(7.2)
and also
7.1.1 Related work
For standard inpainting, i.e. when the index set of the observed pixels or the observation mask
is known, the reconstruction problem can be solved by one of several existing methods for image reconstruction from a sparse set of observations. Many of these methods, such as [2, 9, 10, 22, 32, 37, 41], were developed for the additive and Gaussian noise model and often in the context of compressed sensing [16, 26] reconstruction. However, it must be noted that in compressed sensing the observation operator needs to satisfy conditions to lead to incoherent observations [17]. For the removal of impulse noise, a common approach is to estimate the support set of the noisy pixels using an outlier detection method, to obtain an estimate of
and then apply the reconstruction method.
For the additive and Gaussian noise model, there exist several methods for reconstructing an image with missing data and removing impulse noise. Early approaches for estimating the missing values used median ltering, which discards outliers [38]. The adaptive median lter (AMF) [38] and adaptive center weighted median lter [21] were developed to detect the positions of noisy pixels with, respectively, salt-and-pepper and random valued impulse noise. Median lters rely on the accuracy of the neighboring pixels and inherently cannot deal with a large percentage of outliers. In [25] an improved outlier detector based on thresholding, a measure called the rank-ordered logarithmic difference with edge preserving regularization (ROLD-EPR) was proposed, for random valued impulse noise.
Two-phase methods for estimating the image involve a mask estimation step, in which the observation mask
is estimated using outlier detection, and a recon­struction step in which a standard convex optimization procedure is used with the estimate of impulse noise is a sum of an regularizers, leading to an using either the
[15, 19]. Another commonly used formulation for denoising with
-norm data delity term and total variation (TV) [20, 48]
-TV optimization problem [15]. Sparse regularization
or
norm regularizers on the impulse noise term or support set of
0
the outliers was used in [15, 24, 60] and [62]. An iteratively reweighted least squares (IRLS) based method for mixed impulse and Gaussian noise removal was proposed in [46, 47]withan
data delity term corresponding to the observed pixels, an
2
term
corresponding to the noisy pixels and TV regularization on the image.
Several of the above methods are based on alternating minimization, where two
or more variables are iteratively estimated through a Gauss–Seidel method [44].
7-3
Vascular and Intravascular Imaging Trends, Analysis, and Challenges, Volume 1
https://t.me/medicina_free
A related approach is the augmented Lagrangian (AL)/alternating direction method of multipliers (ADMM) framework [29], which has been used extensively in recent work on image reconstruction/restoration because of its mathematical elegance and computational speed [37, 63].
For the Poisson noise model, there exist methods for image reconstruction from a partial set of pixels with the observation matrix known, such as [33, 34, 40]. Another method proposed in [8] lled in missing data through the minimization of the image gradient and an approximate solution of the mean curvature ow equation. Patch­based dictionary learning methods for the Poisson model were proposed in [35] for image denoising and in [36] for inpainting.
For the Rayleigh speckle noise model, existing methods such as the classical interpolation methods for ultrasound [54], as well as those based on TV regulariza­tion after logarithmic compression [6, 49], all require the sampling matrix to be known. A denoising method for ultrasound with an outlier shrinkage step applied on the log-transformed image was proposed in [43].
7.1.2 Contributions and organization
In [5], we proposed a method to estimate the image
and the observation mask simultaneously, for the additive and Gaussian noise model. We formulated the masking operation as a summation after logarithmic compression, and applied a TV regularizer on the term corresponding to the logarithm of the image, and an
-norm
0
regularizer on the term corresponding to the mask. The TV regularizer encourages the estimate of
to be piece-wise smooth, while the
-norm regularizer encourages
0
the mask term to be sparse. The problem was solved iteratively using a Gauss–Seidel alternating minimization scheme. This method was extended to the multiplicative and Rayleigh distributed speckle noise, and Poisson noise models. The data delity terms corresponding to these statistical models allow the relation between the image and observation mask to remain additive after logarithmic transformation.
In this chapter, we review the blind inpainting method for multiplicative and Rayleigh distributed noise, and present results for outlier detection in ultrasound images. We show that applying the blind inpainting algorithm on ultrasound images of the carotid artery, without loss of pixels, produced outlier maps which corre­sponded roughly to the lumen and can be useful for segmenting the images.
We formulate the estimation problem and the blind inpainting algorithm in section 7.2.2. In section 7.3, we present experimental results on inpainting. Sub­section 7.3.2 presents results for the application of blind inpainting for segmentation of the lumen.
,
7.2 Blind inpainting
We begin with the method for blind inpainting for the additive and Gaussian noise model, which is mathematically simpler, and then elaborate the method for multi­plicative and Rayleigh distributed noise. In section 7.2.1, the observation model is
7-4
η
λ
λ
1
1
Vascular and Intravascular Imaging Trends, Analysis, and Challenges, Volume 1
https://t.me/medicina_free
where
is an additive Gaussian noise term.
G
η=+yAx(),
G
(7.3)
7.2.1 Blind inpainting for additive noise
For standard inpainting with TV regularization on the image and with the observation mask
known, the problem of estimating
as formulated and solved
in [ 2, 22]is
λ
2
(),
is the regularization
> 0
(7.4)
where
ˆ
=∥+TVxAxyxarg min
is the isotropic total variation function and
TV (.)
1
x
22
parameter. Note that this problem assumes the additive and Gaussian noise model and has an
When we need to estimate both the image applied on
where we now have two regularization parameters,
-norm data delity term.
2
, leading to
ˆ
ˆ
=∥++TVxA Ax y x A(, ) argmin
1
xA,
22()2
and the mask
λλ
12
2
λ >,0
12
, a regularizer
( ), (7.5)
ϕ
ϕ(.)
, for each of the two regularizer terms. The problem (7.5) is difcult to solve because it is not separable for our variables
xA
,)
.
In [5], a logarithmic transform was used to convert the masking problem into an
additive and separable one. Since
is a diagonal matrix
=
adiag( )
and the masking operation is element-wise multiplication, the variable of optimization for the observation matrix is the vector of diagonal elements with index i is observed, the corresponding mask element lost,
. Thus, a pixel k in vector
=a 0
i
is dened as the scalar product,
yxa.
ii
i
=a
i
n
{0, 1}
. When a pixel
, and when pixel i is
(7.6)
is
It is not known a priori if a given pixel ykcorresponds to an observed one ( not (
). Rather than have
=a 0
k
dened to be a small value in the order of that depends on the dynamic range of the image, typically greater than or equal to 3. Dening
= alog( )
ii
Assuming that
on equation (7.6) converts it into an additive model,
when the pixel is not observed, where aiis
=a 0
i
−0K
or smaller, K being a positive integer
,
0, if is observed
v =
i
−iK
,otherwise
and
are always positive, applying a logarithmic transformation
yxalog log( ),
i
ii
. (7.7)
7-5
)or
=a
k
(7.8)
g
0
λ
λ
Vascular and Intravascular Imaging Trends, Analysis, and Challenges, Volume 1
https://t.me/medicina_free
where
=
ii
and
xlog
=+yx alog log log ,
i
i
i
ylog( )
i
ii
v=+gu ,
ii
. A small positive bias term
δ=+
δ > 0
is added to
(7.9)
(7.10)
to guarantee positivity. The base of the logarithm used is 10, but we could use the natural logarithm with the only difference being an additive constant term. The problem is now estimating the vectors We assume that our image piece-wise smooth. We apply a TV regularizer on the log-transformed image
-norm regularizer on the log-transformed mask
0
correspond to the non-observed pixels and those elements of 0 correspond to the observed pixels. Since the elements irrespective of their sign, minimizing
and
, given the log-transformed observationg.
and therefore its logarithmic transformation
. The negative elements of
therefore
which are equal to
-norm indicates the number of non-zero
0
minimizes the number of
v
are
,andthe
non-observed pixels.
The problem therefore becomes
λλ
12
2
TV( )
2
2
, (7.11)
0
where
ˆˆ
λ >,0
12
=∥++uv guv u v(, ) argmin
1
uv,
22
are the respective regularization parameters.
Since equation (7.11) is a separable problem, we can apply an iterative alternating method as in [12]. We apply an iterative alternating minimization to solve equation (7.11), by isolating the terms in each variable keeping the other xed, leading to a Gauss–Seidel scheme. Solving for
at iteration t,
tt() ()
ˆ
=∥+u guv uarg min
1
u
22
λ
1
2
TV( ).
2
(7.12)
This is a TV regularized denoising problem, the solution of which can be computed efciently using an algorithm such as Chambolles algorithm [18].
Similarly, for
at iteration t we have
tt() ()
ˆ
=∥+vguvvarg min
1
u
22
λ
2
2 2
.
0
(7.13)
This problem although non-convex, has a solution given by the hard threshold [27]
where
tt()
ˆ
Hvgu( ), (7.14)
=−
λ
2
2
is the hard threshold operator and is dened element-wise as
(.)
λ
2
0, if g u 2 ,
t
()
v
=
i
()
t
()
gu
i
i
,otherwise.
()
t
()
i
i
()
−⩽
λ
2
(7.15)
The steps (7.12) and (7.13) are run alternatingly until the stopping criterion is satised. Continuation schemes can be used on the regularization parameters
λ,
12
in which they are multiplied by a factor greater than one, until they reach a certain
7-6
,
Vascular and Intravascular Imaging Trends, Analysis, and Challenges, Volume 1
https://t.me/medicina_free
maximum value, as was done in [59]. The estimates of the image and mask are computed by inverting the logarithmic transformation,
ˆ
u
ˆ
10
=
and
ˆ
ˆ
v
. The
=
10
conditions for convergence [29, 34] do not require equation (7.12) to be solved exactly, as long as the error sequence decreases and the parameter μ is positive.
7.2.2 Blind inpainting for Rayleigh multiplicative noise
We now extend the method described in the previous sub-section to the multi­plicative and Rayleigh distributed noise model. An intuitive way would be to apply the logarithmic transform to convert the observation model (7.1) into an additive one. Then, we could apply the method described in the previous section. However, this approach does not take into account the statistical model of the noise and the appropriate data delity term for Rayleigh speckle noise.
We therefore extend the blind inpainting method to Rayleigh distributed multiplicative speckle noise by using the appropriate data delity term. For multiplicative noise, multiplying a pixel whose value is 0 will always lead to the corresponding observed pixel being equal to 0 as well. Therefore, we interchange the order of the noisy observation and masking so that our observation
is the result of observing the masked imagexunder
the noise model. The observation model changes from equation (7.1)to,
ηyAx() .
S
(7.16)
For the Rayleigh multiplicative noise, the likelihood function is
2
y
yAx() exp
∣=
p
=
in1
ax
ii
i
⎛ ⎜
⎜ ⎝
y
i
ax
2( )
ii
⎞ ⎟
. (7.17)
⎟ ⎠
It is straightforward to show that (see [51] for more details) the associated data delity term between
In equation (7.18), we see that there appears a term with the product
andxis
J
r
2
y
i
=+
ax
2( )
=
in1
log( ) . (7.18)
ii
axyAx(, )
ii
⎞ ⎟
⎟ ⎠
ax
ii
and a
)
term involving its logarithm. Therefore, as before, we can work with the log­transformed variables
=
J
r
and
log
=++
=
in1
=
2
y
i
⎜ ⎜
2
. Thus equation (7.18) changes to
alog
−+
u
()
ii
euyuv(, , )
v
⎞ ⎟
v
. (7.19)
ii
⎟ ⎠
We now formulate our optimization problem, once again with TV regularization on
and
regularization on
0
. The data delity term
is changed accordingly.
(.)
The problem (7.11) for the additive Gaussian noise model changes to the more general problem
λλ
ˆˆ
=++JTVuv yuv u v(, ) argmin (, , )
uv,
12
()
2
. (7.20)
0
2
7-7
μ
Vascular and Intravascular Imaging Trends, Analysis, and Challenges, Volume 1
https://t.me/medicina_free
Since equation (7.19) involves the sum of a linear term and an exponential term, it is non-separable for
and
. Therefore, we need to use variable splitting [23]tobe able to use the augmented Lagrangian/alternating direction method of multipliers (AL/ADMM) to solve equation (7.20). We therefore introduce two auxiliary variables
andwto act as the arguments of the TV and
regularizer terms,
0
respectively, leading to the constrained problem
λλ
12
JTVyuv z w
min ( , , )
uvzw,,,
subject to , .
++
==
uzvw
()
2
2
0
(7.21)
Using the augmented Lagrangian [39, 45], this problem can be shown to be
equivalent to the minimization problem
λλ
12
++
zw
()
2
2
2
2
2
d,
zw
0
2
,
2
are the so-called Bregman
(7.22)
where
μ ,0
12
JTVyuv z w
min ( , , )
uvzw,,,
μμ
1
+∥−−∥+∥−−∥
uzd vwd
22
are the penalty parameters, and update vectors [37]. This problem is split into four problems at each iteration by gathering all the terms in each variable, and solving for each by keeping the others xed. Thus, the AL algorithm iterates between minimizing the objective function in equation (7.22) with respect to
and
, leading to a Gauss–Seidel process (for more
details, see [2, 3, 31] and references therein) which at iteration t is summarized as
μ
+
ttt
(1) ()
=+
+
ttt
(1) ()
=+
+
tt
(1)
=−+
+
tt
(1)
wvwdwarg min
=−+
Ju yuv u z darg min ( , , )
u
Jvyuvvwdarg min ( , , )
v
μ
1
()
z
22
μ
2
()
w
22
t
(1) ()
+
ddz u
z
t
(1) ()
+
ddw v,.
w
t
=+ −
z
t
=+ −
w
As in the case of Gaussian noise, the
1
()
2
μ
2
()
2
2
λ
t
()
z
()
w
tt
(1) (1)
++
tt
(1) (1)
++
-TV denoising problem from equation
2
1
2
2
t
λ
2
(7.25) is solved using a few iterations of Chambolles algorithm and the
2
t
()
z
2
2
t
()
w
2
TVzuzdzarg min
()
2
0
(7.23)
(7.24)
(7.25)
(7.26)
−ℓℓ
20
regularized denoising problem from equation (7.26) is solved using the hard threshold. The problems involving
, equations (7.23) and (7.24) can be solved
(.)
approximately using a few iterations of Newtons method [44], after plugging in equation (7.19).
7-8
λ
μ
λ
1
Vascular and Intravascular Imaging Trends, Analysis, and Challenges, Volume 1
https://t.me/medicina_free
The proposed method for blind inpainting with Rayleigh multiplicative noise is
summarized in the following algorithm.
Algorithm. Blind inpaintingnon-Gaussian noise
1. Input
2. Initialize parameters
.
(0) (0) (0) (0)
vzwdd,,, ,,
δ > 0
(0) (0)
.
zw
,
>K 0
,
,
λ,
12
, initial estimates
μ,
12
3. Set t = 0.
4. Repeat.
+
5. Compute
6. Compute
7. Compute
8. Compute
t
(1) ()
+
9.
z
t
(1) ()
+
10.
w
11. Update values of
12.
←+tt
t(1)
using Newtons method to solve (7.23).
+
t(1)
using Newtons method to solve (7.24).
+
t(1)
using Chambolles method to solve (7.25).
+
tt
(1)
←−
Hwvd
λμ
/
()
2
t
tt
z
t
w
(1) (1)
(1) (1)
←+ −
dz u
←+ −
dw v
2
++
tt
++
.
λ,
12
()
t
()
.
w
.
.
13. Until the stopping criterion is satised.
14. Set estimates
u
ˆ
= e
v
ˆ
,
= e
.
7.3 Experimental results
In this section we compare our proposed method for blind inpainting, with inpainting using the additive model after logarithmic transformation. In the synthetic experiments with the Lena and Cameraman images, we have the noise free image for reference and use the normalized mean absolute error (NMAE) [28], which is dened as
−ˆ∥∥∥xx x/
of the mask is measured in terms of the number of incorrectly estimated mask pixels, which is obtained by the binary exclusive or XOR operation between the estimated mask and the reference. A logical value equal to 1 is obtained at the mask pixels estimated incorrectly, and zero otherwise. Hence, the sum over all the pixels of the logical XOR operation is a measure of the errors in the estimate of the mask. All experiments were performed on MATLAB on an Ubuntu Linux based server with 64 GB of RAM.
Results for the additive and Gaussian noise and Poisson noise models can be
found in [5].
7.3.1 Blind inpainting
To test our proposed method, we generate a random binary mask with a fraction of its elements equal to zero and multiply it element-wise to our image corrupted with multiplicative Rayleigh noise. The criteria used to evaluate the accuracy of estimation are the NMAE, the structural similarity index measure (SSIM) [61] and the fraction of incorrectly estimated mask pixels.
, the gure of merit. The accuracy of the estimation
11
7-9
Vascular and Intravascular Imaging Trends, Analysis, and Challenges, Volume 1
https://t.me/medicina_free
We normalize the Lena image by dividing by the maximum pixel value. To compare our method, we use a logarithmic transformation followed by inpainting using a method for inpainting with additive noise. In this case, we use fast two-phase deblurring [15], since AOP [62], although faster, also divides the observed image by 255, and in the case of Rayleigh speckle, the image is already normalized. We summarize our results for the Lena and cameraman images for different fractions of missing pixels, in table 7.1. We can see that taking into account the statistical model offers an improvement in terms of the NMAE. The existing methods for blind inpainting do not always work well for large fractions of missing pixels, above
0%
or take a long time, over 10 min.
For the Lena image, a cropped region from the noisy image with
of the pixels
0%
missing is shown in gure 7.1(b) and the respective estimates using the proposed
Table 7.1. Inpainting with Rayleigh noise. κ indicates the fraction of missing pixels. (*) The additive model is used after logarithmic transformation [5] (© 2015 IEEE).
Lena Cameraman
κ Method Time (s) NMAE SSIM Mask
err. (%)
0.973 0.664 15.5 1.23 × 10
0.958 0.000 19 17.3 1.53 × 10
0.972 0.317 24.4 1.26 × 10
0.958 4.9 19.1 1.53 × 10
0.972 0.0835 156 1.27 × 10
0.25 Proposed 41 3.12 × 10
Additive(*) 11.3 3.81 × 10
0.5 Proposed 66.8 3.16 × 10
Additive(*) 75.8 3.82 × 10
0.7 Proposed 116 3.17 × 10
06
06
06
06
06
Time (s) NMAE SSIM Mask
05
05
05
05
05
err. (%)
0.975 11.6
0.961 0.001 14
0.974 2.13
0.961 4.29
0.973 0.484
,
Figure 7.1. Blind inpainting with Rayleigh noise with the Lena image: (a) original image (cropped), (b) observed image with Rayleigh noise and method; (d) estimate using inpainting with the additive model after logarithmic transformation; (e) observed image with Rayleigh noise and [
5] (© 2015 IEEE).
of its pixels missing; (c) estimate from (b) using the proposed
50%
of its pixels missing; and (f) estimate from (e) using the proposed method
70%
7-10