Kluwer - Handbook of Biomedical Image Analysis Vol

p ← Z(i, j) − Z(i, j − 1) q ← Z(i, j) − Z(i − 1, j)

end do

end do

Normalize(Z(x, y), Zmax, Zmin) Output Z(x, y)

The subfunction Normalize is a standard math function used in signal and image processing.

We now demonstrate this method by using the following example.

Example 6. Reconstruct the surface of a synthetic vase using the Zheng– Chellappa method.

The experiments are based on the synthetic images that are generated using true depth maps. Figure 5.4(a) shows the same synthetic vase as in the previous section and the reconstruction results using Pentland’s algorithm. The light is from above at (x = 0, y = 0, z = 1). The input image is showed in Fig. 5.4(a). The surface, shown in Figs. 5.4(b), (c), and (d), is the reconstructed depth map from three different directions. Zheng–Chellappa algorithm produces reasonable results as expected for the vase. However, some errors can be seen around the boundary of the vase. In general, the experiment shows that Zheng–Chellappa’s algorithm can reasonably reconstruct the object on the surface. The most important advantage of Zheng–Chellappa’s algorithm is that the optimization approach is not limited to the situation where the reflectance map changes linearly with respect to the surface shape.

Example 7. Reconstruct the surface of a synthetic Mozart using Zheng– Chellappa’s method.

The experiments are also based on the synthetic images that are generated using true depth maps. Figure 5.5(a) shows the synthetic Mozart and the reconstruction results using Zheng–Chellappa’s algorithm. The light is from above at (x = 0, y = 0, z = 1). The input image is showed in Fig. 5.5 (a). The result image, shown in Figs. 5.5(b), (c), and (d), is the reconstructed depth map from three different directions. The recovered surface is well outlined as expected for the human’s head. However, the details of Mozart cannot be accurately recovered using their approach. In our opinion, this is due to the rapid changes and complexity of the input image. Although the results can be improved by prefiltering and smoothing the input image, in general, we conclude

Figure 5.4: Zheng–Chellappa’s linear SFS algorithm applied to the synthetic vase image. (a) is the input image with light source (x = 1, y = 0, z = 1). (b), (c), and (d) are the reconstructed surface from three different directions.

from the experiment that Zheng–Chellappa’s algorithm does encounter some difficulties when the input image is complex. This observation is also true even if we used the simplest light source direction. We expect this experiment to inform the readers that SFS problem is indeed one of the most difficult problems in computer vision. No perfect, or even satisfactory, solution has been proposed yet.

We summarize this section with a few words about the advantage and disadvantage of these two methods we introduced in this section. Pentland’s method uses FFT and IFFT to calculate the depth map. This makes the algorithm relatively nonsensitive to the initial values. However, there are a few disadvantages:

(1) When the light source direction and the viewing direction are similar, the Fourier transforms of p2 and q2 will have a doubling effect in the frequency

Figure 5.5: Zheng–Chellappa’s linear SFS algorithm applied to the synthetic Mozart image. (a) is the input image with light source (x = 0, y = 0, z = 1). (b), (c), and (d) are the reconstructed surface from three different directions.

domain, which will affect the accuracy of the linear approximation. (2) When applying FFT and IFFT to the whole image, Pentland’s algorithm needs more time than Tsai–Shah’s approach. Tsai–Shah’s algorithm uses Newton’s method to solve the quadratic equations. When the initial value is close to the exact solution, Tsai–Shah’s algorithm converges very fast. Actually, given certain good initial values, Tsai–Shah’s algorithm needs several steps to converge. However, it is well known that Newton’s method cannot always guarantee convergence. This disadvantage makes Tsai–Shah’s approach sensitive to initial estimation than Pentland’s.

The discussion in this subsection has also shown us that the linear approach is conceptually simple. The related algorithms are relatively fast and easy to

having the form

ψmn(t) = 2m/2ψmn(2mt − n), n, m Z,

where ψ (t) is the “mother wavelet.” Usually it is not constructed directly but rather from another function called the “scaling function” φ(t) L2(R). The

ing of closed subspaces of L2(R). The space Vm is taken to be the closed linear span of {φ(2mt − n)}n Z . Because of (5.62) (ii), the Vm are nested, i.e. Vm Vm+1 and because of (5.62) (iii), mVm is dense in L2(R).

There are many different types of wavelet bases created and employed for different purposes. They can be classified as time-limited wavelets, such as Haar wavelets and Daubechies wavelets, band-limited wavelets, such as Shannon and Meyer wavelets. Another standard prototype is the Haar system in which

φ(t) = χ[0,1](t), where

is the characteristic function of [0, 1]. It is an easy exercise to show that (5.62) is satisfied. This prototype has poor frequency localization but good time localization. Most of the other examples found, e.g., in [12] and [66], attempt to get fairly good time and frequency localization simultaneously.

The various scales are related by the dilation equation of the scaling function

√∞

n=−∞

√∞

where dn = c1−n(−1)n.

In addition, the Fourier transform of the mother wavelet ψ (t) vanishes in a neighborhood of the origin. We denote by Wm the closed linear span of {ψ (2mt − n)}. This set of functions form an orthogonal basis of L2(R). That is,

Vm = Vm−1 Wm−1

L2(R) = ∞m=−∞ Wm

For f L2(R), we have the projections onto the subspace Vm and Wm respectively given by

∞

n=−∞

where

∞

am,n = 2−m/2 f (x)φ(2mt − n)dx,

−∞∞

bm,n = 2−m/2 f (x)ψ (2mt − n)dx.

−∞

The coefficients aj,n and b j,n at resolution j = m and j = m + 1 are related by a tree algorithm. To see this, we space V1, we have two distinct orthonormal bases:

Let {Vm} be a multiresolution of L2(R); a separable two-dimensional multiresolution is composed of the tensor product spaces

φm,k,l (x, y) : = φm,k(x)φm,l (y).

Like in 1D case, we have

V 2 = V 2− W 2 − = (Vm Vm) W 2 − ,

m m 1 m 1 m 1

Wm2 = (Vm Wm) (Wm Vm) (Wm Wm),

and

L2(R2) = ∞m=−∞ Wm2 .

Wells et al. [67] proved the following theorem.

Theorem 8 (Wells and Zhou). Assume the function f C2( ), where is

|| f − fm||L2 ( ) ≤ C(1/2m)2,

where C is dependent on the diameter of , the first and second moduli of the firstand second-order derivatives of f on .

Formula (5.69) is the one which was used in the wavelet-based SFS method. Now we are ready to introduce this method.

5.4.2 The Wavelet-Based SFS

A wavelet-based method was developed in [31]. Instead of using the constraints in Zheng–Chellappa’s method (see Section 5.3.2.1), the authors introduced a new constraint (5.20). It is said that “the new constraint not only enforces integrability but also introduces a smoothness constraint in an implicit manner.” Now the energy function is defined as

The objective function is first replaced by its approximation in scaling space V0 of Daubechies wavelets. Then the variational problem is solved by an iterative algorithm. We now describe this method.

We assume that the given image size is M × M. The surface Z(x, y), its partial derivatives ∂∂Zx = p(x, y), and ∂∂Zy = q(x, y) have projection to V0, the scaling space at level 0: