Chau Chemometrics From Basics to Wavelet Transform

into sparse expansions. Thus, a signal can be accurately represented by only a small number of significant coefficients, and most coefficients have absolute values small enough to be neglected. Furthermore, the overcomplete representation of signal by WPT allows one to choose the appropriate representation (best basis) for the signal compression. However, the following two problems have to be resolved to guarantee the correct use of the FWT or WPT in practical applications:

1.How to find the best basis

2.How to determine the threshold value for discarding ‘‘unwanted’’ coefficients

Methods for Best-Basis Selection. Best-basis selection means finding the most efficient basis out of a given set of bases in the wavelet packet tree to represent a given signal. Thus, we intend to find a basis through which some coefficients have high values while the remaining ones have low values; in other words, we wish to differentiate coefficients within a given set as much as possible. One way of selecting an efficient basis from all possible orthonormal bases in the wavelet packet tree is to apply the entropy or information criterion because the amount of information is a measure of inequality of distribution. A basis with coefficients giving more or less the same values would yield a low information or high entropy value. The ‘‘best basis’’ can be defined as the basis giving the minimum entropy or maximum information for its distribution of coefficients. This best-basis definition requires the criterion of the coefficients selection and can also explicitly contain the criterion of the coefficient selection. For instance, the best basis can be specified as the basis with the minimal number of coefficients whose absolute value is higher than the assigned threshold.

In order to calculate the entropy value, several methods have been proposed in the literature. The Coifman--Wickerhauser entropy method and the Shannon--Weaver entropy method are most commonly used.

The Coifman--Wickerhauser entropy is expressed as

M

where pm = (wjp,m / w00 )2, M is the number of coefficients in the wjp , and

160 application of wavelet transform in chemistry

where qm = |wjp,m |2/ wjp 2 and |wjp,m | is the absolute value of the wjp,m andwjp is the RMS norm of the wjp . However, Equation (5.14) does not obey the theorem of additive measure of the information. Hence Equation (5.15) is usually used for Shannon--Weaver entropy calculation:

M

or to calculate the entropy value directly by the number of the coefficients whose absolute value is higher than the predefined threshold. As different methods lead to different results, it is hard to say which one is better. The most suitable method in a case should be chosen by considering the intended application and performing trial calculations.

Once the wavelet packet tree is set up, the entropy of each node (basis) can be determined by any of the methods described above. Then, the best basis can be searched by comparing the entropy values between two adjacent levels. Since each node in the binary tree represents a subspace of the original signal and each subspace is the orthogonal sum of its two children nodes, the search procedure can be performed by a comparison between the entropy value and the sum of entropy values of the two immediate descendants. Nodes with fewer entropy values will be selected as part of the best-basis. For example, in Figure 5.5, if the entropy value of w20 is greater than the sum of the entropy values w30 and w31, the two ‘‘children’’ nodes will be chosen as the best basis. If further decomposition is needed up to level j = 4, one should continue to compare the entropy values of w30 and w31 with that of their children nodes w40 and w41, w42 and w43, respectively. However, if the entropy value of w21 is less than the sum of the entropy values w32 and w33, the ‘‘parent’’ node will be chosen as the best basis. Even when decomposition goes on, the comparison stops at this level.

Methods for Determination of Threshold. In both the FWT and WPT compression, it is an important step to suppress the small coefficients by thresholding. A larger ε gives a higher compression ratio but a poorer reconstructed signal. Therefore, determination of the right threshold value is a key step for both compression procedures.

There are many ways to determine the threshold value, and they can be classified as follows:

1. The universal threshold. The threshold value of the method is defined as

ε = (2 log2 (S)) (5.19)

where S = N log2 (N) for the WPT and S = N for the FWT. This threshold must be applied to the data vector f that is normalized to the noise level 1. This means that one has to decompose the data to the resolution level j = 1 to obtain the detail at the first level, d1, and estimate the noise based on, for instance, the median of the d1:

Then, normalize the signal vector f by f/s before performing the data compression.

2.The threshold defined by the desired compression ratio (CR). CR is generally defined by N /N, where N is the data point number of the original signal and N is the number of the coefficients to be retained. In the cases

of a desired CR has been assigned, the threshold can be defined according to the CR value. For example, in Examples 5.1 and 5.2, if the desired CR is assigned as 16, that is, if we must retain 64 largest coefficients, the threshold will have the same value as the 64th coefficient in plot (b) of either Figure 5.3 or 5.7.

3.The threshold defined by the (root) mean-square error of the reconstructed signal. This is the most commonly used method because the (root) mean-square error is a measure of the quality of the compression. A large error means that a significant portion of the useful information of the signal is lost after the compression, while an excessively small error will affect the compression efficiency. An effective way to determine the threshold value for the method is by trial and error, which is generally time-consuming. Fortunately, when the basis of the WT is orthonormal, the mean-square error of the reconstructed signal equals the sum of the squared coefficients suppressed from the vector w. The quality of the compression can be easily evaluated directly by the suppressed coefficients without reconstruction. In

some cases, the threshold value can also be defined by acceptable local pointwise error of signal reconstruction.

4. The minimum description length (MDL) criterion. MDL is a criterion for optimization of the retained largest coefficients and the error of the reconstructed signal. Use of the MDL criterion allows an objective threshold selection, which can be particularly useful for real data where the noise level is difficult to estimate. The MDL is defined by

where N denotes the length of the signal f, n describes the filter, k (0 ≤ k < N) denotes the number of nonzero elements in the vector wnk , I is the N-dimensional identity matrix, k is a thresholding operation that keeps the k largest elements (in absolute value) and sets all other elements to zero, and (I − k )wnk f represents the error between the original signal and the reconstructed signal with the k largest elements.

The first term of the MDL cost function can be considered as the penalty function of retaining the nonzero wavelet coefficients, whereas the second term describes the error between the original signal and the reconstructed signal with the k largest wavelet coefficients. These two terms of the MDL represent opposing demands. We would like to have k as small as possible in order to compress the data. At the same time we would like to minimize the distortion between the estimated and the true signal. But a larger k value usually gives a smaller error. By adding the linear penalty function to the log term, we can observe the minimum of the MDL cost function for a small amount of k values. We are not interested in a k near N, because this makes no sense in terms of data compression.

Besides, there are still other methods to determine the threshold value, and some of which are discussed in Section 5.2.2, because they are used for data denoising.

Example 5.3: Data Compression of an Experimental NMR Spectrum.

Figure 5.8 shows a NMR spectrum of a biological sample measured on a Bruker DMX 500 NMR spectrometer. It consists of 32,768 data points. In order to compress the spectrum using WT, we can first represent the spectrum as a vector c0 and scale its values into the range of 0--1, then perform a FWT to the data vector according to the process described in Section 5.1.1. Figure 5.9 shows the wavelet coefficients obtained with Symmlet10 (L = 20) filters and biggest resolution J = 10. In order to see clearly, the figure was enlarged and clipped into the magnitude range within ±0.01. It

Figure 5.10. Comparison of the experimental NMR spectrum (a) and the reconstructed spectra by WT using 2048 (b), 1024 (c), and 512 (d) coefficients.

Computational Details of Example 5.3

1.Load the experimental NMR signal with 32,768 data points (see Fig. 5.9).

2.Make a wavelet filter---Symmlet10.

3.Set resolution level J = 10 (15 − 5).

4.Perform WT to obtain the wavelet coefficients.

5.Apply thresholding to the wavelet coefficients keeping the 2048, 1024, 512 largest coefficients, respectively.

6.Construct the signal by applying inverse WT to the retained coefficients, respectively.

7.Display Figure 5.9.

8.Display Figure 5.10.

9.Display the RMS between the original signal and the reconstructed signals.

Note: Results in Table 5.1 can be obtained by changing the wavelet filters in step 2, and the results in Table 5.2 can be obtained by changing the resolution level in step 3.

As has been stated above, different wavelet filters and different decomposition levels will result in different effects of compression. In order to obtain optimal filter and resolution level J for the spectrum, the RMS

Table 5.1. RMS between Experimental NMR Spectrum and Reconstructed Signal by WT with Different Wavelet Filters and Resolution Level J = 10 (CR = 64)

between the original measured spectrum and reconstructed signal by different filters and different resolution levels should be investigated.

Table 5.1 summarizes the RMS by different wavelet filters with decomposition level J = 10 and CR = 64. Filters including Haar, Daubechies (L = 4--20), Symmlet (L = 8--20), and coiflet (L = 6--30) are investigated. It can be seen that different filters give different RMS results and the Symmlet (L = 18) gives a slightly better filter for compression of this NMR spectrum, yet the difference among these filters are not significant.

Table 5.2. RMS between Experimental NMR Spectrum and Reconstructed Signal by WT with Wavelet Filters of Coiflet, Daubechies, and Symmlet and Different Decomposition Level (CR = 64)

Figure 5.11. An enlarged plot of the WPT coefficients obtained by applying WPT to the experimental NMR spectrum using the filters of Symmlet6 (L = 12) and resolution level J = 10.

Table 5.2 lists the RMS by coiflet (L = 18), Daubechies (L = 12), and Symmlet (L = 18) filters at different decomposition levels. The CR is also 64. It can be seen that, for all the three filters, the RMS reaches a minimum when decomposition level J is larger than 10. However, the RMS decreases with the increase of the decomposition level within J = 6--10.

WPT can also be used for compression of the NMR spectrum. Figure 5.11 shows the coefficients obtained by Symmlet6 filter and decomposition level J = 10. The best basis was searched by the Coifman-- Wickerhauser entropy method. Figure 5.12 compares the original NMR

5.2. DATA DENOISING AND SMOOTHING

Noise in chemical signals is generally defined as the instantaneously irreproducible signals caused by interfering physical or chemical processes, imperfections in the experimental apparatus, and other irregularities, by which the experimental results are often complicated. Therefore, denoising and smoothing is a problem of interest in all fields of science and technology, and a large number of filtering methods have been developed, such as the Fourier filtering method, the Savitsky--Golay smoothing method, and the Kalman filtering method.

Figure 5.12. Comparison of the experimental NMR spectrum (a) and the reconstructed spectra by WPT using 2048 (b), 1024 (c), and 512 (d) coefficients.

Strictly speaking, denoising removes the small-amplitude components of the transformed signal regardless of the frequency, whereas smoothing minimizes the fluctuation in the signal by removing the high-frequency components regardless of amplitude [see Anal. Chem. 69:78 (1997)].

The underlying philosophy of the denoising by WT is generally thought to resemble the traditional Fourier filtering, in which the high-frequency components are cut off by lowpass filters. But strictly speaking, this kind of filtering should be classified into smoothing according to the definition given above. The principle of WT denoising and smoothing is in fact similar to that of compression by WT, which suppresses the small elements from the coefficient vectors. In the following, both denoising and smoothing are discussed in further detail.

5.2.1.Denoising

According to the principle of the WT denoising, the procedure of denoising with WT can be summarized as follows:

1.Apply a WT to noisy signal fnoisy and obtain the wavelet coefficient vector w.

2.Suppress those elements in w that are thought to be attributed to noise by thresholding.

3.Apply the inverse transform to the suppressed w to obtain the denoised signal fdenoised.