Chau Chemometrics From Basics to Wavelet Transform

A slight change in the notation of f (k ) to f0(k ) is made here so as to indicate the signal before Fourier transformation. This equation can be written as follows:

It should be noted that w nk = e−2πj (nk /4). Here the symbol ξ is used to denote the remainder of the division of (kn) by 4. Then, only ξ is needed to be considered in the following calculation. For nk = 6, we have

w 6 = e−j 2π6/4 = e−j 2π(4/4)e−j 2π2/4 = e−( j 2π/4)2 = w 2

and ξ = 2. Equation (2.50) can now be written as

50 one-dimensional signal processing techniques in chemistry

The matrix containing 1 and w ξ terms can be factorized into two matrices as

Attention should be paid to the interchange of f (1) and f (2) in the matrix on the left-hand side of the equation. Computing f (n) via Equation (2.52) requires four multiplication and eight addition operations of complex numbers. In contrast, finding the value of the same elements through Equation (2.50) requires 16 complex multiplications and 12 complex additions. Hence, the total number of mathematical operations is reduced significantly with the use of Equation (2.52), instead of Equation (2.50). It is obvious that the reduction in operations becomes more dramatic when N is much greater than 4.

In MATLAB software, the fast Fourier transformation (FFT) has its specific statement. One can simply use one command to obtain the FFT result:

x = fft (y )

This makes Fourier transformation very simple using MATLAB. Here, fft (x) is the discrete Fourier transformation (DFT) of vector x. If the length of x is a power of 2, a fast radix-2 fast Fourier transform algorithm is utilized. If not, a slower non-power-of 2 algorithm is employed. For matrices, the FFT operation is applied to each column separately.

2.2.3.3.Fourier Transformation as Applied to Smooth Analytical Signals

The major feature of Fourier transformation is that it transforms analytical signals from the time or space domain into the frequency domain. So it is not strange that it can be applied to smooth noisy analytical signals. The reasoning behind is quite simple. In chemical study, noises are usually generated in instrumental measurement and are called white noises that obey the normal distribution of zero mean and equal variance. In general, noises are of high frequency while analytical signals are of low frequency in the time domain. Hence, after transforming the analytical signals into the frequency domain, if one discards the high-frequency part but keeps the low-frequency part, it is possible to eliminate the white noises present in the signals. An example is provided here to illustrate the treatment.

Figure 2.11 gives an example of showing how to use Fourier transformation to smooth a noisy signal. Plot (a) shows the original noisy signal.

Figure 2.11. Illustration of how Fourier transformation can be applied to smooth an analytical signal: the original noisy signal (a); the dependence of intensity on frequency after Fourier transformation (b); the recovery signal with the cutoff threshold frequency of 10 Hz (c); the recovery signal with a cutoff threshold frequency of 20 Hz (d).

The intensity distribution in the frequency range of 0--40 Hz after Fourier transformation is shown in plot (b). From this plot, it can be seen that the main part of the signal lies in the low-frequency part. The recovery signal obtained with the threshold of cutoff frequency of 10 Hz is shown in plot

(c). It can be seen that the recovery signal by inverse Fourier transformation is quite smooth with a slight distortion. Plot (d) shows the recovery signal with a cutoff threshold frequency of 20 Hz. From the plot, we can see that the recovery signal seems to be better than the one shown in plot

(c). Thus, the threshold of the cutoff frequency is an important parameter for smoothing in Fourier transformation. In order to make it easier for the readers to understand the smoothing procedure by Fourier transformation, a MATLAB source code is given as follows:

>%demonstration of FT applied to smoothing >x1=[0.001:.001:1.024];

>for k=101:1024

> x1b(k)=exp(-(k-100)/176) sin(2 pi (k-100)/160); >end

52 one-dimensional signal processing techniques in chemistry

>x2=[x1b]; >y=fft(x2);

>subplot(221),plot(x1,x2);grid on >xlabel(‘Time (s)’);ylabel(‘amplitude’) >axis([0 1.024 -1 1]) >freq=(0:160)/(1024 .001);

>subplot(222),plot(freq, abs(y(1:161)),‘.’) >xlabel(‘Frequency (Hz)’);ylabel(‘amplitude’) >xx1=ifft(y(1:18),1024); >subplot(223),plot(real(xx1))

>figure(2) >x2=x2+randn(size(x2)) .1; >yy=fft(x2); >subplot(221),plot(x1,x2);grid on

>xlabel(‘Time(s)’);ylabel(‘amplitude’) >axis([0 1.024 -1 1.4]) >freq=(0:40)/(1024 .001); >subplot(222),plot(freq, abs(y(1:41)),‘.’)

>xlabel(‘Frequency (Hz)’);ylabel(‘amplitude’) >axis([0 40/(1024 .001) 0 100]) >xx=ifft(yy(1:10),1024); >subplot(223),plot(x1,real(xx) 2);grid on >axis([0 1.024 -1 1.4]) >xx=ifft(yy(1:20),1024); >subplot(224),plot(x1,real(xx) 2);grid on >axis([0 1.024 -1 1.4])

2.2.3.4. Fourier Transformation as Applied to Convolution and Deconvolution

Convolution and deconvolution calculations are very important in analytical chemistry, especially in electroanalytical chemistry. Application of Fourier transformation for convolution and deconvolution is based mainly on the following convolution law of

where F (ν) and H(ν) correspond to the Fourier transformation of f (t ) and h(t ).

According to the convolution law, we can first apply Fourier transformation to the functions f (t ) and h(t ) to obtain the corresponding functions F (ν) and H(ν) in the frequency domain, and then carry out convolution calculations on functions F (ν) and H(ν). Finally, inverse Fourier transformation is employed to obtain the broadened function g(t ) (convolution calculation).

F (ν) · H(ν) = G(ν)

inverse FT

G(ν) −−−−−−→ g(t )

Conversely, since F (ν) = G(ν)/H(ν), the following procedure is utilized to obtain the original signal f (t ) (deconvolution calculation):

This algorithm is also called inverse filtering. It should be mentioned that the noise disturbance is completely ignored here. In fact, the measured signal is affected not only by the broadening function g(t ) arising from the width of the slit of a device but also by the measurement noise n(t ) as

Thus, the Fourier transformation G(ν) of the measured signal g(t ) should be expressed as follows after considering the additive property of Fourier transformation

Here F (ν) comes from Fourier transformation of an unknown original

signal function f (t ), while F (u) is its estimation from deconvolution calculation. One must also consider the influence of noise when performing deconvolution using Fourier transformation.

54one-dimensional signal processing techniques in chemistry

2.3.NUMERICAL DIFFERENTIATION

Numerical differentiation is a common tool used in analytical chemistry for processing one-dimensional signals because part of the useful information is often ‘‘hidden’’ in the usual measurement plot. The combined chromatographic response, for instance, of two poorly resolved components may not indicate the presence of two coeluting components. The chromatographic profile may appear to be a unimodel and may show no ‘‘shoulders’’ at all. Such ‘‘hidden’’ features can be enhanced by taking the derivative of a signal. A derivative is sensitive to the subtle features of its distribution and is therefore effective in detecting important yet subtle details such as shoulders. In this section, we discuss some common methods used for obtaining derivatives of signals.

2.3.1. Simple Difference Method

The direct-difference method is the simplest numerical differentiation method for analytical signals. For a discrete spectrum xi (i = 1, . . . , n), suppose that wi (i = 1, . . . , n) are its sampling wavelengths, the directdifference method can be expressed as follows:

Here yi (i = 1, . . . , n − 1) represent the series of derivatives of spectrum xi . If the sampling wavelength or time intervals are equal, this equation can be rewritten as

In MATLAB, there is such a function named ‘‘diff’’ for computing derivative spectrum just according to the above equation.

The direct-difference method is simple, but its drawback is also obvious. First, the number of points in the derivative spectrum as deduced in this way is less than one compared to the original discrete spectrum. This leads to the whole derivative spectrum moving half a point forward compared to the original one. Thus, the maximum and/or minimum obtained by the direct-difference method is not exactly the one of zero gradient. Hence, this method is good or acceptable for spectrum of high resolution but not for that of low resolution if one wants to obtain the exact maximum and/or minimum point from the original spectrum. Figure 2.12 shows the results obtained from the ‘‘diff’’ function of MATLAB for both highand low-resolution spectra.

Figure 2.12. The high- (a) and low- (c) resolution spectrum and their derivative spectra (b,d) as obtained by the MATLAB ‘‘diff" function.

2.3.2. Moving-Window Polynomial Least-Squares

Fitting Method

In Section 2.1.2, we discussed the Savitsky--Golay filter for smoothing, which is essentially a method based on moving-window polynomial leastsquares fitting. The filter can also be used for numerical differentiation. As it is based on polynomial fitting, one can directly perform derivation on the polynomial and then obtain the weighting expression for the central point in the moving window to give the derivative spectrum. It is worth noting that the derivative spectrum deduced by this method is better than that obtained by the direct-difference method because there is no point shift in the direct-difference method does.

Every point in a spectrum to be differentiated can be expressed by a polynomial as

xji +j = a0 + a1j + a2j 2 + · · · + ak j k

In this way, Savitsky and Golay collected ai (i = 1, . . . , k ) with different orders of derivative and with different window sizes in their tables (see Tables 2.3--Table 2.6) for convenience. For instance, we can find the second-derivative spectrum through the use of a window size of N = 2m + 1 = 9 and polynomial of fourth or fifth orders through the following expression (see Table 2.5):

x0 i = ( − 126x−i −44 + 371x−i −33 + 151x−i −22 − 211x−i −11 − 370x0i − 211x1i +1 + 151x2i +2 + 371x3i +3 − 126x4i +4)

2.4.DATA COMPRESSION

Rapid development in computer technology leads to numerous electronic spectral libraries and databases such as digitized IR, NMR, and mass spectra available in the market. In order to identify the spectrum of an unknown compound from a reference library, data searching is required. Thus, fast and highly accurate library search algorithms are desirable. Before carrying out any search, the spectral library must be constructed first from a set of reference spectra together with additional information such as structure, name, connection data, and molecular mass of each individual compound. In order to reduce the storage space of the spectral data, different compression techniques have been developed.

Generally speaking, the goal of data compression is to represent an information source (e.g., a data file, a speech signal, an image, or a video signal) as accurately as possible using the lowest number of bits. Although many methods have been developed for data compression, most of them are used for compression of images and acoustic signals. Only few of these methods have been adopted in compression of chemical signals.

The principles of data compression methods can be classified into transformation, projection, information encoding, vector quantization, functional approximation, and feature extraction. In this section, data compression methods that are commonly used in chemistry are briefly discussed, including B-spline curve-fitting methods [13--15], Fourier transformation [16--19], and factor analysis [20--22].

2.4.1. Data Compression Based on B-Spline Curve Fitting

The problem of curve fitting is defined as representing a signal by a linear combination of a group of functions. In the case of B-spline curve fitting,