Figure 22.2 Horizontal domain

Figure 22.3 Vertical domain

Figure 22.4 Temporal domain

Figure 22.5 Spatial domain

At the far left of Figure 22.1 is a sketch of a twodimensional spatial domain of a single image. Some image processing operations, such as certain kinds of filtering, can be performed separately on the horizontal and vertical axes, and have an effect in the spatial domain – these operations are called separable. Other processing operations cannot be separated into horizontal and vertical facets, and must be performed directly on a two-dimensional sample array. Twodimensional sampling theory applies.

In Chapter 20, Filtering and sampling, on page 191, I described how to analyze a signal that is a function of the single dimension of time, such as an audio signal.

Sampling theory also applies to a signal that is a function of one dimension of space, such as a single scan line (image row) of a video signal. This is the horizontal or transverse domain, sketched in Figure 22.2 in the margin. If an image is scanned line by line, the waveform of each line can be treated as an independent signal. The techniques of filtering and sampling in one dimension, discussed in Chapter 20, apply directly to this case.

Consider a set of points arranged vertically that originate at the same displacement along each of several successive image rows, as sketched in Figure 22.3. Those points can be considered to be sampled by the scanning process itself. Sampling theory can be used to understand the properties of these samples.

A third dimension is introduced when a succession of images is temporally sampled to represent motion. Figure 22.4 depicts samples in the same column and the same row in three successive frames.

Complex filters can act on two axes simultaneously. Figure 22.5 illustrates spatial sampling. The properties of the entire set of samples are considered all at once, and cannot necessarily be separated into independent horizontal and vertical aspects.

Spatial frequency domain

I explained in Image structure, on page 75, how a onedimensional waveform in time transforms to a onedimensional frequency spectrum. This concept can be extended to two dimensions: The two dimensions of space can be transformed into two-dimensional spatial

1

0

Horizontal displacement

(fraction of picture width)

Figure 22.6 Horizontal spatial frequency domain

Vertical frequency, C/PH

0

04

Horizontal frequency, C/PW

Figure 22.7 Vertical spatial frequency domain

subcarrier were included in image data, it would

0 tion. 0

0Horizontal frequency, C/PW

NTSC SUBCARRIER

LUMA

188 Horizontal frequency, C/PW

When spatial frequency is determined analytically using the twodimensional Fourier transform, the result is plotted in the manner of Figure 22.8, where low vertical frequencies – that is, low y values – are at the bottom. When spatial frequency is computed numerically using discrete transforms, such as the 2-D discrete Fourier transform (DFT), the fast Fourier transform (FFT), or the discrete cosine transform (DCT), the result is usually presented in a matrix, where low vertical frequencies are at the top.

If an image comprises rows with identical content, all of the power will be concentrated on the horizontal axis of spatial frequency. If the content of successive scans lines varies slightly, the power will spread to nonzero vertical frequencies. An image of diagonal bars would occupy a single point in spatial frequency, displaced from the x-axis and displaced from the y-axis.

The spatial frequency that corresponds to half the vertical sampling rate depends on the number of picture lines. A 480i system has approximately 480 picture lines: 480 samples occupy the height of the picture, and the Nyquist frequency for vertical sampling is 240 C/PH. No vertical frequency in excess of this can be represented without aliasing.

In most images, successive rows and columns of samples (of R’, G’, B’, or of luma) are very similar; low frequencies predominate, and image power tends to cluster toward spatial frequency coordinates [0, 0]. Figure 22.8 sketches the spatial frequency spectrum of luma in a 480i system. If the unmodulated NTSC colour subcarrier were an image data signal, it would take the

An optical transfer function

(OTF) includes phase. The magnitude of an OTF is MTF; MTF disregards phase.

Figure 22.9 Two samples, vertically arranged

indicated location. In composite NTSC, chroma is modulated onto the subcarrier; the resulting modulated chroma can be thought of as occupying a particular region of the spatial frequency plane, as described in Spatial frequency spectra of NTSC, in Chapter 6 of Composite NTSC and PAL: Legacy Video Systems. In NTSC encoding, modulated chroma is then summed with luma; this causes the spectra to be overlaid. If the luma and chroma spectra overlap, cross-colour and cross-luma interference artifacts can result.

In optics, the terms magnitude frequency response and bandwidth are not used. An optical component, subsystem, or system is characterized by its modulation transfer function (MTF), a one-dimensional plot of horizontal or vertical spatial frequency response. (Depth of modulation is a single point quoted from this graph.) Technically, the MTF is the Fourier transform of the point spread function (PSF) or line spread function (LSF). By definition, the MTF relates to light intensity. Since negative light power is physically unrealizable, an MTF is measured by superimposing a high-frequency sinusoidal (modulating) wave onto a constant level, then taking the ratio of output modulation to input modulation.

Comb filtering

In Finite impulse response (FIR) filters, on page 207,

I described FIR filters operating in the single dimension of time. If the samples are from a scan line of an image, the frequency response can be considered to represent horizontal spatial frequency (in units of C/PW), instead of temporal frequency (in cycles per second, or hertz).

Consider a sample from a digital image sequence, and the sample immediately below, as sketched in Figure 22.9 in the margin. If the image has 640 active (picture) samples per line, and these two samples are presented to a comb filter like that of Figure 20.19, on page 206, but having 639 zero-samples between the two “ones,” then the action of the comb filter will be identical to the action of a filter having two taps weighted [1, 1] operating in the vertical direction. In Figure 20.12, on page 203, I graphed the frequency response of a one-dimensional [1, 1] filter. The graph in