Raster scanning

Flicker is sometimes redundantly called large-area flicker. Take care to distinguish flicker, described here, from twitter, to be described on page 89. See Fukuda, Tadahiko (1987), “Some Characteristics of Peripheral Vision,” NHK Tech. Monograph No. 36 (Tokyo: NHK Science and Technical Research Laboratories).

I introduced the pixel array on page 3. This chapter outlines the basics of this process of raster scanning, whereby the samples of the pixel array are sequenced uniformly in time to form scan lines, which are in turn sequenced in time throughout each frame interval. In Chapter 13, Introduction to component SD, on

page 129, I will present details on scanning in conventional “525-line” and “625-line” video. In Introduction to composite NTSC and PAL, on page 135, I will introduce the colour coding used in these systems. In Chapter 15, Introduction to HD, on page 141, I will introduce scanning in high-definition television.

Flicker, refresh rate, and frame rate

A sequence of still pictures, captured and displayed at a sufficiently high rate, can create the illusion of motion.

The historical CRT display used for television emits light for a small fraction of the frame time: The display has a short duty cycle; it is black most of the time. If the flash rate – or refresh rate – is too low, flicker is perceived. The flicker sensitivity of vision is dependent upon display and viewing conditions: The brighter the environment, and the larger the angle subtended by the picture, the higher the flash rate must be to avoid flicker. Because picture angle influences flicker, flicker depends upon viewing distance.

Most modern displays – including LCDs and plasma displays – do not flash, and cannot flicker. Nonetheless, they may be subject to various motion impairments.

In a “flashing” display, the brightness of the displayed image itself influences the flicker threshold to some

The fovea has a diameter of about 1.5 mm, and subtends a visual angle of about 5°.

Figure 8.1 A dual-bladed shutter in a film projector flashes each frame twice. Rarely, three bladed shutters are used; they flash each frame thrice.

Television refresh rates were originally chosen to match the local AC power line frequency. See Frame, field, line, and sample rates, on page 389.

Farrell, Joyce E., et al. (1987), “Predicting flicker thresholds for video display terminals,” in Proc. Society for Information Display

28 (4): 449–453.

Table 8.1 Refresh rate refers to the shortest interval over which the entire picture is updated. Flash rate refers to the rate at which the picture height is covered at the display. Different refresh rates and flash rates are used in different applications.

extent, so the brighter the image the higher the refresh rate must be. In a very dark environment, such as the cinema, flicker sensitivity is completely determined by the luminance of the image itself. Peripheral vision has higher temporal sensitivity than central (foveal) vision, so the flicker threshold increases to some extent with wider viewing angles. Table 8.1 summarizes refresh rates used in film, video, and computing.

In the darkness of a cinema, a flash rate of 48 Hz is sufficient to overcome flicker. In the early days of motion pictures, 24 frames per second were found to be sufficient for good motion portrayal. So, a conventional film projector uses a dual-bladed shutter, depicted in Figure 8.1, to flash each frame twice. Higher realism can be obtained with material at 48 frames per second or higher displayed with single-bladed shutters, but such schemes are nonstandard.

In the dim viewing environment typical of television, such as a living room, a flash rate of 60 Hz suffices. The interlace technique, to be described on page 88, provides for video a function comparable to the dualbladed shutter of a film projector: Each frame is flashed as two fields. Refresh is established by the field rate (twice the frame rate). For a given data rate, interlace doubles the apparent flash rate, and provides improved motion portrayal by doubling the temporal sampling rate. Scanning without interlace is called progressive.

CRT computer displays used in office environments historically required refresh rates above 70 Hz to overcome flicker (see Farrell, 1987). CRTs have now been supplanted by LCD displays, which don’t flicker. High refresh rates are no longer needed to avoid flicker.

The word raster is derived from the Greek word rustum (rake), owing to the resemblance of a raster to the pattern left on newly raked earth.

Line is a heavily overloaded term. Lines may refer to the total number of raster lines: Figure 8.2 shows “525-line” video, which has 525 total lines. Line may refer to a line containing picture, or to the total number of lines containing picture – in this example, 480. Line may denote the AC power line, whose frequency is very closely related to vertical scanning. Finally, lines is

a measure of resolution, to be described in Resolution, on page 97.

Introduction to scanning

A moment ago, I outlined how refresh rate for television was chosen so as to minimize flicker. In Viewing distance and angle, on page 100, I will outline how spatial sampling determines the number of pixels in the image array. Video scanning represents pixels in sequential order, so as to acquire, convey, process, or display every pixel during the fixed time interval associated with each frame. In analog video, information in the image plane was scanned left to right at a uniform rate during

a fixed, short interval of time – the active line time. Scanning established a fixed relationship between

a position in the image and a time instant in the signal. Successive lines were scanned at a uniform rate from the top to the bottom of the image, so there was also a fixed relationship between vertical position and time. The stationary pattern of parallel scanning lines disposed across the image area is the raster.

Samples of a digital image matrix are usually conveyed in the same order that the image was historically conveyed in analog video: first the top image row (left to right), then successive rows. Scan line is an oldfashioned term; the term image row is now preferred. Successive pixels lie in image columns.

In cameras and displays, a certain time interval is consumed in advancing the scanning operation – historically, horizontal retracing – from one line to the next; several line times are consumed by vertical retrace, from the bottom of one scan to the top of the next. A CRT’s electron gun had to be switched off (blanked) during these intervals, so these intervals were (and are) called blanking intervals. The horizontal blanking interval occurs between scan lines; the vertical blanking interval (VBI) occurs between frames (or fields). Figure 8.2 shows the blanking intervals of “525-line” video.

Figure 8.2 Blanking intervals for “525-line”video are indi-

cated here by a dark region surrounding a light-shaded rectangle that represents the

picture. The vertical blanking 525 480 interval (VBI) consumes about

8% of each field time; horizontal blanking consumes about 15% of each line time.

VERTICAL

BLANKING INTERVAL (≈8%)

HORIZONTAL

BLANKING INTERVAL (≈15%)

The count of 480 picture lines in Figure 8.2 is a recent standard; some people will quote numbers between 481 and 487. See Picture lines, on page 379.

VITS: Vertical interval test signal VITC: Vertical interval timecode

The horizontal and vertical blanking intervals required for a CRT were large fractions of the line time and frame time: In SD, vertical blanking consumes roughly 8% of each frame period. In HD, that fraction is reduced to about 4%.

In a video interface, whether analog or digital, synchronization information (sync) is conveyed during the blanking intervals. In principle, a digital video interface transmit just the active pixels accompanied by the minimum necessary sync information. Instead, digital video interfaces use interface rates that match the blanking intervals of historical analog equipment. What would otherwise be excess data capacity is put to good use conveying audio signals, captions, test signals, error detection or correction information, or other data or metadata.

Scanning parameters

In progressive (or sequential) scanning, the image rows are scanned in order, from top to bottom, at a picture rate sufficient to portray motion. Figure 8.3 at the top of the facing page indicates four basic scanning parameters:

•Total lines (LT) comprises all of the scan lines, including the vertical blanking interval and the picture lines.

•The image has NR image rows, containing the picture. (Historically, this was the active line count, LA.)

•Samples per total line (STL) comprises the sample intervals in the total line, including horizontal blanking.

•The image has NC image columns, containing the picture. (Historically, some number of samples per active

line (SAL) were permitted to take values different from the blanking level.

The production aperture, sketched in Figure 8.3,

comprises the array NC columns by NR rows. The samples in the production aperture comprise the pixel

array; they are active. All other sample intervals comprise blanking; they are inactive or “blanked.”

The vertical blanking interval in analog signals typically carried vertical interval information such as VITS, VITC, and closed captions. Consumer display equipment must blank these lines. Both vertical and horizontal blanking intervals in a digital video interface may be used to convey ancillary (ANC) data such as audio.

Figure 8.3 The Production aperture comprises the image array, NC columns by NR rows.

Blanking intervals lie outside the production aperture; here,

blanking intervals are darkly LT shaded. The product of NC and NR yields the active pixel count per frame. Sampling rate (fS) is the product of STL, LT, and

frame rate.

STL

NR

NC

PRODUCTION APERTURE

(NC ›‹ NR)

Figure 8.4 The Clean aperture should remain subjectively free from artifacts arising from filtering. The clean aperture excludes blanking transition samples, indicated here by black bands outside the left and right edges of the picture width, defined by the count of samples per picture width (SPW).

SPW

CLEAN

APERTURE

All standard SD and HD image formats have NC and NR both even. The horizontal center of the picture lies midway between the central two luma samples, and the vertical center of the picture lies vertically midway between the central two image rows.

See Transition samples, on page 378.

Only pixels in the image array are represented in acquisition, processing, storage, and display. However, some processing operations (such as spatial filtering) use information in a small neighbourhood surrounding the subject pixel. In the absence of any better information, we take the pixel of the image array to lie on black. At the left-hand edge of the picture, if the video signal of the leftmost pixel has a value greatly different from black, an artifact called ringing is liable to result when that transition is processed through an analog or digital filter. A similar circumstance arises at the righthand picture edge. In studio video, the signal builds to full amplitude, or decays to blanking level, over several transition samples ideally having a raised cosine envelope.

Active samples encompass not only the picture, but also the transition samples; see Figure 8.4 above. Studio equipment should maintain the widest picture possible within the production aperture, subject to appropriate blanking transitions.

I have treated the image array as an array of pixels, without regard for the spatial distribution of light