Table 8.2 Scanning in computing has no standardized notation, but these notations are widely used.

Computing Video notation notation

1080i30

Frame rate

Image rows

(picture lines i for interlace, in frame) p for progressive

Figure 8.9 Modern image format notation gives the count of image rows (active/picture lines); p for progressive or i for interlace (or psf for progressive segmented-frame); then a rate. I write frame rate, but some people write field rate. Aspect ratio is not part of the notation.

pixel array every field time – in a 480i29.97 system, 1⁄480 of the picture height in 1⁄60 s, or one picture height in 8 seconds. Owing to interlace, half of that image’s vertical information will be lost! At other rates, some portion of the vertical detail in the image will be lost. With interlaced scanning, vertical motion can cause serious motion artifacts. Techniques to avoid such artifacts will be discussed in Deinterlacing, on page 413.

Scanning notation

In computing, a display format may be denoted by a pair of numbers: the count of pixels across the width of the image – that is, the count of image columns – and the count of pixels down the height of the image – that is, the count of image rows. Square sampling is implicit. This notation does not indicate refresh rate. (Alternatively, a display format may be denoted symbolically – VGA, SVGA, XGA, etc., as in Table 8.2. This notation is utterly opaque to the majority of computer users.)

Traditionally, video scanning was denoted by the total number of lines per frame (picture lines plus sync and vertical blanking overhead), a slash, and the field rate in hertz. (Interlace was implicit unless a slash and 1:1 was appended to indicate progressive scanning; a slash and 2:1 make interlace explicit.) For SD, 525/59.94/2:1 scanning is used in North America and Japan; 625/50/2:1 prevails in Europe, Asia, and

Australia. Before the advent of HD, these were the only scanning systems used for broadcasting.

Digital technology enabled new scanning and display standards. Conventional scanning notation could not adequately describe the new scanning systems. A new notation, depicted in Figure 8.9, emerged: Scanning is denoted by the count of image rows, followed by p for progressive or i for interlace, followed by the frame rate. I write the letter i in lowercase italics to avoid potential confusion with the digit 1. For consistency,

I write the letter p in lowercase italics. Traditional video notation (such as 625/50) is inconsistent, juxtaposing lines per frame with fields per second. Some people – particularly historical proponents of interlaced scanning – seem intent upon carrying this confusion into the future, by denoting the old 525/59.94 as 480i59.94.

I strongly prefer using frame rate.

Since all 480i systems have a frame rate of 29.97 Hz, I use 480i as shorthand for 480i29.97. Similarly, I use 576i as shorthand for 576i25.

Because some people write 480p60 when they mean 480p59.94, the notation 60.00 should be used to emphasize a rate of exactly 60 Hz.

Poynton, Charles (1996), “Motion portrayal, eye tracking, and emerging display technology,” in Proc. 30th SMPTE Advanced Motion Imaging Conference (New York: SMPTE), 192–202.

Historically, this technique was called 3-2 pulldown, but since the adoption of SMPTE RP 197 in 1998, it is now more accurately called 2-3 pulldown.

In my notation, conventional 525/59.94/2:1 video is denoted 480i29.97; conventional 625/50/2:1 video is denoted 576i25. HD systems include 720p60 and 1080i30. Film-friendly versions of HD are denoted 720p24 and 1080p24. Aspect ratio is not explicit in the new notation: 720p, 1080i, and 1080p are implicitly 16:9 since there are no 4:3 standards for these systems, but 480i or 480p could potentially have either conventional 4:3 or widescreen 16:9 aspect ratio.

Motion portrayal

In Flicker, refresh rate, and frame rate, on page 83, I outlined the perceptual considerations in choosing

refresh rate. In order to avoid objectionable flicker, it was historically necessary with CRT displays to flash an image at a rate higher than the rate necessary to portray its motion. Different applications have adopted different refresh rates, depending on the image quality requirements and viewing conditions. Refresh rate is generally engineered into a video system; once chosen, it cannot easily be changed.

Flicker is minimized by any display device that produces steady, unflashing light, or pulsed light lasting for the majority of the frame time. You might regard

a nonflashing display to be more suitable than a device that flashes; many modern devices do not flash. However, with a display having a pixel duty cycle near 100% – that is, an on-time approaching the frame time – if the viewer’s gaze tracks an element that moves across the image, that element will be seen as smeared. This problem becomes more severe as eye tracking velocities increase, such as with the wide viewing angle of HD. (For details of temporal characteristics of image acquisition and display, see my SMPTE paper.)

Film at 24 frames per second is transferred to interlaced video at 60 fields per second by 2-3 pulldown. The first film frame is transferred to two video fields, then the second film frame is transferred to three video fields; the cycle repeats. The 2-3 pulldown is normally used to produce video at 59.94 Hz, not 60 Hz; the film is run 0.1% slower than 24 frames per second. The scheme is detailed in 2-3 pulldown, on page 405. The 2-3 technique can be applied to transfer to progressive video at 59.94 or 60 frames per second.

The progressive segmented-frame

(PsF) technique is known in consumer SD systems as quasiinterlace. PsF is not to be confused with point spread function, PSF.

Some camera sensors sample the entire scene at the same instant (global shutter, GS). With other sensors, sampling of the scene proceeds vertically at a uniform rate through the frame time (rolling shutter, RS).

Film at 24 frames per second is transferred to 576i video using 2-2 pulldown: Each film frame is scanned into two video fields (or frames); the film is run 4% faster than the capture rate. The process is described more fully in Chapter 34, 2-3 pulldown, on page 405.

Segmented-frame (24PsF)

A scheme called progressive segmented-frame is sometimes used in HD equipment that handles images at 24 frames per second. The scheme, denoted 24PsF, uses progressive scanning: The entire image is sampled at once, and vertical filtering to reduce twitter is both unnecessary and undesirable. However, lines are rearranged to interlaced order for studio distribution and recording. The scheme offers some compatibility with interlaced processing and recording equipment.

Video system taxonomy

Insufficient channel capacity was available at the outset of television broadcasting to transmit three separate colour components. The NTSC and PAL techniques were devised to combine (encode) the three colour components into a single composite signal. Composite video remains in use in consumers’ premises. However, modern video equipment – including all consumer digital video equipment, and all HD equipment – uses component video, either Y’PBPR analog components or Y’CBCR digital components.

A video system can be classified as component HD, component SD, or composite SD. Independently,

a system can be classified as analog or digital. Table 8.3 indicates the six classifications, with the associated

Table 8.3 Video systems are classified as analog or digital, and component or composite (or S-video). SD may be represented in component, hybrid (S-video), or composite forms. HD is always in component form.

By NTSC, I do not mean 525/59.94 or 480i; by PAL, I do not mean 625/50 or 576 i! See Introduction to composite NTSC and PAL, on page 135. Although secam is

a composite technique in that luma and chroma are combined, it has little in common with NTSC and PAL. SECAM is obsolete.

Transcoding refers to the technical aspects of conversion; signal modifications for creative purposes are not encompassed by the term.

colour encoding schemes. Composite NTSC and PAL video encoding is used only in 480i and 576i systems; HD systems use only component video. S-video is

a hybrid of component analog video and composite analog NTSC or PAL; in Table 8.3, S-video is classified in its own seventh (hybrid) category.

Conversion among systems

In video, encoding traditionally referred to converting a set of R’G’B’ components into an NTSC or PAL composite signal. Encoding may start with R’G’B’, Y’CBCR, or Y’PBPR components, or may involve matrixing from R’G’B’ to form luma (Y’) and intermediate [U, V]

components. Quadrature modulation then forms modulated chroma (C); luma and chroma are then summed. Decoding historically referred to converting an NTSC or PAL composite signal to R’G’B’. Decoding involves luma/chroma separation, quadrature demodulation to recover [U, V], then scaling to recover [CB, CR] or

[PB, PR], or matrixing of luma and chroma to recover

R’G’B’. Encoding and decoding have now become general terms that may refer to JPEG, M-JPEG, MPEG, or other encoding or decoding processes.

Transcoding traditionally referred to conversion among different colour encoding methods having the same scanning standard. Transcoding of component video involves chroma interpolation, matrixing, and chroma subsampling. Transcoding of composite video involves decoding, then reencoding to the other standard. With the emergence of compressed storage and digital distribution, the term transcoding is now applied toward various methods of recoding compressed bitstreams, or decompressing then recompressing.

Scan conversion refers to conversion among scanning standards having different spatial structures, without the use of temporal processing. In consumer video and desktop video, scan conversion is commonly called scaling.

If the input and output frame rates differ, the operation is said to involve frame rate conversion. Motion portrayal is liable to be impaired.

In radio frequency (RF) technology, upconversion refers to conversion of a signal to a higher carrier frequency; downconversion refers to conversion of a signal to a lower carrier frequency.

Watkinson, John (1994), The Engineer’s Guide to Standards Conversion (Petersfield, Hampshire, England: Snell & Wilcox, now Snell Group).

Historically, upconversion referred to conversion from SD to HD; downconversion referred to conversion from HD to SD. Historically, these terms referred to conversion of a signal at the same frame rate as the input; nowadays, frame rate conversion might be involved. High-quality upconversion and downconversion require spatial interpolation. That, in turn, is best performed in a progressive format: If the source is interlaced, intermediate deinterlacing is required, even if the target format is interlaced.

Standards conversion is the historical term denoting conversion among scanning standards having different frame rates. Historically, the term implied similar pixel count (such as conversion between 480i and 576i), but nowadays a standards converter might incorporate upconversion or downconversion. Standards conversion requires a fieldstore or framestore; to achieve high quality, it requires several fieldstores and motioncompensated interpolation. The complexity of standards conversion between different frame rates has made it difficult for broadcasters and consumers to convert European material for use in North America or Japan, or vice versa.