DV compression

Concerning DVC recording of SD, see DV recording, in Chapter 9 of

Composite NTSC and PAL: Legacy Video Systems. DVCPRO and DVCPRO 50 have been standardized as SMPTE D-7. JVC’s Digital-S was standardized as D-9. See

Professional DV variants, on page 512.

DV denotes the compression and data packing scheme introduced for consumer digital video cassette (DVC) recorders and later adapted for professional use. DV compression uses discrete cosine transform (DCT), quantization, and variable-length encoding (VLE) comparable to JPEG; however, DV does not conform to the JPEG standard: Optimizations have been made to accommodate interlaced scanning, constant bit-rate (CBR) operation, and other features related to videotape recording. Interlace is handled by allowing the encoder to dynamically choose between frame and field modes. Constant bit-rate is achieved by dynamically altering the quantization matrices to avoid exceeding the available capacity.

Consumer DVC has a data rate of 25 Mb/s. I call this DV25. DV25 coding was adopted for studio use in D-7 (DVCPRO) and DVCAM; it was extended to 50 Mb/s (DV50), used in the DVCPRO50 and D-9 systems, and then to 100 Mb/s (DV100), used in D-11 (DVCPRO HD).

DV25 and DV50 compress either 480i29.97 or 576i25 SD video according to BT.601. DV100 compresses HD video according to BT.709. Chroma subsampling in DV is a smorgasbord; see Table 46.1.

Table 46.1 DV chroma subsampling

SMPTE 314M defines DV25 and DV50 for studio use. The Blue Book, and IEC standards, use the word decimated instead of discarded.

IEC 61834-1, cited in Chapter 9,

Videotape recording, of Composite NTSC and PAL: Legacy Video Systems, prescribes the subsampling schemes for consumer DV.

DV chroma subsampling

Different versions of DV use different subsampling schemes: 4:2:2, 4:1:1, 4:2:0, and even 3:1:1 and 3:1:0! In DV, a macroblock comprises an 8× 8 block of CB samples, an 8× 8 block of CR samples, and the requisite number and arrangement of 8× 8 blocks of luma samples.

SMPTE 314M declares that in subsampling 4:2:2 to 4:1:1, “every other pixel is discarded.” Obviously, high image quality requires that proper filtering be performed before discarding samples. In DV, CB and CR samples coincide with luma both horizontally and vertically. However, in the 4:2:0 scheme used in 576i consumer equipment, CR samples are not sited at the same locations as CB samples. Instead, CB and CR samples are sited in line-alternate vertical positions throughout each field: Each CB sample is centered two image rows below an associated CR sample.

The DV format originated for consumer applications. Consumer DV25 aggregates 27 macroblocks into

a superblock (SB), covering 48 image rows, whose arrangement is depicted in Figure 46.1 above. In 480i29.97 DV25, there are 50 superblocks per frame; in 576i25 DV25, there are 60 superblocks per frame.

Figure 46.2 Chroma samples in 4:1:1 DV require special treatment at the right-hand edge of the picture, in the region circled in Figure 46.1. Here I show chroma as if square-sampled; actually, each 4:1:1 chroma sample is about four times wider than tall.

For consumer 576i25 DV25, and the 576i25 variant of DVCAM, 4:2:0 chroma subsampling was chosen, allegedly to offer some compatibility with DVB. The 4:2:0 superblock structure is shown in the middle sketch of Figure 46.1; in 4:2:0, each superblock comprises a regular array of 9× 3 macroblocks.

For consumer 480i29.97 DV25, 4:1:1 subsampling was chosen. This was sufficient to achieve NTSC chroma bandwidth, and avoided conflict between subsampling and interlace. BT.601 video adapted to 4:1:1 subsampling has 180 active chroma samples per line; however, 180 isn’t divisible by 8, so a regular superblock structure wasn’t possible. To form 8× 8 chroma blocks, CB and CR samples at the right-hand edge of the image are treated strangely, as detailed in Figure 46.2 in the margin: Pairs of vertically adjacent 4× 8 arrays of chroma samples from the right end of each line are abutted horizontally to form 8× 8 blocks.

The studio version of DV25 (used in D-7) uses 4:1:1 chroma subsampling in both 480i29.97 and 576i25; the strange 4:1:1 superblock structure is used.

DV50 has twice the data rate of DV25. DV50 uses 4:2:2 chroma subsampling for both 480i29.97 and 576i25. Owing to 4:2:2 subsampling, each macroblock has four blocks (instead of the six blocks of either 4:1:1 or 4:2:0); a frame has twice as many macroblocks as DV25, and twice as many superblocks. The 4:2:2 superblock structure is depicted in the top sketch of

Figure 46.1. Today’s DV50 recorders are implemented using two DV25 encoders, processing alternate 24-row bands of the image; they record the resulting two channels to parallel tracks.

DV frame/field modes

ISO JPEG assumes a progressive source image: All image rows are processed in spatial order. In video, the two fields of an interlaced frame can be woven together and processed as if progressive, where every 8× 8 block has alternate rows taken from opposite fields. Providing interfield motion is limited, this scheme works well. In the presence of interfield motion, though, the high vertical frequency AC coefficients produced by the DCT will have high magnitude. This produces low quality for a given data rate. Higher quality can be obtained if the

See weave, on page 414.

This scheme involves two 8× 4 DCTs; it should be called 2-8-4-DCT! MPEG-2 has a similar scheme: An encoder can choose frame DCT coding or field DCT coding for each macroblock in an interlaced MPEG-2 sequence.

fields are coded separately. DV dynamically adapts to the degree of interfield motion by allowing the compressor to choose, on a block-by-block basis, whether to use 8-8-DCT mode or 2-4-8-DCT mode.

In 8-8-DCT (“frame”) mode, opposite fields are woven together and subject to a single DCT as an 8× 8 block.

In 2-4-8-DCT (“field”) mode, two 8× 4 arrays of samples from like fields are formed. The element-by- element sums of the two 8× 4 arrays, and their element- by-element differences, are subject to separate DCTs. The sum matrix is associated with a DC term; the difference array has only AC terms. Each of the sum and difference DCT arrays is zigzag scanned, and the coefficients are alternated for joint VLE encoding.

DV standards do not dictate how an encoder is to choose between 8-8-DCT and 2-4-8-DCT modes. Typically, luma differences are analyzed in the spatial domain to detect interfield motion, and 2-4-8-DCT mode is chosen if interfield motion is significant.

Picture-in-shuttle in DV

It is a practical necessity that a VCR recover a usable picture in fast forward and rewind (“shuttle”). When

a digital VCR is operating in shuttle mode, isolated sync blocks are read from the tape. In DV, recovering a usable picture in shuttle mode is made possible by having each sync block correspond to one coded macroblock (CM); an entire macroblock can thereby be reconstructed individually from an isolated sync block. This scheme precludes predictive coding of the DCT’s DC terms: DC terms are represented directly in each CM.

The correspondence of sync blocks with coded macroblocks seems to require that every macroblock be coded into a fixed number of bits. In the DV system, every macroblock is associated with a 77-byte coded macroblock, and a sync block contains one CM. Each CM contains the DC term, and several AC terms, of each block’s DCT. However, that’s not the whole story.

DV overflow scheme

When a portion of an image is devoid of detail, a few low-frequency AC coefficients of its DCT may have significant magnitude, but nearly all high-frequency AC

CMs in a segment are denoted a through e.

terms will have very small magnitude. When a portion of an image has a lot of fine detail, many highfrequency AC terms will have large magnitude. To reproduce the image accurately requires that these terms be recorded. Generally, increasing amounts of detail require increased data capacity.

It is unusual for an image to contain detail everywhere; typically, complexity is spatially concentrated in an image. A compression algorithm should adapt to the spatial distribution of image detail, by allocating bits where they are needed. If fine detail is distributed throughout an image – in a full-frame image of the leaves of a tree, perhaps – then even quite large reconstruction errors are likely to be imperceptible.

The requirement for picture-in-shuttle seems to preclude allocation of data capacity to the regions where more bits are required. However, DV implements an overflow scheme whereby the bits resulting from compression of a handful of macroblocks are shared among a handful of CMs. Should the VLE-coded AC coefficients for a complex block (augmented by

a 4-bit EOB) require more bits than the fixed capacity assigned to that block in the CM, the overflow bits “spill” into other blocks whose capacity was not filled. Overflow data first spills into space that might remain in other blocks of the same CM; any remaining bits spill into available space in any of four other CMs associated with diverse regions of the image. The set of five coded macroblocks that share overflow space is called a segment; a segment has a fixed capacity of 385 bytes. The macroblocks of a segment are spatially distributed throughout the frame: No two macroblocks are taken from the same row or the same column of superblocks. This distribution exploits the statistical likelihood that only one or two of the macroblocks will be complex. When a DV VCR is in shuttle mode, it is likely that

a single CM will be recovered individually, without any of the other four CMs of its segment. Overflow data for the macroblock is quite likely to be missing. However, overflow data is limited to high-frequency AC coefficients. In shuttle playback, missing overflow coefficients are replaced by zero. This causes loss of picture detail in the reconstructed picture; however, the absence of these coefficients does not seriously degrade picture