Figure 16.6 Example GoP

I0B1B2P3B4B5P6B7B8

Figure 16.7 Example 9-picture GoP without B-pictures

I0P1P2P3P4P5P6P7P8

able for distribution, but owing to its inability to be edited without impairment at arbitrary points, MPEG is generally unsuitable for production. In the specialization of MPEG-2 called I-frame only MPEG-2, every GoP is a single I-frame. This is conceptually equivalent to Motion-JPEG, but has the great benefit of an international standard. (Another variant of MPEG-2, the simple profile, has no B-pictures.)

I have introduced MPEG as if all elements of every P-picture and all elements of every B-picture are coded similarly. But even a picture that is generally well predicted by the past reference picture may have a few regions that cannot effectively be predicted. In MPEG, the image is tiled into macroblocks of 16× 16 luma samples, and the encoder is given the option to code any particular macroblock in intra mode – that is, independently of any prediction. A compact code signals that a macroblock should be skipped, in which case the motion-compensated prediction is used without modification. In a B-picture, the encoder can decide on

a macroblock-by-macroblock basis to code using forward prediction, backward prediction, or biprediction. Formally, an I-picture contains only I-macro- blocks; a P-picture has at least one P-macroblock, and a B-picture has at least one B-macroblock.

Reordering

In a sequence without B-pictures, I- and P-pictures are encoded then stored or transmitted in the obvious order. However, when B-pictures are used, the decoder typically needs to access the past anchor picture and the future anchor picture to reconstruct a B-picture.

Consider an encoder about to compress the sequence in Figure 16.6 (where anchor pictures I0, P3, and P6 are written in boldface). The coded B1 and B2 pictures may be backward predicted from P3, so the encoder must buffer the uncompressed B1 and B2 pictures until P3 is coded: Only when coding of P3 is complete can coding of B1 start. Using B-pictures incurs a penalty in encoding delay. (If the sequence were coded without B-pictures, as depicted in Figure 16.7, transmission of the coded information for P1 would not be subject to this two-picture delay.) Coding delay

Figure 16.8 GoP reordered for transmission

I0P3B1B2P6B4B5(I9)B7B8

ISO/IEC 11172-1, Coding of moving pictures and associated audio for digital storage media at up to about 1,5 Mbit/s – Part 1: Systems [MPEG-1].

(latency) can make MPEG with B-pictures unsuitable for realtime two-way applications such as teleconferencing.

If the coded 9-picture GoP of Figure 16.6 were transmitted in the order shown, then the decoder would have to hold the coded B1 and B2 data in a buffer while receiving and decoding P3; only when decoding of P3 was complete could decoding of B1 start. The encoder must buffer the B1 and B2 pictures no matter what; however, to minimize buffer memory at the decoder, MPEG-2 specifies that coded B-picture information is transmitted after the coded reference picture.

Figure 16.8 indicates picture data as reordered for transmission. I have placed I9 in parentheses because it belongs to the next GoP (the GoP header precedes it). Here, B7 and B8 follow the GoP header.

MPEG-1

The original MPEG effort resulted in a standard now called MPEG-1, which was deployed in multimedia applications. MPEG-1 was optimized for the coding of progressive 352× 240 images at 30 frames per second (240p30). MPEG-1 has no provision for interlace. When 480i29.97 or 576i25 video is coded with MPEG-1 at typical data rates, the first field of each frame is coded as if it were progressive, and the second field is dropped. At its intended data rate of about 1.5 Mb/s, MPEG-1 delivers VHS-quality images.

For broadcast, MPEG-1 has been superseded by MPEG-2. An MPEG-2 decoder must decode MPEG-1 constrained-parameter bitstream (CPB) sequences – to be discussed in the caption to Table 47.1, on page 515 – so I will not discuss MPEG-1 further.

MPEG-2

The MPEG-2 effort was initiated to extend MPEG-1 to interlaced scanning, to larger pictures, and to data rates much higher than 1.5 Mb/s. MPEG-2 is standardized in a series of documents from ISO/IEC; MPEG-2 is widely deployed for the distribution of digital television (DTV) including SD and HD (for example, ATSC), and is the (only) video compression scheme for DVD.

Many MPEG terms – such as frame, picture, and macroblock – can refer to elements of the source video, to the corresponding elements in the coded bitstream, or to the corresponding elements in the reconstructed video. It is generally clear from context which is meant.

ISO/IEC 15938-1, Information technology – Multimedia content description interface – Part 1: Systems.

ISO/IEC 21000, Multimedia framework (MPEG-21).

MPEG IMX is a Sony trademark; IMX is not an MPEG designation.

MPEG-2 accommodates both progressive and interlaced material. A video frame can be coded directly as a frame-structured picture. Alternatively, a video frame (typically originated from an interlaced source) may be coded as a pair of field-structured pictures – a top-field picture and a bottom-field picture. The two fields are time-offset by half the frame time, and are intended for interlaced display. Field pictures always come in pairs having opposite parity (top/bottom or bottom/top). Both pictures in a field pair have the same picture coding type (I, P, or B), except that an I-field may be followed by a P-field (in which case the pair effectively serves as an I-frame).

The MPEG IMX variant of MPEG-2, for studio use, is described below. The HDV variant of MPEG-2, for consumer use, is described on page 161. MPEG-2 video compression is detailed starting on page 513.

Other MPEGs

While the MPEG-2 work was underway, an MPEG-3 effort was launched to address HD. The MPEG-3 committee concluded early on that MPEG-2, at high data rate, would accommodate HD; consequently, the MPEG-3 effort was abandoned. I’ll discuss MPEG-4 below. MPEG numbers above 4 are capricious.

MPEG-7, titled Multimedia Content Description Interface, standardizes a description of various types of multimedia information (metadata). In my view, MPEG-7 is not relevant to handling studioor distribu- tion-quality video signals.

According to ISO, MPEG-21 “defines an open framework for multimedia delivery and consumption, with both the content creator and content consumer as focal points. The vision for MPEG-21 is to define a multimedia framework to enable transparent and augmented use of multimedia resources across a wide range of networks and devices used by different communities.” In my view, MPEG-21 is not relevant to handling studioor distribution-quality video.

MPEG IMX

Sony’s original Digital Betacam for 480i and 576i SD used proprietary motion-JPEG-like compression. The first products were videotape recorders denoted

ISO/IEC 14496, Information technology – Coding of audiovisual objects.

ISO/IEC 14496-12:2004, Information technology – Coding of audio-visual objects – Part 12:

ISO base media file format.

Betacam SX, having a data rate of about 18 Mb/s. Follow-on products adopted I-frame-only MPEG-2 422P@ML, denoted MPEG IMX, having data rates of 30 Mb/s, 40 Mb/s, or 50 Mb/s (IMX30, IMX40, IMX50). MPEG IMX videotape recorders were commercialized. Today it is common to place or wrap IMX compressed video in an MXF file.

XDCAM is Sony’s designation for a line of products using a variety of compression systems and a variety of physical media. MPEG IMX compression is one of the compression systems available in the XDCAM line.

Recording on optical disc media is possible.

MPEG-4

The original goal of the MPEG-4 effort was video coding at very low bit rates. The video compression system that resulted is standardized as MPEG-4 Part 2

Visual; it differes from MPEG-2 and from H.264. ISO/IEC co-published the ITU-T H.264 standard as MPEG-4 Part 10, so the term MPEG-4 alone is ambiguous.

MPEG-4 Part 2 defines the advanced simple profile (ASP), implemented by DivX and Xvid. ASP is not useful for professional quality video. Even in ASP’s intended application domain, low bit-rate video, H.264 – to be described in a moment – has proven to have better performance. Consequently, ASP has fallen out of favour.

MPEG-4 Part 2 also defines a profile called Simple Studio Profile (SStP). This profile is used in Sony’s HDCAM SR, at very high bit rates (the other end of the bit rate spectrum for which MPEG-4 was conceived). HDCAM SR is widely used in HD, both on tape and in files. Apart from HDCAM SR, the simple studio profile of MPEG-4 sees very limited use.

Part 12 of the MPEG-4 suite of standards defines the

ISO Base Media File Format, which defines a general container structure for time-based media files. The format is used in desktop video (most commonly with MPEG-4 Part 2/ASP video, in mp4 files), but is rarely if ever used in professional video distribution.