VP8 compression

VP3, a distant predecessor to VP8, was made available by On2 as open source. VP3 subsequently developed into Theora. On2 licensed VP6 and VP7 to Adobe as the basis for Flash 8 video; subsequently, H.264 was incorporated into Flash 9. On2 licensed VP7 to Skype.

IP-based means based upon internet (TCP/IP) protocols. H.264 IP means intellectual property (patent) rights associated with H.264.

In 2010, Google acquired a company called On2 that had, over a decade or more, developed a series of proprietary software-based codecs for video distribution. Google made the VP8 codec open-source and used it as the basis for a proposal called WebM for web (IP-based) distribution of video to consumers. WebM comprises video encoded by the VP8 codec and audio encoded by the Vorbis codec, both wrapped in the Matroska file wrapper.

The VP8 codec is broadly based upon the principles of MPEG-2 and H.264 discussed in earlier chapters, although Google intends VP8 to be unencumbered by MPEG-2 and H.264 intellectual property rights (IPR, in this case, patent rights). Patents on elements of VP8 were issued to On2; Google permits their royalty-free use. Google’s license to VP8 requires that the user not litigate any IP that addresses VP8 (“mutual nonassert”). There’s no guarantee or indemnity that Google’s VP8 implementation does not infringe patents not controlled by Google – perhaps even patents in the MPEG-2 or H.264 pools.

It is a technical and commercial problem with VP8 that the descriptive standard is not comprehensive: The definitive specification of VP8 is effectively its reference code. In places, there is opaque code that raises the question, should the VP8 “standard” be defined by what was apparently intended, or by what is executed by the code? In the absence of a written standard, implementors are forced to treat the reference code as definitive, even if performance or interoperability suffer.

Google documents refer to

Y’CBCR as YUV.

Every picture is accompanied by a 1-bit flag show_frame, signalling whether to display the frame. That flag can cause a decoded frame to be placed into one of the reference frames but not displayed. Under unusual circumstances, using this mechanism can simulate a B-frame.

VP8 has no 8× 8 intra luma prediction.

Algorithmic features

As mentioned earlier, the VP8 codec is broadly based upon the principles of MPEG-2 and H.264. To make the most of what follows, you should be familiar with Introduction to video compression (on page 147), and with JPEG/M-JPEG, DV, MPEG-2, and H.264, described in the preceding four chapters.

VP8 codes only progressive, 8-bit, 4:2:0 Y’CBCR video. No provision is made for interlace.

VP8 has what it calls key-frames (comparable to MPEG-2 I-frames), and inter-frames (like MPEG-2 P-frames). VP8 has no B-frames: All decoded frames are potentially available for predictions. A VP8 decoder has three reference frames: the golden frame, the previous frame, and the altref (“alternate reference”) frame.

The bitstream is partitioned into segments. Within a segment there is a 4-byte frame header, and between one and nine partitions denoted I, II, III, and so on.

A partition is a sequence of bytes representing aspects of video (akin to the separation of VCL NAL units and non-VCL NAL units in H.264). Partition I conveys prediction modes and motion vectors, per macroblock, in raster order. Partitions beyond I convey quantized transform coefficients (in VP8, sometimes termed texture). Macroblock rows can be mapped to a single partition, or to 2, 4, or 8 partitions each of which can be processed in parallel. (Entropy contexts, to be described, are shared among partitions; binary arithmetic decoding can be parallelized to some extent, but encoding can’t be.)

VP8 subdivides 16× 16 macroblocks into subblocks of 4× 4 pixels. There are 24 subblocks in each Y’CBCR 4:2:0 macroblock. Unlike H.264, VP8 has no 8x8 luma blocks. Chroma prediction is performed on 8× 8 chroma blocks.

VP8 has two luma intra prediction modes – i16x16 and i4x4 – which reference previously decoded pixels in the same frame. Using intra prediction precludes parallelism.

The bitstream identifies one of four methods through which the intra prediction for each block can be obtained:

• V_PRED: Prediction values are replicated down the block from the row above.

Every entry in a Walsh-Hadamard matrix is either +1 or -1.

•H_PRED: Prediction values are replicated across the block from the column to the left.

•DC_PRED: Prediction values are all set to the average value of the row above and the column to the left; this is called “DC” chroma prediction.

•TM_PRED: Prediction values are extrapolated from the row above and the column to the left using (fixed) second differences from the upper-left corner. (This mode is roughly comparable to H.264’s planar

prediction.)

VP8’s core transform is a 4× 4 DCT approximated by 16-bit integer coefficients. The decoder uses exact 16-bit arithmetic; there is no decoder drift.

For the 16× 16 luma prediction mode, luma processing involves a second level (Y2) transform: After the 16 luma subblocks have been transformed by the DCT, the 16 DC coefficients are collected and

a (twenty-fifth) 4× 4 transform is performed on those coefficients. The second-level transform is not a DCT, but a Walsh-Hadamard transform (WHT).

There are six quantizers, each with its own levels. Which quantizer is used depends upon the “plane” (first-order luma, second-order [Y2] luma, or chroma), and whether the coefficient is DC or AC.

Quantizer level is a 7-bit number that indexes an entry in one of the quantization tables. Quantization is potentially region-adaptive: The encoder associates each macroblock with one of four classes; each class has a different quantization parameter set.

VP8 implements a sophisticated arithmetic coding scheme, simpler than CABAC, but having comparable performance and lighter processing load. The encoder constructs estimates of probabilities of various syntax elements and parameter values. A default baseline parameter set is maintained; upon the occurrence of a keyframe, probability distributions are reset to the baseline. Probabilities are updated as each frame is processed; the encoder signals whether upon comple-

tion of decoding the updated set is to become the new baseline (“persistent”) or is to be discarded (“one-time”).

VP8 has an adaptive in-loop deblocking filter having quality and complexity roughly comparable to that of H.264’s deblocking filter.

Some multimedia formats used in PCs use multiple files – for example, one file for video and another for audio. Such schemes effectively push the multiplexing operation to the player software. Such schemes are prone to failure to play one kind of essence, or to have essences fall out of sync.

In the section MPEG-4, on

page 159, I briefly discussed the ISO Base Media File Format. That format serves as a container format for MPEG-4 Part 2/ASP video. That format is generally agreed to be inapplicable to professional video.

Multimedia encompasses video and audio, potentially accompanied by other elements such as subtitles, coded in a manner suitable for synchronous presentation to the viewer. Many video compression systems are in use; for consumer use, MPEG-2 and H.264 are widely used. Many audio compression systems are in use; in the consumer domain, Dolby Digital (AC-3) and MPEG-1 Level III (MP3) are widely used.

Multimedia broadcasting or distribution requires that the various elements – essences, in the lingo of multimedia – are multiplexed into a single file or stream where the video and audio elements can subsequently be synchronized so as to be presented simultaneously.

In multimedia computing, multiplexing is accomplished by structuring the various components into a container file. Microsoft’s AVI, Apple’s QuickTime, and Matroska (used in WebM) are examples. Such container formats are fairly well suited for computers, but not usually well suited to broadcast and sometimes even not very well suited to dedicated, high-performance playback from media such as DVD and Blu-ray disc.

The Systems part of the MPEG-1 standard from 1992 established a multiplexing structure. That scheme was extended in MPEG-2, and the MPEG-2 scheme is now widely used in computing, in broadcasting, and in consumer video applications (including consumer camcorders using hard drive or flash media). MPEG-2 Part 1, Systems, defines two multiplexing mechanisms, the program stream (PS) and the transport stream (TS). Both can be regarded as MPEG “containers,” whose structure is the subject of the remainder of this chapter.