JFIF mandates Y’CBCR with BT.601 luma-chroma matrixing; however, 4:2:0 chroma subsampling in JFIF is sited interstitially both horizontally and vertically, unlike BT.601.

Hamilton, Eric (1992), JPEG File Interchange Format, Version 1.02 (Milpitas, Calif.: C-Cube Microsystems). This informal document was endorsed by ECMA, who made slight modifications and in June 2009 published ECMA TR/98 having the same title.

To use JFIF for BT.709 HD Y’CBCR, to conform to the standard you must recode to 601Y’CBCR, compress, transfer, decompress, and finally recode to 709Y’CBCR.

Metadata Example 4: JPEG/JFIF

At its inception, the JPEG committee decided to avoid colour space wars: Its scope was established as compressing and decompressing image data, without concern for what colours the data represented. They accommodated one-channel greyscale image data, three-channel image data such as RGB, and four-channel image data such as CMYK. Chroma subsampling was recognized as providing a big compression gain – a 2:1 factor in the case of 4:2:0 – so allowance was made to enable luma-chroma encoding as a preprocessing step.

JPEG development culminated before the World Wide Web emerged – JPEG’s original target application was colour facsimile! The founders expected that system integrators would make provisions outside JPEG for reliable colour transfer. However, JPEG was rapidly adopted as a method of exchanging colour images in files, not just compressing them.

It became clear to a JPEG proponent, C-Cube, that confusion regarding colour spaces in the exchange of JPEG files threatened to inhibit commercialization.

C-Cube quickly drafted a document (paper metadata) defining a file format called JFIF, stating that image data was to be coded in the Y’CBCR colour space of BT.601 (but without the footroom and headroom). JFIF is clear on the arithmetic. However, implementation according to JFIF calls for a CB/CR reference range of ±128, that is, 257 integers – but 8-bit coding permits only 256 values! Meeting the specification produces grey having noninteger CB and CR values; decoded grey is bound to be coloured. Implementations (particularly the widely used libjpeg) use the standard scale factors but clip the CB/CR range to +127, thereby clipping pure blue and pure red. JFIF explicitly mandates the BT.601 luma-chroma matrix. BT.601 says nothing of primaries, and JFIF does not say what primaries are intended. JFIF was embraced by the computer industry at exactly the time that the

sRGB standard was being formulated using BT.709 primaries. In practice, JFIF uses BT.709 primaries.

The JFIF specification states, “RGB components calculated by linear conversion from YCbCr shall not be gamma corrected (gamma = 1.0).” This passage does not mean that linear-light components are encoded; instead, decoding is intended to conform to BT.601

Here I describe aspects of MPEG-2’s sequence display extension. Identical metadata is conveyed in H.264’s Annex E,

Video usability information (VUI).

DeMarsh, LeRoy E. (1993), “TV display phosphors/primaries: Some history," in SMPTE J. 102 (12): 1095–1098.

practice, which (after Y’CBCR-to-R’G’B’ dematrixing) imposes a 2.4-power function on R’G’B’ components to produce display tristimulus values.

So, we have the following mess:

•The spec says “BT.601 Y’CBCR” but contrary to BT.601, and for no good reason, “full-swing” is used.

•The spec says CB and CR values range ±128, a range unattainable in 8-bit integer arithmetic. Implementa-

tions cope by clipping pure blue and pure red.

•The spec says “gamma = 1.0,” but the intention is clearly not linear-light coding. The spec is otherwise silent on “gamma,” but a 2.4-power law EOCF is implicit.

•The spec says “BT.601,” evidently taking that to define colour primaries, but BT.601 is silent on colour primaries. In practice, the primaries of BT.709 are used.

You may be thinking, “this is just a story about

a poorly written specification for paper metadata, and about nonconformant implementations.” That assessment is mainly correct. The next example is analagous – but brings the poorly conceived metadata into the professional video data stream.

Metadata Example 5: Sequence display extension

In a manner roughly comparable with MPEG-2’s RFF flag, MPEG-2’s sequence display extension provides a decoder with information concerning how the image

data is intended to be displayed. The standard provides (in printed form, as paper metadata) tables giving RGB primary chromaticities (color primaries), transfer functions (transfer characteristics), and luma-chroma matrices (matrix coefficients). The bitstream conveys enumerated codes that serve as indexes into these tables. The same scheme is adopted in H.264. The tables (simplified, and augmented with my annotations) are summarized in Tables 18.1, 18.2, and 18.3.

Color primaries code 4 designates NTSC 1953 primaries. As far as I am aware, no extant recorded video material uses those primaries; they had been abandoned before the introduction of the first VTR. A bitstream containing that value is nonsensical: If the code is encountered, it ought to be ignored by all decoders. (A well-meaning technician may have set the code thinking, “I’m broadcasting NTSC; this is the only setting that says NTSC; I’d better use it.”)

2 Unspecified

3 Reserved/future

4 BT.470-6/NTSC 1953

5 EBU Tech. 3213

6 SMPTE RP 145

7 SMPTE 240M

8 ”Generic film“

Table 18.1 Color primaries.

Entries shaded in red are obsolete; the NTSC 1953 entry is utterly obsolete. Codes 5 and 6 are unsuitable for HD. Code 8 “Generic film” (shaded in magenta) is inscrutable. No matter which code you place in the bitstream at encoding, your material will almost certainly be presented with BT.709 primaries (bolded).

MPEG and H.264 have the pervasive conceptual model that only bitstreams and decoders are standardized. A compliant encoder emits only legal bistreams. Apart from that, no aspect of the encoder is standardized. So, transfer characteristics should be specified as EOCFs, not OECFs!

Apple, Inc. (2011), QuickTime File

Format Specification (July): 141.

MPEG cites the ITU-R document incorrectly: The cited document is a Report (Rep.), not a Recommendation (Rec.).

Issues with SDE metadata must be handled by software developers. Here’s what Apple says concerning the duplicate BT.709 codes in transfer characteristics:

QuickTime writers should map [code] 6 to 1 when converting from transfer_characteristics …

The MPEG-2 specification cites ITU-R Rec. BT.470-6. Concerning that reference, Apple writes,

This information is both incomplete and obsolete.

We could add, “erroneous.”

In the light of all this confusion, how should an encoder be configured?

•For SD material, set the primaries to EBU 3213 for 576i material and to SMPTE RP 145 for 480i material; set transfer characteristics to BT.709 and matrix coefficients to BT.601.

•For HD, declare BT.709 everywhere in the SDE.

A problem for ATSC encoders is that North American HD material is almost all mastered with SMPTE RP 145 primaries, and you’re tempted to declare that; however, ATSC specifications call for BT.709 primaries, and virtually all consumer receivers display with BT.709 primaries. My suggestion is to declare BT.709, for two reasons: to be ATSC compliant, and to prepare for the future when regional primary sets are relics of that past.

What should a decoder do?

•For SD formats, if BT.709 is declared for color primaries, it’s probably intended and should be respected; otherwise, expect EBU 3213 for 576i material and SMPTE RP 145 for 480i material. Any other code is nonsensical and should be treated as BT.709. Treat transfer characteristics as BT.709 no matter what is declared. Expect matrix coefficients to be BT.601; BT.709 could potentially be correct but should be treated with suspicion. Any other code is almost certainly wrong.

•For HD formats, expect BT.709 across the board. Any other codes are highly suspect.

I summarize these recommendations for decoder processing in Tables 18.4, 18.5, and 18.6.