- •Contents
- •Figures
- •Tables
- •Preface
- •Acknowledgments
- •1. Raster images
- •Aspect ratio
- •Geometry
- •Image capture
- •Digitization
- •Perceptual uniformity
- •Colour
- •Luma and colour difference components
- •Digital image representation
- •Square sampling
- •Comparison of aspect ratios
- •Aspect ratio
- •Frame rates
- •Image state
- •EOCF standards
- •Entertainment programming
- •Acquisition
- •Consumer origination
- •Consumer electronics (CE) display
- •Contrast
- •Contrast ratio
- •Perceptual uniformity
- •The “code 100” problem and nonlinear image coding
- •Linear and nonlinear
- •4. Quantization
- •Linearity
- •Decibels
- •Noise, signal, sensitivity
- •Quantization error
- •Full-swing
- •Studio-swing (footroom and headroom)
- •Interface offset
- •Processing coding
- •Two’s complement wrap-around
- •Perceptual attributes
- •History of display signal processing
- •Digital driving levels
- •Relationship between signal and lightness
- •Algorithm
- •Black level setting
- •Effect of contrast and brightness on contrast and brightness
- •An alternate interpretation
- •Brightness and contrast controls in LCDs
- •Brightness and contrast controls in PDPs
- •Brightness and contrast controls in desktop graphics
- •Symbolic image description
- •Raster images
- •Conversion among types
- •Image files
- •“Resolution” in computer graphics
- •7. Image structure
- •Image reconstruction
- •Sampling aperture
- •Spot profile
- •Box distribution
- •Gaussian distribution
- •8. Raster scanning
- •Flicker, refresh rate, and frame rate
- •Introduction to scanning
- •Scanning parameters
- •Interlaced format
- •Interlace and progressive
- •Scanning notation
- •Motion portrayal
- •Segmented-frame (24PsF)
- •Video system taxonomy
- •Conversion among systems
- •9. Resolution
- •Magnitude frequency response and bandwidth
- •Visual acuity
- •Viewing distance and angle
- •Kell effect
- •Resolution
- •Resolution in video
- •Viewing distance
- •Interlace revisited
- •10. Constant luminance
- •The principle of constant luminance
- •Compensating for the CRT
- •Departure from constant luminance
- •Luma
- •“Leakage” of luminance into chroma
- •11. Picture rendering
- •Surround effect
- •Tone scale alteration
- •Incorporation of rendering
- •Rendering in desktop computing
- •Luma
- •Sloppy use of the term luminance
- •Colour difference coding (chroma)
- •Chroma subsampling
- •Chroma subsampling notation
- •Chroma subsampling filters
- •Chroma in composite NTSC and PAL
- •Scanning standards
- •Widescreen (16:9) SD
- •Square and nonsquare sampling
- •Resampling
- •NTSC and PAL encoding
- •NTSC and PAL decoding
- •S-video interface
- •Frequency interleaving
- •Composite analog SD
- •15. Introduction to HD
- •HD scanning
- •Colour coding for BT.709 HD
- •Data compression
- •Image compression
- •Lossy compression
- •JPEG
- •Motion-JPEG
- •JPEG 2000
- •Mezzanine compression
- •MPEG
- •Picture coding types (I, P, B)
- •Reordering
- •MPEG-1
- •MPEG-2
- •Other MPEGs
- •MPEG IMX
- •MPEG-4
- •AVC-Intra
- •WM9, WM10, VC-1 codecs
- •Compression for CE acquisition
- •AVCHD
- •Compression for IP transport to consumers
- •VP8 (“WebM”) codec
- •Dirac (basic)
- •17. Streams and files
- •Historical overview
- •Physical layer
- •Stream interfaces
- •IEEE 1394 (FireWire, i.LINK)
- •HTTP live streaming (HLS)
- •18. Metadata
- •Metadata Example 1: CD-DA
- •Metadata Example 2: .yuv files
- •Metadata Example 3: RFF
- •Metadata Example 4: JPEG/JFIF
- •Metadata Example 5: Sequence display extension
- •Conclusions
- •19. Stereoscopic (“3-D”) video
- •Acquisition
- •S3D display
- •Anaglyph
- •Temporal multiplexing
- •Polarization
- •Wavelength multiplexing (Infitec/Dolby)
- •Autostereoscopic displays
- •Parallax barrier display
- •Lenticular display
- •Recording and compression
- •Consumer interface and display
- •Ghosting
- •Vergence and accommodation
- •20. Filtering and sampling
- •Sampling theorem
- •Sampling at exactly 0.5fS
- •Magnitude frequency response
- •Magnitude frequency response of a boxcar
- •The sinc weighting function
- •Frequency response of point sampling
- •Fourier transform pairs
- •Analog filters
- •Digital filters
- •Impulse response
- •Finite impulse response (FIR) filters
- •Physical realizability of a filter
- •Phase response (group delay)
- •Infinite impulse response (IIR) filters
- •Lowpass filter
- •Digital filter design
- •Reconstruction
- •Reconstruction close to 0.5fS
- •“(sin x)/x” correction
- •Further reading
- •2:1 downsampling
- •Oversampling
- •Interpolation
- •Lagrange interpolation
- •Lagrange interpolation as filtering
- •Polyphase interpolators
- •Polyphase taps and phases
- •Implementing polyphase interpolators
- •Decimation
- •Lowpass filtering in decimation
- •Spatial frequency domain
- •Comb filtering
- •Spatial filtering
- •Image presampling filters
- •Image reconstruction filters
- •Spatial (2-D) oversampling
- •Retina
- •Adaptation
- •Contrast sensitivity
- •Contrast sensitivity function (CSF)
- •24. Luminance and lightness
- •Radiance, intensity
- •Luminance
- •Relative luminance
- •Luminance from red, green, and blue
- •Lightness (CIE L*)
- •Fundamentals of vision
- •Definitions
- •Spectral power distribution (SPD) and tristimulus
- •Spectral constraints
- •CIE XYZ tristimulus
- •CIE [x, y] chromaticity
- •Blackbody radiation
- •Colour temperature
- •White
- •Chromatic adaptation
- •Perceptually uniform colour spaces
- •CIE L*a*b* (CIELAB)
- •CIE L*u*v* and CIE L*a*b* summary
- •Colour specification and colour image coding
- •Further reading
- •Additive reproduction (RGB)
- •Characterization of RGB primaries
- •BT.709 primaries
- •Leggacy SD primaries
- •sRGB system
- •SMPTE Free Scale (FS) primaries
- •AMPAS ACES primaries
- •SMPTE/DCI P3 primaries
- •CMFs and SPDs
- •Normalization and scaling
- •Luminance coefficients
- •Transformations between RGB and CIE XYZ
- •Noise due to matrixing
- •Transforms among RGB systems
- •Camera white reference
- •Display white reference
- •Gamut
- •Wide-gamut reproduction
- •Free Scale Gamut, Free Scale Log (FS-Gamut, FS-Log)
- •Further reading
- •27. Gamma
- •Gamma in CRT physics
- •The amazing coincidence!
- •Gamma in video
- •Opto-electronic conversion functions (OECFs)
- •BT.709 OECF
- •SMPTE 240M OECF
- •sRGB transfer function
- •Transfer functions in SD
- •Bit depth requirements
- •Gamma in modern display devices
- •Estimating gamma
- •Gamma in video, CGI, and Macintosh
- •Gamma in computer graphics
- •Gamma in pseudocolour
- •Limitations of 8-bit linear coding
- •Linear and nonlinear coding in CGI
- •Colour acuity
- •RGB and R’G’B’ colour cubes
- •Conventional luma/colour difference coding
- •Luminance and luma notation
- •Nonlinear red, green, blue (R’G’B’)
- •BT.601 luma
- •BT.709 luma
- •Chroma subsampling, revisited
- •Luma/colour difference summary
- •SD and HD luma chaos
- •Luma/colour difference component sets
- •B’-Y’, R’-Y’ components for SD
- •PBPR components for SD
- •CBCR components for SD
- •Y’CBCR from studio RGB
- •Y’CBCR from computer RGB
- •“Full-swing” Y’CBCR
- •Y’UV, Y’IQ confusion
- •B’-Y’, R’-Y’ components for BT.709 HD
- •PBPR components for BT.709 HD
- •CBCR components for BT.709 HD
- •CBCR components for xvYCC
- •Y’CBCR from studio RGB
- •Y’CBCR from computer RGB
- •Conversions between HD and SD
- •Colour coding standards
- •31. Video signal processing
- •Edge treatment
- •Transition samples
- •Picture lines
- •Choice of SAL and SPW parameters
- •Video levels
- •Setup (pedestal)
- •BT.601 to computing
- •Enhancement
- •Median filtering
- •Coring
- •Chroma transition improvement (CTI)
- •Mixing and keying
- •Field rate
- •Line rate
- •Sound subcarrier
- •Addition of composite colour
- •NTSC colour subcarrier
- •576i PAL colour subcarrier
- •4fSC sampling
- •Common sampling rate
- •Numerology of HD scanning
- •Audio rates
- •33. Timecode
- •Introduction
- •Dropframe timecode
- •Editing
- •Linear timecode (LTC)
- •Vertical interval timecode (VITC)
- •Timecode structure
- •Further reading
- •34. 2-3 pulldown
- •2-3-3-2 pulldown
- •Conversion of film to different frame rates
- •Native 24 Hz coding
- •Conversion to other rates
- •Spatial domain
- •Vertical-temporal domain
- •Motion adaptivity
- •Further reading
- •36. Colourbars
- •SD colourbars
- •SD colourbar notation
- •Pluge element
- •Composite decoder adjustment using colourbars
- •-I, +Q, and Pluge elements in SD colourbars
- •HD colourbars
- •References
- •38. SDI and HD-SDI interfaces
- •Component digital SD interface (BT.601)
- •Serial digital interface (SDI)
- •Component digital HD-SDI
- •SDI and HD-SDI sync, TRS, and ancillary data
- •Analog sync and digital/analog timing relationships
- •Ancillary data
- •SDI coding
- •HD-SDI coding
- •Interfaces for compressed video
- •SDTI
- •Switching and mixing
- •Timing in digital facilities
- •Summary of digital interfaces
- •39. 480i component video
- •Frame rate
- •Interlace
- •Line sync
- •Field/frame sync
- •R’G’B’ EOCF and primaries
- •Luma (Y’)
- •Picture center, aspect ratio, and blanking
- •Halfline blanking
- •Component digital 4:2:2 interface
- •Component analog R’G’B’ interface
- •Component analog Y’PBPR interface, EBU N10
- •Component analog Y’PBPR interface, industry standard
- •40. 576i component video
- •Frame rate
- •Interlace
- •Line sync
- •Analog field/frame sync
- •R’G’B’ EOCF and primaries
- •Luma (Y’)
- •Picture center, aspect ratio, and blanking
- •Component digital 4:2:2 interface
- •Component analog 576i interface
- •Scanning
- •Analog sync
- •Picture center, aspect ratio, and blanking
- •R’G’B’ EOCF and primaries
- •Luma (Y’)
- •Component digital 4:2:2 interface
- •Scanning
- •Analog sync
- •Picture center, aspect ratio, and blanking
- •R’G’B’ EOCF and primaries
- •Luma (Y’)
- •Component digital 4:2:2 interface
- •43. HD videotape
- •HDCAM (D-11)
- •DVCPRO HD (D-12)
- •HDCAM SR (D-16)
- •JPEG blocks and MCUs
- •JPEG block diagram
- •Level shifting
- •Discrete cosine transform (DCT)
- •JPEG encoding example
- •JPEG decoding
- •Compression ratio control
- •JPEG/JFIF
- •Motion-JPEG (M-JPEG)
- •Further reading
- •46. DV compression
- •DV chroma subsampling
- •DV frame/field modes
- •Picture-in-shuttle in DV
- •DV overflow scheme
- •DV quantization
- •DV digital interface (DIF)
- •Consumer DV recording
- •Professional DV variants
- •47. MPEG-2 video compression
- •MPEG-2 profiles and levels
- •Picture structure
- •Frame rate and 2-3 pulldown in MPEG
- •Luma and chroma sampling structures
- •Macroblocks
- •Picture coding types – I, P, B
- •Prediction
- •Motion vectors (MVs)
- •Coding of a block
- •Frame and field DCT types
- •Zigzag and VLE
- •Refresh
- •Motion estimation
- •Rate control and buffer management
- •Bitstream syntax
- •Transport
- •Further reading
- •48. H.264 video compression
- •Algorithmic features, profiles, and levels
- •Baseline and extended profiles
- •High profiles
- •Hierarchy
- •Multiple reference pictures
- •Slices
- •Spatial intra prediction
- •Flexible motion compensation
- •Quarter-pel motion-compensated interpolation
- •Weighting and offsetting of MC prediction
- •16-bit integer transform
- •Quantizer
- •Variable-length coding
- •Context adaptivity
- •CABAC
- •Deblocking filter
- •Buffer control
- •Scalable video coding (SVC)
- •Multiview video coding (MVC)
- •AVC-Intra
- •Further reading
- •49. VP8 compression
- •Algorithmic features
- •Further reading
- •Elementary stream (ES)
- •Packetized elementary stream (PES)
- •MPEG-2 program stream
- •MPEG-2 transport stream
- •System clock
- •Further reading
- •Japan
- •United States
- •ATSC modulation
- •Europe
- •Further reading
- •Appendices
- •Cement vs. concrete
- •True CIE luminance
- •The misinterpretation of luminance
- •The enshrining of luma
- •Colour difference scale factors
- •Conclusion: A plea
- •Radiometry
- •Photometry
- •Light level examples
- •Image science
- •Units
- •Further reading
- •Glossary
- •Index
- •About the author
Tungsten illumination can’t have a colour temperature higher than tungsten’s melting point, 3695 K.
The reciprocal of correlated colour temperature is somewhat more perceptually uniform than correlated colour temperature itself. Cinematographers use units of mirek (micro reciprocal kelvin [MK-1]), that is, 106/t, where t is in units of kelvin [K]. Mirek units are more perceptually uniform than kelvin. For typical video or cinema acquisition, CCT typically ranges from 2000 K to 10,000 K; that is, from 500 to 100 mirek.
The mirek unit is sometimes called reciprocal megakelvin, and was historically called mired (“micro reciprocal degree”) .
Chromatic adaptation
Human vision adapts to the viewing environment. An image viewed in isolation – such as a 35 mm slide, or motion picture film projected in a dark room – creates its own white reference; a viewer will be quite tolerant of variation in white point. However, if the same image is viewed alongside an external white reference, or with a second image, differences in white point can be objectionable. Complete adaptation seems to be confined to colour temperatures from about 5000 K to 6500 K. Tungsten illumination, at about 3200 K, almost always appears somewhat yellow.
Perceptually uniform colour spaces
As I outlined in Perceptual uniformity, on page 30,
a system is perceptually uniform if a small perturbation to a component value is approximately equally perceptible across the range of that value.
Luminance is not perceptually uniform. On page 259, I described how luminance can be transformed to lightness, denoted L*, which is nearly perceptually uniform:
|
|
116 |
|
3 |
Y |
|
|
Y |
|
|
|
24 |
|
3 |
||||||
|
|
; |
|
≤ |
|
|||||||||||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|||||
|
|
|
YN |
|
|
|
|
|
|
|||||||||||
|
12 |
|
|
|
YN |
116 |
||||||||||||||
L* (Y) = |
|
|
|
|
|
1 |
|
|
|
|
|
|
|
|
|
|
|
|
Eq 25.3 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
3 |
|
|
|
||||
|
|
Y |
3 |
|
|
24 |
|
Y |
||||||||||||
|
116 |
|
|
|
|
− 16; |
|
|
|
|
|
|
< |
|
|
|
||||
|
|
|
|
|
|
Y |
||||||||||||||
|
|
|
Y |
|
|
|
116 |
|
||||||||||||
|
|
|
|
|
N |
|
|
|
|
|
|
|
|
|
|
|
|
|
N |
|
280 |
DIGITAL VIDEO AND HD ALGORITHMS AND INTERFACES |
L*u*v* and L*a*b* are often written CIELUV and CIELAB; they are usually pronounced SEA-love and SEA-lab. The u* and v* quantities of colour science – and the u’ and v’ quantities, to be described – are unrelated to the U and V colour difference components of video.
Extending this concept to colour, XYZ and RGB tristimulus values, and xyY (chromaticity and luminance), are far from perceptually uniform. Finding a transformation of XYZ into a reasonably perceptually uniform space occupied the CIE for a decade, and in the end no single system could be agreed upon. In 1976, the CIE standardized two systems, L*u*v* and L*a*b*, which
I will now describe. In both systems, perceptual difference is approximated as Euclidean distance.
CIE L*u*v*
Computation of CIE L*u*v* starts with a projective transformation of [X, Y, Z] into intermediate u’ and v’ quantities:
u’ = |
4 X |
|
|
v’ = |
9 Y |
|
|
|
; |
|
Eq 25.4 |
||
X + 15Y |
|
|
||||
|
+ 3 Z |
|
X + 15Y + 3 Z |
|||
Equivalently, u’ and v’ can be computed from x and y chromaticity:
u’ = |
|
4 x |
v’ = |
9 y |
||
|
|
; |
|
Eq 25.5 |
||
|
|
|
||||
3 |
− 2x + 12 y |
|
3− 2x + 12 y |
|||
To recover X and Z tristimulus values from u’ and v’, use these relations:
X = |
9u’ |
Y; |
Z = |
12 − 3u’− 20v’ |
Y |
Eq 25.6 |
|
|
|||||
|
4v’ |
|
4v' |
|
||
To recover x and y chromaticity from u’ and v’, use these relations:
x = |
9u’ |
; |
y = |
4v’ |
Eq 25.7 |
6u’− 16v’+ 12 |
6u’− 16v’+ 12 |
The primes in the CIE 1976 u’ and v’ quantities denote the successor to the obsolete 1960 CIE u and v quantities. u=u’; v=2⁄3v’ – that is, the 1960 v quantity underestimated visual perceptibility, and was multiplied by a factor of 1.5 to form the 1976 system. (To compute 1960 v, replace the numerator 9y in Eq 25.5 by 6y.) The primes are not formally related to the primes in R’, G’, B’, and Y’, though all imply some degree of perceptual uniformity.
Since u’ and v’ are formed by a projective transformation, u’ and v’ coordinates are associated with
a chromaticity diagram similar to the CIE 1931 2° [x, y] chromaticity diagram on page 274. You can use the [u’, v’] diagram if you want to produce 2-D plots that are more suggestive of the perceptibility of colour differences than an [x, y] plot would be. However,
[u’, v’] are subsequently multiplied by L* (see Equation 25.8 below) to form [u*, v*]. That multiplication effectively enlarges the perceptual increment as luminance decreases. Perceptual differences in a [u’, v’] diagram are dependant upon luminance, but that fact is
CHAPTER 25 |
THE CIE SYSTEM OF COLORIMETRY |
281 |
∆E* is pronounced delta E-star.
not evident from the diagram: Be careful not to draw strong conclusions from the diagram.
To compute u* and v*, first compute L*. Then compute u’n and v’n from your reference white Xn, Yn, and Zn. (The subscript n suggests normalized.) The u’n and v’n coordinates for several common white points are given in Table 25.1, White references, on page 279. (The [xn,yn] coordinates for a colour temperature of infinity are about [0.237, 0.237]; the [u’n,v’n] coordinates are about [0.177, 0.397].) Finally, compute u* and v*:
u* = 13 L*(Y) (u’ − u’N ); |
v* = 13 L*(Y) (v’ − v’N ) |
Eq 25.8 |
Gamut refers to the range of colours available in an imaging system. For gamuts typical of image reproduction, u* and v* values each range approximately ±100.
Euclidean distance in L*u*v* – denoted ∆E* – uv
estimates the perceptibility of colour differences:
∆ |
* |
= |
* |
− |
* 2 |
+ * |
− |
* 2 |
+ |
* |
− |
* |
2 |
|
Euv |
|
L2 |
|
L1 |
u2 |
|
u1 |
v |
2 |
|
v1 |
Eq 25.9 |
If ∆E* is unity or less, the colour difference is uv
assumed to be imperceptible. However, L*u*v* does not achieve perceptual uniformity, it is merely an approxi-
mation. ∆E* values between about 1 and 4 may or uv
may not be perceptible, depending upon the region of
colour space being examined. ∆E* values greater than uv
4 are likely to be perceptible; whether such differences are objectionable depends upon circumstances.
A polar-coordinate version of the [u*, v*] pair can be used to express chroma and hue:
|
|
|
2 |
|
2 |
|
|
−1 v* |
|
|
C* |
= |
u* |
|
+v* |
|
; |
h = tan |
|
Eq 25.10 |
|
|
|
|
||||||||
uv |
|
|
|
|
|
|
uv |
|
u* |
|
|
|
|
|
|
|
|
|
|
|
|
In addition, there is a “psychometric saturation” term:
s |
= |
C* |
Eq 25.11 |
|
L* |
||||
uv |
|
|
Chroma, hue, and saturation defined here are not directly related to saturation and hue in the HSB, HSI, HSL, HSV, and IHS systems used in computing and in digital image processing: Most of the published descriptions of these spaces, and most of the published formulæ, disregard the principles of colour science. In
282 |
DIGITAL VIDEO AND HD ALGORITHMS AND INTERFACES |
Eq 25.12
Eq 25.13
Eq 25.14
particular, the quantities called lightness and value are wildly inconsistent with their definitions in colour science.
CIE L*u*v* exhibits reasonable perceptual uniformity. L*u*v* has been common in video because the mapping of XYZ, xyY, and RGB to the u’v’ coordinates is projective: Straight lines in any of these spaces map to straight lines in u’v’. Despite the convenience and utility of L*u*v*, colour scientists today generally agree that better perceptual performance is exhibited by L*a*b*, which I will now describe.
CIE L*a*b* (CIELAB)
The quantities a* and b* are computed as follows:
|
* |
= |
125 |
|
* |
|
X |
|
* |
|
Y |
|
|
* |
|
50 |
|
* |
|
Y |
|
* |
|
Z |
|
a |
|
|
L |
|
|
|
− L |
|
; |
b |
|
= |
|
L |
|
|
|
− L |
|
|
|||||
|
29 |
X |
|
|
|
|
Z |
||||||||||||||||||
|
|
|
|
|
|
|
|
Y |
|
|
|
|
29 |
|
Y |
|
|
|
|
||||||
|
|
|
|
|
|
|
n |
|
|
|
n |
|
|
|
|
|
|
|
|
n |
|
|
|
n |
|
The coefficients are approximately 4.310 and 1.724. My definition is written in an unusual way, using L* instead of the traditional auxiliary function f. The definition of L* involves a linear segment having C1 continuity with a power function segment. That linear segment is incorporated (by way of L*) into a* and b*.
The reference L* range from black to white is zero to 100. For the BT.709 primaries typical of SD and HD, a* and b* are contained within the ranges [-87…+97] and [-108…+95] respectively, not including any undershoot, overshoot, or “illegal” or “invalid” CBCR values.
As in L*u*v*, one unit of Euclidean distance in L*a*b* –
denoted ∆E* – approximates the perceptibility of ab
colour differences:
∆ |
|
* |
= |
|
* |
− |
* 2 |
+ |
|
|
* |
− |
|
* 2 |
+ |
|
|
* |
− |
|
* 2 |
E |
|
L |
L |
|
a |
|
a |
|
|
b |
|
b |
|
||||||||
|
ab |
|
|
|
|
|
2 |
|
|
|
|
|
|
|
|||||||
|
|
|
2 |
|
1 |
|
|
|
1 |
|
2 |
|
1 |
||||||||
If ∆E*ab is unity or less, the colour difference is taken to be imperceptible. However, L*a*b* does not achieve perceptual uniformity: It is merely an approximation.
A polar-coordinate version of the [a*, b*] pair can be used to express chroma and hue:
|
|
a* |
2 |
+b* |
2 |
|
|
−1 b* |
|
C* |
= |
|
|
; |
h = tan |
|
|||
|
|
|
|||||||
ab |
|
|
|
|
|
|
ab |
|
a* |
|
|
|
|
|
|
|
|
|
|
CHAPTER 25 |
THE CIE SYSTEM OF COLORIMETRY |
283 |
