- •Contents
- •Figures
- •Tables
- •Preface
- •Acknowledgments
- •1. Raster images
- •Aspect ratio
- •Geometry
- •Image capture
- •Digitization
- •Perceptual uniformity
- •Colour
- •Luma and colour difference components
- •Digital image representation
- •Square sampling
- •Comparison of aspect ratios
- •Aspect ratio
- •Frame rates
- •Image state
- •EOCF standards
- •Entertainment programming
- •Acquisition
- •Consumer origination
- •Consumer electronics (CE) display
- •Contrast
- •Contrast ratio
- •Perceptual uniformity
- •The “code 100” problem and nonlinear image coding
- •Linear and nonlinear
- •4. Quantization
- •Linearity
- •Decibels
- •Noise, signal, sensitivity
- •Quantization error
- •Full-swing
- •Studio-swing (footroom and headroom)
- •Interface offset
- •Processing coding
- •Two’s complement wrap-around
- •Perceptual attributes
- •History of display signal processing
- •Digital driving levels
- •Relationship between signal and lightness
- •Algorithm
- •Black level setting
- •Effect of contrast and brightness on contrast and brightness
- •An alternate interpretation
- •Brightness and contrast controls in LCDs
- •Brightness and contrast controls in PDPs
- •Brightness and contrast controls in desktop graphics
- •Symbolic image description
- •Raster images
- •Conversion among types
- •Image files
- •“Resolution” in computer graphics
- •7. Image structure
- •Image reconstruction
- •Sampling aperture
- •Spot profile
- •Box distribution
- •Gaussian distribution
- •8. Raster scanning
- •Flicker, refresh rate, and frame rate
- •Introduction to scanning
- •Scanning parameters
- •Interlaced format
- •Interlace and progressive
- •Scanning notation
- •Motion portrayal
- •Segmented-frame (24PsF)
- •Video system taxonomy
- •Conversion among systems
- •9. Resolution
- •Magnitude frequency response and bandwidth
- •Visual acuity
- •Viewing distance and angle
- •Kell effect
- •Resolution
- •Resolution in video
- •Viewing distance
- •Interlace revisited
- •10. Constant luminance
- •The principle of constant luminance
- •Compensating for the CRT
- •Departure from constant luminance
- •Luma
- •“Leakage” of luminance into chroma
- •11. Picture rendering
- •Surround effect
- •Tone scale alteration
- •Incorporation of rendering
- •Rendering in desktop computing
- •Luma
- •Sloppy use of the term luminance
- •Colour difference coding (chroma)
- •Chroma subsampling
- •Chroma subsampling notation
- •Chroma subsampling filters
- •Chroma in composite NTSC and PAL
- •Scanning standards
- •Widescreen (16:9) SD
- •Square and nonsquare sampling
- •Resampling
- •NTSC and PAL encoding
- •NTSC and PAL decoding
- •S-video interface
- •Frequency interleaving
- •Composite analog SD
- •15. Introduction to HD
- •HD scanning
- •Colour coding for BT.709 HD
- •Data compression
- •Image compression
- •Lossy compression
- •JPEG
- •Motion-JPEG
- •JPEG 2000
- •Mezzanine compression
- •MPEG
- •Picture coding types (I, P, B)
- •Reordering
- •MPEG-1
- •MPEG-2
- •Other MPEGs
- •MPEG IMX
- •MPEG-4
- •AVC-Intra
- •WM9, WM10, VC-1 codecs
- •Compression for CE acquisition
- •AVCHD
- •Compression for IP transport to consumers
- •VP8 (“WebM”) codec
- •Dirac (basic)
- •17. Streams and files
- •Historical overview
- •Physical layer
- •Stream interfaces
- •IEEE 1394 (FireWire, i.LINK)
- •HTTP live streaming (HLS)
- •18. Metadata
- •Metadata Example 1: CD-DA
- •Metadata Example 2: .yuv files
- •Metadata Example 3: RFF
- •Metadata Example 4: JPEG/JFIF
- •Metadata Example 5: Sequence display extension
- •Conclusions
- •19. Stereoscopic (“3-D”) video
- •Acquisition
- •S3D display
- •Anaglyph
- •Temporal multiplexing
- •Polarization
- •Wavelength multiplexing (Infitec/Dolby)
- •Autostereoscopic displays
- •Parallax barrier display
- •Lenticular display
- •Recording and compression
- •Consumer interface and display
- •Ghosting
- •Vergence and accommodation
- •20. Filtering and sampling
- •Sampling theorem
- •Sampling at exactly 0.5fS
- •Magnitude frequency response
- •Magnitude frequency response of a boxcar
- •The sinc weighting function
- •Frequency response of point sampling
- •Fourier transform pairs
- •Analog filters
- •Digital filters
- •Impulse response
- •Finite impulse response (FIR) filters
- •Physical realizability of a filter
- •Phase response (group delay)
- •Infinite impulse response (IIR) filters
- •Lowpass filter
- •Digital filter design
- •Reconstruction
- •Reconstruction close to 0.5fS
- •“(sin x)/x” correction
- •Further reading
- •2:1 downsampling
- •Oversampling
- •Interpolation
- •Lagrange interpolation
- •Lagrange interpolation as filtering
- •Polyphase interpolators
- •Polyphase taps and phases
- •Implementing polyphase interpolators
- •Decimation
- •Lowpass filtering in decimation
- •Spatial frequency domain
- •Comb filtering
- •Spatial filtering
- •Image presampling filters
- •Image reconstruction filters
- •Spatial (2-D) oversampling
- •Retina
- •Adaptation
- •Contrast sensitivity
- •Contrast sensitivity function (CSF)
- •24. Luminance and lightness
- •Radiance, intensity
- •Luminance
- •Relative luminance
- •Luminance from red, green, and blue
- •Lightness (CIE L*)
- •Fundamentals of vision
- •Definitions
- •Spectral power distribution (SPD) and tristimulus
- •Spectral constraints
- •CIE XYZ tristimulus
- •CIE [x, y] chromaticity
- •Blackbody radiation
- •Colour temperature
- •White
- •Chromatic adaptation
- •Perceptually uniform colour spaces
- •CIE L*a*b* (CIELAB)
- •CIE L*u*v* and CIE L*a*b* summary
- •Colour specification and colour image coding
- •Further reading
- •Additive reproduction (RGB)
- •Characterization of RGB primaries
- •BT.709 primaries
- •Leggacy SD primaries
- •sRGB system
- •SMPTE Free Scale (FS) primaries
- •AMPAS ACES primaries
- •SMPTE/DCI P3 primaries
- •CMFs and SPDs
- •Normalization and scaling
- •Luminance coefficients
- •Transformations between RGB and CIE XYZ
- •Noise due to matrixing
- •Transforms among RGB systems
- •Camera white reference
- •Display white reference
- •Gamut
- •Wide-gamut reproduction
- •Free Scale Gamut, Free Scale Log (FS-Gamut, FS-Log)
- •Further reading
- •27. Gamma
- •Gamma in CRT physics
- •The amazing coincidence!
- •Gamma in video
- •Opto-electronic conversion functions (OECFs)
- •BT.709 OECF
- •SMPTE 240M OECF
- •sRGB transfer function
- •Transfer functions in SD
- •Bit depth requirements
- •Gamma in modern display devices
- •Estimating gamma
- •Gamma in video, CGI, and Macintosh
- •Gamma in computer graphics
- •Gamma in pseudocolour
- •Limitations of 8-bit linear coding
- •Linear and nonlinear coding in CGI
- •Colour acuity
- •RGB and R’G’B’ colour cubes
- •Conventional luma/colour difference coding
- •Luminance and luma notation
- •Nonlinear red, green, blue (R’G’B’)
- •BT.601 luma
- •BT.709 luma
- •Chroma subsampling, revisited
- •Luma/colour difference summary
- •SD and HD luma chaos
- •Luma/colour difference component sets
- •B’-Y’, R’-Y’ components for SD
- •PBPR components for SD
- •CBCR components for SD
- •Y’CBCR from studio RGB
- •Y’CBCR from computer RGB
- •“Full-swing” Y’CBCR
- •Y’UV, Y’IQ confusion
- •B’-Y’, R’-Y’ components for BT.709 HD
- •PBPR components for BT.709 HD
- •CBCR components for BT.709 HD
- •CBCR components for xvYCC
- •Y’CBCR from studio RGB
- •Y’CBCR from computer RGB
- •Conversions between HD and SD
- •Colour coding standards
- •31. Video signal processing
- •Edge treatment
- •Transition samples
- •Picture lines
- •Choice of SAL and SPW parameters
- •Video levels
- •Setup (pedestal)
- •BT.601 to computing
- •Enhancement
- •Median filtering
- •Coring
- •Chroma transition improvement (CTI)
- •Mixing and keying
- •Field rate
- •Line rate
- •Sound subcarrier
- •Addition of composite colour
- •NTSC colour subcarrier
- •576i PAL colour subcarrier
- •4fSC sampling
- •Common sampling rate
- •Numerology of HD scanning
- •Audio rates
- •33. Timecode
- •Introduction
- •Dropframe timecode
- •Editing
- •Linear timecode (LTC)
- •Vertical interval timecode (VITC)
- •Timecode structure
- •Further reading
- •34. 2-3 pulldown
- •2-3-3-2 pulldown
- •Conversion of film to different frame rates
- •Native 24 Hz coding
- •Conversion to other rates
- •Spatial domain
- •Vertical-temporal domain
- •Motion adaptivity
- •Further reading
- •36. Colourbars
- •SD colourbars
- •SD colourbar notation
- •Pluge element
- •Composite decoder adjustment using colourbars
- •-I, +Q, and Pluge elements in SD colourbars
- •HD colourbars
- •References
- •38. SDI and HD-SDI interfaces
- •Component digital SD interface (BT.601)
- •Serial digital interface (SDI)
- •Component digital HD-SDI
- •SDI and HD-SDI sync, TRS, and ancillary data
- •Analog sync and digital/analog timing relationships
- •Ancillary data
- •SDI coding
- •HD-SDI coding
- •Interfaces for compressed video
- •SDTI
- •Switching and mixing
- •Timing in digital facilities
- •Summary of digital interfaces
- •39. 480i component video
- •Frame rate
- •Interlace
- •Line sync
- •Field/frame sync
- •R’G’B’ EOCF and primaries
- •Luma (Y’)
- •Picture center, aspect ratio, and blanking
- •Halfline blanking
- •Component digital 4:2:2 interface
- •Component analog R’G’B’ interface
- •Component analog Y’PBPR interface, EBU N10
- •Component analog Y’PBPR interface, industry standard
- •40. 576i component video
- •Frame rate
- •Interlace
- •Line sync
- •Analog field/frame sync
- •R’G’B’ EOCF and primaries
- •Luma (Y’)
- •Picture center, aspect ratio, and blanking
- •Component digital 4:2:2 interface
- •Component analog 576i interface
- •Scanning
- •Analog sync
- •Picture center, aspect ratio, and blanking
- •R’G’B’ EOCF and primaries
- •Luma (Y’)
- •Component digital 4:2:2 interface
- •Scanning
- •Analog sync
- •Picture center, aspect ratio, and blanking
- •R’G’B’ EOCF and primaries
- •Luma (Y’)
- •Component digital 4:2:2 interface
- •43. HD videotape
- •HDCAM (D-11)
- •DVCPRO HD (D-12)
- •HDCAM SR (D-16)
- •JPEG blocks and MCUs
- •JPEG block diagram
- •Level shifting
- •Discrete cosine transform (DCT)
- •JPEG encoding example
- •JPEG decoding
- •Compression ratio control
- •JPEG/JFIF
- •Motion-JPEG (M-JPEG)
- •Further reading
- •46. DV compression
- •DV chroma subsampling
- •DV frame/field modes
- •Picture-in-shuttle in DV
- •DV overflow scheme
- •DV quantization
- •DV digital interface (DIF)
- •Consumer DV recording
- •Professional DV variants
- •47. MPEG-2 video compression
- •MPEG-2 profiles and levels
- •Picture structure
- •Frame rate and 2-3 pulldown in MPEG
- •Luma and chroma sampling structures
- •Macroblocks
- •Picture coding types – I, P, B
- •Prediction
- •Motion vectors (MVs)
- •Coding of a block
- •Frame and field DCT types
- •Zigzag and VLE
- •Refresh
- •Motion estimation
- •Rate control and buffer management
- •Bitstream syntax
- •Transport
- •Further reading
- •48. H.264 video compression
- •Algorithmic features, profiles, and levels
- •Baseline and extended profiles
- •High profiles
- •Hierarchy
- •Multiple reference pictures
- •Slices
- •Spatial intra prediction
- •Flexible motion compensation
- •Quarter-pel motion-compensated interpolation
- •Weighting and offsetting of MC prediction
- •16-bit integer transform
- •Quantizer
- •Variable-length coding
- •Context adaptivity
- •CABAC
- •Deblocking filter
- •Buffer control
- •Scalable video coding (SVC)
- •Multiview video coding (MVC)
- •AVC-Intra
- •Further reading
- •49. VP8 compression
- •Algorithmic features
- •Further reading
- •Elementary stream (ES)
- •Packetized elementary stream (PES)
- •MPEG-2 program stream
- •MPEG-2 transport stream
- •System clock
- •Further reading
- •Japan
- •United States
- •ATSC modulation
- •Europe
- •Further reading
- •Appendices
- •Cement vs. concrete
- •True CIE luminance
- •The misinterpretation of luminance
- •The enshrining of luma
- •Colour difference scale factors
- •Conclusion: A plea
- •Radiometry
- •Photometry
- •Light level examples
- •Image science
- •Units
- •Further reading
- •Glossary
- •Index
- •About the author
X, Y, and Z are pronounced big-X, big-Y, and big-Z, or cap-X, cap-Y, and cap-Z, to distinguish them from little-x and little-y, to be described in a moment.
Grassmann’s Third Law:
Sources of the same colour produce identical effects in an additive mixture regardless of their spectral composition.
Thornton, William A. (1999), “Spectral sensitivities of the normal human visual system, color-matching functions and their principles, and how and why the two sets should coincide,” in Color Research and Application 24 (2): 139–156 (Apr.).
property of vision; it is best considered as a consequence
_ _
of the mathematical process by which the x(λ), y(λ), and
_
z(λ) curves are constructed.
CIE XYZ tristimulus
_
Weighting an SPD under the y(λ) colour-matching function yields luminance (symbol Y), as I described on page 205. When luminance is augmented with two
other values, computed in the same manner as lumi-
_ _
nance but using the x(λ) and z(λ) colour-matching functions, the resulting values are known as XYZ tristimulus values (denoted X, Y, and Z). XYZ values correlate to the spectral sensitivity of human vision. Their amplitudes – always nonnegative – are proportional to intensity.
Tristimulus values are computed from a continuous
_ _
SPD by integrating the SPD under the x(λ), y(λ), and
_
z(λ) colour-matching functions. In discrete form, tristimulus values are computed by a matrix multiplication, as illustrated in Figure 25.6 opposite.
Human colour vision follows a principle of superposition known as Grassmann’s Third Law: The set of tristimulus values computed from the sum of a set of SPDs is identical to the sum of the tristimulus values of each SPD. Due to this linearity of additive colour mixture, any set of three components that is a nontrivial linear combination of X, Y, and Z – such as R, G, and B – is also a set of tristimulus values. (In Transformations between RGB and CIE XYZ, on page 307, I will introduce related CMFs that produce R, G, and B tristimulus values.)
Luminance can be considered to be a distinguished tristimulus value that is meaningful on its own, and, exceptionally, carries units of cd·m-2. Apart from luminance, tristimuli come in sets of three, as the word suggests, and have no units.
This chapter accepts the CIE Standard Observer rather uncritically. Although the CIE Standard Observer is very useful and widely used, some researchers believe that it exhibits some problems and ought to be improved. For one well-informed and provocative view, see Thornton.
272 |
DIGITAL VIDEO AND HD ALGORITHMS AND INTERFACES |
X |
|
0.0143 |
0.0004 |
0.0679 |
|
T |
|
82.75 |
|
400 nm |
|
|
|
|
|
|
|
|
|
||||||||
|
|
|
|
|
|
|
|||||||
Y |
= |
0.0435 0.0012 0.2074 |
|
• |
|
91.49 |
|
|
|
|
|||
Z |
|
0.1344 0.0040 0.6456 |
|
|
|
93.43 |
|
|
|
|
|||
|
|
0.2839 |
0.0116 |
1.3856 |
|
|
|
86.68 |
|
|
|
|
|
|
|
0.3483 |
0.0230 |
1.7471 |
|
|
|
104.86 |
|
|
|
|
|
|
|
0.3362 |
0.0380 |
1.7721 |
|
|
|
117.01 |
|
450 nm |
|
|
|
|
|
0.2908 |
0.0600 |
1.6692 |
|
|
|
117.81 |
|
|
|
|
|
|
|
0.1954 |
0.0910 |
1.2876 |
|
|
|
114.86 |
|
|
|
|
|
|
|
0.0956 |
0.1390 |
0.8130 |
|
|
|
115.92 |
|
|
|
|
|
|
|
0.0320 |
0.2080 |
0.4652 |
|
|
|
108.81 |
|
|
|
|
|
|
|
0.0049 |
0.3230 |
0.2720 |
|
|
|
109.35 |
|
500 nm |
|
|
|
|
|
0.0093 |
0.5030 |
0.1582 |
|
|
|
107.80 |
|
|
|
|
|
|
|
0.0633 |
0.7100 |
0.0782 |
|
|
|
104.79 |
|
|
|
|
|
|
|
0.1655 |
0.8620 |
0.0422 |
|
|
|
107.69 |
|
|
|
|
|
|
|
0.2904 |
0.9540 |
0.0203 |
|
|
|
104.41 |
|
|
|
|
|
|
|
0.4334 |
0.9950 |
0.0087 |
|
|
|
104.05 |
|
550 nm |
|
|
|
|
|
0.5945 |
0.9950 |
0.0039 |
|
|
|
100.00 |
|
|
|
|
|
|
|
0.7621 |
0.9520 |
0.0021 |
|
|
|
96.33 |
|
|
|
|
|
|
|
0.9163 |
0.8700 |
0.0017 |
|
|
|
95.79 |
|
|
|
|
|
|
|
1.0263 |
0.7570 |
0.0011 |
|
|
|
88.69 |
|
|
|
|
|
|
|
1.0622 |
0.6310 |
0.0008 |
|
|
|
90.01 |
|
600 nm |
|
|
|
|
|
1.0026 |
0.5030 |
0.0003 |
|
|
|
89.60 |
|
|
|
|
|
|
|
0.8544 |
0.3810 |
0.0002 |
|
|
|
87.70 |
|
|
|
|
|
|
|
0.6424 |
0.2650 |
0.0000 |
|
|
|
83.29 |
|
|
|
|
|
|
|
0.4479 |
0.1750 |
0.0000 |
|
|
|
83.70 |
|
|
|
|
|
|
|
0.2835 |
0.1070 |
0.0000 |
|
|
|
80.03 |
|
650 nm |
|
|
|
|
|
0.1649 |
0.0610 |
0.0000 |
|
|
|
80.21 |
|
|
|
|
|
|
|
0.0874 |
0.0320 |
0.0000 |
|
|
|
82.28 |
|
|
|
|
|
|
|
0.0468 |
0.0170 |
0.0000 |
|
|
|
78.28 |
|
|
|
|
|
|
|
0.0227 |
0.0082 |
0.0000 |
|
|
|
69.72 |
|
|
|
|
|
|
|
0.0114 |
0.0041 |
0.0000 |
|
|
|
71.61 |
|
700 nm |
|
|
|
|
|
Figure 25.6 Calculation of tristimulus values by matrix multiplica- |
|||||||||||
|
|
tion starts with a column vector representing the SPD. The |
|||||||||||
|
|
31-element column vector in this example is a discrete version of |
|||||||||||
|
|
CIE Illuminant D |
|
sampled at 10 nm intervals. The SPD is matrix- |
|||||||||
|
|
|
65 |
|
|
|
|
|
_ |
_ |
_ |
||
multiplied by a discrete version of the CIE x(λ), y(λ), and z(λ) colour-matching functions (CMFs) of Figure 25.5, here in a 31× 3 matrix (which is sometimes denoted A). The superscript T denotes
the matrix transpose operation. The result of the matrix multiplica- |
|
tion is a set of XYZ tristimulus components. |
_ |
|
|
In the caption to Figure 25.5, I mentioned that y(λ) is scaled to |
|
unity at 560 nm. In the 10 nm approximation given here, the value is not exactly unity owing to the CIE’s interpolation procedure.
CHAPTER 25 |
THE CIE SYSTEM OF COLORIMETRY |
273 |
y 520
0.8
540
0.7
500
0.6 |
560 |
0.5
580
0.4
0.3
480
0.2
0.1
460
440 400
0.0 
600
620
640
700
0.0 |
0.1 |
0.2 |
0.3 |
0.4 |
0.5 |
0.6 |
0.7 |
x |
Figure 25.7 CIE 1931 2° [x, y] chromaticity diagram. The spectral locus is a horseshoe-shaped path swept by a monochromatic source as it is tuned from 400 nm to 700 nm. The line of purples traces SPDs that combine longwave and shortwave power but have no mediumwave power. All colours lie within the horseshoe-shaped region: Points outside this region are not colours.
This diagram is not a slice through [X, Y, Z] space! Instead, points in [X, Y, Z] project onto the plane of the diagram in a manner comparable to the perspective projection. White has [X, Y, Z] values near [1, 1, 1]; it projects to a point near the center of the diagram, in the region of [1⁄3, 1⁄3]. Attempting to project black, at XYZ coordinates [0, 0, 0], would require dividing by zero in Equation 25.1: Black has no place in a chromaticity diagram.
274 |
DIGITAL VIDEO AND HD ALGORITHMS AND INTERFACES |
The x and y symbols are pronounced little-x and little-y.
1
y = 1- x
1
Spectral locus
Line of purples
Alychne
Blackbody locus
White point
Figure 25.8 CIE [x, y] chart features.
CIE [x, y] chromaticity
It is convenient, for both conceptual understanding and for computation, to have a representation of “pure” colour in the absence of lightness. The CIE standardized a procedure for normalizing XYZ tristimulus values to obtain two chromaticity values x and y.
Chromaticity values are computed by this projective transformation:
x = |
X |
; |
y = |
Y |
Eq 25.1 |
X + Y + Z |
X + Y + Z |
A third chromaticity coordinate, z, is defined, but is redundant since x + y + z = 1. The x and y chromaticity coordinates are abstract values that have no direct physical interpretation.
A colour can be specified by its chromaticity and luminance, in the form of an xyY triple. To recover X and Z tristimulus values from [x, y] chromaticities and luminance, use the inverse of Equation 25.1:
X = |
x |
Y; |
Z = |
1− x − y |
Y |
Eq 25.2 |
|
|
|||||
|
y |
|
y |
|
||
A colour plots as a point in an [x, y] chromaticity diagram, plotted in Figure 25.7 opposite.
In Figure 25.8 in the margin, I sketch several features of the [x, y] diagram. The important features lie on, or below and to the left of, the line y=1-x.
When a narrowband (monochromatic) SPD comprising power at just one wavelength is swept across the range 400 nm to 700 nm, it traces the inverted-U (or horseshoe) shaped spectral locus in [x, y] coordinates.
The sensation of purple cannot be produced by
a single wavelength; it requires a mixture of shortwave and longwave light. The line of purples on a chromaticity diagram joins the chromaticity of extreme blue (violet), containing only shortwave power, to the chromaticity of extreme red, containing only longwave power.
There is no unique physical or perceptual definition of white. Many important sources of illumination are blackbody radiators, whose chromaticity coordinates lie on the blackbody locus (sometimes called the Planckian
CHAPTER 25 |
THE CIE SYSTEM OF COLORIMETRY |
275 |
