- •Contents
- •Figures
- •Tables
- •Preface
- •Acknowledgments
- •1. Raster images
- •Aspect ratio
- •Geometry
- •Image capture
- •Digitization
- •Perceptual uniformity
- •Colour
- •Luma and colour difference components
- •Digital image representation
- •Square sampling
- •Comparison of aspect ratios
- •Aspect ratio
- •Frame rates
- •Image state
- •EOCF standards
- •Entertainment programming
- •Acquisition
- •Consumer origination
- •Consumer electronics (CE) display
- •Contrast
- •Contrast ratio
- •Perceptual uniformity
- •The “code 100” problem and nonlinear image coding
- •Linear and nonlinear
- •4. Quantization
- •Linearity
- •Decibels
- •Noise, signal, sensitivity
- •Quantization error
- •Full-swing
- •Studio-swing (footroom and headroom)
- •Interface offset
- •Processing coding
- •Two’s complement wrap-around
- •Perceptual attributes
- •History of display signal processing
- •Digital driving levels
- •Relationship between signal and lightness
- •Algorithm
- •Black level setting
- •Effect of contrast and brightness on contrast and brightness
- •An alternate interpretation
- •Brightness and contrast controls in LCDs
- •Brightness and contrast controls in PDPs
- •Brightness and contrast controls in desktop graphics
- •Symbolic image description
- •Raster images
- •Conversion among types
- •Image files
- •“Resolution” in computer graphics
- •7. Image structure
- •Image reconstruction
- •Sampling aperture
- •Spot profile
- •Box distribution
- •Gaussian distribution
- •8. Raster scanning
- •Flicker, refresh rate, and frame rate
- •Introduction to scanning
- •Scanning parameters
- •Interlaced format
- •Interlace and progressive
- •Scanning notation
- •Motion portrayal
- •Segmented-frame (24PsF)
- •Video system taxonomy
- •Conversion among systems
- •9. Resolution
- •Magnitude frequency response and bandwidth
- •Visual acuity
- •Viewing distance and angle
- •Kell effect
- •Resolution
- •Resolution in video
- •Viewing distance
- •Interlace revisited
- •10. Constant luminance
- •The principle of constant luminance
- •Compensating for the CRT
- •Departure from constant luminance
- •Luma
- •“Leakage” of luminance into chroma
- •11. Picture rendering
- •Surround effect
- •Tone scale alteration
- •Incorporation of rendering
- •Rendering in desktop computing
- •Luma
- •Sloppy use of the term luminance
- •Colour difference coding (chroma)
- •Chroma subsampling
- •Chroma subsampling notation
- •Chroma subsampling filters
- •Chroma in composite NTSC and PAL
- •Scanning standards
- •Widescreen (16:9) SD
- •Square and nonsquare sampling
- •Resampling
- •NTSC and PAL encoding
- •NTSC and PAL decoding
- •S-video interface
- •Frequency interleaving
- •Composite analog SD
- •15. Introduction to HD
- •HD scanning
- •Colour coding for BT.709 HD
- •Data compression
- •Image compression
- •Lossy compression
- •JPEG
- •Motion-JPEG
- •JPEG 2000
- •Mezzanine compression
- •MPEG
- •Picture coding types (I, P, B)
- •Reordering
- •MPEG-1
- •MPEG-2
- •Other MPEGs
- •MPEG IMX
- •MPEG-4
- •AVC-Intra
- •WM9, WM10, VC-1 codecs
- •Compression for CE acquisition
- •AVCHD
- •Compression for IP transport to consumers
- •VP8 (“WebM”) codec
- •Dirac (basic)
- •17. Streams and files
- •Historical overview
- •Physical layer
- •Stream interfaces
- •IEEE 1394 (FireWire, i.LINK)
- •HTTP live streaming (HLS)
- •18. Metadata
- •Metadata Example 1: CD-DA
- •Metadata Example 2: .yuv files
- •Metadata Example 3: RFF
- •Metadata Example 4: JPEG/JFIF
- •Metadata Example 5: Sequence display extension
- •Conclusions
- •19. Stereoscopic (“3-D”) video
- •Acquisition
- •S3D display
- •Anaglyph
- •Temporal multiplexing
- •Polarization
- •Wavelength multiplexing (Infitec/Dolby)
- •Autostereoscopic displays
- •Parallax barrier display
- •Lenticular display
- •Recording and compression
- •Consumer interface and display
- •Ghosting
- •Vergence and accommodation
- •20. Filtering and sampling
- •Sampling theorem
- •Sampling at exactly 0.5fS
- •Magnitude frequency response
- •Magnitude frequency response of a boxcar
- •The sinc weighting function
- •Frequency response of point sampling
- •Fourier transform pairs
- •Analog filters
- •Digital filters
- •Impulse response
- •Finite impulse response (FIR) filters
- •Physical realizability of a filter
- •Phase response (group delay)
- •Infinite impulse response (IIR) filters
- •Lowpass filter
- •Digital filter design
- •Reconstruction
- •Reconstruction close to 0.5fS
- •“(sin x)/x” correction
- •Further reading
- •2:1 downsampling
- •Oversampling
- •Interpolation
- •Lagrange interpolation
- •Lagrange interpolation as filtering
- •Polyphase interpolators
- •Polyphase taps and phases
- •Implementing polyphase interpolators
- •Decimation
- •Lowpass filtering in decimation
- •Spatial frequency domain
- •Comb filtering
- •Spatial filtering
- •Image presampling filters
- •Image reconstruction filters
- •Spatial (2-D) oversampling
- •Retina
- •Adaptation
- •Contrast sensitivity
- •Contrast sensitivity function (CSF)
- •24. Luminance and lightness
- •Radiance, intensity
- •Luminance
- •Relative luminance
- •Luminance from red, green, and blue
- •Lightness (CIE L*)
- •Fundamentals of vision
- •Definitions
- •Spectral power distribution (SPD) and tristimulus
- •Spectral constraints
- •CIE XYZ tristimulus
- •CIE [x, y] chromaticity
- •Blackbody radiation
- •Colour temperature
- •White
- •Chromatic adaptation
- •Perceptually uniform colour spaces
- •CIE L*a*b* (CIELAB)
- •CIE L*u*v* and CIE L*a*b* summary
- •Colour specification and colour image coding
- •Further reading
- •Additive reproduction (RGB)
- •Characterization of RGB primaries
- •BT.709 primaries
- •Leggacy SD primaries
- •sRGB system
- •SMPTE Free Scale (FS) primaries
- •AMPAS ACES primaries
- •SMPTE/DCI P3 primaries
- •CMFs and SPDs
- •Normalization and scaling
- •Luminance coefficients
- •Transformations between RGB and CIE XYZ
- •Noise due to matrixing
- •Transforms among RGB systems
- •Camera white reference
- •Display white reference
- •Gamut
- •Wide-gamut reproduction
- •Free Scale Gamut, Free Scale Log (FS-Gamut, FS-Log)
- •Further reading
- •27. Gamma
- •Gamma in CRT physics
- •The amazing coincidence!
- •Gamma in video
- •Opto-electronic conversion functions (OECFs)
- •BT.709 OECF
- •SMPTE 240M OECF
- •sRGB transfer function
- •Transfer functions in SD
- •Bit depth requirements
- •Gamma in modern display devices
- •Estimating gamma
- •Gamma in video, CGI, and Macintosh
- •Gamma in computer graphics
- •Gamma in pseudocolour
- •Limitations of 8-bit linear coding
- •Linear and nonlinear coding in CGI
- •Colour acuity
- •RGB and R’G’B’ colour cubes
- •Conventional luma/colour difference coding
- •Luminance and luma notation
- •Nonlinear red, green, blue (R’G’B’)
- •BT.601 luma
- •BT.709 luma
- •Chroma subsampling, revisited
- •Luma/colour difference summary
- •SD and HD luma chaos
- •Luma/colour difference component sets
- •B’-Y’, R’-Y’ components for SD
- •PBPR components for SD
- •CBCR components for SD
- •Y’CBCR from studio RGB
- •Y’CBCR from computer RGB
- •“Full-swing” Y’CBCR
- •Y’UV, Y’IQ confusion
- •B’-Y’, R’-Y’ components for BT.709 HD
- •PBPR components for BT.709 HD
- •CBCR components for BT.709 HD
- •CBCR components for xvYCC
- •Y’CBCR from studio RGB
- •Y’CBCR from computer RGB
- •Conversions between HD and SD
- •Colour coding standards
- •31. Video signal processing
- •Edge treatment
- •Transition samples
- •Picture lines
- •Choice of SAL and SPW parameters
- •Video levels
- •Setup (pedestal)
- •BT.601 to computing
- •Enhancement
- •Median filtering
- •Coring
- •Chroma transition improvement (CTI)
- •Mixing and keying
- •Field rate
- •Line rate
- •Sound subcarrier
- •Addition of composite colour
- •NTSC colour subcarrier
- •576i PAL colour subcarrier
- •4fSC sampling
- •Common sampling rate
- •Numerology of HD scanning
- •Audio rates
- •33. Timecode
- •Introduction
- •Dropframe timecode
- •Editing
- •Linear timecode (LTC)
- •Vertical interval timecode (VITC)
- •Timecode structure
- •Further reading
- •34. 2-3 pulldown
- •2-3-3-2 pulldown
- •Conversion of film to different frame rates
- •Native 24 Hz coding
- •Conversion to other rates
- •Spatial domain
- •Vertical-temporal domain
- •Motion adaptivity
- •Further reading
- •36. Colourbars
- •SD colourbars
- •SD colourbar notation
- •Pluge element
- •Composite decoder adjustment using colourbars
- •-I, +Q, and Pluge elements in SD colourbars
- •HD colourbars
- •References
- •38. SDI and HD-SDI interfaces
- •Component digital SD interface (BT.601)
- •Serial digital interface (SDI)
- •Component digital HD-SDI
- •SDI and HD-SDI sync, TRS, and ancillary data
- •Analog sync and digital/analog timing relationships
- •Ancillary data
- •SDI coding
- •HD-SDI coding
- •Interfaces for compressed video
- •SDTI
- •Switching and mixing
- •Timing in digital facilities
- •Summary of digital interfaces
- •39. 480i component video
- •Frame rate
- •Interlace
- •Line sync
- •Field/frame sync
- •R’G’B’ EOCF and primaries
- •Luma (Y’)
- •Picture center, aspect ratio, and blanking
- •Halfline blanking
- •Component digital 4:2:2 interface
- •Component analog R’G’B’ interface
- •Component analog Y’PBPR interface, EBU N10
- •Component analog Y’PBPR interface, industry standard
- •40. 576i component video
- •Frame rate
- •Interlace
- •Line sync
- •Analog field/frame sync
- •R’G’B’ EOCF and primaries
- •Luma (Y’)
- •Picture center, aspect ratio, and blanking
- •Component digital 4:2:2 interface
- •Component analog 576i interface
- •Scanning
- •Analog sync
- •Picture center, aspect ratio, and blanking
- •R’G’B’ EOCF and primaries
- •Luma (Y’)
- •Component digital 4:2:2 interface
- •Scanning
- •Analog sync
- •Picture center, aspect ratio, and blanking
- •R’G’B’ EOCF and primaries
- •Luma (Y’)
- •Component digital 4:2:2 interface
- •43. HD videotape
- •HDCAM (D-11)
- •DVCPRO HD (D-12)
- •HDCAM SR (D-16)
- •JPEG blocks and MCUs
- •JPEG block diagram
- •Level shifting
- •Discrete cosine transform (DCT)
- •JPEG encoding example
- •JPEG decoding
- •Compression ratio control
- •JPEG/JFIF
- •Motion-JPEG (M-JPEG)
- •Further reading
- •46. DV compression
- •DV chroma subsampling
- •DV frame/field modes
- •Picture-in-shuttle in DV
- •DV overflow scheme
- •DV quantization
- •DV digital interface (DIF)
- •Consumer DV recording
- •Professional DV variants
- •47. MPEG-2 video compression
- •MPEG-2 profiles and levels
- •Picture structure
- •Frame rate and 2-3 pulldown in MPEG
- •Luma and chroma sampling structures
- •Macroblocks
- •Picture coding types – I, P, B
- •Prediction
- •Motion vectors (MVs)
- •Coding of a block
- •Frame and field DCT types
- •Zigzag and VLE
- •Refresh
- •Motion estimation
- •Rate control and buffer management
- •Bitstream syntax
- •Transport
- •Further reading
- •48. H.264 video compression
- •Algorithmic features, profiles, and levels
- •Baseline and extended profiles
- •High profiles
- •Hierarchy
- •Multiple reference pictures
- •Slices
- •Spatial intra prediction
- •Flexible motion compensation
- •Quarter-pel motion-compensated interpolation
- •Weighting and offsetting of MC prediction
- •16-bit integer transform
- •Quantizer
- •Variable-length coding
- •Context adaptivity
- •CABAC
- •Deblocking filter
- •Buffer control
- •Scalable video coding (SVC)
- •Multiview video coding (MVC)
- •AVC-Intra
- •Further reading
- •49. VP8 compression
- •Algorithmic features
- •Further reading
- •Elementary stream (ES)
- •Packetized elementary stream (PES)
- •MPEG-2 program stream
- •MPEG-2 transport stream
- •System clock
- •Further reading
- •Japan
- •United States
- •ATSC modulation
- •Europe
- •Further reading
- •Appendices
- •Cement vs. concrete
- •True CIE luminance
- •The misinterpretation of luminance
- •The enshrining of luma
- •Colour difference scale factors
- •Conclusion: A plea
- •Radiometry
- •Photometry
- •Light level examples
- •Image science
- •Units
- •Further reading
- •Glossary
- •Index
- •About the author
Pratt, William K. (1991),
Digital Image Processing, Second Edition (New York: Wiley)., 64. The error is corrected in the third [2001] and fourth [2007] editions.
SMPTE EG 28, Annotated Glossary of Essential Terms for Electronic Production.
The error propagated into the digital imageprocessing community. Pratt’s textbook states:
N.T.S.C. formulated a color coordinate system for transmission composed of three tristimulus values YIQ. The Y tristimulus value is the luminance of a color.
The video quantities are certainly not tristimulus values, which are, by CIE’s definition, proportional to intensity.
Loose nomenclature on the part of video engineers has misled a generation of digital image processing, computer software, and computer hardware engineers.
The enshrining of luma
The term luma was used in video for a long time, without having had a precise interpretation.
I campaigned among video engineers, and among computer graphics experts, for adoption of the term luma to designate the nonlinear video quantity. The term offers no impediment to video engineers – in fact, it slides off the tongue more easily than luminance. By virtue of its being a different word from luminance, the word luma invites readers from other domains to investigate fully before drawing conclusions about its relationship with luminance.
With the help of Fred Kolb, my campaign succeeded: In 1993, SMPTE adopted Engineering Guideline EG 28,
Annotated Glossary of Essential Terms for Electronic Production. EG 28 defines the term luma, and clarifies the two conflicting interpretations of the term luminance. While a SMPTE EG is not quite a SMPTE “Standard,” at long last the term has received official recognition. There’s no longer any excuse for sloppy use of the term luminance by the authors of video engineering papers and books. Had the term luma been widespread 20 years ago when A.R. Smith was writing about YIQ, or when Foley and van Dam were preparing
Computer Graphics: Principles and Practice, this whole mess would have been avoided. But EG 28 was unavailable at the time.
It is a shame that today’s SMPTE, ISO/IEC, ITU-R, and ITU-T standards persist in using the incorrect word luminance, without ever mentioning the ambiguity – even conflict – with the CIE standards of colour science.
570 |
DIGITAL VIDEO AND HD ALGORITHMS AND INTERFACES |
When I say NTSC and PAL, I refer to colour encoding, not scanning. I do not mean 480i and 576i, or 525/59.94 and 625/50!
ITU-R Rec. BT.601-5, Studio encoding parameters of digital television for standard 4:3 and widescreen 16:9 aspect ratios.
Chroma components are properly ordered B’-Y’ then R’-Y’, or CB then CR. Blue associates with U, and red with V; U and V are in alphabetic order.
Colour difference scale factors
To represent colour, luma is accompanied by two colour difference – or chroma – components, universally based on blue minus luma and red minus luma, where blue, red, and luma have all been subject to gamma correction: B’-Y’ and R’-Y’. Different scale factors are applied to the basic B’-Y’ and R’-Y’ components for different applications. Y’PBPR scale factors are optimized for component analog video. Y’CBCR scale factors are optimized for component digital video, such as 4:2:2 studio video, JPEG, and MPEG. Correct use of the Y’UV and Y’IQ scale factors is limited to the formation of composite NTSC and PAL video.
Y’CBCR scaling as defined by BT.601 is appropriate for component digital video. Y’CBCR chroma is almost always subsampled using one of three schemes: 4:2:2, or 4:2:0, or 4:1:1.
Y’UV scaling is properly used only as an intermediate step in the formation of composite NTSC or PAL video signals. Y’UV scaling is not appropriate when the components are kept separate. However, the Y’UV nomenclature is now used rather loosely, and sometimes – particularly in computing – it denotes any scaling of B’-Y’ and R’-Y’.
In about 1991, digital disk recorders (DDRs) were introduced that were able to transfer files across Ethernet using IP protocols. Abekas introduced the extension yuv for these files. But the scale factors typically used (then and now) actually correspond to Y’CBCR . Use of the yuv extension reinforced the misleading YUV nomenclature.
Subsampling is a digital technique, properly performed only on component digital video – that is, on Y’CBCR . Subsampling is inappropriate for Y’UV in all but very specialized applications (namely, digital encoding of 4fSC NTSC or PAL composite video). If you see
a system described as Y’UV 4:2:2, you have a dilemma. Perhaps the person who wrote the description is unfamiliar with the principles of component video, and the scale factors actually implemented in the equipment (or the software) are correct. But you must allow for the possibility that the engineers who designed or implemented the system used the wrong scale factors! If the
APPENDIX A |
YUV AND LUMINANCE CONSIDERED HARMFUL |
571 |
Hamilton, Eric (1992), JPEG File Interchange Format, Version 1.02 (Milpitas, Calif.: C-Cube Microsystems).
wrong equations were used, then colour accuracy will suffer; however, this can be difficult to diagnose.
Proper Y’CBCR scaling is usual in Motion-JPEG, and in MPEG. However, the Y’CBCR scaling used in stillframe JPEG/JFIF in computer applications usually uses full-range luma and chroma excursions, without any headroom or footroom. The chroma excursion is 256⁄255 of the luma excursion. The scaling is almost exactly that of Y’PBPR, but is unfortunately described as Y’CBCR : Now even Y’CBCR is ambiguous! It is far too late for proper Y’CBCR scaling to be incorporated into JFIF; compressed stillframe and motion imagery in computing is bound to suffer a conversion process.
Y’IQ coding has been obsolete in studio practice for at least three decades. It should now be banished in favour of Y’CBCR.
Conclusion: A plea
Using the term luminance for video Y’ is tantamount to using the word cement instead of concrete to describe the primary construction material of a sidewalk. Lay people don’t care, and experts can live with it, but people in the middle – in this case, the programmers and engineers who are reimplementing video technology in the computer domain – are liable to draw the wrong conclusions from careless use of terms, and thereby make inaccurate colour. The accurate exchange of images is compromised, and users suffer.
I urge video engineers and computer graphics specialists to avoid the terms YUV, Y’UV, YIQ, Y’IQ, and luminance, except in the highly specialized situations where those terms are technically correct. The appropriate terms are almost always Y’CBCR and luma.
572 |
DIGITAL VIDEO AND HD ALGORITHMS AND INTERFACES |
Introduction to radiometry
and photometry |
B |
ANSI/IESNA RP-16, Nomenclature and Definitions for Illuminating Engineering.
CIE No 17.4 (E-1.1) (1987), International Lighting Vocabulary, 4th Edition (Vienna, Austria: Commission Internationale de L’Éclairage).
The domain of radiometry involves optical power and its spatial and angular distributions. Photometry is, in essence, radiometry weighted by the spectral response of vision. These fields involve several subtle concepts, masked by a bewildering array of symbols and units.
I strive to sort out some of the confusion, and make some suggestions concerning units and nomenclature. Table B.1 below summarizes radiometric quantities, symbols, and units (in the left columns) and the corresponding photometric quantities, symbols, and units (to the right). The symbol for a photometric quantity is just the symbol for the corresponding radiometric quantity, with the addition of the subscript v (for visual). Some people add the subscript e to the radiometric symbols.
Differentiate |
Radiometric (Radiant) |
|
|
Photometric (Luminous) |
|
flux with |
|
|
|
|
|
Quantity (Symbol) |
Unit |
|
Quantity (Symbol) |
Unit |
|
respect to |
|
||||
|
|
|
|
|
|
– |
radiant flux, |
watt [W] |
|
luminous flux |
lumen [lm] |
|
power (Φ , F, P) |
|
|
(Φv, Fv, Pv) |
|
area |
irradiance (E), |
W·m–2 |
|
illuminance (Ev), |
lm·m–2 |
|
radiant exitance |
|
|
luminous exitance |
= lux [lx] |
|
(M) |
|
|
(Mv) |
|
solid angle |
radiant intensity (I) |
W·sr –1 |
|
luminous intensity |
lm·sr –1 |
|
|
|
|
(Iv) |
= candela [cd] |
area |
radiance (L) |
W·sr –1·m–2 |
|
luminance (Lv) |
cd·m–2 |
and solid angle |
|
|
|
|
= nit [nt] |
|
|
|
|
|
|
Table B.1 Quantities, symbols, and units of radiometry and photometry. The symbol Lv (or L) is used for absolute luminance (e.g., in photometry). The symbol Y is used for relative luminance (e.g., in colour science and in video).
573
