- •Contents
- •Figures
- •Tables
- •Preface
- •Acknowledgments
- •1. Raster images
- •Aspect ratio
- •Geometry
- •Image capture
- •Digitization
- •Perceptual uniformity
- •Colour
- •Luma and colour difference components
- •Digital image representation
- •Square sampling
- •Comparison of aspect ratios
- •Aspect ratio
- •Frame rates
- •Image state
- •EOCF standards
- •Entertainment programming
- •Acquisition
- •Consumer origination
- •Consumer electronics (CE) display
- •Contrast
- •Contrast ratio
- •Perceptual uniformity
- •The “code 100” problem and nonlinear image coding
- •Linear and nonlinear
- •4. Quantization
- •Linearity
- •Decibels
- •Noise, signal, sensitivity
- •Quantization error
- •Full-swing
- •Studio-swing (footroom and headroom)
- •Interface offset
- •Processing coding
- •Two’s complement wrap-around
- •Perceptual attributes
- •History of display signal processing
- •Digital driving levels
- •Relationship between signal and lightness
- •Algorithm
- •Black level setting
- •Effect of contrast and brightness on contrast and brightness
- •An alternate interpretation
- •Brightness and contrast controls in LCDs
- •Brightness and contrast controls in PDPs
- •Brightness and contrast controls in desktop graphics
- •Symbolic image description
- •Raster images
- •Conversion among types
- •Image files
- •“Resolution” in computer graphics
- •7. Image structure
- •Image reconstruction
- •Sampling aperture
- •Spot profile
- •Box distribution
- •Gaussian distribution
- •8. Raster scanning
- •Flicker, refresh rate, and frame rate
- •Introduction to scanning
- •Scanning parameters
- •Interlaced format
- •Interlace and progressive
- •Scanning notation
- •Motion portrayal
- •Segmented-frame (24PsF)
- •Video system taxonomy
- •Conversion among systems
- •9. Resolution
- •Magnitude frequency response and bandwidth
- •Visual acuity
- •Viewing distance and angle
- •Kell effect
- •Resolution
- •Resolution in video
- •Viewing distance
- •Interlace revisited
- •10. Constant luminance
- •The principle of constant luminance
- •Compensating for the CRT
- •Departure from constant luminance
- •Luma
- •“Leakage” of luminance into chroma
- •11. Picture rendering
- •Surround effect
- •Tone scale alteration
- •Incorporation of rendering
- •Rendering in desktop computing
- •Luma
- •Sloppy use of the term luminance
- •Colour difference coding (chroma)
- •Chroma subsampling
- •Chroma subsampling notation
- •Chroma subsampling filters
- •Chroma in composite NTSC and PAL
- •Scanning standards
- •Widescreen (16:9) SD
- •Square and nonsquare sampling
- •Resampling
- •NTSC and PAL encoding
- •NTSC and PAL decoding
- •S-video interface
- •Frequency interleaving
- •Composite analog SD
- •15. Introduction to HD
- •HD scanning
- •Colour coding for BT.709 HD
- •Data compression
- •Image compression
- •Lossy compression
- •JPEG
- •Motion-JPEG
- •JPEG 2000
- •Mezzanine compression
- •MPEG
- •Picture coding types (I, P, B)
- •Reordering
- •MPEG-1
- •MPEG-2
- •Other MPEGs
- •MPEG IMX
- •MPEG-4
- •AVC-Intra
- •WM9, WM10, VC-1 codecs
- •Compression for CE acquisition
- •AVCHD
- •Compression for IP transport to consumers
- •VP8 (“WebM”) codec
- •Dirac (basic)
- •17. Streams and files
- •Historical overview
- •Physical layer
- •Stream interfaces
- •IEEE 1394 (FireWire, i.LINK)
- •HTTP live streaming (HLS)
- •18. Metadata
- •Metadata Example 1: CD-DA
- •Metadata Example 2: .yuv files
- •Metadata Example 3: RFF
- •Metadata Example 4: JPEG/JFIF
- •Metadata Example 5: Sequence display extension
- •Conclusions
- •19. Stereoscopic (“3-D”) video
- •Acquisition
- •S3D display
- •Anaglyph
- •Temporal multiplexing
- •Polarization
- •Wavelength multiplexing (Infitec/Dolby)
- •Autostereoscopic displays
- •Parallax barrier display
- •Lenticular display
- •Recording and compression
- •Consumer interface and display
- •Ghosting
- •Vergence and accommodation
- •20. Filtering and sampling
- •Sampling theorem
- •Sampling at exactly 0.5fS
- •Magnitude frequency response
- •Magnitude frequency response of a boxcar
- •The sinc weighting function
- •Frequency response of point sampling
- •Fourier transform pairs
- •Analog filters
- •Digital filters
- •Impulse response
- •Finite impulse response (FIR) filters
- •Physical realizability of a filter
- •Phase response (group delay)
- •Infinite impulse response (IIR) filters
- •Lowpass filter
- •Digital filter design
- •Reconstruction
- •Reconstruction close to 0.5fS
- •“(sin x)/x” correction
- •Further reading
- •2:1 downsampling
- •Oversampling
- •Interpolation
- •Lagrange interpolation
- •Lagrange interpolation as filtering
- •Polyphase interpolators
- •Polyphase taps and phases
- •Implementing polyphase interpolators
- •Decimation
- •Lowpass filtering in decimation
- •Spatial frequency domain
- •Comb filtering
- •Spatial filtering
- •Image presampling filters
- •Image reconstruction filters
- •Spatial (2-D) oversampling
- •Retina
- •Adaptation
- •Contrast sensitivity
- •Contrast sensitivity function (CSF)
- •24. Luminance and lightness
- •Radiance, intensity
- •Luminance
- •Relative luminance
- •Luminance from red, green, and blue
- •Lightness (CIE L*)
- •Fundamentals of vision
- •Definitions
- •Spectral power distribution (SPD) and tristimulus
- •Spectral constraints
- •CIE XYZ tristimulus
- •CIE [x, y] chromaticity
- •Blackbody radiation
- •Colour temperature
- •White
- •Chromatic adaptation
- •Perceptually uniform colour spaces
- •CIE L*a*b* (CIELAB)
- •CIE L*u*v* and CIE L*a*b* summary
- •Colour specification and colour image coding
- •Further reading
- •Additive reproduction (RGB)
- •Characterization of RGB primaries
- •BT.709 primaries
- •Leggacy SD primaries
- •sRGB system
- •SMPTE Free Scale (FS) primaries
- •AMPAS ACES primaries
- •SMPTE/DCI P3 primaries
- •CMFs and SPDs
- •Normalization and scaling
- •Luminance coefficients
- •Transformations between RGB and CIE XYZ
- •Noise due to matrixing
- •Transforms among RGB systems
- •Camera white reference
- •Display white reference
- •Gamut
- •Wide-gamut reproduction
- •Free Scale Gamut, Free Scale Log (FS-Gamut, FS-Log)
- •Further reading
- •27. Gamma
- •Gamma in CRT physics
- •The amazing coincidence!
- •Gamma in video
- •Opto-electronic conversion functions (OECFs)
- •BT.709 OECF
- •SMPTE 240M OECF
- •sRGB transfer function
- •Transfer functions in SD
- •Bit depth requirements
- •Gamma in modern display devices
- •Estimating gamma
- •Gamma in video, CGI, and Macintosh
- •Gamma in computer graphics
- •Gamma in pseudocolour
- •Limitations of 8-bit linear coding
- •Linear and nonlinear coding in CGI
- •Colour acuity
- •RGB and R’G’B’ colour cubes
- •Conventional luma/colour difference coding
- •Luminance and luma notation
- •Nonlinear red, green, blue (R’G’B’)
- •BT.601 luma
- •BT.709 luma
- •Chroma subsampling, revisited
- •Luma/colour difference summary
- •SD and HD luma chaos
- •Luma/colour difference component sets
- •B’-Y’, R’-Y’ components for SD
- •PBPR components for SD
- •CBCR components for SD
- •Y’CBCR from studio RGB
- •Y’CBCR from computer RGB
- •“Full-swing” Y’CBCR
- •Y’UV, Y’IQ confusion
- •B’-Y’, R’-Y’ components for BT.709 HD
- •PBPR components for BT.709 HD
- •CBCR components for BT.709 HD
- •CBCR components for xvYCC
- •Y’CBCR from studio RGB
- •Y’CBCR from computer RGB
- •Conversions between HD and SD
- •Colour coding standards
- •31. Video signal processing
- •Edge treatment
- •Transition samples
- •Picture lines
- •Choice of SAL and SPW parameters
- •Video levels
- •Setup (pedestal)
- •BT.601 to computing
- •Enhancement
- •Median filtering
- •Coring
- •Chroma transition improvement (CTI)
- •Mixing and keying
- •Field rate
- •Line rate
- •Sound subcarrier
- •Addition of composite colour
- •NTSC colour subcarrier
- •576i PAL colour subcarrier
- •4fSC sampling
- •Common sampling rate
- •Numerology of HD scanning
- •Audio rates
- •33. Timecode
- •Introduction
- •Dropframe timecode
- •Editing
- •Linear timecode (LTC)
- •Vertical interval timecode (VITC)
- •Timecode structure
- •Further reading
- •34. 2-3 pulldown
- •2-3-3-2 pulldown
- •Conversion of film to different frame rates
- •Native 24 Hz coding
- •Conversion to other rates
- •Spatial domain
- •Vertical-temporal domain
- •Motion adaptivity
- •Further reading
- •36. Colourbars
- •SD colourbars
- •SD colourbar notation
- •Pluge element
- •Composite decoder adjustment using colourbars
- •-I, +Q, and Pluge elements in SD colourbars
- •HD colourbars
- •References
- •38. SDI and HD-SDI interfaces
- •Component digital SD interface (BT.601)
- •Serial digital interface (SDI)
- •Component digital HD-SDI
- •SDI and HD-SDI sync, TRS, and ancillary data
- •Analog sync and digital/analog timing relationships
- •Ancillary data
- •SDI coding
- •HD-SDI coding
- •Interfaces for compressed video
- •SDTI
- •Switching and mixing
- •Timing in digital facilities
- •Summary of digital interfaces
- •39. 480i component video
- •Frame rate
- •Interlace
- •Line sync
- •Field/frame sync
- •R’G’B’ EOCF and primaries
- •Luma (Y’)
- •Picture center, aspect ratio, and blanking
- •Halfline blanking
- •Component digital 4:2:2 interface
- •Component analog R’G’B’ interface
- •Component analog Y’PBPR interface, EBU N10
- •Component analog Y’PBPR interface, industry standard
- •40. 576i component video
- •Frame rate
- •Interlace
- •Line sync
- •Analog field/frame sync
- •R’G’B’ EOCF and primaries
- •Luma (Y’)
- •Picture center, aspect ratio, and blanking
- •Component digital 4:2:2 interface
- •Component analog 576i interface
- •Scanning
- •Analog sync
- •Picture center, aspect ratio, and blanking
- •R’G’B’ EOCF and primaries
- •Luma (Y’)
- •Component digital 4:2:2 interface
- •Scanning
- •Analog sync
- •Picture center, aspect ratio, and blanking
- •R’G’B’ EOCF and primaries
- •Luma (Y’)
- •Component digital 4:2:2 interface
- •43. HD videotape
- •HDCAM (D-11)
- •DVCPRO HD (D-12)
- •HDCAM SR (D-16)
- •JPEG blocks and MCUs
- •JPEG block diagram
- •Level shifting
- •Discrete cosine transform (DCT)
- •JPEG encoding example
- •JPEG decoding
- •Compression ratio control
- •JPEG/JFIF
- •Motion-JPEG (M-JPEG)
- •Further reading
- •46. DV compression
- •DV chroma subsampling
- •DV frame/field modes
- •Picture-in-shuttle in DV
- •DV overflow scheme
- •DV quantization
- •DV digital interface (DIF)
- •Consumer DV recording
- •Professional DV variants
- •47. MPEG-2 video compression
- •MPEG-2 profiles and levels
- •Picture structure
- •Frame rate and 2-3 pulldown in MPEG
- •Luma and chroma sampling structures
- •Macroblocks
- •Picture coding types – I, P, B
- •Prediction
- •Motion vectors (MVs)
- •Coding of a block
- •Frame and field DCT types
- •Zigzag and VLE
- •Refresh
- •Motion estimation
- •Rate control and buffer management
- •Bitstream syntax
- •Transport
- •Further reading
- •48. H.264 video compression
- •Algorithmic features, profiles, and levels
- •Baseline and extended profiles
- •High profiles
- •Hierarchy
- •Multiple reference pictures
- •Slices
- •Spatial intra prediction
- •Flexible motion compensation
- •Quarter-pel motion-compensated interpolation
- •Weighting and offsetting of MC prediction
- •16-bit integer transform
- •Quantizer
- •Variable-length coding
- •Context adaptivity
- •CABAC
- •Deblocking filter
- •Buffer control
- •Scalable video coding (SVC)
- •Multiview video coding (MVC)
- •AVC-Intra
- •Further reading
- •49. VP8 compression
- •Algorithmic features
- •Further reading
- •Elementary stream (ES)
- •Packetized elementary stream (PES)
- •MPEG-2 program stream
- •MPEG-2 transport stream
- •System clock
- •Further reading
- •Japan
- •United States
- •ATSC modulation
- •Europe
- •Further reading
- •Appendices
- •Cement vs. concrete
- •True CIE luminance
- •The misinterpretation of luminance
- •The enshrining of luma
- •Colour difference scale factors
- •Conclusion: A plea
- •Radiometry
- •Photometry
- •Light level examples
- •Image science
- •Units
- •Further reading
- •Glossary
- •Index
- •About the author
Eq 30.6
Eq 30.7
xvYCC refers to an IEC standard. x.v.Color and x.v.Colour are Sony’s trademarks for the scheme.
Concerning Pointer, see the marginal note on page 312.
[219, 224, 224], corresponding to the excursions of each of the components:
|
709Y’ |
16 |
|
|
46.559 |
156.629 |
|
|
15.812 |
R’ |
||||||
219 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
CB |
|
= 128 |
|
+ |
|
25.664 |
|
|
86.336 |
112 |
|
· G’ |
|||
|
|
|||||||||||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
CR |
|
128 |
|
|
112 |
101.730 |
|
|
10.270 |
B’ |
|||||
Summing the first row of the matrix yields 219, the luma excursion from black to white. The two entries of 112 reflect the positive CBCR extrema at blue and red.
To recover R’G’B’ in the range [0…+1] from 709Y’CBCR, use the inverse of Equation 30.6:
219R’ |
|
|
256 |
0 |
394150. |
|
|
219709Y’ |
|
16 |
||||||
|
|
|
|
|
||||||||||||
G’ |
|
1 |
|
256 |
|
46.885 |
|
117165. |
|
|
C |
B |
|
|
|
128 |
219 |
|
256 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
219B’ |
|
|
256 |
464430. |
0 |
|
|
CR |
|
|
|
128 |
||||
|
|
|
|
|
||||||||||||
The 709Y’CBCR components are integers in 8 bits; reconstructed R’G’B’ is scaled to the range [0…+1].
Figure 28.2 (on page 339) illustrated that when R’G’B’ components are transformed to luma and colour differences, the unit R’G’B’ cube occupies only a small fraction of the volume of the enclosing cube. In digital video, only about 1⁄4 of Y’CBCR codewords correspond to R’G’B’ values between zero and unity. Certain signalprocessing operations (such as filtering) may produce Y’CBCR codewords that lie outside the RGB-legal cube. These codewords cause no difficulty in the Y’CBCR domain, but potentially present a problem when decoded to R’G’B’. Generally, R’G’B’ values are clipped between 0 and 1.
CBCR components for xvYCC
One method of extending the colour gamut of an R’G’B’ system is to allow components to excurse below zero and above unity. In Wide-gamut reproduction, on page 312, I explained one approach. The xvYCC scheme is based upon BT.709 primaries, but enables the RGB tristimulus components to excurse from -1⁄4 to +4⁄3 .
When transformed to BT.709 Y’CBCR, all of the real surface colours documented by Pointer – that is, all the colours in Pointer’s gamut – produce values that are Y’CBCR-valid. Though BT.1361 was needed to specify the R’G’B’ representation of wide-gamut colours, no
CHAPTER 30 |
COMPONENT VIDEO COLOUR CODING FOR HD |
373 |
There is an SD version of xvYCC, using the BT.601 luma coefficients. That scheme will almost certainly never see any deployment.
Eq 30.8
Eq 30.9
special provisions are necessary to carry those colours across a 709Y’CBCRinterface. The notation “xvYCC Y’CBCR,” or xvYCCY’CBCR, makes it explicit that codewords outside the unit R’G’B’ cube are to be interpreted as wide-gamut colours, instead of being treated as RGB-illegal.
Studio equipment conforming to BT.1361 is not yet deployed, and is not anticipated for several years. Wide-gamut acquisition and production equipment will begin to replace film over the next decade or so; however, wide-gamut consumer displays are not expected in that time frame. When these begin to be deployed, it is unlikely that they will all have the same gamut; electronics associated with each display will have to process the colour signals according to the properties of each display. In the long term, gamut mapping strategies comparable to those in the desktop colour management community will have to be deployed.
Y’CBCR from studio RGB
In studio equipment, 8-bit R’G’B’ components usually use the same 219 excursion as the luma component of Y’CBCR. To encode Y’CBCR from R’G’B’ in the range [0…219] using 8-bit binary arithmetic, scale the encoding matrix of Equation 30.6 by 256⁄219:
|
709Y’ |
16 |
|
|
|
|
54.426 |
183.091 |
|
|
18.483 |
|
219 |
R’ |
||||
219 |
|
|
|
+ |
1 |
|
|
|
|
|
|
|
|
|
|
|||
|
C |
B |
|
= 128 |
|
|
|
|
30.000 |
|
100.922 |
130.922 |
· |
219G’ |
||||
256 |
|
|
||||||||||||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||
|
C |
|
|
|
|
130.922 |
118.918 |
|
|
|||||||||
|
|
R |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
To decode to R’G’B’ in the range [0…219] from BT.709 Y’CBCR using 8-bit binary arithmetic:
219R’ |
|
|
256 |
0 |
394150. |
|
|
219709Y’ |
|
16 |
||||||
|
|
|
|
|
||||||||||||
G’ |
|
1 |
|
256 |
|
46.885 |
|
117165. |
|
|
C |
B |
|
|
|
128 |
219 |
|
256 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
219B’ |
|
|
256 |
464430. |
0 |
|
|
CR |
|
|
|
128 |
||||
|
|
|
|
|
||||||||||||
Y’CBCR from computer RGB
In computing it is conventional to use 8-bit R’G’B’ components, with no headroom or footroom: Black is at code 0 and white is at 255. To encode Y’CBCR from
374 |
DIGITAL VIDEO AND HD ALGORITHMS AND INTERFACES |
Eq 30.10
Eq 30.11
R’G’B’ in the range [0…255] using 8-bit binary arithmetic, the matrix of Equation 30.6 is scaled by 256⁄255:
709Y’ |
16 |
|
|
|
|
46.742 |
157.243 |
|
|
15.874 |
|
255 |
R’ |
||||||
219 |
|
|
|
+ |
1 |
|
|
|
|
|
|
|
|
|
|
|
|||
|
C |
B |
|
= 128 |
|
|
|
|
25.765 |
|
|
86.674 |
112.439 |
· |
255G’ |
||||
256 |
|
|
|||||||||||||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||
|
C |
|
|
|
|
112.439 |
102.129 |
|
|
||||||||||
|
|
R |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
To decode R’G’B’ in the range [0…255] from BT.709 Y’CBCR using 8-bit binary arithmetic:
|
255 |
R’ |
|
|
298082. |
0 |
|
458942. |
|
709Y’ |
|
|
16 |
||||||
|
|
= |
1 |
|
|
|
|
|
|
|
|
219 |
|
|
|
|
|||
|
255G’ |
|
298082. |
|
|
54.592 |
|
136425. |
· |
C |
B |
|
|
128 |
|||||
256 |
|
|
|
||||||||||||||||
|
|
|
|
|
540775. |
0 |
|
|
C |
|
|
|
|
||||||
|
255B’ |
|
|
298082. |
|
|
R |
|
|
128 |
|
||||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||
Conversions between HD and SD
The differences among the EBU, SMPTE, and BT.709 primaries are negligible for practical purposes. New equipment should be designed to BT.709. Also, SD and HD have effectively converged to the transfer function specified in BT.709. Consequently, R’G’B’ coding uses essentially identical parameters worldwide, for SD and HD. (The sRGB standard for desktop computing uses the primaries of BT.709, but uses a different transfer function.)
Unfortunately, as I have mentioned, the luma coefficients differ dramatically between SD and HD. This wouldn’t matter if HD systems were isolated! However, in practice, SD is upconverted and HD is downconverted, both at the studio and at consumers’ premises. Serious colour reproduction errors arise if differences among luma coefficients are not taken into account in conversions.
In principle, downconversion can be accomplished by decoding 709Y’CBCR to R’G’B’ using a suitable 3× 3 matrix (such as that in Equation 30.7, on page 373), then encoding R’G’B’ to 601Y’CBCR using another 3× 3 matrix (such as that in Equation 29.6, on page 364).
CHAPTER 30 |
COMPONENT VIDEO COLOUR CODING FOR HD |
375 |
The two 3× 3 matrices can be combined so that the conversion can take place in one step:
|
601Y' |
1 |
0.099312 |
0.191700 |
|
709Y' |
|||
Eq 30.12 |
219 |
|
|
0.989854 |
|
|
219 |
|
|
|
C |
|
= 0 |
−0.110653 |
• |
C |
|
||
|
|
B |
|
|
−0.072453 |
|
|
B |
|
|
|
CR |
|
0 |
0.983398 |
|
CR |
|
|
|
|
|
|
|
|
|
|
|
|
Equations 30.12 and 30.13 are written without interface offsets of +16 for luma and +128 for CB and CB: If the offsets are present, remove them, transform, then reapply them.
Eq 30.13
In the first row of the matrix, the coefficient 0.099312 adds about one-tenth of BT.709’s CB into BT.601’s luma. This is a consequence of BT.709’s blue luma coefficient being just 0.0722, compared to 0.114 for BT.601. The coefficient 0.1917 adds about one-fifth of BT.709’s CR into BT.601’s luma; this is a consequence of BT.709’s red luma coefficient being 0.2126, compared to 0.299 for BT.601. Clearly, failure to perform this colour transform produces large colour errors.
To convert from SD to HD, the matrix of Equation 30.12 is inverted:
709Y' |
1 |
−0.115550 |
−0.207938 |
601Y' |
||||
|
219 |
|
|
|
|
219 |
|
|
|
C |
|
= 0 |
1.018640 |
0.114618 |
• |
C |
|
|
B |
|
|
0.075049 |
|
|
B |
|
|
CR |
|
0 |
1.025327 |
|
CR |
|
|
|
|
|
|
|
|
|
|
|
Unfortunately, to upconvert or downconvert a subsampled representation such as 4:2:2 or 4:2:0 requires chroma interpolation, colour transformation, then chroma subsampling. This is computationally intensive.
Colour coding standards
ITU-R Rec. BT.709 defines Y’PBPR for component analog HD and Y’CBCR for component digital HD. The parameters of Y’PBPR and Y’CBCR for the 1280× 720 and
1920× 1080 systems are defined by the SMPTE standards cited below.
ITU-R Rec. BT.709,Basic parameter values for the HDTV standard for the studio and for international programme exchange.
SMPTE ST 274, 1920× 1080 Scanning and Analog and Parallel Digital Interfaces for Multiple Picture Rates.
SMPTE ST 296, 1280× 720 Progressive Image Sample Structure – Analog and Digital Representation and Analog Interface.
376 |
DIGITAL VIDEO AND HD ALGORITHMS AND INTERFACES |
