- •Contents
- •Figures
- •Tables
- •Preface
- •Acknowledgments
- •1. Raster images
- •Aspect ratio
- •Geometry
- •Image capture
- •Digitization
- •Perceptual uniformity
- •Colour
- •Luma and colour difference components
- •Digital image representation
- •Square sampling
- •Comparison of aspect ratios
- •Aspect ratio
- •Frame rates
- •Image state
- •EOCF standards
- •Entertainment programming
- •Acquisition
- •Consumer origination
- •Consumer electronics (CE) display
- •Contrast
- •Contrast ratio
- •Perceptual uniformity
- •The “code 100” problem and nonlinear image coding
- •Linear and nonlinear
- •4. Quantization
- •Linearity
- •Decibels
- •Noise, signal, sensitivity
- •Quantization error
- •Full-swing
- •Studio-swing (footroom and headroom)
- •Interface offset
- •Processing coding
- •Two’s complement wrap-around
- •Perceptual attributes
- •History of display signal processing
- •Digital driving levels
- •Relationship between signal and lightness
- •Algorithm
- •Black level setting
- •Effect of contrast and brightness on contrast and brightness
- •An alternate interpretation
- •Brightness and contrast controls in LCDs
- •Brightness and contrast controls in PDPs
- •Brightness and contrast controls in desktop graphics
- •Symbolic image description
- •Raster images
- •Conversion among types
- •Image files
- •“Resolution” in computer graphics
- •7. Image structure
- •Image reconstruction
- •Sampling aperture
- •Spot profile
- •Box distribution
- •Gaussian distribution
- •8. Raster scanning
- •Flicker, refresh rate, and frame rate
- •Introduction to scanning
- •Scanning parameters
- •Interlaced format
- •Interlace and progressive
- •Scanning notation
- •Motion portrayal
- •Segmented-frame (24PsF)
- •Video system taxonomy
- •Conversion among systems
- •9. Resolution
- •Magnitude frequency response and bandwidth
- •Visual acuity
- •Viewing distance and angle
- •Kell effect
- •Resolution
- •Resolution in video
- •Viewing distance
- •Interlace revisited
- •10. Constant luminance
- •The principle of constant luminance
- •Compensating for the CRT
- •Departure from constant luminance
- •Luma
- •“Leakage” of luminance into chroma
- •11. Picture rendering
- •Surround effect
- •Tone scale alteration
- •Incorporation of rendering
- •Rendering in desktop computing
- •Luma
- •Sloppy use of the term luminance
- •Colour difference coding (chroma)
- •Chroma subsampling
- •Chroma subsampling notation
- •Chroma subsampling filters
- •Chroma in composite NTSC and PAL
- •Scanning standards
- •Widescreen (16:9) SD
- •Square and nonsquare sampling
- •Resampling
- •NTSC and PAL encoding
- •NTSC and PAL decoding
- •S-video interface
- •Frequency interleaving
- •Composite analog SD
- •15. Introduction to HD
- •HD scanning
- •Colour coding for BT.709 HD
- •Data compression
- •Image compression
- •Lossy compression
- •JPEG
- •Motion-JPEG
- •JPEG 2000
- •Mezzanine compression
- •MPEG
- •Picture coding types (I, P, B)
- •Reordering
- •MPEG-1
- •MPEG-2
- •Other MPEGs
- •MPEG IMX
- •MPEG-4
- •AVC-Intra
- •WM9, WM10, VC-1 codecs
- •Compression for CE acquisition
- •AVCHD
- •Compression for IP transport to consumers
- •VP8 (“WebM”) codec
- •Dirac (basic)
- •17. Streams and files
- •Historical overview
- •Physical layer
- •Stream interfaces
- •IEEE 1394 (FireWire, i.LINK)
- •HTTP live streaming (HLS)
- •18. Metadata
- •Metadata Example 1: CD-DA
- •Metadata Example 2: .yuv files
- •Metadata Example 3: RFF
- •Metadata Example 4: JPEG/JFIF
- •Metadata Example 5: Sequence display extension
- •Conclusions
- •19. Stereoscopic (“3-D”) video
- •Acquisition
- •S3D display
- •Anaglyph
- •Temporal multiplexing
- •Polarization
- •Wavelength multiplexing (Infitec/Dolby)
- •Autostereoscopic displays
- •Parallax barrier display
- •Lenticular display
- •Recording and compression
- •Consumer interface and display
- •Ghosting
- •Vergence and accommodation
- •20. Filtering and sampling
- •Sampling theorem
- •Sampling at exactly 0.5fS
- •Magnitude frequency response
- •Magnitude frequency response of a boxcar
- •The sinc weighting function
- •Frequency response of point sampling
- •Fourier transform pairs
- •Analog filters
- •Digital filters
- •Impulse response
- •Finite impulse response (FIR) filters
- •Physical realizability of a filter
- •Phase response (group delay)
- •Infinite impulse response (IIR) filters
- •Lowpass filter
- •Digital filter design
- •Reconstruction
- •Reconstruction close to 0.5fS
- •“(sin x)/x” correction
- •Further reading
- •2:1 downsampling
- •Oversampling
- •Interpolation
- •Lagrange interpolation
- •Lagrange interpolation as filtering
- •Polyphase interpolators
- •Polyphase taps and phases
- •Implementing polyphase interpolators
- •Decimation
- •Lowpass filtering in decimation
- •Spatial frequency domain
- •Comb filtering
- •Spatial filtering
- •Image presampling filters
- •Image reconstruction filters
- •Spatial (2-D) oversampling
- •Retina
- •Adaptation
- •Contrast sensitivity
- •Contrast sensitivity function (CSF)
- •24. Luminance and lightness
- •Radiance, intensity
- •Luminance
- •Relative luminance
- •Luminance from red, green, and blue
- •Lightness (CIE L*)
- •Fundamentals of vision
- •Definitions
- •Spectral power distribution (SPD) and tristimulus
- •Spectral constraints
- •CIE XYZ tristimulus
- •CIE [x, y] chromaticity
- •Blackbody radiation
- •Colour temperature
- •White
- •Chromatic adaptation
- •Perceptually uniform colour spaces
- •CIE L*a*b* (CIELAB)
- •CIE L*u*v* and CIE L*a*b* summary
- •Colour specification and colour image coding
- •Further reading
- •Additive reproduction (RGB)
- •Characterization of RGB primaries
- •BT.709 primaries
- •Leggacy SD primaries
- •sRGB system
- •SMPTE Free Scale (FS) primaries
- •AMPAS ACES primaries
- •SMPTE/DCI P3 primaries
- •CMFs and SPDs
- •Normalization and scaling
- •Luminance coefficients
- •Transformations between RGB and CIE XYZ
- •Noise due to matrixing
- •Transforms among RGB systems
- •Camera white reference
- •Display white reference
- •Gamut
- •Wide-gamut reproduction
- •Free Scale Gamut, Free Scale Log (FS-Gamut, FS-Log)
- •Further reading
- •27. Gamma
- •Gamma in CRT physics
- •The amazing coincidence!
- •Gamma in video
- •Opto-electronic conversion functions (OECFs)
- •BT.709 OECF
- •SMPTE 240M OECF
- •sRGB transfer function
- •Transfer functions in SD
- •Bit depth requirements
- •Gamma in modern display devices
- •Estimating gamma
- •Gamma in video, CGI, and Macintosh
- •Gamma in computer graphics
- •Gamma in pseudocolour
- •Limitations of 8-bit linear coding
- •Linear and nonlinear coding in CGI
- •Colour acuity
- •RGB and R’G’B’ colour cubes
- •Conventional luma/colour difference coding
- •Luminance and luma notation
- •Nonlinear red, green, blue (R’G’B’)
- •BT.601 luma
- •BT.709 luma
- •Chroma subsampling, revisited
- •Luma/colour difference summary
- •SD and HD luma chaos
- •Luma/colour difference component sets
- •B’-Y’, R’-Y’ components for SD
- •PBPR components for SD
- •CBCR components for SD
- •Y’CBCR from studio RGB
- •Y’CBCR from computer RGB
- •“Full-swing” Y’CBCR
- •Y’UV, Y’IQ confusion
- •B’-Y’, R’-Y’ components for BT.709 HD
- •PBPR components for BT.709 HD
- •CBCR components for BT.709 HD
- •CBCR components for xvYCC
- •Y’CBCR from studio RGB
- •Y’CBCR from computer RGB
- •Conversions between HD and SD
- •Colour coding standards
- •31. Video signal processing
- •Edge treatment
- •Transition samples
- •Picture lines
- •Choice of SAL and SPW parameters
- •Video levels
- •Setup (pedestal)
- •BT.601 to computing
- •Enhancement
- •Median filtering
- •Coring
- •Chroma transition improvement (CTI)
- •Mixing and keying
- •Field rate
- •Line rate
- •Sound subcarrier
- •Addition of composite colour
- •NTSC colour subcarrier
- •576i PAL colour subcarrier
- •4fSC sampling
- •Common sampling rate
- •Numerology of HD scanning
- •Audio rates
- •33. Timecode
- •Introduction
- •Dropframe timecode
- •Editing
- •Linear timecode (LTC)
- •Vertical interval timecode (VITC)
- •Timecode structure
- •Further reading
- •34. 2-3 pulldown
- •2-3-3-2 pulldown
- •Conversion of film to different frame rates
- •Native 24 Hz coding
- •Conversion to other rates
- •Spatial domain
- •Vertical-temporal domain
- •Motion adaptivity
- •Further reading
- •36. Colourbars
- •SD colourbars
- •SD colourbar notation
- •Pluge element
- •Composite decoder adjustment using colourbars
- •-I, +Q, and Pluge elements in SD colourbars
- •HD colourbars
- •References
- •38. SDI and HD-SDI interfaces
- •Component digital SD interface (BT.601)
- •Serial digital interface (SDI)
- •Component digital HD-SDI
- •SDI and HD-SDI sync, TRS, and ancillary data
- •Analog sync and digital/analog timing relationships
- •Ancillary data
- •SDI coding
- •HD-SDI coding
- •Interfaces for compressed video
- •SDTI
- •Switching and mixing
- •Timing in digital facilities
- •Summary of digital interfaces
- •39. 480i component video
- •Frame rate
- •Interlace
- •Line sync
- •Field/frame sync
- •R’G’B’ EOCF and primaries
- •Luma (Y’)
- •Picture center, aspect ratio, and blanking
- •Halfline blanking
- •Component digital 4:2:2 interface
- •Component analog R’G’B’ interface
- •Component analog Y’PBPR interface, EBU N10
- •Component analog Y’PBPR interface, industry standard
- •40. 576i component video
- •Frame rate
- •Interlace
- •Line sync
- •Analog field/frame sync
- •R’G’B’ EOCF and primaries
- •Luma (Y’)
- •Picture center, aspect ratio, and blanking
- •Component digital 4:2:2 interface
- •Component analog 576i interface
- •Scanning
- •Analog sync
- •Picture center, aspect ratio, and blanking
- •R’G’B’ EOCF and primaries
- •Luma (Y’)
- •Component digital 4:2:2 interface
- •Scanning
- •Analog sync
- •Picture center, aspect ratio, and blanking
- •R’G’B’ EOCF and primaries
- •Luma (Y’)
- •Component digital 4:2:2 interface
- •43. HD videotape
- •HDCAM (D-11)
- •DVCPRO HD (D-12)
- •HDCAM SR (D-16)
- •JPEG blocks and MCUs
- •JPEG block diagram
- •Level shifting
- •Discrete cosine transform (DCT)
- •JPEG encoding example
- •JPEG decoding
- •Compression ratio control
- •JPEG/JFIF
- •Motion-JPEG (M-JPEG)
- •Further reading
- •46. DV compression
- •DV chroma subsampling
- •DV frame/field modes
- •Picture-in-shuttle in DV
- •DV overflow scheme
- •DV quantization
- •DV digital interface (DIF)
- •Consumer DV recording
- •Professional DV variants
- •47. MPEG-2 video compression
- •MPEG-2 profiles and levels
- •Picture structure
- •Frame rate and 2-3 pulldown in MPEG
- •Luma and chroma sampling structures
- •Macroblocks
- •Picture coding types – I, P, B
- •Prediction
- •Motion vectors (MVs)
- •Coding of a block
- •Frame and field DCT types
- •Zigzag and VLE
- •Refresh
- •Motion estimation
- •Rate control and buffer management
- •Bitstream syntax
- •Transport
- •Further reading
- •48. H.264 video compression
- •Algorithmic features, profiles, and levels
- •Baseline and extended profiles
- •High profiles
- •Hierarchy
- •Multiple reference pictures
- •Slices
- •Spatial intra prediction
- •Flexible motion compensation
- •Quarter-pel motion-compensated interpolation
- •Weighting and offsetting of MC prediction
- •16-bit integer transform
- •Quantizer
- •Variable-length coding
- •Context adaptivity
- •CABAC
- •Deblocking filter
- •Buffer control
- •Scalable video coding (SVC)
- •Multiview video coding (MVC)
- •AVC-Intra
- •Further reading
- •49. VP8 compression
- •Algorithmic features
- •Further reading
- •Elementary stream (ES)
- •Packetized elementary stream (PES)
- •MPEG-2 program stream
- •MPEG-2 transport stream
- •System clock
- •Further reading
- •Japan
- •United States
- •ATSC modulation
- •Europe
- •Further reading
- •Appendices
- •Cement vs. concrete
- •True CIE luminance
- •The misinterpretation of luminance
- •The enshrining of luma
- •Colour difference scale factors
- •Conclusion: A plea
- •Radiometry
- •Photometry
- •Light level examples
- •Image science
- •Units
- •Further reading
- •Glossary
- •Index
- •About the author
Tables
|
|
Part 1 – Introduction |
1 |
|
|
|
||
3 |
Linear-light and |
3.1 |
Typical contrast ratios |
30 |
|
|||
|
perceptual uniformity |
|
|
|
|
|
|
|
4 |
Quantization |
4.1 |
Decibel examples |
39 |
|
|
||
5 |
Contrast, brightness, |
5.1 |
Effect of adjusting contrast and brightness 57 |
|||||
|
contrast, and |
|
|
|
|
|
|
|
|
brightness |
|
|
|
|
|
|
|
8 |
Raster scanning |
8.1 |
Refresh rate |
84 |
|
|
|
|
|
|
8.2 |
Scanning in computing |
92 |
|
|||
|
|
8.3 |
Video systems are classified |
94 |
||||
11 |
Picture rendering |
11.1 |
End-to-end power functions |
119 |
||||
13 |
Introduction to |
13.1 |
Gratuitous differences |
132 |
|
|||
|
component SD |
|
|
|
|
|
|
|
15 |
Introduction to HD |
15.1 |
ATSC A/53 Table 3 |
143 |
|
|||
|
|
15.2 |
HD scanning parameters 145 |
|||||
16 |
Introduction to video |
16.1 |
Approximate compression ratios 149 |
|||||
|
compression |
16.2 |
Approximate compression ratios of |
|||||
|
|
|
M-JPEG |
150 |
|
|
|
|
17 |
Streams and files |
17.1 |
Files and streams |
163 |
|
|
||
18 |
Metadata |
18.1 |
Color primaries |
177 |
|
|
||
|
|
18.2 |
Transfer characteristics |
177 |
|
|||
xxxi
|
|
18.3 |
Matrix coefficients |
177 |
|
|
|
|
|
|
|
18.4 |
Color primaries interpretation |
179 |
|
||||
|
|
18.5 |
Transfer characteristics interpretation |
179 |
|||||
|
|
18.6 |
Matrix coefficients interpretation |
179 |
|
||||
|
|
Part 2 – Theory 189 |
|
|
|
|
|
|
|
23 |
Perception and visual |
23.1 |
Power functions in perception |
251 |
|
||||
|
acuity |
|
|
|
|
|
|
|
|
25 |
The CIE system |
25.1 |
White references |
279 |
|
|
|
|
|
|
of colorimetry |
|
|
|
|
|
|
|
|
26 Colour science for video |
26.1 |
BT.709 primaries |
292 |
|
|
|
|
||
|
|
26.2 |
Luminance and chromaticities of BT.709 |
||||||
|
|
|
colourbars |
292 |
|
|
|
|
|
|
|
26.3 |
NTSC primaries (obsolete) |
293 |
|
|
|||
|
|
26.4 |
EBU Tech. 3213 primaries |
293 |
|
|
|||
|
|
26.5 |
SMPTE RP 145 primaries |
293 |
|
|
|
||
|
|
26.6 |
SMPTE “Free Scale” default primaries |
294 |
|||||
|
|
26.7 |
AMPAS ACES primaries 295 |
|
|
|
|||
|
|
26.8 |
SMPTE/DCI P3 primaries |
295 |
|
|
|
||
|
|
26.9 |
Example primaries |
297 |
|
|
|
|
|
28 |
Luma and colour |
28.1 |
luma coefficients, EOCF, and primary |
|
|||||
|
differences |
|
chromaticities 351 |
|
|
|
|
||
|
|
28.2A |
Colour difference systems for analog SD 353 |
||||||
|
|
28.2B |
Colour difference systems for digital SD and |
||||||
|
|
|
computing |
354 |
|
|
|
|
|
|
|
28.2C |
Colour difference systems for HD |
354 |
|||||
|
|
Part 3 – Practical matters |
355 |
|
|
|
|
||
32Frame, field, line, and sample rates
32.1 Derivation of 13.5 MHz common sampling rate 394
33 |
Timecode |
33.1A |
Timecode bit assignment table 403 |
|
|
|
33.1B |
Timecode flag bits |
403 |
|
|
33.1C |
Timecode binary group flags 404 |
|
35 |
Deinterlacing |
35.1 |
Weston deinterlacer |
416 |
xxxii |
DIGITAL VIDEO AND HD ALGORITHMS AND INTERFACES |
|
|
|
Part 4 – Studio standards |
425 |
|
|
|
|
|||
38 |
SDI and HD-SDI |
38.1 |
Analog video levels |
429 |
|
|
|
|
|||
|
interfaces |
38.2 |
Digital video levels |
429 |
|
|
|
|
|||
|
|
|
38.3 |
Timing reference sequence (TRS) |
435 |
||||||
|
|
|
38.4 |
Protection bits for SAV and EAV |
435 |
||||||
|
|
|
38.5 |
Line number and CRC in HD-SDI |
437 |
||||||
|
|
|
38.6 |
Digital to analog timing relationships |
438 |
||||||
|
|
|
38.7 |
SD and HD interface standards 444 |
|
||||||
39 |
480i |
component video |
39.1 |
480i |
line assignment |
446 |
|
|
|
||
40 |
576i |
component video |
40.1 |
576i |
line assignment |
458 |
|
|
|
||
41 |
1280× 720 HD |
41.1 |
720p scanning parameters |
|
467 |
|
|
||||
|
|
|
41.2 |
1280× 720 line assignment |
468 |
|
|
||||
42 |
1920× 1080 HD |
42.1 |
1920× 1080 scanning parameters |
474 |
|||||||
|
|
|
42.2 |
1080i and 1080p line assignment |
475 |
||||||
43 |
HD videotape |
43.1 |
Digital videotape formats for HD |
481 |
|||||||
|
|
|
Part 5 – Video compression |
489 |
|
|
|
|
|||
46 |
DV compression |
46.1 |
DV chroma subsampling |
505 |
|
|
|||||
47 |
MPEG-2 video |
47.1 |
MPEG-2 profiles |
515 |
|
|
|
|
|||
|
compression |
47.2 |
MPEG-2 main and 4:2:2 profiles |
516 |
|||||||
|
|
|
47.3 |
GoP restrictions in SMPTE ST 308M |
517 |
||||||
|
|
|
47.4 |
MPEG-2 aspect ratio information |
517 |
||||||
|
|
|
47.5 |
MPEG-2 frame rate code |
518 |
|
|
||||
|
|
|
47.6 |
2-3 pulldown sequence in MPEG-2 |
519 |
||||||
|
|
|
47.7 |
MPEG macroblock types |
523 |
|
|
||||
|
|
|
47.8 |
MPEG picture coding types |
523 |
|
|
||||
|
|
|
47.9 |
MPEG-2 prediction modes |
523 |
|
|
||||
48 |
H.264 video |
48.1 |
H.264 features 539 |
|
|
|
|
|
|||
|
compression |
48.2 |
H.264 levels 540 |
|
|
|
|
|
|
||
|
|
|
48.3 |
Two hypothetical coding schemes |
544 |
||||||
|
|
|
48.4 |
An example of exponential Golomb coding 545 |
|||||||
|
|
|
48.5 |
Exp-Golomb coding can be generalized 545 |
|||||||
|
|
|
48.6 |
AVC-Intra profile/level combinations |
548 |
||||||
xxxiii
|
|
Part 6 – Distribution standards 553 |
|
50 |
MPEG-2 storage and |
50.1 |
MPEG-2 PCR counts per frame 558 |
|
transport |
|
|
|
|
Appendices 565 |
|
B |
Introduction to |
B.1 |
Quantities, symbols, and units of radiometry and |
|
radiometry and |
|
photometry 573 |
|
photometry |
B.2 |
Light level examples 578 |
|
|
B.3 |
Conversion of illuminance into lux 579 |
|
|
B.4 |
Conversion of luminance into cd·m–2 579 |
xxxiv |
DIGITAL VIDEO AND HD ALGORITHMS AND INTERFACES |
