Stereoscopic (“3-D”) video

The term S3D (“ess-three-dee”) distinguishes stereoscopic 3-D from imagery having depth cues (particularly, perspective) but only one view. Computer-generated imagery (CGI) produces images synthesized from scene geometry; CGI can relatively easily produce stereo views. Some people consider the term S3D to be redundant – that which is stereoscopic is necessarily 3D.

Stereoscopic 3-D (S3D) refers to acquisition, processing, storage, distribution, and display of imagery in two views, one intended for the left eye and one for the right. The views are typically acquired from cameras acquiring the same scene from positions a short lateral distance apart. Stereo viewing presents an illusion. Unlike viewing the real world, the views do not change when the viewer moves his or her head. Nonetheless, for very carefully crafted material, the effect can be convincing, and in some cases, can add to storytelling.

Acquisition

Two cameras are most often used; however, many other arrangements have been demonstrated such as one lens and two imagers, and two lenses and one imager.

To acquire images from a real scene in professional content creation, two cameras are typically used, each including an imager and signal processing. To produce “normal” stereo the optical axes of the cameras are displaced by the same distance the typical viewer’s eyes are separated – the interocular distance (also known as interpupillary distance), which for adults is between about 52 mm and 75 mm, with a mean of about

63.5 mm (2.5 in). Various effects can be achieved by changing the interaxial distance of the cameras: setting a wide camera interaxial distance collapses depth, and upon display makes the scene look smaller than it is; setting a narrow camera interaxial distance expands depth and upon viewing magnifies the scene. Misaligned cameras can lead to viewer discomfort.

S3D display

S3D display can be achieved with a dedicated display for each eye, in the manner of the historical View-

Associating red with left conforms to the nautical convention that red signifies the port (left) side.

Master. Many virtual reality systems from the 1990s and 2000s used the technique, sometimes in combination with head tracking; however, consumers are not comfortable with head-mounted display equipment! Viewing at a distance is a commercial necessity.

For normal television viewing distance of about 3 m, several schemes are in use that multiplex the two views at the display device and separate the views at each viewers’ pair of eyes: anaglyph, temporal multiplexing, polarization, wavelength multiplexing, parallax barrier autostereoscopy, and lenticular autostereoscopy. These techniques are outlined in the sections to follow.

The techniques to be described are almost always used with a single “native” 2-D display (either direct view, or projector). In this case, all of the techniques have the disadvantage that at best 50% of the light of the native 2-D display is available (and frequently, much less). Consequently, stereo 3-D display systems tend to be dim.

Anaglyph

Imagery is created placing the red component of the left view into the red primary, and the green and blue components of the right view, into the three components of what would otherwise be a 2-D video stream. (Clearly, several assumptions that enable chroma subsampling and MPEG or H.264 encoding are broken.)

The display presents the left-eye image using the red primary and the right-eye image using green and blue.

The viewer wears glasses having a colour filter over each eye. A red filter is placed over the left eye – the left eye only sees the red primary of the signal, containing the left image. A cyan filter is placed over the right eye – the right eye sees dichromatic combinations of the green and blue components of the right image. Full colour is not present for every pixel for each eye; nonetheless, the viewer’s visual system largely compensates the loss (albeit with some discomfort). The red/cyan scheme is most common, but anaglyph display can use other combinations of colours.

Owing to the ease of recording and transmission using standard 2-D video infrastructure (admittedly outside of its usual assumptions), the anaglyph scheme was used sporadically for years in both cinema and tele-

An excellent outline of the physics of polarized light is given in this book: Reinhard, Erik et al. (2008), Color Imaging: Fundamentals and Applications (Wellesley, Mass.: A K Peters).

vision, but has mostly fallen into disuse and is now generally considered a novelty.

Temporal multiplexing

Two views can be multiplexed in time: The display operates at (at least) twice the frame rate of the imagery and alternately presents the left-eye image and the right.

The viewer wears active shuttered glasses, synchronized with the display such that the right eye is blocked while the left image is displayed and the left eye is blocked while the right image is displayed.

Shutter synchronization is typically achieved through an infrared (IR) light beam that is pulsed at the frame rate, flooding the viewing area. Each set of glasses includes an IR receiver. (Bluetooth radio frequency synchronization has been proposed.)

The scheme dominates 3-D consumer television, and has limited use in cinema (XPAND 3D).

Polarization

Many S3D display schemes involve polarized light. The simplest forms of polarization – those used commercially – are linear polarization (LP) and circular polarization (CP). The viewer wears passive polarized glasses; filters for two eyes have opposite polarizations.

Polarization can be time-multiplexed: The display operates at (at least) twice the frame rate of the imagery, and alternately presents the left-eye image (in one polarization) and the right-eye image (in the opposite polarization).

In the RealD system common in theatres, a “Z screen” is inserted in the light path at the projector, between the projection lens and the port glass. The Z screen is an electro-optical device that rapidly switches the polarity of circular polarization. The imager produces the leftand right-eye images time-sequentially; the

Z screen is actuated in synchrony. (In the RealD system deployed in theatres as I write, there are three left-right cycles per 1/24 s – that is, the display’s modulator produces images at 144 Hz.) The technique has not been commercialized for direct-view displays.

Polarized projection can potentially produce both views at the same time – for example, by using a pair of projectors (or two image modulators sharing the same

In the system as commercialized, each image has 1060 rows, not 1080 as you might expect: 40 black rows lie between the two.

Opposite polarization of alternate image rows is typically achieved using a film pattern retarder (FPR).

The wavelength multiplex scheme could simultaneously present left and right images. However, that mode hasn’t been commercialized.

projection lens). However, such solutions are unpopular owing to their high cost. A single 4 K (4096× 2160) projector can be adapted to display a 2 K (2048× 1060) left image on the top and a like-sized right image on the bottom, then fitted with an optical device to oppositely polarize the two images and combine them for simultaneous display. The scheme has been commercialized for cinema by Sony.

Polarized projection requires that the screen preserve polarization. Typical cinema screens depolarize, so “silver” – actually, aluminized – screens are used.

For direct-view displays, polarization can be accomplished by fabricating polarizers of opposite polarity over alternate image rows of the display. Obviously, in 3-D operation, vertical resolution is halved compared to the native display capability. Such a display can be used for normal 2-D viewing without glasses (though with at best 50% of the 2-D light available).

A big advantage of polarized systems is the fact that the glasses are passive and inexpensive.

Wavelength multiplexing (Infitec/Dolby)

This technique was invented by Helmut Jorke at Daimler-Benz in Germany. The display operates at twice the frame rate of the imagery (or higher), and presents first the left-eye image, then the right, through different optical filters. The wavelength compositions of each

pair (e.g., GLEFT and GRIGHT) are designed to be mostly nonoverlapping. The characteristics of the optical filters

are compensated by signal processing to produce roughly metameric pairs – that is, although the wavelength composition of the pair of reds differ, the colours look roughly the same.

The viewer wears passive glasses, where each eye has a different optical filter roughly matching that of the projector. The left eye’s filter rejects the wavelengths

corresponding to RRIGHT, GRIGHT, and BRIGHT; the right eye’s filter rejects the wavelengths corresponding to

RLEFT, GLEFT, and BLEFT.

The Infitec scheme uses passive (albeit somewhat expensive) glasses, and does not require a polarizationpreserving screen.

Dolby commercialized the scheme for 3-D cinema. It has not been commercialized for direct-view displays.