A depth map can fairly easily be created for CGI content, including computer games in consumers’ premises. However, there is no widely available standard for conveying the depth map from the computer to the display. Depth map techniques do not directly deal with occlusion, so visual performance is limited.

Autostereoscopic displays

Autostereoscopy refers to techniques that present stereoscopic imagery without the requirement for the viewer to wear glasses. Two techniques have received limited commercialization: the parallax barrier technique, and the lenticular technique.

Autostereoscopic displays typically create reasonable stereo across a small volume of the viewing space. The major problem is that the “sweet spot” is typically fairly small, and outside the sweet spot, the stereo effect is either dramatically reduced or vanishes entirely. Also, autostereoscopic displays sometimes have (unintended) viewing positions where the views are reversed, causing apparent depth inversion known as pseudostereo.

Parallax barrier display

Two views are displayed interleaved column-by-column on the same display surface. A short distance in front of the display lies a set of barriers that form slots through which, at normal viewing distance, alternate image columns can be viewed. The geometry of the barrier (pitch and position) is designed so that at a chosen optimal viewing location, one set of columns is visible to the left eye and the other is “shadowed” by the barrier; the situation is reversed for the right eye.

The technique has been commercialized in handheld devices (3-D cameras and cellphones).

Lenticular display

Two or more (n) views are interleaved on the display surface in n columns. A set of lenses is placed, one lenslet per n columns, over the display. The geometry of each lens is arranged to project the interleaved columns out into the space in front of the display. In the case of two views (n = 2), the left and right images lie in alternate beams.

Philips has demonstrated lenticular autostereoscopic display where several views (n ≈ 9) are generated at the display by signal processing based upon a single 2-D image accompanied by a “depth map” (2-D + Z) that is encoded during postproduction or produced in graphics generation (for example, in PC gaming). The technique has had limited deployment in digital signage, but has not been commercialized for consumer use.

The stereo high profile is related to the multiview profile (MVP) of H.264: Both are documented in Appendix H of the current revision.

Some people might quote multiples as low as 1.2 or as high as 1.8.

Dave LeHoty describes his home HDMI system as “1.3a with

a steenkin’ asterisk,” alluding to the wide variety of versions and options that makes system integration difficult for the expert, let alone for the average consumer.

Recording and compression

For a given image format (e.g., 1920× 1080), S3D obtained through a pair of views obviously involves double the data rate of a single view. The challenges in transport and interface centre around the high data rate. Professional acquisition and postproduction usually involves doubling the data rate (and often doubling up the production equipment). For consumer recording and distribution, 3-D systems have been devised that use less than twice the data rate of 2-D imagery.

Many techniques have been devised to record S3D content and to transport S3D content through broadcasting distribution chains. Some distribution networks squeeze the left and right views 2:1 and abut them horizontally side-by-side (SbS) onto a single signal that can be conveyed through ordinary distribution networks.

The Blu-ray standard has been augmented with

a mechanism to compress S3D content using the stereo high profile of H.264. The motion estimation and motion-compensated interpolation schemes of H.264 were devised to compactly code a sequence of images having a high degree of spatial correlation, where differences between the images are a consequence of elapsed time between their exposures. The left and right images of a stereo pair exhibit a high degree of spatial correlation, where differences between the images are a consequence of position shifts (disparity) induced by parallax. In typical SHP use, the right image is predicted by the left image after “motion” compensation by disparity vectors (comparable to motion vectors). Typical stereo can be coded at between about 1.3 or 1.6 times the data rate of 2-D imagery.

Consumer interface and display

Previous sections have discussed acquisition and display of S3D imagery. Here, we’ll discuss interface to the consumer display.

HDMI version 1.4a has a mandatory frame packing 3D structure that packs left and right eye 1920× 1080 images into a 1920× 2205 “container”

having 45 blanking (black) lines separating the images. There are progressive and interlaced versions.

HDMI 1.4a also describes an interface using

a 1920× 1080 container to convey a 960× 1080 left eye

I once visited a consumer electronics retailer where a stereoscopic 3-D movie was being played from

a Blu-ray disc and conveyed across HDMI in side-by-side format – but displayed on a receiver whose 3-D processing was disabled. I adjusted my vergence to free-view the 3-D imagery, horizontally squished 2:1. A salesman approached and said, “That’s not 3-D.” I said, “Well, I’m seeing depth.” Without missing

a beat, and with full confidence, he said, “No, you’re not.”

image and a 960× 1080 right eye image, both horizontally squeezed 2:1, abutted side-by-side (SbS). Horizontal resolution suffers.

Finally, HDMI 1.4a describes an interface using

a 1920× 1080 container to convey a 1920× 540 left eye image and a 1920× 540 right eye image, both vertically squeezed 2:1, abutted top-and-bottom (TaB). Vertical resolution suffers.

There are many schemes. Confusion abounds.

Ghosting

Most of the display techniques that I have described exhibit the problem that light intended for the left eye “leaks” into the right eye, and vice versa. You could call it “crosstalk.” Image artifacts created by such unwanted light are called ghosts.

There are several reasons for ghosting. In most displays, generation of light in response to the video signal is not instantaneous. For example, the LCD material of LCD displays takes a certain time to respond to the drive signal; the phosphors of PDPs have a certain decay time. When LCD and PDP displays are used for 3-D display using temporal multiplexing, if the display is still decaying while the opposite shutter opens, ghosting will result. In polarized displays, the polarizers (at both the display and the glasses) typically have incomplete extinction. In the Infitec scheme, practical optical filters have a certain degree of unwanted spectral overlap.

Reduction of ghosting to tolerable levels involves compensating the image data prior to its reaching the display. (In cinema, the processing is called ghostbusting). If a bright left-eye image element is anticipated to leak into the right, light can be artificially subtracted from the corresponding spatial location in the right image. Compensation is necessarily imperfect, though: If the corresponding location in the right image is black, no light can be subtracted, and the crosstalk persists. In cinema, compensation can potentially be accomplished either in mastering or in the projector’s signal processing. Movie creators don’t want to create separate masters for each 3-D display technology, so the second option is now usual.

Vergence movements ideally involve rotation of the eyeballs with respect to the plane that joins their centres.

Presbyopia is age-related loss of accommodation owing to the lens becoming less pliant. Even for people having normal vision, presbyopia typically makes reading glasses necessary beyond age 50.

Vergence and accommodation

The region of the human retina intersected by the optical axis is the fovea; it is a cluster of tightly packed cone photoreceptor cells. The fovea has an angular diameter of about 1°; it covers a small fraction of the visual field – a few tenths of a percent of the area corresponding to an HD image at normal viewing distance.

The oculomotor system of the eye includes muscles attached to the eyeball. The muscles “steer” the optical axis of each eye so that the fovea images light from the region of interest in the visual field. A few times per second, the muscles operate and the gaze shifts to

a new point; the movement is called a saccade.

In normal binocular viewing of an actual scene, eye movements are made such that the optical axes of both eyes meet at the depth of the scene element of interest. The oculomotor system’s control of the distance at which the optical axes meet is known as vergence.

The lens of the human eye is enclosed in a capsule that is somewhat pliable: The lens can change shape. Within the eyeball, surrounding the lens, is a muscle – the ciliary muscle. When the muscle is in its relaxed state, the lens capsule is at its flattest; the focal length of the lens is at its maximum. As the ciliary muscle contracts, the lens capsule becomes more spherical; focal length decreases, focussing on nearer objects. Muscle control over the lens is called accommodation; it is analagous to focusing of a camera lens.

In normal human vision viewing real objects, the vergence and accommodation systems work in concert. In a stereo display of the kinds that I have described, both the left and right images are formed on the display surface, and that surface is a fixed distance away from the viewer. To keep the images sharp requires accommodation to the distance of the display screen – not to the apparent distance of the object that is formed by the stereo display system. As apparent depth departs

from the screen distance – either to longer distance (“behind” the screen) or closer distance (“in front of” the screen), conflict between vergence and accommodation (V-A conflict) is likely to be experienced subconsciously by the viewer. Researchers have proven that V-A conflict is a major contributor to viewer discomfort in stereo 3-D.