substantial number (between 3 and 4 percent; Valberg et al., 1986b) of LGN cells with clear M–S inputs that were for a long time regarded as a missing link in opponent colour theories (Gouras and Zrenner, 1981). This cell type may now have been identified morphologically (Dacey and Packer, 2003).

After this introduction to a quantitative model of the responses of retinal and geniculate opponent cells across several stimulus dimensions, let us see how these ideas relate to psychophysical measures of unique hues, the Abney effect and the Bezold– Bru¨cke phenomenon (Abney, 1910, 1913; von Bezold, 1873; von Bru¨cke, 1878).

The opponent model and color perception

In the following we shall see how the functionally important nonlinearity in coneopponent cells can be used to construct a physiological model of color vision. While linear models account nicely for near threshold data and for relative small changes in stimulus intensity and chromaticity, a nonlinear model is valid over a much larger stimulus range. Furthermore, a nonlinear model can successfully account for many color phenomena where linear models fail.

Munsell color scaling

Using the model of Figure 6.9, Figure 6.18 shows the combined response magnitudes of macaque geniculate ‘L–M’ cells to chromatic stimuli of Munsell hue 5R and different chroma and values. Solid circles connected with lines represent constant Munsell chroma. Figure 6.18(a) shows the chromatic responses of I-cells and Figure 6.18(b) that of D-cells to the same stimuli. The Increment cells in A are more responsive to the lighter colors of higher value, leading to tilted iso-chroma lines with positive slopes, while Decrement cells are the more responsive to darker colors of lower value, giving chroma lines with negative slopes. Because of these tendencies to prefer colors of different lightness, the averaged responses of these two types of cells, shown in Figure 6.18(c), yield close to optimal chroma lines, parallel to the achromatic axis with a few exceptions for the darkest colors.

Results similar to those of Figure 6.18 were also obtained for value–chroma planes of other hues, confirming that, along the reddish-greenish dimension of color space, the output of both I- and D-opponent cells are necessary to account for chroma scaling of colors of different lightness.

Unique colors

The colored wedges in the diagram of Figure 6.19(a) represent the model responses to the four unique hue stimuli, Y, R, B and G of increasing chroma (denoted 5Y, 5R,

Figure 6.18 (a, b) Model responses of macaque I- and D-cells to the chromati stimuli of Munsell hue 5R, varying in chroma and value. I-cells (a) have the higher firing rate to bright colors while D- cells (b) respond better to dark colors. (c) The linearly combined responses of (a) and (b) [as in Figure 6.9(c)] give close to equal spacing of chroma.

5PB and 5G in the Munsell hue system). For a given light adaptation, these stimuli correspond to the unique hues as determined by human subjects. The dashed ellipse in the same figure represents the loci of colors of constant chroma 8. The response magnitudes to the Munsell color stimuli have been computed for each opponent direction and plotted along the x- and y-axes. The response magnitude of the summed

Figure 6.19 (a) The loci of the unique hues yellow, red, blue and green in a diagram combining the chromatic model responses, N, of the six opponent LGN cell types mentioned in the text. (b) The combined model response of the macaque opponent cells to colors in the Munsell color system, varying in hue and chroma (value 5). Loci of constant hue and varying chroma are approximately straight lines, while equal chroma is represented by quasi-elliptical loci. See also Color Plate Section.

I- and D-, ‘L–M’ cells is plotted along the positive x-axis and the response of the ‘M– S’ cells along the positive y-axis. ‘M–L’ cells are represented by the negative x-axis, and ‘S–L’ cells by the negative y-axis. This figure shows only the chromatic responses, i.e. the difference in firing rates between a chromatic stimulus and an achromatic stimulus of the same reflectance factor (same Munsell value).

In order to simplify the computation, the variability of the response within each class of opponent cells has been dealt with in Figure 6.19(a) by using data from cells that represent typical averages of each type (Valberg et al., 1986b). This is equivalent to narrowing the spectral bandwidth of the ‘population response’ represented by each axis, and for this reason the unique hues have been represented by fan-like distributions in the figure.

Several properties of Figure 6.19(a) should be noted. The center of the coordinate axes represents the response to achromatic stimuli (that need not be zero) for all cell types. This magnitude has been subtracted from each response to obtain a ‘pure chromatic response’. Activation responses are related to an increase in the magnitude of a certain percept, whereas inhibitory responses of the same cells imply less of the same, not the opposite quality.

It is immediately clear from Figure 6.19(a) that the cone opponent directions, represented by the four cardinal coordinate axes, cannot individually represent the unique hues. However, as is shown in Figure 6.19(b), a constant unique hue, and all other constant hues, approximate straight lines radiating from the white point, and can be approximated by a constant ratio of opponent cells responses. It may come as a surprise that unique red, for instance, falls nearly midway between the L–M and M–S axes. Chromatic stimuli that activate only one group of opponent cells, without causing any differential excitation in the other independent cell types, do not correspond to unique hues. Such stimuli, with responses along one axis and zero response in the orthogonal directions, have binary hues; we perceive their color as being composed of two unique hues. Increasing the chroma of one of these binary hues (e.g. a bluish red of Munsell hue 5RP along the L–M axis) would lead to differentiated responses in only the ‘L–M’ cells, whereas the orthogonal ‘M–S’ and ‘S–L’ cell types would react as if the stimulus were achromatic of the same value. For our adaptation condition, the same would be true for stimuli of hue 10G (activating only ‘M–L’ cells, stimuli of hue 2.5GY activating ‘M–S’ cells, and stimuli of hue 2.5P activating only ‘S–L’ cells). The neural correlate to white (and other achromatic colors) appears to be related somehow to normalized responses in all opponent cell types. Each hue – whether it is of elementary, binary or any other hue – depends on vector coding of separate opponent mechanisms. The hue lines radiating from the white point in Figure 6.19(b) are closer to straight lines in this cell representation than in the CIE chromaticity diagram (see Figures 5.28 and 5.29). This is a consequence of the nonlinear stimulus–response relationships of opponent cells. The fact that straight lines in the diagram of Figure 6.19(b) are reproduced as curved lines in the CIE (x,y) diagram gives an obvious explanation of the well-known Abney effect (see below).

The fact that unique hues are not represented by the four cardinal axes in the diagram may come as a disappointment to those who have grown up with the opponent theory of color vision. Nevertheless, and not surprisingly, the ratios of the ‘chromatic responses’ of different types of opponent cells are clearly important for hue perception. A straight line, radiating from the white point, represents a constant

ratio of the responses of orthogonal cell types. At the same time, it represents a single hue where distance from the white point is proportional to chroma. These results are in accordance with earlier psychophysical data which also demonstrated non-correspon- dence between cardinal axes and unique hues (Valberg, 1971; Krauskopf et al., 1982; Burns et al., 1984; De Valois et al., 1997).

Under other experimental conditions, with a different adaptation or with a different surround, a unique red color, for instance, would be associated with another wavelength or spectral composition. Despite this, there is a theoretical possibility that the same local and lateral feedback that contributes to color constancy would counteract adaptation changes in the receptors and single cells – and thus provide for the same response ratio between opponent cells whenever we see the same hue. We do not yet know to what extent these opponent cells’ relative firing patterns are linked to the physical properties of the stimulus projected onto their receptive fields, or are being modified (adapted) by long-range interactions, as would be necessary for ‘color constancy’ to occur (but see Valberg et al., 1985a). If the phenomenon of color constancy is, in part, governed by processes beyond the retinal and geniculate levels (according to the results of Land et al., 1983), changing adaptation would lead to unique red being represented by different response vectors under altered states of adaptation. So far, unique hue directions have not been found to have a particular status in the cortex.

In conclusion, in a normalized viewing situation, there is a covariation of a specific chromatic response ratio of opponent cells and our perceiving a constant hue irrespective of chroma. Generally, however, we cannot know which hue is associated with a particular ratio, only that the hue stays the same as long as the cell response ratio does. The chroma scale of increasing color strength corresponds to equal increments along each hue vector [but not the same increment for all vectors; see Figure 6.19(b)]. Theoretically, a general correlate to color strength can be approximated by a linear combination of the cardinal axes p1 and p2 of Figure 6.19(b), transforming the elliptical shapes to circular shapes:

F1 ¼ 1:5p1 0:5p2

F2 ¼ 0:7p2

The p1 axis is negative in the M–L direction and the p2 axis is negative in the S–L direction. As shown in Figure 5.32, such a transformation leads to steps of equal hue approximating equal angles, and equal color differences becoming nearly equal geometrical differences. However attractive such a mathematical/cortical solution might appear, nature nevertheless may have adopted an alternative strategy in primates. For instance, neurones tuned to several chromatic directions other than the cardinal axes have been found in primary visual cortex of primates (Lennie et al., 1990) and at later stages (Kiper et al., 1997; Wachtler et al., 2003).

Under normal conditions, no color is at the same time perceived as red and green or as yellow and blue. The mutual exclusiveness of the unique red and green hues, and

of unique yellow and blue, has no simple geometrical counterpart in the cell response spaces of Figures 5.32 and 6.19 (thus being different from complementary colors in stimulus space). Although the cardinal directions p1 and p2 (as well as F1 and F2 of Figure 5.32) seem to account for variations in chromatic scaling and color discrimination, unique hues are still without a neural representation, and their physiological origin remains enigmatic. The argument could be made, of course, that the reason for our bewilderment about colour qualities, and particularly about the unique hues, is that monkeys are different from humans, or because neurons in anesthetized animals respond differently from those in awake animals. The first objection is hard to maintain, considering the many demonstrations of psycho-physiological similarities of color vision in primates. Except at the peripheral levels of visual processing, the second argument may, however, be relevant.

The Abney effect

Anyone with experience of additive color mixtures will have noticed that, when white light is added to a colored light of high purity, the hue of the mixture will change (Abney, 1910, 1913). As noted earlier, provided luminance is kept constant, even a small amount of white added to unique red will give the mixture a bluish-red appearance. Increasing amounts of white will intensify the blue shift up to a point, after which the blue component diminishes. When white is added to a saturated orange, the hue becomes more reddish, and when added to yellow–green, it becomes more green. These hue shifts, also known as the Kohlrausch effect, are represented in the chromaticity diagrams of Figures 5.28 and 5.29.

The calculations for hues by the opponent model are shown in Figures 6.19(b) and 5.32(c). Perfectly straight lines in Figures 6.19(b) and 5.32(c), like those representing slightly yellowish-red or a yellowish-green, and purples and blue-greens, are reproduced as curved hue lines in the (x,y)-diagram of the CIE system of Figures 5.28 and 5.29. Although not all the hues are represented as perfectly straight lines by the model, straightening them would lead to a slightly stronger curvature than they already have in the chromaticity diagram, in the direction predicted by the Abney effect. We therefore conclude that the nonlinearity of the cell responses in the model describes the Abney effect adequately.

The Bezold–Bru¨cke phenomenon

Another well-established phenomenon is that, when the luminance of foveal chromatic light increases from zero, there is first an achromatic interval between light detection and the identification of its hue, after which chromatic strength increases and the perceived hue steadily changes. Long-wavelength red light becomes more yellowish and short-wavelength violet light becomes more bluish. This latter

phenomenon is known as the Bezold–Bru¨cke phenomenon (von Bezold, 1873; von Bru¨cke, 1878; Purdy, 1931a; Stabell and Stabell 1982b).

It is less well known that this hue shift is accompanied by significant changes in the perceived chromatic content of the stimulus. When the luminance of a spectral light increases, the chromatic strength first increases and then decreases in a wavelengthspecific manner (Haupt, 1922; Purdy, 1931b; Vimal et al., 1987). Such nonlinearities of color vision occur, for instance, in colored light sources and in self-luminous color displays of various kinds where luminance ratios relative to an adaptive surround or a background can vary over a much larger range than for (related) surface colors. Here we shall describe how the combined changes of hue and saturation affect the color appearance of near-monochromatic lights over a 4–5 log unit range of intensity above the chromatic threshold.

A common theoretical framework for explaining these phenomena has long been sought by the vision science community. Bezold–Bru¨cke hue shifts have been explained by the saturation of the intensity–response curves of the cones themselves (Walraven, 1961; Cornsweet, 1978), or by the relative activities of ‘chromatic and achromatic signals’ (Judd, 1951b; Hurvich and Jameson, 1955). Explanations in terms of compressive nonlinearities of receptor excitations (e.g. von Helmholtz, 1962) are rather general, and such models have not been demonstrated to predict the experimental data for the combined hue and saturation appearances of individual chromatic lights.

In the discussion below, these hue shifts are linked to the Abney effect by a common physiological mechanism. The hue shifts are readily accounted for at a postreceptoral level by the combination of non-monotonic intensity–response curves of different types of cone-opponent cells. From the assumption that both the chromatic content of a stimulus and its hue are related to the relative activation of parvoand koniocellular opponent cells (Valberg et al., 1986a), it follows that the dependence of the perceived magnitudes on stimulus intensity must be understood in the same framework.

In order to demonstrate the validity of this hypothesis, let us first present psychophysical data on the combined color shifts that occur when the relative luminance changes. Figures 6.20 and 6.21 exemplify the estimates of hue and chromatic strength of several different stimuli, in which the luminance changes relative to a white adapting reference background. The experimental results show significant inter-observer variation in the details, but the general trend is in accordance with the physiological conjecture that chromatic changes can be accounted for by the relative activation of opponent cells.

The observers viewed 4 stimuli of increasing luminance flashed on a screen for 300 ms in alternation with a white (100 cd/m2) adaptation field of 1.2 s duration. The white stimulus in this successive contrast paradigm served as a constant reference and set the adaptation level during the experiment. The apparatus and the procedure were the same as that used in our neurophysiological recordings described earlier (see

Figure 6.20 The hues of the spectral colors change as their luminance ratio relative to an adaptation light increases. In this experiment, the hue of a light with a given wavelength is assessed at different luminance ratios. A wavelength of 572 nm normally appears yellow–green when the luminance ratio is low. As the luminance ratio increases, it become less green and approaches the yellow elementary hue at a high luminance ratio. No wavelength maintains a constant hue for luminance changes over a range of 3–4 log units (although 589 nm comes close to doing this for subjects B.L.M. and A.V.). These data were obtained in a stimulus situation like that used in the neurophysiological experiments described in Figure 6.7.

Figures 4.14 and 6.7 and Lee et al., 1987). The CIE 2 (x,y) chromaticity coordinates of the white light were (0.388, 0.406). The presentation was monocular and foveal. The luminance of the chromatic stimuli was changed with neutral density filters over a range of 5 log units in steps of 0.3 log unit from 0.1 to 10 000 cd/m2. The surround was dark in the experiments reported here. No artificial pupil was used.

Figure 6.21 Chromatic strength also changes with luminance ratio. Starting at low values, color strength increases with luminance ratio, typically reaching a maximum which is followed by a slow decline. Maximum color strength is reached at different luminance ratios for different wavelengths: at low ratios for short-wavelength, blue lights (e.g. 454 nm) and at high ratios for mid-spectral yellow (589 nm).

The observer was asked to estimate two attributes of each stimulus, the hue and the chromatic strength, but only one attribute in each session. Hue was judged in terms of the contribution of two neighboring elementary (unique) hues (yellow and red, red and blue, blue and green, or green and yellow) in terms of two values that added up to 10. Elementary hues, 10Y, 10R, 10B, 10G in Figure 6.20, obeyed the usual neither–nor criterion (e.g. elementary yellow, 10Y, is neither reddish nor greenish). In this way a reddish orange light could be characterized, for example, as 3Y and 7R . Stimulus luminance were presented in a pseudo-random order.

In scaling the chromatic content of a stimulus, the observer was instructed not to pay attention to the intensity changes in the stimuli, but to estimate only ‘the perceived chromatic difference relative to an equally bright achromatic stimulus’. This is essentially the definition of Munsell chroma (Wyszecki and Stiles, 1982), but in this

context we call it ‘chromatic strength’. We deliberately did not use the term ‘saturation’, since saturation is often used in the sense of a ratio between chromatic and achromatic components of light (either perceptual or colorimetric).

The magnitude of the perceived chromatic strength of the stimulus was rated by the observer on a scale from 0 to 100, where 100 was supposed to be the absolute maximum for the most saturated hue. In one case a number greater than 100 was necessary. Further details are found in Valberg et al. (1991b).

Following an established tradition, the hue ratings in Figure 6.20 are plotted on the abscissa while the ordinate refers to the ratio between the luminance of the stimulus

(L) and that of the white (100 cd/m2) adaptation field (Lb). With increasing luminance ratios above 1.0 (the ratio when the chromatic stimulus and the white adaptation field both had a luminance of 100 cd/m2) the curves of the stimuli between 531 and 649 nm converge towards yellow. For luminance ratios below 1.0, orange and greenish or greenish-yellow hues dominate increasingly, although some interindividual variation was seen.

Even though we measured color changes as a function of (a successive) luminance ratio, and not as a function of absolute luminance, the directions of hue shifts are in general agreement with earlier observations reported in the literature for comparable luminance ranges. However, several studies have concluded that the elementary hues are invariant with intensity, whereas others have provided evidence that they generally do vary (Purdy, 1931a; Savoie, 1973; Nagy, 1979). None of the stimuli of Figure 6.20 were hue-invariant for all observers. There is, of course, a slight possibility that invariance was missed due to the sampling of wavelengths, but it is also possible that hue-invariant wavelengths are only found for a restricted luminance range; we used a constant white adaptation stimulus and investigated a much larger luminance range than usual. Note also the different positions of the stimuli on the hue axes of Figure 6.20 for the three subjects. Both position and the relative hue shifts with intensity were subject to inter-observer variation.

In Figure 6.21 the ratings of chromatic strength are illustrated for a few selected wavelengths and for the same observers as in Figure 6.20. Reproducibility was good for all observers, and similar to that shown for subject PM. The curves show a rise and a fall of chromatic strength as luminance increases. Short-wavelength stimuli reached maximum chromatic strength at low luminance ratios (L/Lb) of about 0.1 or below. For the 500 nm stimulus, maximum chromatic strength was found at a luminance ratio close to 0.3 for all observers, and for the midspectral yellow stimulus of 589 nm, the maximum has moved further to the right to about 10 times the luminance for the white adapting reference. Such a phenomenon of hue shift with intensity can be observed in colored light bulbs, where the bright filament appears nearly white and the darker colored glass, at some distance away from the filament, has a much higher color strength.

The ratings of hue and chromatic strength for some of the wavelengths of Figures 6.20 and 6.21 are combined in polar plots in Figure 6.22. In this diagram,

Figure 6.22 In this figure the results of Figures 6.20 and 6.21 are combined in a polar diagram. In these plots, each curve represents the hue and color strength of a given wavelength for a range of different luminance ratios. Subjective color strength is proportional to the distance from the center, and constant hue corresponds to a particular radial angle. The elementary hues, here denoted 5Y, 5R, 5PB and 5G, have the same position as on the hue circle of the Munsell system. The small arrows point to the coordinates for the subjective impression of color strength and hue for a luminance ratio Y ¼ L=Lb ¼ 0:1. Since color strength decreases for lower luminance ratios, all the curves converge towards the origin as Y drops from 0.1 to 0. For stimuli of constant chromaticity we perceive a change in color strength and hue as luminance ratio increases from zero.

the relative chromatic strengths are proportional to radial distance from the origin. These distances have been normalized so that for all observers they have equal length for the 649 nm stimulus at a luminance ratio, L/Lb, of 0.1, as indicated by the circle. Luminance increases from the origin as is indicated by the chevrons in the top figure.

From this figure, we see that, as luminance increases from zero, short-wavelength violet light becomes more bluish (and finally bluish-green for AV), whereas longwavelength stimuli usually turn more yellowish at high relative luminances. All five

NM--L NL--M). Whenever these differences were both zero, some putative higherorder mechanism would signal a correlate for an achromatic color, with the relative responses of I- and D-cells determining the actual gray percept. A sum of the responses of the L/M cell types, in some combination with opponent cells with S-cone inputs, might correlate with brightness (since the response of a chromatic light would be higher than that for the achromatic one of the same luminance). Where in the visual pathway such a possible separation of ‘white responses’, ‘chromatic responses’, and ‘brightness responses’ would occur, if they occur at all, is an unsolved issue, but nothing like it has been observed in the retina and the LGN.

The response magnitudes of the model (composed of the six most typical cells, of which examples are shown in Figure 6.10) to stimuli of constant chromaticities but different luminance ratios are shown in the polar plot of Figure 6.24(a) and (b). While Figure 6.24(a) covers the same luminance range as Figure 6.22, Figure 6.24(b) extends the responses of Figure 6.24(a) to very high luminance ratios. The curves in the diagram are projections on to a plane, and radius vector in this plane is proportional to chromatic strength. The orientation of the vectors relative to the coordinate axes is related to hue. The figure shows that color strength initially grows as luminance increases from zero, reaches a maximum for a luminance ratio that is characteristic for each hue, and then decreases again. The model predicts [Figure 6.24(b)] that at very high intensity all chromatic response is lost, and the stimulus will appear achromatic.

The predictions of the model for the combined changes in chromatic strength and hue are very similar to the experimental results plotted in Figure 6.22. In accordance with the measured Bezold–Bru¨cke hue shifts, the model predicts that a stimulus that appears reddish at low luminance will turn more orange and finally yellowish at very high relative intensity. At some intermediate intensity, chromatic strength reaches a maximum, and for the highest intensities, the light becomes whitish. The typical directions of the hue shifts in Figure 6.22 are well predicted for all hues, with the exception of the 500 nm stimulus. This overall agreement demonstrates that the Bezold–Bru¨cke hue shift and the related change of chromatic strength have a common origin in the non-monotonic responses of retinal and geniculate opponent cells.

We have described the perceived changes of chromatic strength and hue which occur as stimuli of constant chromaticity increase in luminance. The successive contrast paradigm resembles normal viewing conditions, with eyes successively fixating objects of different color and relative luminance. Together with the color scaling, the Abney effect and the Bezold–Bru¨cke phenomenon can be well accounted for by the nonlinear and non-monotonic responses of opponent cells in the retina and the LGN.

Judd, as well as Hurvich and Jameson, has referred the correlate of hue perception and the Bezold–Brucke phenomenon to the relative activity of two neighboring, red–green and yellow–blue ‘primary processes’ (Judd, 1951b; Hurvich, 1981). Here,

we have related the joint hue and chroma changes to a combination of cone-opponent system responses, which, individually, exhibit no simple relationship to the elementary hue qualities.

The decreasing chromatic strength of chromatic stimuli at high luminance has interesting theoretical implications. It has been suggested that it is caused by the relative activity of separate chromatic and achromatic processes, the achromatic signal being dominant at high intensities. Our model, however, points to an alternative unifying explanation for the Bezold–Bru¨cke phenomenon and the related changes of chromatic strength. These phenomena, and the Abney effect, were here accounted for by combining non-monotonic outputs of low-level opponent cells, a process that may take place at the cortical level. The non-monotonic, composite responses of the model of Figure 6.24(a) fit the same functions of luminance as do psychophysical chromatic strength and hue in Figure 6.22. These responses are modeled by a sum of activating and inhibiting cone inputs to opponent cells. There is no need to invoke an early achromatic mechanism to explain these results; both achromatic and chromatic response components can be derived from combinations of outputs from the same opponent cells. However, one would like to find the physiological substrate that isolates and processes the ‘achromatic component’, but this appears currently to be as difficult as localizing the physiological correlates to unique hues. The ideas presented here support the notion that, once a chromatic threshold is overcome at low luminance, it is the response magnitude of the combination of chromatic responses of opponent cells that alone sets the correlate for chromatic strength and hue. In addition, the excitatory profile across these color-selective cells probably contains the code for lightness and brightness attributes.

We can thus view the joint changes in chromatic strength and hue as a result of low-level, opponent cell activities and their combination at a higher, cortical level. Both chromatic and achromatic colors and intensity attributes are implicit (multiplexed) in the activity of these early cells, and cortical mechanisms may derive several perceptual attributes of the retinal image by analyzing such multiple converging and diverging pathways. The ‘cell opponent stage’ of Figure 6.24(a) suggests that, after the outputs of I- and D-channels have converged on some cortical mechanism, they may again be related in new opponent pathways. Theoretically, this is one simple possibility for arriving at an equidistant color space [see Figure 5.31 and

3

Figure 6.24 (a) Predictions made by the neural color vision model for the changes in perceived hue and color strength of eight stimuli of constant chromaticity (shown in c) when luminance ratio is raised from 0 (origin) to 0.1 (solid arrows) and beyond. The distance from the center of the diagram is proportional to model color strength, and the radial angle represents hue. The units for cell responses are scaled in such a way that they correspond to Munsell chroma. The model gives a similar picture of variations of hue and chromatic strength as Figure 6.22. (b) Extrapolation beyond the experimental data shows that for extremly high luminance ratios monochromatic stimuli will appear achromatic.