Ординатура / Офтальмология / Английские материалы / Seeing_De Valois_2000

74 Robert Shapely

FIGURE 9 Reverse correlation measurements of the time evolution of orientation tuning in macaque V1 neurons (Ringach, Hawken, and Shapley, unpublished results). Here the results are obtained by reverse correlation in the orientation domain. To study dynamics of orientation tuning we used as stimuli the set of sine gratings of optimal spatial frequency at many orientations (around the clock in 10 steps). The dynamical stimuli used consisted of a rapidly changing sequence of sinusoidal gratings. The responses of the neurons are the cross-correlations of their spike trains with the sequence of images—this gives the neurons’ orientation-tuning functions, as functions of time. In each graph, the horizontal axis goes from 0-180˚ orientation. This figure displays orientation tuning function for two neurons in macaque V1 layer 4B. Results for each cell occupy a di erent row, as marked. The five di erent panels for each neuron correspond to give di erent delays between neural response and stimulus onset. Note that already in layer 4B there is evidence for complex interactions in the orientation domain: rebound inhibition in both cells can be observed around 65-ms delay, and in cell 1, there is clear evidence for flanking inhibition around the peak at 55 ms. In cell 2’s response, there is an especially clear indication of an elevated response orthogonal to the main peak, at a later time, at 65 ms in this case.

feedback changes the neural activity of cortical cells significantly, to that same degree we are forced to question whether the idea of a receptive field is adequate to describe the behavior of a cortical cell. This then is the reason for considering recent work on the dynamics of orientation tuning (Ringach, Hawken, & Shapley, 1997), in which the importance of nonlinear cortical feedback has become evident.

D. Orientation Dynamics

To study dynamics of orientation tuning in simple cells, we (Ringach et al., 1997) used as stimuli sine gratings of optimal spatial frequency at many orientations (around the clock in 10 steps) and at several spatial phases (responses were averaged over spatial phase). The dynamical stimuli used consisted of a rapidly changing sequence of sinusoidal gratings. The responses of the neurons are measured as the cross-correlations of their impulse train outputs with the sequence of images—this gives the neurons’ orientation tuning functions, as functions of time. Thus the method used permitted the measurement of orientation tuning when the test stimuli are embedded within a rich stimulus set of temporally adjacent stimuli. We also

could check linearity of spatial summation directly by creating images that were linear combinations of the original set, and measuring whether the spatiotemporal correlation functions were changed by the process of superposition of stimuli—for a linear system, there would be no change. We studied neurons in all layers of macaque V1, keeping track of their locations through standard methods of track reconstruction (Ringach et al., 1997).

Our preliminary results indicate that most cells in the input layers 4C and have simple dynamics and are relatively broadly tuned for orientation. By simple dynamics I mean that after a time delay, the response simply has a single peak in time and, after the peak, simply relaxes back to baseline. However, cells in layers 2, 3, 4B, 5, and 6 showed complicated dynamics: rebound responses, sharpening of the orientation tuning with time, and/or transient peaks of activity at o -optimal orientation. Furthermore, we found that these neurons with complicated dynamics are usually much more sharply tuned in orientation. Examples of more sharply tuned cells with complicated dynamics are given in Figure 9, which displays orientation tuning functions, derived from subspace correlation measurements, for two neurons in macaque V1 layer 4B. Results for each cell occupy a di erent row, as marked. The five di erent panels for each neuron correspond to five di erent delays between neural response and stimulus onset. Note that already in layer 4B there is evidence for complex interactions in the orientation domain: rebound inhibition in both cells can be observed around 65-ms delay, and in cell 1, clear evidence for flanking inhibition around the peak at 55 ms. In cell 2’s response, there is an especially clear indication of an elevated response orthogonal to the main peak, at a later time, at 65 ms in this case.When we compare these measurements to orientation tuning curves obtained by conventional methods on the same neurons, we find good qualitative agreement on the peak orientation, but conventional measurements completely miss the dynamic aspects of the orientation tuning: rebound suppression at the peak orientation, and o -peak excitation later in the response. These results indicate that lateral interaction in the orientation domain is prevalent. The nature of this interaction is not yet understood, but peaks far from the main tuning peak, so evident in cell 2 at 65 ms, for instance, suggest that we must consider recurrent or feedback inhibitory mechanisms in a model for the cortex.

This work has implications about the importance of “the receptive field” as a valid way of understanding cortical function. The implication is that lateral interactions in the cortex shape all aspects of cellular functions. Much work supports this view but uses a very di erent paradigm: the study of cortical visual interactions across space, also known as “contextual e ects.”This includes the work on Center– Surround interactions (reviewed by Allman, 1985), orientation contrast (Knierim & Van Essen, 1992; Sillito, Grieve, Jones, Cudeiro, & Davis, 1995; Levitt & Lund, 1997), and figure–ground e ects (Lamme, 1995; Zipser, Lamme, & Schiller, 1996), among others. The work of Bullier and colleagues (reviewed in Salin & Bullier, 1995) indicates the importance of feedback connections from extrastriate areas into V1. This body of work suggests that the cells in the primary visual cortex are influ-

76 Robert Shapley

enced by lateral and feedback interactions in a very significant way, much more than was imagined when the receptive field concept was first applied to cortex. In perception of natural scenes, even more than in laboratory experiments on simplified stimuli, the e ects of context may play an important role in visual cortical cell responses. This must be quantified in future experiments.

It was a good first approximation to see whether we could treat the visual cortex like the retina, as a passive filter of visual information, which is how I view the receptive field concept. However, as researchers learn more and more about how the cortex works, this approximation seems less useful. It is not a lost cause, and we don’t have to give up the scientific analysis of the cortex just because the concept of “receptive field” that works well in the retina does not apply to cortex. Rather, we need to use other tools, perhaps nonlinear dynamical system theory, perhaps some di erent analytical tools, to comprehend what is going on in the cortex. It is also important to realize that understanding the network architecture of the cortex is critical for building models. In the future, theory and experiment, structure and function will all have to be integrated in a concerted scientific probe of the cortex’s contribution to visual perception.

References

Adelson, E. H., & Bergen, J. R. (1985). Spatiotemporal energy models for the perception of motion.

Journal of the Optical Society of America, Series A, 2, 284–299.

Allman, J. (1985). Stimulus-specific responses from beyond the classical receptive field: Neurophysiological mechanisms for local-global comparison in visual neurons. Annual Review of Neuroscience, 8, 407–430.

Ben-Yishai, R., Bar-Or, R. L., & Sompolinsky, H. (1995). Theory of orientation tuning in the visual cortex. Proceedings of National Academy of Sciences, 92, 3844–3848.

Bonds, A. B. (1989). Role of inhibition in the specification of orientation selectivity of cells in the cat striate cortex. Visual Neuroscience, 2, 41–55.

Burr, D. C., Ross, J., Morrone, & M. C. (1986). Seeing objects in motion. Proceedings of the Royal Society of London, series B, 227, 249–265.

Cleland, B. G., Dubin, M. W., & Levick, W. R. (1971). Simultaneous recording of input and output of lateral geniculate neurones. Nature, New Biology, 9, 191–192, 231.

DeAngelis, G. C., Ohzawa, I., & Freeman, R. D. (1993). Spatiotemporal organization of simple-cell receptive fields in the cat’s striate cortex. II. Linearity of temporal and spatial summation. Journal of Neurophysiology, 69, 1118–1135.

Derrington, D. M., & Lennie, P. (1984). Spatial and temporal contrast sensitivities of neurones in lateral geniculate nucleus of macaque. Journal of Physiology, 257, 219–240.

De Valois, R. L., Albrecht, D. G., Thorell, L. G. (1982). Spatial frequency selectivity of cells in macaque visual cortex. Vision Research, 22, 545–559.

De Valois, R. L., Morgan, H., & Snodderly, D. M. (1974). Psychophysical studies of monkey vision. 3. Spatial luminance contrast sensitivity tests of macaque and human observers. Vision Research, 14, 75–81.

Enroth-Cugell, C. (1993). The world of retinal ganglion cells. In D. K. Lam & R. M. Shapley (Ed.), Contrast sensitivity (pp. 149–179). Cambridge, MA: MIT Press.

Enroth-Cugell, C., & Pinto, L. H. (1972). Properties of the surround response mechanism of cat retinal ganglion cells and centre-surround interaction. Journal of Physiology, 220 (2), 403–439.

Enroth-Cugell, C., Hertz, B. G., & Lennie, P. (1977). Convergence of rod and cone signals in the cat’s retina. Journal of Physiology (London), 269, 297–318.

Enroth-Cugell, C., & Robson, J. G. (1966). The contrast sensitivity of retinal ganglion cells of the cat.

Journal of Physiology (London), 187, 517–552.

Enroth-Cugell, C., & Robson, J. G. (1984). Functional characteristics and diversity of cat retinal ganglion cells: Basic characteristics and quantitative description. Investigative Ophthalmalogy and Visual Science, 25, 250–267.

Hartline, H. K. (1940).The receptive fields of optic nerve fibers. American Journal of Physiology, 130, 690– 699.

Hawken, M. J., & Parker, A. J. (1990). Detection and discrimination mechanisms in the striate cortex of the Old World monkey. In C. Blakemore (Ed.), Vision: Coding and e ciency. Cambridge, UK: Cambridge University Press.

Hochstein, S., & Shapley, R. (1976). Quantitative analysis of retinal ganglion cell classification. Journal of Physiology (London), 262, 237–264.

Hubel, D. H., & Wiesel, T. N. (1962). Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. Journal of Physiology (London), 160, 106–154.

Hubel, D. H., & Wiesel,T. N. (1968). Receptive fields and functional architecture of monkey striate cortex. Journal of Physiology (London), 195, 215–243.

Jones, J. P., & Palmer, L. A. (1987). An evaluation of the two-dimensional Gabor filter model of simple receptive fields in cat striate cortex. Journal of Neurophysiology, 58, 1233–1258.

Kaplan, E., & Shapley, R. M. (1984). The source of S(slow) potentials in the mammalian LGN. Experimental Brain Research, 55, 111–116.

Knierim, J. J., & van Essen, D. C. (1992). Neuronal responses to static texture patterns in area V1 of the alert macaque monkey. Journal of Neurophysiology, 67, 961–980.

Ku er, S.W. (1953). Discharge patterns and functional organization of mammalian retina. Journal of Neurophysiology, 16, 37–68.

Lamme, V. A. (1995). The neurophysiology of figure-ground segregation in primary visual cortex. Journal of Neuroscience, 15, 1605–1615.

Levitt, J. B., & Lund, J. S. (1997). Contrast dependence of contextual e ects in primate visual cortex.

Nature, 387, 73–76.

McLean, J., Raab, S., & Palmer, L. A. (1994). Contribution of linear mechanisms to the specification of local motion by simple cells in areas 17 and 18 of the cat. Visual Neuroscience, 11, 271–294.

Ma ei, L., & Fiorentini, A. (1973). The visual cortex as a spatial frequency analyser. Visual Neuroscience, 13, 1255–1267.

Movshon, J. A., Thompson, I. D., & Tolhurst, D. J. (1978). Receptive field organization of complex cells in the cat’s striate cortex. Journal of Physiology (London), 283, 79–99.

Parker, A. J., & Newsome, W. T. (1998). Sense and the signal neuron: Probing the physiology of perception. Annual Review of Neuroscience, 21, 227–277.

Reid, R. C., & Shapley, R. M. (1992). Spatial structure of cone inputs to receptive fields in primate lateral geniculate nucleus. Nature, 356, 716–718.

Reid, R. C., Soodak, R. E., & Shapley, R. M. (1987). Linear mechanisms of directional selectivity in simple cells of cat striate cortex. Proceedings of the National Academy of Sciences, 84, 8740–8744.

Reid, R. C., Soodak, R. E., & Shapley, R. M. (1991). Directional selectivity and spatiotemporal structure of receptive fields of simple cells in cat striate cortex. Journal of Neurophysiology, 66, 509 – 529.

Reid, R. C., Victor, J. D., Shapley, R. M. (1997). The use of m-sequences in the analysis of visual neurons: linear receptive field properties. Visual Neuroscience, 14, 1015–27.

Ringach, D. L., Hawken, M. J., & Shapley, R. M. (1997). The dynamics of orientation tuning in the macaque monkey striate cortex. Nature, 387, 281–284.

Rodieck, R. W. (1965). Quantitative analysis of cat retinal ganglion cell response to visual stimuli. Vision Research, 5(11), 583–601.

78 Robert Shapely

Rodieck, R. W., & Stone, J. (1965). Analysis of receptive fields of cat retinal ganglion cells. Journal of Neurophysiology, 28, 832–849.

Sakai, H. M., & Naka, K. (1995). Response dynamics and receptive-field organization of catfish ganglion cells. Journal of General Physiology, 105, 795–814.

Sakuranaga, M., Ando, Y., & Naka, K. (1987). Dynamics of the ganglion cell response in the catfish and frog retinas. Journal of General Physiology, 90, 229–259.

Salin, P. A., & Bullier, J. (1995). Corticocortical connections in the visual system: Structure and function. Physiology Review, 75, 107–154.

Shapley, R. M. (1994). Linearity and non-linearity in cortical receptive fields. In Higher order processing in the visual system: Ciba Symposium (vol. 184, pp. 71–87). Chichester: Wiley.

Shapley, R., & Victor, J. D. (1979). Nonlinear spatial summation and the contrast gain control of cat retinal ganglion cells. Journal of Physiology (London), 290, 141–161.

Sillito, A. M. (1975).The contribution of inhibitory mechanisms to the receptive field properties of neurones in the striate cortex of the cat. Journal of Physiology (London), 250, 305–329.

Sillito, A. M., Grieve, K. L., Jones, H. E., Cudeiro, J., & Davis, J. (1995). Visual cortical mechanisms detecting focal orientation discontinuities. Nature, 378, 492–496.

Skottun, B. C., De Valois, R. L., Grosof, D. H., Movshon, J. A., Albrecht, D. G., Bonds, A. B. (1991). Classifying simple and complex cells on the basis of response modulation. Vision Research, 31, 1079–1086.

Somers, D. C., Nelson, S. B., & Sur, M. (1995). An emergent model of orientation selectivity in cat visual cortical simple cells. Journal of Neuroscience, 15, 448–465.

Soodak, R. E., Shapley, R. M., & Kaplan, E. (1991). Fine structure of receptive field centers of X and Y Cells of the cat. Visual Neuroscience, 15, 5448–5465.

Spekreijse, H. (1969). Rectification in the goldfish retina: Analysis by sinusoidal and auxiliary stimulation. Vision Research, 9, 1461–1472.

Spitzer, H., & Hochstein, S. (1985). Simpleand complex-cell response dependences on stimulation parameters. Journal of Neurophysiology, 53, 1244–1265.

Tolhurst, D. J., & Dean, A. F. (1990). The e ects of contrast on the linearity of spatial summation of simple cells in the cat’s striate cortex. Experimental Brain Research, 79, 582–588.

Tolhurst, D. J., & Heeger, D. J. (1997). Contrast normalization and a linear model for the directional selectivity of simple cells in cat striate cortex. Visual Neuroscience, 14, 19–25.

Victor, J. D. (1988). The dynamics of the cat retinal Y cell subunit. Journal of Physiology (London), 405, 289–320.

Victor, J. D. (1992). Nonlinear systems analysis in vision: Overview of kernel methods. In R. Pinter & B. Nabet (Eds.), Nonlinear vision: Determination of neural receptive fields, function and networks (pp. 1– 37). Cleveland, OH: CRC Press.

Victor, J. D., & Shapley, R. (1979). Receptive field mechanisms of cat X and Y retinal ganglion cells.

Journal of General Physiology, 74, 275–298.

Vogels, R., & Orban, G. A. (1991). Quantitative study of striate single unit responses in monkeys performing an orientation discrimination task. Experimental Brain Research, 84, 1–11.

Watson, A. B., & Ahumada, A. J., Jr. (1985). Model of human visual-motion sensing. Journal of the Optical Society of America, Series A, 2, 322–341.

Zipser, K., Lamme,V. A., & Schiller, P. H. (1996). Contextual modulation in primary visual cortex. Journal of Neuroscience, 15, 7376–7389.

80 Wilson S. Geisler and Duane G. Albrecht

sity of the cells along all of the stimulus dimensions. Thus, for example, the intensity response functions of cones are all very similar from cell to cell, whereas the contrast response functions of cortical neurons are quite di erent from cell to cell.

A number of di erent explanations have been proposed for this emergence of stimulus selectivity along the visual pathway. One explanation is that this progressive selectivity is part of a hierarchical process, ultimately leading to single neurons that respond uniquely to specific real-world objects (Barlow, 1972, 1995): “The Neuron Doctrine.” A second explanation is that this selectivity reflects a low redundancy code that is well matched to the statistics of natural images (Barlow, 1961, 1989; Field, 1987; Olshausen & Field, 1997): “Sparse Coding.”

A third explanation is that this selectivity is a critical step in segregating objects from their context: “Object Segregation.” Objects of interest within the natural environment are generally located within a very complex context of other objects. In order to recognize an object of interest, the parts of the object must be separated from the parts of other objects. For example, to recognize a longhorn bull behind a barbed wire fence, it is necessary to separate the image features that define the wire fence from those that define the bull. Fortunately, context is much less of a problem on a local scale; it is relatively easy to identify the orientation, position, and color of local image contours. The selectivity of visual cortical neurons permits recognition of these local image properties, thus allowing subsequent grouping mechanisms to bind together the contours that define the fence separately from those that define the bull.

These three di erent explanations for stimulus selectivity are not necessarily incompatible. Object segregation or sparse coding could be a first step in producing single neurons tuned to real-world objects. On the other hand, the processing following object segregation or sparse coding could be highly distributed. Further, having neurons matched to the statistics of the natural environment must surely be advantageous for both sparse coding and object segregation, given the constraint of limited resources. However, the goals of sparse coding and object segregation are quite di erent; hence, the specific selectivities, and how they are implemented, could well be di erent. It is important to keep all three explanations in mind, given that one’s theoretical viewpoint can substantively influence the direction of future research.

In this chapter we will rely upon a wealth of psychophysical and physiological research to develop the topic of spatial vision with two themes in mind. The first theme concerns how stimulus selectivity develops along the visual pathway.The second theme concerns how the anatomical and physiological mechanisms of stimulus selectivity contribute to visual performance, and ultimately, object recognition and scene interpretation.

II. SINGLE NEURONS AND BEHAVIOR

A. Levels of Analysis

Research in perception and cognition has been performed at many di erent levels of analysis: the organism as a whole, the subregions of the brain, the individual neu-

rons, and the components of individual neurons.The main focus in this chapter will be on the responses of the organism as a whole (i.e., visual psychophysics) and on the responses of the individual neurons (i.e., single neuron electrophysiology).

Focusing on the behavioral responses of the organism as a whole is justified because the ultimate goal of perception and cognition research is to understand behavior. Indeed, behavioral abilities define the phenomena of interest in perception and cognition, and hence provide the groundwork for directing and interpreting the measurements at all the other levels.

Specifically, behavioral findings often serve as a guide to neurophysiological and anatomical studies. To begin with, note that there is an enormous amount of information available in the visual stimulus. The visual system is not capable of transmitting and utilizing all of this information because it has finite neural resources. Some of this information will be used by the nervous system and some of this information will be lost. Behavioral performance can tell us what information is used and what is lost. If behavioral performance indicates that a certain kind of information is lost, then there should be some stage in the neural processing where the loss occurs, and neurophysiological measurements should be able to demonstrate this loss. If behavioral performance indicates that a certain kind of information is not lost, then the information must be preserved at every sequential level of the nervous system between the stimulus and the behavior, and neurophysiological measurements should be able to demonstrate the presence of the information.

Consider, for example, the task of color discrimination. There is an enormous number of di erent wavelength distributions that can be presented to a human observer. Behavioral studies have shown that most of these wavelength distributions cannot be distinguished. This fact indicates that substantial chromatic information has been lost in the nervous system. Neurophysiological measurements have demonstrated that most of this loss occurs at the level of the photoreceptors. On the other hand, behavioral studies have also shown that su cient chromatic information is transmitted to support trichromacy. Neurophysiological studies have used this fact as a basis for designing chromatic stimuli and locating the neural systems responsible for trichromatic color discrimination.

The justification for focusing on the responses of individual neurons is somewhat more involved. All of the information transmitted by a neuron is contained in the temporal sequence of action potentials generated by the neuron; this information represents the net sum of all the synaptic inputs to the neuron, plus all of the prior neural information processing that those inputs represent. Because of the all-or-none property of action potentials and their short duration, it is relatively easy to accurately measure the temporal sequence (with a resolution finer than milliseconds) and hence all of the transmitted information (the noise in the recording instrument has little or no e ect on the accuracy of the measurements). This fact makes it possible to quantitatively relate physiological responses to perception and cognition in a way that is not possible by current methods for measuring the responses of subregions of the brain or subcomponents of individual neurons.

82 Wilson S. Geisler and Duane G. Albrecht

When measuring the responses of a subregion in the brain using one of the neuroimaging techniques, the responses from the individual neurons are measured indirectly through metabolic activity, with relatively poor spatial and temporal resolution. Although certain questions can be answered with these techniques, it is very di cult to know how to relate the measurements to behavioral performance in a quantitative fashion.1 Similarly, although the responses of some particular subcomponents of individual neurons can be measured (such as the activity of a single synapse), a complete characterization of all the components is not possible at this point in time, and thus it is di cult to relate the responses of the subcomponents to the responses of the neuron as a whole or to behavioral responses.

There are, of course, limitations to the inferences that can be drawn about visual information processing by measuring action potentials from single neurons. To begin with, not all neurons at all levels of the visual system produce action potentials. However, these neurons are generally local circuit neurons. Neural communications that extend over distances greater than a few millimeters are carried via action potentials. Thus, measurement of action potential responses should be su - cient to characterize the output of the information processing that is transmitted from one region of the brain to another. Further, using intracellular recording (and similar techniques) it is generally possible to measure the information transmitted by the voltage potential responses of nonspiking neurons, with reasonable accuracy. A second limitation concerns the variation in sampling probabilities that may occur with di erent cell sizes and di erent subpopulation densities. However, anatomical measurements of subpopulation densities can help to lessen this uncertainty; and, increasing the sample size of measured neurons can increase the probability of obtaining measurements from small subpopulations. A third limitation is that it is not possible to record the individual responses of many neurons simultaneously. However, by measuring the responses of many neurons, one at a time, to the same set of stimuli, it is possible to make inferences about the information processing of

1 Neuroimaging techniques are useful because they allow simultaneous measurement of large populations of neurons and they can be performed on human beings (without invasive procedures). However, there are a number of factors which make neuroimaging data di cult to interpret. First, there is considerable uncertainty about the degree to which the measurements correspond to neural activity, because most techniques measure neural responses indirectly via the level of metabolic activity. Second, because the signals are summed over a relatively long temporal interval (on the order of seconds) it is di cult to obtain useful measurements of the rapid neural information processing that ordinarily occurs during perception and cognition (on the order of milliseconds). Third, because the signals are summed over space and time, many di erent patterns of neural activity within a given region could produce exactly the same summed level of metabolic activity, and thus it is impossible to quantitatively characterize the information transmitted by the population of neurons. Fourth, neural circuits for a given behavioral information-processing task are identified when the di erence in metabolic activity between a baseline condition and a test condition exceeds a certain criterion level. Thus, to the extent that the neural circuits for a given information-processing task are distributed throughout di erent regions of the brain, or to the extent that the neural circuits for di erent information-processing tasks are intermingled, the changes in metabolic activity will fail to exceed the criterion. (For further discussion of this issue see Wandell, 1999.)

the population as a whole. These inferences depend upon knowing the degree of statistical independence of the responses across neurons. New techniques that allow recording from two or three neurons simultaneously suggest that the responses are relatively uncorrelated.

B. Linking Hypotheses

The ultimate goal of measuring the anatomical and physiological properties of the neurons in the visual pathway is to understand behavioral performance. For example, it is possible to measure the responses of visual cortex neurons as a function of spatial frequency and contrast, but ultimately we would like to understand how the responses of these neurons contribute to spatial frequency discrimination, contrast discrimination, and other behavioral performances. To understand how neural responses contribute to behavioral performance, it is necessary to have a linking hypothesis so that predictions for behavioral performance can be generated from the responses of the neurons at a particular level of the visual system (Brindley, 1960; Teller, 1984). A linking hypothesis can be regarded as a specific model of the neural processing which occurs subsequent to the selected level.

Two general categories of linking hypotheses have been widely considered. One consists of the Bayesian ideal-observer models; these models assume that all of the information available in the responses of the neurons is extracted and optimally combined with knowledge about stimulus probabilities. The other consists of simple pooling models; these models assume that the responses of the neurons are combined using a simple operation such as simple summation, Minkowski summation, or “winner take all.”

An advantage of the Bayesian ideal-observer models is that all of the available neural information is used, and hence if the behavioral performance exceeds the predicted performance, then one can be certain there is something missing in the neurophysiological data (e.g., an undetected subpopulation of neurons). Conversely, if the behavioral performance falls below that of the ideal observer, then either there must be something inaccurate about the neurophysiological data, or there must be some loss of information in subsequent neural stages. For some situations, the ideal combination of the neural responses is relatively simple but in general, it is complex and varies from situation to situation. Although the human visual system is sophisticated, it has many complex tasks to perform (with limited resources) and hence it probably does not perform any individual task in the optimal fashion.

An advantage of the simple pooling rules is that they are easy to understand, and they could be implemented with simple neural circuits. However, for many situations, the simple pooling rules probably do not adequately represent the complexity of later neural processing.

Given a linking hypothesis, it is then possible to compare neural performance and behavioral performance along stimulus dimensions of interest. For some linking hypotheses it is possible to compare absolute levels of performance. One example