Luminance

Receptors

ÓDirection sensitive cell

Figure 8.8 The structure of a simple model that explains a cell’s selectivity for a certain direction of movement. The cell at the bottom of the figure receives signals from a set of cells with excitatory and inhibitory inputs; these intermediate cells are inhibited by a bright object (a luminance increment) that moves from the right towards the left (an inhibitory signal is projected ahead of the moving edge), and they are activated by a bright object (a luminance increment) approaching from the opposite direction.

intermediate cells just ahead of the border and keep them inhibited as long as the light is activating the receptors above. Thus the bottom I-cell, summing the inputs from the previous layer, will be activated maximally by one direction of movement and inhibited by movement in the opposite direction.

Vision in depth

Perception of depth in three-dimensional space requires cells that code for disparity, i.e. or the degree of non-correspondence between same scene images in the two eyes (Figure 3.7). Of course, this cannot happen before the signals from the two eyes are compared. This comparison is made by the binocular cells of the cortex. We find such cells in layer 4C in V1 and also in area V2. It seems that MC cells and the complex cells of V1 have an important role in stereopsis (see also Figures 1.2 and 3.9), but V3 and adjoining areas may also subserve depth vision.

The binding problem

Different aspects of vision, such as the perception of color, movement and the expression in a person’s face would seem to arise in different, specialized parts of the brain. Damage to one of these areas, for instance by stroke, can result in bizarre perceptual phenomena such as, for instance, color without form, or form vision

without movement perception. Considering the reports of such and similar phenomena, one is led to conclude that, in order to perceive something as a meaningful whole, many specialized areas of the brain must contribute in parallel to the percept.

After several feature dimensions have been multiplexed in the activity of cones and retinal ganglion cells, is it reasonable to assume a separation at a later stage in which each cortical area processes a different object feature, or a set of features? Several aspects of the visual world need to be sorted out: first, there is the information about the objects themselves and their surface properties (their size, shape, color, texture, whether they are familiar or unfamiliar); second, the interrelations between the objects making up the visual scene (constancy of size, color, etc., figure/ground, grouping, movement speed and direction, etc.); and third, their relation to the observer’s space (spatial position, orientation, etc.). Are all these dimensions and their sub-modalities separated out at some stage in the visual process? One might argue that the large degree of selectivity for object features in neurons in the different cortical areas and modules confers a certain identity on cells belonging to each module. This identity defines one or several object-related properties. Do these properties need to be physically integrated at some later stage, in order to correlate and compare them with all other aspects of the visual scene, or are they somehow autonomous? In a hierarchical model, object attributes may be processed at the lowest possible level and the result passed on to higher neural levels (Lennie, 1998), such as the information about light increments and decrements in I- and D-channels. If this information were only to converge and be combined linearly at a later stage, the feature identity would be lost. If each neuron in an area, say V1, were sensitive only to a small region of a multidimensional feature space, a composite object might be represented by an abstract multidimensional vector, as illustrated in Figure 8.9. These vectors might be more or less fuzzy, ranging from sharply defined arrows in Figure 8.9(a) for segregated pathways to diffuse, overlapping distributions of multiplexed stimulus features in Figure 8.9(b).

The segregated pathway hypothesis that the perceived attributes of a composite stimulus are associated with distributed neural activity of many separate modules and areas of the brain, each dealing with a particular sub-modality or property (Livingstone and Hubel, 1988), leaves us with the problem of explaining how all the distinct features are brought together to be associated with the same object. This is referred to as the ‘binding problem’ (Singer, 1993). In recent years, several models for such integrative processes in the brain have been proposed.

Properties such as color, movement, form and orientation are perceived as belonging to an object. Since color is conveyed by the relatively slow opponent channels while movement is signaled by the faster, phasic MC pathways, the hypothesis of parallel processing might lead us to expect that color will be separated from form with the color lingering behind the moving object. Such strange situations have, in fact, been reported for patients with brain injuries. However, since this form of separation does not occur normally, one may assume that visual cohesion relies on coincidence detectors that can signal some sort of synchrony between the distributed

Figure 8.9 (a) A multidimensional vector space where each dimension represents a set of object features. Each vector represents the neural responses to one particular feature. The orientation vector represents the activity of all 20 or so orientation-selective cells, while the color vector represents the activity of Increment and Decrement cells responding along the different cardinal directions of cone space. A complex object is represented by a combination of these and other vectors to form an abstract resultant state vector represented by the thick arrow. This resultant vector represents an abstract response state of the system and need not necessarily be represented by a particular cell, as in the convergent structure of a hierarchical model. (b) serves to illustrate the possibility that the vectors representing feature dimensions need not be sharply defined, but that they can be relatively broadly tuned. A particular object may thus be represented by the response state of an ensemble of fuzzy vectors.

neural activities. The cortical processes in the separate areas, the correlates for form, color, movement and direction, must run in step.

The populations of nerve cells whose responses correlate with the object’s various attributes must be activated simultaneously and run in synchrony. It has been suggested that for cells that respond to different aspects of the same retinal location, or of the same object, the responses oscillate in phase, and that the activity of cells

responding to another object oscillates out of phase with the first. Perhaps cells that are spatially distributed but nevertheless ‘bound together’ by virtue of being associated with the same complex stimulus somehow achieve temporally synchronized firing. Synchrony of cell firing might bind distributed activity in a temporary functional unit. It has been speculated that such synchronization could be mediated by oscillatory brain activities in the 30–70 Hz range that have been discovered in many animals. However, since the response properties of cells become increasingly complex the further downstream post-retinal processing occurs, we cannot exclude the possibility that parallel and serial processing are followed by neural integration, leading to some degree of hierarchical convergence.

Mirror neurons

How would you go about constructing a neural system that would make it possible to interpret and understand the actions of other beings? The answer may lie in the discovery of so-called ‘mirror cells’ (Rizzolatti et al., 1996). While recording from cells in the ventral premotor area of a monkey, it was found that some cells fired when the monkey saw the experimenter or another monkey taking food from a plate to put into their own mouth. The strange thing about this was that these were the same cells that responded when the observer monkey himself was allowed to make a grasping movement and feed itself some raisins. Further experimentation led to the notion that these cells were ‘mirror cells’, responding to the observation of a familiar action carried out by somebody else. How did these cells recognize a vicariously performed action? Might these findings be extrapolated to suggest that similar responses in mirror neurons elsewhere in the neural system can bring about a correspondence in state of mind and thereby constitute the neural correlate of recognizing emotions and even empathy? Functional MRI experiments on mental imagery with closed eyes have suggested that the same brain structures are indeed active when conciously viewing a visual event and when recalling and imaging the same event.

The ‘split brain’

The right half of the body is governed by the left half of the brain, and the right field of view is projected (via the left retinal hemifield) to the left side of the visual cortex. In some interesting psychophysical experiments, R. Sperry (Nobel Laureate in 1981) analyzed the function of the two brain halves in patients with ‘split brains’. These were patients suffering from epilepsy, and in whom the connections between the two halves had been cut in order to prevent the spread of epileptic seizures from one half of the brain to the other half. These ‘split brain’ patients behaved relatively