Ординатура / Офтальмология / Английские материалы / Seeing_De Valois_2000

284 Andrew Derrington

which, by generating a positive response when the subunit’s response would be negative, convert the ensemble from a half-wave rectified squarer to a full squarer. This di erence provides a simple explanation of the fact that the simple cell possesses and the complex cell lacks a quasi-linear receptive field wherein the response to simple stimuli readily predicts the main features of the response to more complex and larger stimuli (Hubel & Wiesel, 1962, 1968). The fact that the complex cell’s output contains only terms that correspond to the squares of the outputs of quasi-linear filters means that the linear component of its output is zero.

Another way of expressing the di erence between the models is to say that the direction-selective complex cell receptive field model can be decomposed into two simple cell receptive fields. The simple cell receptive fields are aligned so that the linear components of their responses cancel one another and the nonlinear components reinforce one another. Consequently, the linear response of the complex cell receptive field should be zero; it should give the same response to increments and to decrements of luminance throughout its receptive field.

Although the qualitative similarity between the responses of direction-selective cortical neurones and those of the motion-energy filter is encouraging, the number of cells that have been examined in detail is small. At this stage it would be premature to exclude other model motion-detectors; however, one qualitative feature of motion-selective neurones in striate cortex that has been very widely confirmed is that cells are selective for the orientation and direction of motion of elongated contours. Experiments with random 2-D textures, which inevitably contain large numbers of components of di erent spatial frequency and orientation, have yielded confusing results, which at least in some cases can be explained by the mixture of excitatory and inhibitory e ects of the di erent orientation and spatial-frequency components of the random pattern (Morrone, Burr, & Ma ei, 1982). When they are stimulated with simpler 2-D patterns composed by summing only two sinusoidal components, cells in monkey striate cortex respond when the pattern is orientated so that one of its components falls within the range of orientations to which the cell is sensitive and moves in the cell’s preferred direction, even when, as is inevitably the case when a pattern consists of two di erently oriented components both of which are moving, the pattern moves along a di erent axis of motion to the component.

Cortical area middle temporal (MT), which is the destination of some of the projections from striate cortex and which appears to be specialized for analyses of motion (Dubner & Zeki, 1971) contains cells that respond selectively to the axis of motion of a pattern, regardless of the orientations and axes of motion of its components (Movshon, Adelson, Gizzi, & Newsome, 1985). In the conscious monkey, cells in area MT respond to the motion of patterns of dots in which the direction of motion perceived is determined by a small proportion of the dots that move coherently in a consistent direction while the directions of motion of the remaining dots are randomly distributed. The proportion of dots that must move coherently in order to excite a neurone is comparable to the proportion needed to reach

the monkey’s behavioral threshold for reporting motion in a particular direction (Britten, Shadlen, Newsome, & Movshon, 1992), supporting the idea that visual area MT is crucially implicated in the business of seeing motion.

Consideration of the early processing of moving stimuli in the mammalian visual cortex supports the idea that the motion of 2-D stimuli is analyzed in two stages. In the first stage the motion of orientated components, which will not, in general, be moving along the same axis as the stimulus, is analyzed by a range of sensors, each selective for motion along an axis orthogonal to its preferred orientation. In the second stage, the signals from these oriented 1-D sensors are combined to compute the axis and speed of motion of the stimulus as a whole. Two-stage analyses of motion have also been used in analyzing television pictures of objects in motion. In the first stage the speed of motion of luminance gradients in the image, in each case measured along the axis along which the gradient is steepest, is computed; in the second stage the signals from di erent parts of the image that constitute the same object are combined to compute the correct 2-D axis of motion and the speed of motion (Fennema & Thompson, 1979). The next section deals with the relationships between 1-D and 2-D analyses of the motion of 2-D patterns.

IV. INTEGRATING MOTION SIGNALS FROM DIFFERENT AXES: TWO-DIMENSIONAL VECTORS

A. What Is the Problem in Going from 1-D to 2-D Motion?

The problem the visual system faces in calculating the axis and speed of motion of a 2-D pattern or object from the 1-D signals generated by mechanisms that sense the motion of its oriented spatial frequency components or features has been exemplified in at least three di erent ways: (a) as a problem of viewing a stimulus through an aperture that only reveals some of its features; (b) as a problem of calculating the motion of an object from the motions of its oriented features; and (c) as a problem of interpreting signals from orientation-selective motion sensors or receptive fields. Each of these approaches emphasizes di erent aspects of the general problem, so I shall deal with each of them briefly before discussing the problem in general terms.

The aperture problem presents the 1-dimensional motion signal as a consequence of viewing an extended object through an aperture that only shows a small part of the object in which there is only one orientation present. Naturally, the only motion that can be seen through this aperture is orthogonal to the orientation of the visible feature. Apertures that reveal features that have di erent orientations but are part of the same object will, of course, reveal motions in di erent directions. Figure 14a and b show two examples of the aperture problem. Apertures on each of the sides of a moving rectangle show features moving in di erent directions, neither of which coincides with the true axis of motion of the rectangle unless the axis of motion is parallel to one of its sides.

Localized analyses of motion carried out on an entire image can also produce a phenomenon very similar to the aperture problem. In the absence of features, there

286 Andrew Derrington

(a,b) The edges of the tilted square appear to move along an axis orthogonal to their orientation even though the square is moving in a di erent direction. (c) The plaid has horizontally and vertically oriented features but its components (d & e) are obliquely oriented gratings.

is no motion signal and where features are present the local motion signal tends to be strongest along an axis orthogonal to the local orientation of the feature. When speed is computed from local spatial and temporal gradients, the computation is most easily carried out along the axis of steepest spatial gradient, in which case it computes the motion along the axis orthogonal to the feature that generates the gradient (Fennema & Thompson, 1979). Each of the (minuscule) areas over which the spatial and temporal gradients are analyzed constitutes an aperture with its attendant aperture problem.

Orientation-selective motion filters operate on limited regions of the image, which means that they are susceptible to the aperture problem. In addition they are selective for orientation: each filter senses only features of its preferred orientation and consequently only signals motion along an axis orthogonal to that orientation. The fact that each filter is orientation-selective means that it can sense the motion of oriented spatial frequency components that do not themselves give rise to image

features, as when two sinusoidal gratings of di erent orientations are added together to produce a plaid pattern. In the example shown in Figure 14c, the two components that make up the plaid pattern are oblique, 30 either side of vertical, but all the features—edges, dark and light lines—are either vertical or horizontal. Motion of the pattern would potentially give rise to motion signals in sensors tuned to all these orientations. The visual system then has the problem of computing the true motion of the pattern from the signals generated in the various filters.

In all three cases outlined above, the problem is that the local motion measurements give estimates of velocity along the axis of the oriented motion detector or along the axis orthogonal to the local feature. Consequently, the magnitude of these 1-D motion vectors is given by the product of the true motion vector and the cosine of the angle between the true 2-D motion vector and the 1-D motion vector. So

where is the magnitude of the 1-D vector along the axis , V is the magnitude, and is the direction of the 2-D vector. The problem for the visual system is to estimate V and from measurements of along di erent axes, that is, for di erent values of . The values of are given by the local orientation of the features in the image (or are inherent properties of the oriented motion filters), the values ofare measured from the moving image. Equation 4 shows that one measurement of constrains the relationship between V and . Two measurements along di erent axes (i.e., with di erent values of ) are su cient to specify both V and by the intersection of constraints, as illustrated graphically in Figure 5.

Thus although 1-D motion vectors are ambiguous cues to a pattern’s true (2-D) velocity because each 1-D vector is consistent with a range of 2-D vectors, any two 1-D vectors give enough information to resolve the 2-D velocity (Fennema & Thompson, 1979). The 1-D vectors can be derived from spatial frequency components in the pattern or from features produced by local or global analyses of di erent sorts (Adelson & Movshon, 1982; Derrington & Badcock, 1992; Derrington, Badcock, & Holroyd, 1992; Movshon et al., 1985;Wilson, Ferrera, & Yo, 1992).The next section deals with experiments aimed at showing which 1-D vectors are used by the visual system and how they are combined.

B. How Does the Visual System Compute 2-D Motion from 1-D Motion Vectors?

1. Plaid Pattern Experiments and the Intersection of Constraints

Arguably the simplest 2-D pattern with which to compare 1-D and 2-D motion analysis is the plaid pattern, made by summing two sinusoidal gratings, which has been extensively used in vision research since its introduction by Adelson and Movshon (1982). When the two component gratings are very di erent in contrast, spatial frequency, or temporal frequency, they are seen as separate patterns that slide over one another. However, when their spatial frequency, temporal frequency, and

288 Andrew Derrington

contrast are similar, the gratings cohere perceptually to form a rigidly moving pattern whose speed and direction of motion is determined from the speeds and directions of motion of the component gratings by the intersection of constraints.

The motion of the plaid pattern could also be derived in other ways. Local features in the plaid, gradients, peaks, and troughs in its luminance profile can be used to derive 1-D motion vectors. Any two di erent 1-D vectors allow the 2-D vector to be calculated through the intersection of constraints (Figure 5) (Adelson & Movshon, 1982). Another possibility is that correspondence-based mechanisms could be used to track plaid features in two dimensions (Adelson & Movshon, 1982). However two sets of experiments in which the motion processing of plaid patterns can be predicted from the way in which the plaids’ component gratings are processed suggests that the visual system calculates the motion of the plaid from the motion signals generated by its component gratings.

First, the precision with which subjects can estimate the speed of motion of a plaid appears to be predicted from the speed discrimination thresholds for its component gratings rather than from speed discrimination thresholds for other patterns superficially more similar to the plaid (Welch, 1989). Welch used a plaid made from gratings of the same spatial and temporal frequency. The speed of such a plaid relative to the speed of its component gratings increases with the angle between the gratings. This can be seen from the fact that the component speed is related to the plaid speed by the cosine of the angle between the plaid axis of motion and that of the component (equation 4).

Figure 15 shows Weber fractions for speed discrimination (discrimination threshold divided by speed) obtained with a plaid that moves approximately five times faster than its components compared with thresholds obtained with a grating of the same spatial frequency as the components. When plotted against the velocity of the component grating, both sets of Weber fractions show a distinctive dip at a speed of about 2 /sec.

When the data are plotted against the speed of the pattern, the dip for the plaid occurs at a speed five times higher. Welch argues that the fact that the data coincide when they are plotted as a function of component speed suggests that the visual system analyzes the plaid’s speed by analyzing the speeds of its components. In an ingenious control experiment, she showed that if the angle between the plaid’s components varies randomly from trial to trial, it is very di cult to discriminate the speed of its components when they are presented together in the form of a plaid, but not if they are presented alone. This suggests that although the visual system’s estimate of the speed of the plaid may depend on its estimates of the speeds of its components, the speeds of the components cannot be used directly to make perceptual discriminations (Welch, 1989).

Another piece of evidence that the motion of plaid patterns is computed by the visual system from the perceived motions of their components comes from experiments in which the perceived speed of one of the components of a plaid pattern is reduced either by presenting it with lower contrast (Stone, Watson, & Mulligan,

(a) The Weber fraction for speed discrimination of a plaid is plotted against the speed with which the pattern moves; there is a clear minimum at about 10 deg/sec. When the Weber pattern for a grating is plotted it reaches a minimum at 10 deg/sec. (b) The Weber fraction of the grating and the plaid are plotted in against the speed of the sinusoidal components. The minima for the two patterns now coincide at 2 deg/sec. (Data replotted with permission of the author and from Nature, 337, Welch, L., The perception of moving plaids reveals two motion-processing stages, 734–736. Copyright, 1989, MacMillan Magazines Limited.)

1990) or by adapting to a moving grating of the same spatial frequency and orientation presented in the same retinal position. The reduction in the perceived speed of the plaid’s component changes the perceived axis of motion in the same way that it would have changed if the real speed of the component had been reduced. As Figure 16 shows, a plaid in which one of the components has its apparent velocity reduced by a motion aftere ect appears to move in its original direction if the actual

290 Andrew Derrington

Aftere ect of viewing a moving grating on the perceived speed of a 2 c/deg grating and on the perceived direction of motion of a plaid pattern. The squares show the percentage of trials on which a test grating was judged to be moving faster than a standard grating moving at a speed of 0.38 deg/ sec, plotted as a function of the speed of the test grating. Open squares show results obtained without adaptation; filled squares show results obtained while the patch of retina on which the standard grating was presented was being adapted to a grating moving in the same direction at 0.5 deg/sec. The circles show the percentage of trials on which a plaid pattern containing a 2 c/deg grating moving 45 upwards at 0.38 deg/sec and a 2 c/deg grating moving 45 downwards at variable speed was judged to be moving downwards as a function of the speed of the downward-moving component. Empty circles show results obtained without adaptation; the plaid is judged to be moving horizontally when the two components have equal speed. Filled circles show results obtained during adaptation to a grating moving 45 upwards at 0.5 deg/ sec; the plaid appears to move horizontally when the upward-moving grating has its speed reduced to match the perceived speed of the upward-moving grating. (Data replotted from Vision Research, 31, Derrington, A. M., & Suero, M. Motion of complex patterns is computed from the perceived motions of their components, 139–149. Copyright 1991, with permission of Elsevier Science.)

speed of the other component of the plaid is reduced to match the perceived speed of the component whose perceived speed has been reduced (Derrington & Suero, 1991). This is absolutely consistent with the idea that the visual system computes the motion of the plaid from the motions of its component gratings.

Although these studies are consistent with the idea that the visual system computes the motion of plaid patterns from the motions of their sinusoidal components using the intersection of constraints, they do not exclude alternative hypotheses. For example, one weakness of Welch’s study is the absence of comparison mea-

surements for a grating with the same spatial period as the plaid along its axis of motion. It could be that the dip in the Weber fractions in Figure 19 depends on the temporal frequency of the pattern, which inevitably matches the temporal frequency of the components, rather than on its speed. Similarly, the studies in which the apparent speed of one component was reduced, used symmetrical plaids for which the intersection of constraints algorithm predicts qualitatively the same results as other simpler alternatives, such as a vector sum (Derrington & Suero, 1991; Stone et al., 1990; Wilson et al., 1992). In the next section we consider some results that are not consistent with the simple version of the intersection of constraints model.

2. Experiments That Contradict the Simple Version of the IOC

Two sets of experimental results, which will be dealt with in this section, are inconsistent with the hypothesis that the visual system computes the motion of plaid patterns from the motions of their component gratings using the intersection of constraints algorithm. First, thresholds for discriminating the direction of motion of plaid patterns suggest that in computing the direction of motion of plaid patterns the visual system is likely to use 1-D motion vectors derived from local features such as edges and from second-order analyses of the plaid, although the computation could be carried out using the intersection of constraints algorithm, and it could also use 1-D vectors derived from the component gratings (Derrington & Badcock, 1992; Derrington et al., 1992). Second, the perceived axes of motion of some types of plaid pattern deviate systematically from the axis predicted by the intersection of constraints, which has led to the proposal that the visual system uses a di erent algorithm, based on summing 1-D motion vectors, to compute the axis of motion of 2-D patterns (Wilson et al., 1992).

a. Plaid and Component Motion Identification Thresholds

Subjects are able to discriminate correctly between leftward and rightward motion of a horizontally moving plaid pattern when it is moving so slowly that the motion of its component gratings cannot be discriminated (Derrington & Badcock, 1992). This threshold di erence probably occurs because the plaid moves faster than the component gratings, but even so, it could not happen if the visual system computed the motion of the plaid from the motions of its component gratings, because that would make the threshold for discriminating the motion of the component gratings the lower limit on the threshold for discriminating the motion of the plaid. Analysis of thresholds for identifying axes of motion over vertical and oblique directions shows that when spatial frequency or contrast is high it is easier to discriminate the motion of the plaid than to discriminate the direction of motion of its component gratings (Cox & Derrington, 1994).

These observations lead inescapably to the conclusion that in detecting the motion of plaid patterns the visual system must use something other than, or in addition to, the motion vectors derived from the component gratings. Two obvi-

292 Andrew Derrington

ous sets of oriented features that would give rise to 1-D motion vectors with the same magnitude as the plaid pattern’s 2-D vector would be the plaid’s local vertical and horizontal edges and the vertical and horizontal second-order features associated with the plaid’s contrast envelope (Derrington & Badcock, 1992).

It is di cult to demonstrate unequivocally that the visual system uses local features, but it is possible to show that it uses motion vectors derived from a secondorder analysis of the plaid’s contrast envelope. The test uses a moving plaid stimulus in which first-order and second-order analyses signal opposite directions of motion. The test stimulus and the results are shown in Figure 17 (Derrington et al., 1992).

The test takes advantage of the fact that the horizontal spatial period of the contrast envelope is half the horizontal spatial period of the luminance profile. Thus when the pattern is made to move in jumps of increasing size, Figure 17a shows that the direction of motion of the contrast envelope reverses when the jump size is between 0.25 and 0.5 periods of the luminance profile. Figure 17b and d show the test stimulus and its horizontal luminance profile. The horizontal contrast envelope (Figure 17c) is a rectified version of the luminance profile and so has half the spatial period. Figure 17e and f show space–time plots of the luminance profile and the contrast profile jumping repeatedly leftwards in jumps of 0.375 of the period of the luminance profile. The luminance profile appears to be jumping leftwards, and the contrast profile appears to be jumping rightwards.

Figure 17a shows that when the jump size exceeds 0.25 cycles the percentage of trials on which the observer is correct drops below chance (0.5), indicating that the observer systematically reports reversed motion, suggesting that the motion percept is derived from the contrast envelope rather than from first-order features. We also know that first-order features are important because even in this special case where first-order and second-order analyses signal opposite directions of motion, the observer reports motion in the first-order direction at high temporal frequencies and at high spatial frequencies. Thus, the data in Figure 17a suggest very strongly that observers use second-order motion signals in analyzing the motion of plaids.

b. Errors in the Perceived Axis of Motion: The Vector Sum Model

When a plaid pattern consists of two gratings whose axes of motion make an acute angle and whose speeds are very di erent from one another, the axis of motion of the plaid falls outside the angle formed by the axes of the components. Such a plaid is known as a Type II plaid (Ferrera & Wilson, 1987). An example of the intersection of constraints diagram for a type II plaid is shown in Figure 18a. The axis of motion of type II plaids is systematically misperceived: it appears to be closer to the axes of motion of its components than it actually is (Ferrera & Wilson, 1987). The misperception is more extreme for type II plaids presented for short durations or in peripheral vision. If they are presented for only 60 ms, type II plaids appear to move approximately along the axis predicted by summing the 1-D vectors of their component gratings, and if they are presented in the periphery, the perceived axis of motion may deviate by 40 from the true axis of motion (Yo & Wilson, 1992).