Ординатура / Офтальмология / Английские материалы / books.google.com / Visual Perception Fundamentals of awareness multi-sensory integration and high-order perception_Martinez-Conde_2006

182

(Bischof and Di Lollo, 1995; Francis et al., 2004) or the stimulus termination asynchrony (STA) between the target and the mask (the difference in time between the end of the target and the end of the mask) (Macknik and Livingstone, 1998).

(5)Visual masking can be mediated exclusively through cortical processes.

(A)Visual masking can be experienced dichoptically. That is, a mask presented solely to one eye can inhibit a target presented solely to the other eye (Stigler, 1926; Kolers and Rosner, 1960; Schiller, 1968; Weisstein, 1971; McFadden and Gummerman, 1973; Macknik and Martinez-Conde 2004a; Tse, et al., 2005).

(B)Metacontrast masking can be experienced transcallosally. That is, a mask presented solely to one hemi-retina can inhibit a the target presented solely to the other bilaterally symmetric hemi-retina (Stigler, 1926; Schiller, 1968; McFadden and Gummerman, 1973).

(6)Visual masking is stronger with masks that are parallel or co-curvilinear to the target (Werner, 1935, 1940).

These six general findings establish that: (1) the perceived brightness of a stimulus depends on its intensity, its duration, and its temporal, spatial, and contrast relationships to other stimuli, and

(2) cortical processes could be at least partly responsible for these interactions.

Studies of visibility at the spatial edge

In a lateral inhibitory network such as the retina, the lateral geniculate nucleus (LGN), or area V1, the spatial edges of stimuli excite neurons strongly, whereas the interiors of stimuli evoke relatively little response (Hubel and Wiesel, 1959; Ratliff and Hartline, 1959; Battersby and Wagman, 1962; De Weerd et al., 1995; Livingstone et al., 1996; Paradiso and Hahn, 1996). Owing to the very nature of lateral inhibition, both excitatory and inhibitory neural signals are greatest at the spatial edges of the stimulus (Ratliff et al., 1959). One visual correlate of this effect is the Mach band (Mach, 1865, 1965).

Human psychophysical studies of brightness perception and visual masking have shown that inhibition is strongest at the edge of the mask, rather than within its interior (Crawford, 1940; Rushton and Westheimer, 1962; Westheimer, 1965, 1967, 1970). This effect is known as the Westheimer function. My colleagues and I have replicated and extended these results using a

psychophysical visual masking experiment (in which the target and the mask were not present on the screen at the same time, unlike in Westheimer’s experiments). We based the visibility assay on a length-discrimination task, in which the subject was required to determine the longer of two target bars (of 30 ms in duration) when presented at various SOAs with a visual mask of 50 ms in duration and various sizes. See Fig. 2 for a description of forward and backward masking conditions. See Fig. 3 for a description of the experimental results of spatial masking. Notice in Fig. 3 that the mask overlapped spatially the target in every condition. Masking strength was therefore a function of the distance from the mask’s edge to the target’s edge (see inset of Fig. 3). The results indicate that inhibition from the mask increases as

183

the mask’s edge approaches the target. The neural signal from the mask that generates the inhibition is therefore strongest at the mask’s edge than within its interior.

Studies of visibility at the temporal edge

Our psychophysical studies suggest that the temporal edges of targets — the points in time at which they turn on and off — are more important to their visibility than their midlife periods (Macknik and Livingstone, 1998). Fig. 4A–C shows the performance of 25 human subjects in a backward masking task, plotted as a function of SOA, ISI, and STA. Fig. 4A shows performance as a function of SOA. The point of maximum backward masking (drop lines) does not occur at a constant

d

Target Duration (ms)

Inter-Stimulus Interval (ms)

SOA, but rather varies for different mask and target durations. Since SOA is determined as a function of target onset, this result suggested that backward masking was not correlated with target onset. Fig. 4D shows that optimum backward masking tends to occur at later times for longer target durations, suggesting that the termination of the target correlates with the timing of maximum backward masking (slope of linear regression ¼ 0.9870.06, po0.01). The data were therefore replotted as a function of the ISI (Fig. 4B) and the STA (Fig. 4C); since these parameters are defined as a function of target termination, a correlation in either of these parameter spaces would suggest that backward masking is tied somehow to the target’s termination. We found that all backward masking conditions were correlated with an STA of about 100 ms. This suggested that the termination of the mask, 100 ms after the target disappeared, had a crucial impact on the visibility of the target, thus causing backward masking. I will discuss the effect of the mask further below; the point to take home from this result is that some (presumed physiological) event, occurring 100 ms after the target turns off, is important for target visibility.

Previous perceptual studies of backward masking had concluded, incorrectly, that there was a point of minimum visibility at some particular delay between the onsets of target and mask (the ‘‘SOA Law’’) (Weisstein and Haber, 1965; Smith and Schiller, 1966; Bridgeman, 1971; Matin, 1975; Breitmeyer and Ganz, 1976; Francis, 1995). The source of this confusion was that these studies did not vary the duration of the target and the mask, which would presumably have made evident that SOAs were not stable over varying target durations. A few earlier psychophysics studies did vary either target or mask duration, but never both (Alpern, 1953; Sperling, 1965; Breitmeyer, 1978). They reported that mask or target duration modulated the apparent brightness of targets, but did not discuss the effect of target or mask duration on the timing of peak visual masking. Nevertheless, the figures published from these earlier studies show a trend consistent with our findings (that the SOA of peak masking strength varies as a function of target duration).

185

We also examined the psychophysics of forward masking and found that irrespective of target and mask duration, visual masking was strongest when the mask terminated just before the beginning of the target (Fig. 4F). This suggested that some (presumed physiological) event, associated with the onset of the target, was inhibited during forward masking.

Together, our forward and backward masking results suggest that the temporal edges of the target (its onset and termination) are most important to its visibility. However, the factor at the target’s termination that was critical to its visibility did not occur until about 100 ms after the target was turned off. I have examined the neural correlates of this effect in the next section of this chapter.

Since the signals responsible for target visibility appear concentrated at its temporal edges, it follows that masks may be most inhibitory at their temporal edges too. Previous studies had suggested that the onset and termination of the mask are more inhibitory to visual targets than the mask’s midlife time points. Crawford (1947) measured psychophysically the detection threshold of spots of light flashed for 5 ms using method of adjustment (the subject could vary the spot’s luminance), before, during, and after the background flashed to various luminances for 500 ms. For the purposes of this discussion, the spot can be considered the target and the background flash, the mask.

The results indicated that the subjects increased the brightness of the target (in order to detect them) at time points near the beginning and end of the mask, rather than during the mask’s midlife. However, Crawford did not truly address the issue of whether the termination of the mask was more suppressive than its midlife, since he did not vary the duration of the mask. Thus, he could not know if the target inhibition related to the termination of the mask might be the delayed result of the onset of the mask. Margaret Livingstone and I addressed this issue more directly by varying the duration of the mask (Macknik and Livingstone, 1998) (Fig. 4). For the two short-duration masks (50 and 90 ms), we found that most of the inhibition was conveyed at time points near the masks’ termination time. At first sight, this result seemed in conflict with previous studies suggesting that inhibition

186

from the mask was strongest at both its onset and termination. To address this issue, Susana MartinezConde, Michael Haglund, and I conducted a psychophysical experiment to test the strength of visual masking before, during, and after the presentation of nonoverlapping masks, using a wide range of durations (100, 300, and 500 ms) (see Fig. 5). The results indicated that masks are most powerful in their ability to mask targets at their temporal edges (their onsets and terminations).

In summary, both the spatial and temporal edges of a target are critical to its visibility. Thus, in partial answer to Question #1 above (What are the stimulus parameters that are important to visibility?), spatiotemporal edges appear to be important to visibility. In addition, the spatiotemporal edges of the mask drive the strongest suppression of the target.

What types of neural activity best maintain visibility?

Studies of excitatory and inhibitory neural responses at the spatial edge

We optically imaged the neural correlates of excitatory and inhibitory signals from spatial edges in monkey area V1 (Fig. 6). Our results showed that neural signals from the edges are stronger than the signals from the interior, in agreement with previous physiological studies in the early visual system (Hubel and Wiesel, 1959; Ratliff and Hartline, 1959; Battersby and Wagman, 1962; De Weerd et al., 1995; Livingstone et al., 1996; Paradiso and Hahn, 1996).

We next wondered whether physiological inhibition might also be strongest at the spatial edges

(Werner, 1935; Crawford, 1940; Rushton and Westheimer, 1962; Ratliff, 1965; Westheimer, 1965; Nakayama, 1971). To answer this question, we varied the distance from the mask’s edge to the target’edge. We recorded from 26 LGN neurons in the awake monkey, while presenting a visual masking illusion (the standing wave of invisibility, SWI, defined in Fig. 7) in which a flickering target (a white or black bar of 50-ms duration) is rendered invisible by a mask (two bars of 100-ms duration that flank the target to either side). The mask flickered in counterphase alternation to the target (Macknik and Livingstone, 1998; Macknik et al., 2000a) (Fig. 8).

We found that, as the distance between the target and the mask increases, the strength of the inhibition from the mask decreased. This decrease matched the perceptual decrease in visual masking found for increased mask distances (Werner, 1935). The results supported the idea that the spatial edges of the mask convey the strongest inhibitory signals to the target.

Studies of neural responses at the temporal edges

Our psychophysical studies of temporal edges suggested that there are neural events at the beginning

188

The Standing Wave of Invisibility Illusion

Forward Masking Backward Masking

Time

Fig. 7. The time course of events during the standing wave of invisibility illusion. A flickering target (a bar) is preceded and succeeded by two counterphase flickering masks (two bars that abut and flank the target, but do not overlap it) that are presented with optimal forward and backward timing.

of stimuli and after the termination of stimuli that both convey the visibility of the target and the suppressive effects from the mask. Our physiological recordings from V1 of anesthetized monkeys showed that the neural correlates of these events are the transient onset-response to the onset of stimuli, and the transient after-discharge to the termination of stimuli (Fig. 9). Here I discuss the role that these neural events play in visibility and visual masking.

Previous studies had suggested that forward masking is physiologically best described as a function of ISI (Schiller, 1968; Judge et al., 1980). This was confirmed by our psychophysical study (Fig. 4F). Our physiological studies of forward and backward masking, moreover, showed that the neural correlate of forward masking is the inhibition of the onset-response, and the neural correlate of backward masking is the inhibition of the afterdischarge (Fig. 9). These results were consistent with observations from previous physiological studies (Schiller, 1968; Bridgeman, 1975; Bridgeman, 1980; Judge et al., 1980), although these studies had not drawn conclusions about the role of the after-dis- charge. We proposed that the neural correlates of the temporal edges of targets were transient bursts of spikes of the onset-response and after-discharge: the suppression of these transient responses correlated to invisibility during visual masking.

It may seem somewhat counterintuitive that the after-discharge contributes to target visibility, since real-world objects do not usually turn off and should therefore not generate after-discharges. But

Fig. 8. Responses from an on-center type LGN neuron to the standing wave of invisibility, the timing of which is depicted in the colored bars at the top of the figure (‘‘T’’ ¼ target and ‘‘M’’ ¼ mask). Inset pictures on the right represent the stimulus configuration on the display the monkey fixated on (cross near the top of each screen represents the fixation point); the mask distance from the target varied from zero in the top histogram to 8.571 of visual angle in the bottom. Stimulus size, position, and sign of stimulus varied, depending on the optimal parameters of each cell’s receptive field. Each histogram represents separately the results of increasing the mask’s distance (as drawn in the inset). Black traces in each row represent the tar- get-only condition for this cell (all black traces are identical). Purple traces represent the response to the mask alone at each distance, and the blue traces in each histogram represent the firing of the cell to both the target and the mask presented cyclically (SWI illusion condition). Suppression of the target portion of the response in the SWI condition (blue lines) is the neural correlate of visual masking. As it is evident in the blue traces, increase of the mask distance decreases its inhibitory effect on the target (in correlation to perception). Notice that this neuron shows some response to the mask, although the distance between the mask and the target was 8.571 (much larger than the extent of the receptive field). This long-range effect occurred in 16 cells (62%). For all but 1 of these 16 cells the cell type was on-center, indicating that the long-range effect was probably due to light scattering within the eye. Reprinted from Macknik et al. (2000a), with kind permission from the National Academy of Sciences of the United States of America.

189

250 spikes/sec

300 milliseconds

Forward Masking

Backward Masking

the eyes, when open, are rarely stationary (Yarbus, 1967), so real-world stimuli do turn on and off several times per second from the viewpoint of each visual receptive field (Martinez-Conde et al., 2000, 2002). If images are artificially stabilized on the retina, they fade quickly (Day, 1915; Riggs and Ratliff, 1952; Coppola and Purves, 1996). Coppola and Purves (1996) showed that a stabilized image

fades in as little as 80 ms, consistent with the idea that the initial onset-response can produce only a transient visible image. Moreover, Yarbus (1967) showed that after visual fading, stabilized images will reappear as a positive image if they are then turned off. This is consistent with our suggestion that the after-discharge also contributes to visibility. In our own studies, we found that

190

microsaccades drive visibility during fixation (Martinez-Conde et al., 2004; Martinez-Conde et al., 2006), and that the neuronal activity that best correlates with microsaccades is transient bursts of spikes in the LGN and V1 (Martinez-Conde et al., 2000, 2002). See Susana Martinez-Conde’s chapter in the previous vloume (Vol. 154) of Progress in Brain Research for a more in-depth review.

In summary, our results indicate that transient bursts of spikes are generated at the spatiotemporal edges of stimuli. The suppression of these transient bursts (but not the suppression of sustained tonic activity) leads to suppression of visibility in the early visual system. Thus, in partial answer to Question #2 above (What types of neural activity best communicate visibility?), it seems that transient bursts of spikes are important to the neural code for visibility.

What brain areas must be active to maintain visual awareness?

The search for the neural correlates of consciousness requires the localization of circuits in the brain that are sufficient to maintain awareness. To this end, brain areas have been sought within the ascending visual hierarchy that correlate, or more importantly, fail to correlate, with visual perception (Crick and Koch, 1990; Milner, 1995; He et al., 1996; Logothetis et al., 1996; Farah and Feinberg, 1997; Sheinberg and Logothetis, 1997; Tong et al., 1998; Zeki and Ffytche, 1998; Lee and Blake, 1999; Lamme et al., 2000; Polonsky et al., 2000; Thompson and Schall, 2000; Dehaene et al., 2001; Pascual-Leone and Walsh, 2001; Macknik and Martinez-Conde, 2004a; Lee et al., 2005; Moutoussis et al., 2005; Tse et al., 2005). Presumably, the circuits of the brain that are critical to the visibility of targets are circuits whose activity is suppressed during visual masking. A corollary to this is that, if we can identify circuits in which the target response is not suppressed during target masking, we can rule out that circuit as significant to maintaining visual awareness. This section discusses our research to identify parts of the brain whose activity correlates with visual masking.

One of the main reasons that most models of visual masking propose cortical circuits is that

‘‘dichoptic’’ visual masking exists (Kolers and Rosner, 1960; Weisstein, 1971; McFadden and Gummerman, 1973; Olson and Boynton, 1984; McKee et al., 1994, 1995; Harris and Willis, 2001; Macknik and Martinez-Conde, 2004a; Tse et al., 2005). To be clear about the jargon: ‘‘monocular’’ means ‘‘with respect to a single eye,’’ and ‘‘monoptic’’ means either ‘‘monocular’’ or, ‘‘not different between the two eyes.’’ ‘‘Binocular’’ means ‘‘with respect to both eyes’’ and ‘‘dichoptic’’ means ‘‘different in the two eyes.’’ Thus, in dichoptic visual masking, the target is presented to one eye and the mask to the other eye, and the target is nevertheless suppressed. Since excitatory binocular processing within the geniculocortical pathway occurs first in the primary visual cortex (Minkowski, 1920; Le Gros Clark and Penman, 1934; Hubel, 1960), it has been assumed that dichoptic masking must originate from cortical circuits. The anatomical location in which dichoptic masking first begins is thus critical to our evaluation of most models of masking. It is also important to our understanding of neurons of the LGN and their relationship to the subcortical and cortical structures that feed back onto them. In order to establish where dichoptic masking first begins, we first compared the perception of monoptic to dichoptic visual masking in humans over a wide range of timing conditions never before tested (Macknik and Martinez-Conde, 2004a) (see Fig. 10). We found that dichoptic masking was as robust as monoptic masking, and that dichoptic masking exhibited the same timing characteristics previously discovered in humans for monoptic masking (Crawford, 1947; Macknik and Livingstone, 1998; Macknik et al., 2000a).

Next, we conducted recordings of LGN and V1 neurons in the awake monkey while presenting monoptic and dichoptic stimuli. To our knowledge, these were the first dichoptic masking experiments to be conducted physiologically. We found that monoptic masking occurs in all neurons of the early visual system, while dichoptic masking occurs solely cortically in a subset of binocular neurons (see Fig. 11). We also discovered that in the first binocular neurons in the visual system, excitatory responses to monocular targets are inhibited strongly only by inhibitory masks presented to the same eye, whereas interocular

Target = 10 ms

Mask = 300 ms

Time Between Onset of Mask and Onset of Target (ms)

Fig. 10. Psychophysical examination of dichoptic vs. monoptic visual masking in humans. Human psychophysical measurements of visual masking when 10-ms duration target and 300-ms duration mask were presented to both eyes together (monoptic masking) and to the two eyes separately (dichoptic masking). The probability of discriminating correctly the length of two targets is diminished, in the average responses from seven subjects, when they are presented near the times of mask onset and termination. This is true regardless of whether the target and the mask are presented to both eyes (open squares), or if the target was presented to one eye only and the mask was presented to the other (target ¼ left, mask ¼ right: closed upright triangles; target ¼ right, mask ¼ left: closed upside-down triangles). Open circles signify when the target was displayed with both shutters closed, showing that the stimuli were not visible through the shutters. When the mask and the target were presented simultaneously, both eyes’ shutters were necessarily open (dichoptic presentations using shutters are impossible when both stimuli are presented at the same time), and so between 0 and 250 ms all four conditions were equivalent. Dichoptic masking is nevertheless evident when the target was presented before the mask was presented ( 250 to –50 ms on the abscissa), as well as when the target was presented after the mask had been terminated (300–500 ms on the abscissa). Reprinted from Macknik and Martinez-Conde (2004a), with kind permission from the MIT Press.

inhibition is surprisingly weak. Therefore, the circuits responsible for monoptic and dichoptic masking must exist in at least two places independently, one in monocular circuits and another in binocular circuits. Furthermore, the earlier circuits do not exhibit masking as a function of feedback from the later circuits, as proposed by Enns (2002). If they did, then the feedback connections would convey strong dichoptic masking from the

191

Fig. 11. Summary statistics of monoptic vs. dichoptic masking responses in the LGN and area V1. Monoptic (black bars) and dichoptic (white bars) masking magnitude as a function of cell type: LGN, V1 monocular, V1 binocular (nondichoptic masking responsive), and V1 binocular (dichoptic masking responsive) cells. Inset shows the linear regression of the dichoptic masking magnitude seen in V1 binocular neurons (dichoptic masking responsive) as a function of their degree of binocularity (all neurons displayed here were significantly binocular as measured by their relative response to monocularly presented target stimuli to the two eyes sequentially): BI of 0 would indicate that the cells were monocular, while a BI of 1 means both eyes were equally dominant. Reprinted from Macknik and MartinezConde (2004a), with kind permission from the MIT Press.

later circuits, and the earlier circuits would inherit this trait with the feedback (Fig. 12). Since these earlier levels exhibit no dichoptic masking, we can conclude that visual masking in monoptic levels is not due to feedback from dichoptic levels.

No extant theories of visual masking propose that monoptic and dichoptic masking are generated by two different circuits: one that lies in binocular cells and another that lies solely (and specifically) within the monocular cells of V1 (and not the LGN or retina). Therefore, we believe that our results support the most parsimonious conclusion that the circuit underlying visual masking is simple lateral inhibition, which is fundamental to all known circuits of the visual system.