Ординатура / Офтальмология / Английские материалы / books.google.com / Visual Perception Fundamentals of awareness multi-sensory integration and high-order perception_Martinez-Conde_2006

192

A Without Feedback

Left eye Monopt L

Dichopt

Right eye Monopt R

Left eye MonoptDichopt

Dichopt

Right eye MonoptDichopt

Fig. 12. Overriding issues when considering the viability of feedback mechanisms. (A) A general model of early visual system binocular integration without invoking feedback mechanisms. (B) If significant feedback exists between the initial dichoptic levels of processing and earlier levels, it means that, by definition, the earlier levels should therefore behave in the same way as the dichoptic levels (i.e., they will become dichoptic by virtue of the feedback).

The monoptic masking results presented in Fig. 11 show that the strength of monoptic masking increases, in an iterative fashion, with each successive stage of processing in the visual system. Correspondingly, Hubel and Wiesel (1961) found that inhibitory surrounds were stronger in the LGN than in the retina. We proposed that lateral inhibition mechanisms gather strength iteratively in successive stages of the visual hierarchy, as a general principle. The result that dichoptic inhibition is weak in area V1 may reflect such a general principle, given that V1 binocular neurons represent the first stage where dichoptic inhibition could exist in the ascending visual system. If this idea is

correct, downstream binocular neurons in the visual hierarchy should show iteratively stronger dichoptic masking suppression effects. Whether these effects represent the discovery of a general principle of inhibitory iterative processing in the visual system, or not, we believe that dichoptic masking effects must become stronger downstream of V1, to account for the fact that the overall psychophysical magnitude of dichoptic visual masking is equivalent to that of monoptic masking (as shown in Fig. 11).

To search for the neural correlates of masking higher up in the visual hierarchy, we turned to whole brain imaging (functional magnetic resonance imaging, fMRI) techniques. Masking illusions are known to evoke reliable blood-oxygen level dependent (BOLD) signals that correlate with perception within the human visual cortex (Dehaene et al., 2001; Haynes et al., 2005; Haynes and Rees, 2005). Since the psychophysical strengths of monoptic and dichoptic masking are equivalent (Schiller, 1965; Macknik and Martinez-Conde, 2004a), we should be able to find the point in the ascending visual hierarchy in which monoptic and dichoptic masking activities are both extant, and thus determine the first point in the visual hierarchy at which awareness of visibility could potentially be maintained. Previous to this level, target responses will not be well inhibited during dichoptic masking: if these prior areas were sufficient to maintain visual awareness, the target would be perceptually visible during dichoptic masking conditions.

We mapped the retinotopic visual areas with fMRI in human subjects and measured BOLD signal in response to monoptic and dichoptic visual masking within each subject’s individually mapped retinotopic areas (Fig. 13). Our results show that dichoptic masking does not correlate with visual awareness in area V1, but begins only downstream of area V2, within areas V3, V3A/B, V4 and later (Fig. 14). These results agree with previous electrophysiological results in monkeys using both visual masking and binocular rivalry stimuli (Logothetis et al., 1996; Sheinberg and Logothetis, 1997; Macknik and Martinez-Conde, 2004a), as well as with one fMRI study of binocular rivalry in humans (Moutoussis et al., 2005). Furthermore, we found that the iterative increase in lateral inhibition, which we previously

discovered in the subcortical visual system and in area V1 for monoptic masking, continued in the extrastriate visual areas for dichoptic masking (Fig. 14C). This fact plays an important role in localizing the circuits responsible for visibility and perception. For instance, if the brain areas that maintained visual awareness exhibited only weak suppression (i.e., areas that are early in the visual hierarchy such as the LGN and area V1), then target masking would be incomplete and targets would not be rendered invisible during masking. Since dichoptic visual masking is as strong as

monoptic visual masking perceptually and targetderived neural activity is only weakly suppressed by dichoptic masks prior to area V3, it follows that the circuits responsible for visibility lie downstream of area V2, or else targets would not appear to be suppressed during dichoptic masking.

Having determined the lower boundary in the visual hierarchy for the perception of visual masking of simple targets, we set out to determine if there was also an upper boundary. To do this, we isolated the parts of the brain that showed both an increase in BOLD signal when nonillusory visible targets

194

were displayed and a decrease in BOLD signal when the same targets were rendered less visible by visual masking. Surprisingly, only areas within the occipital lobe showed differential activation between visible and invisible targets (Fig. 15).

In addressing Question ]3 (What brain areas must activate to achieve awareness of visibility?), our combined results suggest that visual areas beyond V2, within the occipital lobe, are responsible for maintaining our awareness of simple targets (Fig. 16). This conclusion may not hold true for complex visual stimuli or attended stimuli (see Discussion).

Our results show that masking in the early visual system is not caused by feedback from higher cortical areas that also cause dichoptic masking. It follows that the circuit that causes masking must be ubiquitous enough and simple enough that it exists at many or possibly all levels of the visual system. As lateral inhibition is the basis for all the receptive field structure that we know, it must therefore exist at all levels of the visual system that have receptive fields. Thus, lateral inhibition is a candidate circuit. This idea is strengthened by our findings that lateral inhibition is strengthened iteratively at each progressive level of the early visual system.

What specific neural circuits within the relevant brain areas maintain visibility?

We previously showed that the parts of the target that are most important to conveying its visibility are its spatial edges. We also found that, temporally, the parts of the target’s lifetime most important to its visibility are its onset and termination, rather than its midlife. That is, the target’s temporal edges seemed to convey the

strongest signal concerning the target’s visibility. Similarly, the parts of the mask that were most important to its ability to suppress the perception of the target were the mask’s spatiotemporal edges. We moreover found that the neural correlates of the spatiotemporal edges of stimuli (both targets and masks) were transient bursts of spikes that occurred after the stimulus turned on and off, within neurons with receptive fields positioned at the spatial edges of the stimulus on the retinotopic map. Brain areas downstream of area V2, but nevertheless lying within the occipital lobe, were sufficient to maintain our awareness of visibility of simple unattended targets. Finally, we showed that visual masking and visual awareness do not rely on feedback mechanisms from higher cortical areas.

On the basis of these findings, we proposed that the peculiar timing conditions associated with visual masking may be explained through a simple lateral inhibitory network, in which the transient responses to the mask’s spatiotemporal edges inhibit the transient responses to the target’s spatiotemporal edges. Because the target and the mask do not overlap each other spatially, the circuit underlying masking must be called ‘‘lateral inhibition,’’ as defined by Hartline and Ratliff (Hartline, 1949; Ratliff, 1961; Ratliff et al., 1974). The circuit has three properties:

(1)Excitatory input and output.

(A)Monosynaptic connections between the retina and the LGN, and between the LGN and cortex, are excitatory (Cleland et al., 1971; Levick et al., 1972; Reid and Alonso, 1995).

(2)Self-inhibition.

(3)Lateral inhibition as a function of excitation (thus inhibition follows excitation in time).

(A)Spatiotemporal responses of neurons in the LGN and cortex show that excitation precedes inhibition (Ferster, 1986; Golomb et al., 1994). Fig. 17 shows a descriptive model of how a basic lateral inhibitory circuit that has these properties might account for the spatiotemporal properties found in masking.

(a)Several previous groups have suggested that masking might be explained by lateral inhibition (Bridgeman, 1971; Lawwill, 1973; Anbar and Anbar, 1982; Francis, 1997, 1998). However, various aspects of these models do not match the timing parameters of visual masking we discovered, such as the importance of the target’s after-discharge to its visibility (Macknik and Livingstone, 1998; Macknik et al., 2000a).

(i)For instance, Breitmeyer (1984) noted that Bridgeman’s model predicts that when the target and the mask are presented with nonoptimal stimulus onset

Fig. 15. Localization of visibility-correlated responses to the occipital lobe. (A) An individual brain model from all perspectives, including both hemispheres flat-mapped, overlaid with the functional activation from 17 subjects. The green-shaded areas are those portions of the brain that did not show significant activation to Target Only stimuli. The blue voxels exhibited significant target activation (Target Only activation4Mask Only activation). Yellow voxels indicate a significant difference found between Control (target-visible) and SWI (target-invis- ible) conditions, indicating potentially effective visual masking, and thus a correlation with perceived visibility. (B) Response time-course plots from Control vs. SWI conditions in the occipital cortex. (C) Response time-course plots from Control vs. SWI conditions in nonoccipital cortex. (D) Response timecourse plots from the nonillusory conditions (Target Only and Mask Only combined) in occipital vs. nonoccipital cortex. Error bars in (B), (C), and (D) represent SEM between subjects. Reprinted from Tse et al. (2005), with kind permission from the National Academy of Sciences of the United States of America.

asynchronies (to produce masking), the apparent contrast of the target should beat (called ‘‘ringing’’ by Breitmeyer, 1984) in time (like two tuning forks of slightly different pitches) because of poor phase alignment of target and mask resonance. If this is true, then it follows that one should be able to

tune the optimal SOA by noting the beat frequency of the target. At nonoptimal stimulus onset asynchronies, however, the percept of the target is quite steady (Breitmeyer, 1984).

Our model is the first to specifically propose an explanation of how the spatiotemporal edges of stimuli may interact to cause various novel

illusions of invisibility (below) (Macknik and Martinez-Conde, 2004b). Quantifiable computational models of a lateral inhibitory network from the Herzog’s laboratory have also shown that lateral inhibition circuits can potentially predict novel Gestalt-level masking behavior (Herzog and Fahle, 2002; Herzog et al., 2003a, b). The Herzog’s computational model, based on a Wilson–Cowan algorithm, may behave so as to match our results (not yet tested).

Predictions of the lateral inhibitory model

The SWI illusion

The SWI illusion was the first perceptual prediction of the model. This illusion combined forward and backward masking together in a cyclic version of visual masking, thus suppressing all of the transient responses associated with each flicker of the target. Without the mask, the target is a highly salient flickering bar, but with the mask present, the target becomes virtually invisible (Macknik and Livingstone, 1998). The Enns and McGraw groups quantified the psychophysics of the SWI illusion (Enns, 2002; McKeefry et al., 2005). To the best of our knowledge, this is the first illusion to have been predicted from electrophysiological data, rather than the other way around (Fig. 18).

The temporal fusion illusion

Temporal fusion is a second illusion predicted from the model. In this illusion, two short (i.e., 50 ms) duration targets fuse perceptually into one single, long-duration target due to the presentation of a spatially and temporally abutting mask (100 ms long) during the ISI (i.e., inter-target interval) (Fig. 19). Without the intervening mask, two 50-ms targets, separated by an interval of 100 ms, are easily perceived as two sequential flashes. However, when the nonoverlapping mask (nonoverlapping either spatially or temporally) is presented during the inter-target interval, the on- set-response from the mask suppresses (through backward masking) the after-discharge from the first flash of the target, while the after-discharge from the mask suppresses (through forward masking) the onset-response to the second flash of the target. Therefore, no transient responses remain to

199

A B C

Response

Fig. 18. Optical imaging of the SWI illusion. (A) A visual target with a width of 0.121, and the correlated optical image. (B) Two masking stimuli presented without a target, and the correlated response. (C) The SWI stimulus (note: the target and the masks were never presented on the display at the same time) and the correlated response. Notice that the image of the target in (C) is now missing, compared to the response seen in (A), despite that both targets were displayed in an identical manner. Modified from Macknik and Haglund (1999), with kind permission from the National Academy of Sciences of the United States of America.

indicate that the first flash turned off, and the second flash turned on, rendering an illusory percept of a single, long-duration target. This illusion was originally suggested as a prediction of our model by Prof. David McCormick of Yale Medical School, and we reported this illusion at the SFN meeting in 2000 (Macknik et al., 2000b). The temporal fusion illusion verifies the lateral inhibitory model’s predictions, but it has not yet been quantified psychophysically or physiologically.

Flicker fusion

A third consequence of the model is the wellknown flicker fusion illusion. Flicker fusion (sometimes called ‘‘persistence of vision’’) is essential to everyday vision in the modern world (especially when viewing motion pictures, computer displays, TVs, working under artificial lighting, etc.). In 1824, Peter Mark Roget (who also wrote the famous Thesaurus) first presented the concept of ‘‘persistence of vision’’ to the Royal Society of London, as the ability of the retina to retain an image of an object for 1/20–1/5 s after its removal

200

A

B

Time

C

Time

Perception: Two short flashes

Perception: One long flash

from our field of vision (Roget, 1825). A second principle, the ‘‘phi phenomenon’’ or stroboscopic effect (the basis for the famous Gestalt School of Psychology), is closely related to flicker fusion. It was first studied by Max Wertheimer (the founder of Gestalt Psychology) and Hugo Munsterberg between 1912 and 1916, who found that subjects can perceptually bridge the temporal gap in between two consecutive displays, allowing them to perceive a series of static images in a continuous movement (Munsterberg, 1916; Wertheimer, 1925).

Several excellent studies of the physiology of flicker fusion have been published (De Valois et al., 1974; Merigan, 1980; Merigan et al., 1985; Merigan and Maunsell, 1990; Gur and Snodderly, 1997). However, these studies have focused on understanding or exploiting the difference between

the critical flicker fusion frequencies in the parvocellular vs. magnocellular retino-geniculo-cortical pathways (in order to understand the function of those pathways to perception), rather than on the physiology underlying the circuit that fundamentally causes flicker fusion (whatever the pathway).

It follows from our model that for two briefduration targets presented in close succession, the after-discharge from the first target may interfere (i.e., abolish or diminish) with the onset-response from the second target. Thus the transient responses from a series of flashes would be suppressed, thereby diminishing the salience of the flashes and rendering them less visible. The flashes would therefore be more difficult to differentiate and appear to be fused.

We studied the effect of inhibition at the termination of the stimulus by flashing a target twice, with varied intervals between the first and the second flash (see Fig. 20). We found that with short ISIs, both the after-discharge of the first flash and the onset-response of the second flash were inhibited. This suggested that when flicker fusion occurs perceptually, it may be due to the lack of

Fig. 20. Multiunit recording from upper layers of area V1 in a rhesus monkey. The receptive field was foveal, 0.11 square, and well oriented. The transparently shaded parts of the histograms represent the various phases of the cells’ response to the second flash. The different phases of response are color coded as in Fig. 17D. Notice that the second-flash response is rescued as the inter-stimulus interval (ISI), or the interval between the end of the first-pulse and the beginning of the second-pulse, is increased beyond 30 ms.

201

robust firing to the flickering stimulus (which, in a sense, forwardly and backwardly masks itself). The duration of the inhibitory effect after the first flash coincided in time and had the same duration as the time-out period of the first flash. Fig. 19 shows that suppression of the onset-response from the second flash only occurs with ISIs of 30 ms or shorter (equivalent to 33 Hz periodic). If we separate the flashes by more than 30 ms, the afterdischarge to the first flash and the onset-response to the second flash begin to recover (i.e., equivalent to o3 Hz flicker). These intervals roughly coincide with the minimum flicker fusion threshold in humans for 100% contrast stimuli in the fovea (as we used in anesthetized monkeys in these studies) (Fukuda, 1979; Lennie et al., 1993; Di Lollo and Bischof, 1995; Gorea et al., 2000).

In answer to Question ]4 (What is the structure of the circuits that process visibility?), our experiments suggest that lateral inhibitory networks play a role in suppressing the circuits that maintain visibility. While the nature of the circuits that directly maintain awareness is yet unknown, the knowledge that those circuits are suppressed by lateral inhibition may help us to identify them in the future.

Discussion

This chapter describes our results to date in answer to the following four questions:

(1)What stimulus parameters are important to visibility?

(A)Our results show that spatiotemporal edges are critical to target visibility. When we suppressed spatiotemporal edges, targets became less visible or invisible. Recent research suggests that spatiotemporal corners may be even more important than edges (Troncoso et al., 2005).

(B)Masks produce their strongest suppressive effect at their spatiotemporal edges as well, suggesting that inhibition produced by a mask is a function of the same excitatory activity that makes stimuli visible.