Ординатура / Офтальмология / Английские материалы / The Neuropsychology of Vision_Fahle, Greenlee_2003

72 THOMAS F. MÜNTE AND HANS-JOCHEN HEINZE

The tonic attention paradigm with rapid stimulation from two or several different channels is somewhat different from the classic trial by trial-cueing approach that has been one of the most favoured paradigms in experimental neuropsychology. In this task (the ‘Posner’ task), a cue is presented either centrally (e.g. a leftor right-pointing arrow) or peripherally (e.g. a box at the possible target location) followed several hundred milliseconds later by a target stimulus requiring a simple (i.e. detection) or choice (i.e. discrimination) reaction. Most of the targets appear at the cued, or primed, location. Thus, reaction time and error rate differences between validly and invalidly cued targets can be used to infer the temporal properties of the visual spatial attention system.

In a series of studies, it has been shown that the ERP-effects in this task have many similarities to those obtained with fixed tonic attentional set over an entire experimental run (Mangun and Hillyard 1990, 1991). A central pre-cue indicating the likely site of an upcoming target leads to an amplitude increase for the temporooccipital P1-component in response to the validly primed target compared to the invalidly primed target in both choice and simple reaction tasks. In contrast, the subsequent N1 component was enhanced by validly cued stimuli in the choice reaction task condition only. These findings support models proposing that the behavioural effects of precueing expected target locations can be traced back to changes in sensoriperceptual processing within secondary modality-specific cortex. These areas have been shown to be under control of parietal supervisory systems (Gitelman et al. l999; Mesulam 1990). As expected, in an ERP study using the Posner task in patients with parietal damage, Verleger et al. (1996) found a selective reduction of the attention effect for the left cue/right target combination.

Cross-modal interactions and visual attention

It is quite obvious in everyday life that an integration of information from several modalities is to the organism’s advantage: we might be able to identify the image of the tiger faster if we know where the roar comes from. That our supramodal integration can be fooled is attested by the ventriloquist illusion, where the voice of the ventriloquist is erroneously attributed to his puppet, and the McGurk effect (Campbell et al. 1990a). In a series of elegant studies Driver and Spence (Driver and Spence 1998a,b; Spence and Driver 1996, 1997) have shown that divided attention to concurrent events in different modalities is generally more effective when stimuli have come from the same location. These multiple interactions of visual and auditory modalities at the behavioural level indicate that attention is at least partly dependent on spatial representations that have multimodal properties.

From a cognitive neuroscience perspective the question arises, at what stage of the information-processing cascade do cross-modal interactions take place? Can effects of cross-modal interaction be found in modality-specific cortex or are they confined to supramodal tertiary association cortex? Animal anatomy and physiology indicate that neurons at several levels in the nervous system (midbrain/superior colliculus,

VISUAL PROCESSING IN NORMALS AND NEUROLOGICAL PATIENTS 73

polysensory areas of the parietal, superior temporal, and frontal lobes) are responsive to stimuli from multiple modalities (Stein et al. 1993).

Event-related brain potentials are ideally suited to tackle this question in the human. In an early study, Hillyard and colleagues (1984) rapidly presented visual (flashes) and auditory (tones) stimuli at locations left and right of a fixation point. One group of subjects attended only to visual stimuli, directing their attention to one of the two locations in turn to detect slightly altered target stimuli; a second group of subjects attended only to auditory stimuli and was instructed to ignore the visual stimuli. In spite of the limited electrode array, typical visual and auditory spatial attention effects could be observed when these modalities were attended. Moreover, modulations of ERPs in the unattended modality depended on the direction of spatial attention in the attended modality. This is illustrated in Fig. 3.3 by data from a follow-up study using

Fig. 3.3 Cross-modal spatial attention experiment; The upper left part of the figure illustrates the experimental set-up. Subjects either attended to rapid sequences of auditory stimuli presented over two speakers to the left and right of a fixation aid, or they attended to visual stimuli coming from the same locations. Each subject only attended one modality and attended location

(left /right) was changed from run to run. In the subject group that attended to visual stimuli only (upper right part of figure) the typical temporo-occipital spatial attention effects were observed. The auditory stimuli that had no task relevance for this group showed an attention carry-over effect: auditory stimuli coming from the same location as the attended visual stimuli showed an enhanced negativity (Nd wave). Likewise, in the subject group that always attended to auditory stimuli, a typical auditory attention effect was present (lower right). Again, a carry-over effect to the unattended (visual) modality was observed. The visual stimuli coming from the same location as the attended auditory stimuli showed a frontocentral positive shift in addition to a modulation of the posterior N1 component (after data from Teder-Sälejärvi et al. 1999).

74 THOMAS F. MÜNTE AND HANS-JOCHEN HEINZE

an extended electrode array (Teder-Sälejärvi et al. l999; see also Eimer and Schröger 1998). Again, two groups attended exclusively to either visual or auditory stimuli. In the attend-visual group, ERPs to auditory stimuli showed a typical modulation resembling the effect of spatial attention directed to auditory stimuli albeit with a smaller amplitude. The visual ERPs in the attend-auditory group showed a small but significant enhancement of the posterior and anterior N1 effects as a sign of cross-talk between the modalities. The effects were much smaller than the effects of visual spatial attention proper, though.

Because of the close similarity of attention effects and cross-modal attentional carryover effects, these studies can be taken to suggest that intermodal attention operates by a selective modulation of modality-specific areas. These modulations have to be carried out by a supramodal attentional system.

In another recent study Eimer (1999) asked the question as to whether attention can be directed to opposite locations in different modalities. Attending to the same location in both modalities yielded typical early attentional modulations of the ERPs. These perceptual stage ERP effects were not present in the condition that required attention to opposite sides in the two modalities. Thus, these results disprove theories that posit independent modality-specific systems for spatial selective processing. Rather, the existence of a supramodal attentional control system is supported. Only at postperceptual levels, reflected by ERP effects beyond 200 ms, may attentional control be more flexible.

Attention to objects versus attention to space

A prevailing theme in experimental psychology, reflected also in much of the ERP work cited in the previous section, is that selection of stimuli by the visual system is mostly achieved on the basis of location. This has led to several metaphors—spotlight, gradients, and zoom lens—all of them indicating that a selected subset of visual space is preferentially sampled (Posner 1980; Yantis 1992; Downing 1988; Eriksen and St James 1986).

Over the past 15 years, however, a growing number of studies has suggested that more elaborate levels of representation, such as objects themselves, rather than their spatial location can serve as the basis for attentional selection of information (Duncan 1984; Vecera and Farah 1994, 1997; Kramer et al. 1997). So far, only a few recent ERP studies have tackled this fundamental issue. Clearly, showing a modulation of exogenous components in situations that require an object-based selection of information would place these mechanisms at the front end of the information-processing cascade. On the other hand, late effects would rather indicate that object-based selection is taking place at later postperceptual stages.

One elegant set of studies by Valdes-Sosa et al. (1998) used spatially interspersed dots of different colours as stimuli. Two conditions were used. In one condition, the dots

VISUAL PROCESSING IN NORMALS AND NEUROLOGICAL PATIENTS 75

did not move during the baseline stimulation. This led to the perception of one object composed of many dots. In the other condition, the dots of one colour were rotating in one direction, whereas the dots of the second colour moved in the opposite direction during baseline stimulation. They were thus perceived as two different objects. Critically, the two objects shared the same location. Stimuli were defined as brief displacements of the dots of one colour in several directions, one of which was designated the target direction. Subjects were instructed to attend to displacements of the dots of one (attended) colour and to ignore the displacements or the dots in the other colour. When these displacements were presented in the condition with stationary dots, the stimuli were perceived as displacements of one part of a single object. In the rotating baseline condition, the stimuli can be thought of as displacements of one of two objects. Since the dots of the two colours were interspersed, spatial selection could not be used by the subjects. The results for standard stimuli are shown in Fig. 3.4. Dramatic differences

Two objects

Baseline scene

Parietal

FP

Fig. 3.4 Experiment addressing the role of objects in visual attention. In condition ‘Two objects’ (upper part), stimuli are created by dots of two colours rotating in opposite directions. This creates the perception of two separate objects at the same location. Target stimuli are created by dislocation of one set of dots. Attended colour stimuli are characterized by an enhanced P1 and N1 component, thus indicating very early attentional selection mechanisms. In the control condition ‘One object’ (lower part), stationary dots were used leading to the perception of just one object. The ERPs to the attended stimuli are characterized by a later selection negativity in this condition (after data from Valdes-Sosa et al. 1998).

76 THOMAS F. MÜNTE AND HANS-JOCHEN HEINZE

emerged between the two-object (rotating baseline) and one-object (stationary baseline) conditions. A modulation of the P1 and N1 components was found in the twoobject condition, indicating that very early perceptual processes located in extrastriate cortex are involved in object-based attention. On the other hand, when the baseline compelled the subjects into viewing one object, movement of one set of dots led to a typical selection negativity with an onset of about 150 ms as described previously for colour and motion selection (Anllo-Vento and Hillyard 1996; Hillyard et al. 1984). These results indicate that perceptual organization (into objects) acts as a supplementary attentional selection process at very early levels. Further studies suggest that there might be an interaction between selection based on location and selection based on objects. More work comparing space-based and object-based attention and employing methods of source localization is needed to specify more precisely the time-course and anatomical basis of object-based selection and its independence from spatial attention.

Visual search

Screening a complex visual scene for some relevant item or feature is a prerequisite for the survival of the individuum. Experimental psychologists have shown that, in essence, there are two different search modes. In parallel search, illustrated in Fig. 3.5(a), subjects have to search for a distinguishing feature that is present only for the target item, in this case an extra line extending from the base of one triangle.

Fig. 3.5 Illustration of typical ERP findings in parallel and serial search modes. (a) In the serial search mode subjects are required to detect a target item characterized by the missing of a stimulus feature (in this case an additional horizontal line) shared by all other items. Only half of the stimuli in the experiment contained a target item. (b) In the parallel search mode the task is to detect target items characterized by an additional feature. This latter task can be accomplished much faster, as all items can be screened for the additional feature at the same time (in parallel). (c) ERPS in the parallel search task show a peaked P3 component for the target stimuli. Target and standard stimuli diverge at about 350 ms. (d) This separation occurs much later in the serial search task. Moreover, the serial nature of the task tends to smear out the P3 peak (unpublished data from Münte).

VISUAL PROCESSING IN NORMALS AND NEUROLOGICAL PATIENTS 77

All items of a complex array can be searched for this distinguishing feature in parallel and reaction time does not increase with increased numbers of items in an array. In one type of serial search mode (Fig. 3.5(b)), on the other hand, all items, except for the target item, possess a feature, in this case the additional line. In this condition, an observer has to scan each item serially in order to detect the missing line, leading to a massive increase in reaction time as a function of the number of items within an array.

These tasks have been translated into the electrophysiological domain by Luck and Hillyard (1990, 1994 a,b). They found that the latency and the amplitude of the P3 component differed radically for the two search modes. In the parallel search mode, a high-amplitude P3 with a circumscribed peak occurred for target stimuli, whereas in the serial task the divergence between standard and target stimuli was much later (see Fig. 3.5(c),(d) for similar data). Furthermore, the latency of the P3 varied with the number of distractor items within an array.

Some clinical applications, e.g. on patients with Huntington’s disease (Münte et al. 1997) or brain injury (Heinze et al. 1992), have been reported. In general, it was found that the simple, parallel search mode was more or less preserved in patients with neurological disorders, while the more complex serial search task showed differences between patients with various pathologies and normal controls.

In more recent investigations, Luck and colleagues have concentrated on the so-called N2pc component (Luck and Hillyard 1995; Luck et al. 1996; Luck and Ford 1998). This component is a negative deflection in the 200–300 ms time range that is typically observed at posterior scalp sites contralateral to the location of a target item in a search task. The N2pc reflects the focusing of attention on to the target item to filter out irrelevant information from adjacent distractor items within an array. The functional parallels between the N2pc component and the attentional suppression effects present in single-unit recordings from area V4 in the macaque brain have been discussed by Luck and colleagues (1997).

In one recent series of experiments (see Fig. 3.6), Luck and Ford (1998) compared feature extraction and conjunction tasks. In the feature extraction task (Fig. 3.6(a)), subjects viewed an array containing 10 grey and 2 coloured items and were required to report whether a particular colour was present within an array. This task was performed either alone or in conjunction with a second task that required the subjects to indicate whether a centrally presented letter was a vowel or a consonant. Only a small N2pc component was present for the feature task that was completely abolished when the subjects had to perform the letter classification task in parallel. This was taken to suggest that the feature task does not require the perceptual level attentional mechanism that is indexed by the N2pc component.

In a second, more complex search task subjects had to search for rectangles that had a particular colour and orientation. Again, this task was either executed alone or in conjunction with a central letter discrimination. Only for the conjunction task (colour and orientation) did a sizeable N2pc emerge, which persisted in the dual-task situation

78 THOMAS F. MÜNTE AND HANS-JOCHEN HEINZE

Red

Red

Green

N1

Conjunction

Feature task

Fig. 3.6 Two different varieties of a visual search task. (a) A feature detection task is illustrated. In this task, a target item is characterized by a distinguishing feature, i.e. its colour, that is not shared by any of the distractor items. (b) ERPs from two different versions of the feature detection task. When the feature task is the only task of the subjects, waveforms show a so-called N2pc (for posterior contralateral) component (middle left), which is abolished when subjects have

to simultaneously execute a letter discrimination task presented in the centre of the video screen. This suggests that, although the observers were able to accurately detect the feature target while performing the concurrent letter discrimination task, attention was not necessary for feature detection. (c) A similarly constructed conjunction search experiment is illustrated. In this task, target items are characterized by features (in this case horizontal orientation and red colour) that are shared by the distractor items, necessitating an exhaustive serial search strategy.

(d) A large N2pc wave is seen for the conjunction task when it was performed alone (middle right). When the central letter discrimination task was added, an N2pc was still observed but was considerably delayed and smeared out in time (after data from Luck and Ford 1998).

but was shifted towards a later latency. Because of the nature of the discrimination task, requiring the subjects to combine colour and orientation features of target items, attention was necessary to execute the task as reflected in the higher-amplitude N2pc in the conjunction task alone condition. During the second attention-demanding task, the N2pc was smeared out and shifted in time. Luck and Ford’s interpretation of this latency shift was that subjects accessed the iconic image of the search array after they finished the central task.

VISUAL PROCESSING IN NORMALS AND NEUROLOGICAL PATIENTS 81

region (Milders and Perrett 1993). Some patients were specifically impaired to identify familiar faces; others showed selective deficits for recognition of gaze direction (Campbell et al. 1990b) or lip-reading (Campbell et al. 1996a,b). To account for these selective deficits, neuropsychological models have attempted to fractionate face recognition into several subfunctions (see Chapter 1, this volume). For example, in the model of Bruce and Young (1986), facial features are analysed and specified during an initial phase of structural encoding. The products of this stage are then fed into various independent modules that are concerned with analysis of facial expression and speech as well as face recognition. Such models have been supported by findings in neuropsychological patients as well as by primate studies. In humans, functional imaging studies using PET and fMRI revealed bilateral but right predominant activation in the posterior fusiform gyrus for faces as opposed to other objects or textures (Haxby et al. 1999; Halgren et al. 1999; Kanwisher et al. 1998; Clark et al. 1998).

Electrophysiological studies in humans add further information on the exact time course of the neural activity reflecting face processing. Using subdural electrodes, a surface negative potential at a latency of 200 ms (N200) was recorded from regions of the left and right fusiform and inferior temporal gyri in response to the presentation of faces but not to any other complex stimuli such as scrambled faces, cars, or butterflies (Allison et al. 1994a,b). Jeffreys (1993; Jeffreys and Tukmachi 1992; Jeffreys et al. 1992) described the so-called vertex positive scalp peak at a latency of 150–200 ms, which preferentially responds to faces. Bentin et al. (1996) recorded a negative potential over posterior temporal sites at a latency of 172 ms (N170) that was selectively evoked by faces. In fact, the temporal N170 and the vertex positive peak are probably one and the same component, the different labels merely reflecting the topographical distribution of the dipole. The N170 was reported to be delayed by face inversion and significantly larger to isolated eyes, indicating that N170 might reflect eye-specific instead of whole faceprocessing. This topic was followed up further in a recent study addressing the effects of gaze direction in images of faces. Human faces and isolated eyes served as stimuli with eyes either directed at the subject or away from the subject (Fig. 3.9). Subjects viewed the stimuli flashed upon a video screen in random order and searched for target elements (dots) present only in a minority of the stimuli, while 148 channels of MEG and 32 channels of EEG were recorded. Both isolated eyes and faces showed a very early effect of gaze direction beginning at around 120 ms. This effect could be localized by minimum norm and dipole localization methods to the inferior posterior surface of the temporal lobe. This result can be taken as evidence that gaze direction is analysed very early and independently of the whole facial gestalt within secondary visual areas.

In another recent study, Münte et al. (1998) assessed the electrophysiological differences between matching successively presented pairs of faces (frontal and side-views) for their identity and for their expression (Fig. 3.10). They found that identity matching affected a frontocentral negativity in the 200–400 ms range, whereas the electrophysiological signs of expression matching manifested themselves much later and had a parietal distribution.