Ординатура / Офтальмология / Английские материалы / Computational Maps in the Visual Cortex_Miikkulainen_2005

Fig. 6.11. Reorganization of lateral inhibitory weights after a cortical lesion. The lateral inhibitory weights of a neuron at the bottom-left corner of the lesion are plotted in gray scale from white to black (low to high; orientation preferences or selectivity are not shown). The small white square marks the neuron and the jagged black outline indicates the connectivity before self-organization and pruning, as in Figure 5.12. (a) The connections before the lesion follow the neuron’s orientation preference as usual. (b) Through Hebbian adaptation after the lesion, the connections from neurons in the lesioned area approach zero, because those neurons are no longer active. Because the total inhibitory weight is kept constant by weight normalization, the inhibition concentrates in the connections outside the lesioned zone. This inhibition decreases the responses of the perilesion neurons, giving a computational account for the regressive phase in biology (Section 6.1.2; Merzenich et al. 1990).

Neurons just outside the lesion are most strongly affected by such reorganization: Because they have the largest number of connections from the lesion zone, their lateral inhibition concentrates in the few remaining connections. As a result, during the second phase of reorganization, the lateral inhibition becomes strong outside the lesion, and the previously unmasked activity is suppressed again (Figure 6.10c), resulting in an increased loss of function.

Even after the lateral connections reorganize, the perilesion neurons respond to inputs previously stimulating the lesioned zone. Therefore, through Hebbian adaptation, the afferent weights of these neurons increase in areas that were previously represented within the lesion. Such a reorganization is clearly seen in the topographic map of the afferent receptive fields (Figure 6.12). Gradually, the receptive fields move inward, and the representation of the receptor surface within the lesion zone is partially taken over by the neurons around it. Thus, the network partly compensates for the cortical lesion, and some of the lost function is regained, as in biology (Figure 6.10d). The compensation is not uniform but depends on the orientation preferences. The neurons that prefer orientations perpendicular to the lesion boundary change the most, because they had a large number of lateral connections from the lesioned area. On the other hand, those with preferences parallel to the boundary

150 6 Understanding Plasticity

Retinotopy

OR preference

Fig. 6.12. Reorganization of the orientation map after a cortical lesion. These plots show the retinotopic organization (calculated from settled responses as in Figure 6.7) and the orientation preferences of the central 70 ×70 region of the 142 ×142 cortex. The dotted white line in (a) shows the area that will be lesioned. Immediately after the lesion, the map spreads out slightly into the lesioned area, because the neurons near the lesion boundary respond to inputs previously represented by the lesioned neurons. This expansion can be observed by comparing corresponding areas around the lesion in (a) and (b), such as the lesion’s top boundary. Over time (c), the map expands farther into the lesioned area, regaining some of the lost function. Neurons whose preferred orientations are perpendicular to the lesion boundary change the most because they have the most connections cropped by the boundary. For example, along the top of the lesion, the neurons colored cyan and green on the right side (with vertical preferences) shift their RFs significantly inward, whereas the red, orange, and purple neurons on the left side (with nearly horizontal preferences) do not change much. Thus, the model gives a possible computational explanation for the observed reorganization processes in biology (Section 6.1.2; Merzenich et al. 1990). It further predicts that the specific patterns of expansion depend on the orientation preferences of the neurons around the lesion, and that the extent of recovery depends on how large the lesion is compared with lateral excitation and the afferent receptive fields. An animated demo of the reorganization process can be seen at http://computationalmaps.org.

are less affected by the lesion. Such effects of orientation have not been studied in biology, and are predictions of the model.

6.4.2 Limits of Reorganization

The extent to which the topographic map can reorganize depends on how large the lesion is compared with the extent of the lateral excitatory connections and the anatomical receptive fields. If the receptive fields of neurons on each side of the lesion overlap and the lateral excitation spans the lesion, the receptive fields can shift all the way to the center and cover the region completely. Full functionality would be regained in such a case.

In general, connections passing through the lesioned area do not survive in lesions caused by stroke or surgery, and a complete recovery is not possible in these cases. However, with isolated cell death, such as that in old age, a complete recovery should be expected. This observation could explain why humans can lose a large fraction of their cortical neurons with age without losing any specific representations or functions.

The effects of cortical lesions are reversible in LISSOM, and the original topographic representation will be relearned if the lesioned neurons are restored to normal. Therefore, just as in the case of retinal lesions, the topographic map is dynamic and is maintained in a dynamic equilibrium with external inputs.

6.4.3 Medical Implications

Efforts to understand recovery following acute focal brain damage have traditionally been based on either clinical studies or animal models. LISSOM provides a wellspecified computational model to complement these approaches. The computational model can be used to elucidate the precise mechanisms of recovery and reorganization, which otherwise would not be clear from observing the clinical symptoms. Once these mechanisms are known, it is possible to propose effective treatments and therapeutic strategies.

The LISSOM model suggests two techniques to accelerate recovery following surgery or stroke in the sensory cortices. Normally, the recovery time after surgery would include some immediate recovery, a phase of regression due to reorganized inhibition, and a gradual and slow compensation for the functional loss afterward. The regression phase could be ameliorated by reducing the suppression due to lateral inhibition. Such a technique would selectively deactivate inhibitory interactions around the surgical area using a transient blocker of inhibitory neurotransmitters. Neurons around the area would then fire intensively because of reduced inhibition, and the afferent connections would adapt rapidly to compensate for the lesion. By the time the blocker is absorbed, a substantial number of afferent receptive fields would have shifted and compensated for the lesion. Although the inhibition would still strengthen after the blockade, the recovery would be faster.

Second, the receptive fields of perilesion neurons could be forced to shift and the topographic map reorganized (as in Figure 6.12c) even before surgery. Such an effect could be achieved by intensive and repetitive stimulation of the area expected to lose sensation and by the sensory deprivation of its surroundings. Driven by the excessive stimulation, neurons outside the surgical zone would shift their receptive

152 6 Understanding Plasticity

fields inward, and the area expected to lose sensation would be represented in a much larger area of the cortex. Then, after surgery, the receptive fields would have to move much less to reach their final state and the recovery would again be faster.

In the future, therapy that anticipates functional loss using computational models and compensates for it in advance could form a prelude to neurological surgery, minimizing damage and making recovery as fast and effective as possible.

6.5 Discussion

The LISSOM model shows how receptive fields can be maintained dynamically by excitatory and inhibitory lateral interactions in the cortex. The combined effect of afferent input, lateral excitation, and lateral inhibition determines the neural responses. When the balance of excitation and inhibition is disrupted, neural response patterns change dynamically and receptive fields expand or contract and change their shape. If the perturbations are transient, the weight changes are minimal, and the receptive field properties do not change significantly. If the perturbations persist, the weight changes accumulate and the receptive fields reorganize substantially. Such an ability to reorganize makes the cortex fault tolerant. Apart from injuries, natural cell death occurs in the retina and the brain due to age and disease. If the cortical synapses remain plastic, the remaining neural resources automatically compensate for such losses.

The LISSOM retinal lesion simulations were based on a single eye only. Strictly speaking, such results correspond to experiments where lesion is induced in both eyes (Chino et al. 1995; Gilbert and Wiesel 1992), or in one eye with simultaneous enucleation of the other eye (Chino et al. 1992; Kaas, Krubitzer, Chino, Langston, Polley, and Blair 1990). However, in several recent studies, the other eye was left intact (Calford et al. 1999, 2000, 2003; Waleszczyk et al. 2003), adding an interesting dimension to the experiment and the interpretation of the results. Importantly, Calford et al. (1999, 2000) showed that with monocular retinal lesions, binocular neurons in the corresponding cortical scotoma area reorganize their receptive fields on the lesioned eye as they do with binocular lesions, while their receptive fields on the intact eye remain largely unchanged. This observation is useful because it is difficult to measure both prelesion and postlesion receptive fields of the same neurons; comparing the receptive fields in the two eyes allows obtaining an approximate measure of how they changed. It also suggests that the LISSOM simulations are a valid approximation of monocular lesions as well. However, the LISSOM plasticity studies can also be extended to two eyes, using the model of ocular dominance introduced in Section 5.4. Such a model could be instrumental in understanding in detail how the afferent and lateral contributions from the two eyes interact in the recovery from monocular retinal lesions.

Consistent with recent biological observations (Buonomano and Merzenich 1998; Gilbert 1998; Karni and Bertini 1997), the simulations with the LISSOM model suggest that the cortex is continuously learning and adapting to new visual environments.

Receptive fields and lateral connections constantly adjust in order to match the statistical properties of the visual world. How fast and how locally such adaptation takes place depends on the survival requirements for the animal. If the statistics of the visual world are highly nonstationary, it is important for an animal to adapt constantly to new circumstances, and a high degree of plasticity is required. If the environment is relatively constant, the cortex is probably less plastic or becomes fixed early on. Computational simulations help making such theories concrete, leading to specific predictions that can be tested in further biological experiments.

6.6 Conclusion

The LISSOM model demonstrates that simultaneous adaptation of afferent and lateral connections can explain not only how cortical structures develop, but also how they remain plastic in the adult. The model shows how phenomena such as map reorganization and dynamic receptive fields are produced by lateral interactions and Hebbian learning. The simulated reorganizations are reversible, and demonstrate how a topographic map can be maintained in a dynamic equilibrium with extrinsic input. The model allows predicting the extent and time course of the reorganization, and suggests how recovery after cortical surgery could be hastened by blocking lateral inhibition locally in the cortex and by forcing the topographic map to reorganize prior to surgery.

These computational demonstrations are significant because they show that plasticity does not have to be a special mechanism added for the sole purpose of making the cortex more robust, but can be part of a unified mechanism that the cortex uses to learn and adapt. The next chapter will show how such adaptation and learning can also play a role in normal functional phenomena such as illusions and aftereffects.

7

Understanding Visual Performance: The Tilt

Aftereffect

The results in the previous chapter suggest that the same self-organization processes that determine how the visual cortex develops keep it plastic in the adult. This hypothesis is extended further in this chapter: Such adaptation may be part of the normal function of the visual cortex, resulting in short-term functional phenomena such as the tilt aftereffect (TAE). In this chapter, the psychophysical data and current theories of the TAE are first reviewed, and the simulation method for testing the TAE in LISSOM is described. Adapting lateral connections in the model are shown to result in a TAE that matches human data well, and suggest how the indirect TAE can arise from synaptic normalization. The conclusion is that the TAE is not a flaw in an otherwise well-designed system, but an unavoidable result of a self-organizing process that aims at producing an efficient, sparse encoding of the input through decorrelation. This result establishes a crucial link between developmental processes and adult visual function.

7.1 Psychophysical and Computational Background

In general, humans and animals can accurately estimate the orientation of visual contours such as lines and edges. However, contours presented close together or one after the other in the same location can interact, causing distortions in their apparent orientations. When the lines are presented simultaneously, this effect is known as the tilt illusion, and when they are presented successively, it is known as the tilt aftereffect (Gibson and Radner 1937). This chapter will focus on the tilt aftereffect; possibilities for understanding tilt illusions computationally are briefly discussed in Section 7.5.

7.1.1 Psychophysical Data

The tilt aftereffect is similar to an afterimage from staring at a bright light, but it is due to changes in orientation perception rather than in perceived color or brightness. The effect can be produced in a simple experiment (Figure 7.1). After staring at

156 7 Understanding Visual Performance: The Tilt Aftereffect

Fig. 7.1. Demonstration of the tilt aftereffect. Fixate your gaze on the circle inside the central diagram for at least 30 seconds, moving your eye slightly inside the circle to avoid developing strong afterimages. Now fixate on the diagram at the left. The vertical lines should appear slightly tilted clockwise; this phenomenon is called the direct tilt aftereffect. If you instead fixate upon the horizontal lines at the right, they should appear barely tilted counterclockwise, due to the indirect tilt aftereffect. Adapted from Campbell and Maffei 1971; reprinted from Bednar and Miikkulainen 2000b.

a pattern of tilted lines (the adaptation stimulus), subsequently viewed lines (the test stimulus) appear to have a slight tilt. The direction of the tilt depends on the orientation of the test stimulus: Those with orientations similar to the adaptation lines appear tilted away from them, while those nearly orthogonal to the adaptation stimulus appear to be tilted toward them.

The TAE has been studied extensively in the laboratory (Campbell and Maffei 1971; Gibson and Radner 1937; Mitchell and Muir 1976; Muir and Over 1970). The experimental methods vary somewhat, but subjects are usually tested in a darkened room using oriented stimuli such as lines, bars, or gratings (i.e. multiple parallel bars, as in Figure 7.1). In a typical experiment, subjects are first asked to estimate the orientation of a test stimulus by adjusting another stimulus elsewhere in the visual field until it appears to have the same orientation. They are then asked to fixate on a specific point, and shown an adaptation stimulus for 30–45 seconds. Third, they are shown the test stimulus again, and asked to estimate its orientation. The magnitude of the TAE at that angle between the adaptation and the test stimulus is the difference between the estimated orientation of the test stimulus before and after adaptation.

The results of such experiments are consistent overall, although the detailed shape of the TAE curve varies somewhat between different subjects and different measurement paradigms (Figure 7.2). The maximum angle expansion, called the direct tilt aftereffect, is usually found between 5◦ and 20◦ of difference between adaptation and test stimulus orientations (Howard and Templeton 1966). Larger differences begin to show less of an effect, and eventually reach zero somewhere between 25◦ and 50◦ (Campbell and Maffei 1971; Mitchell and Muir 1976; Muir and Over 1970). Even larger differences (up to 90◦) result in a less pronounced angle contraction effect called the indirect tilt aftereffect, which is largest between 60◦ and 85◦ (Campbell and Maffei 1971; Mitchell and Muir 1976; Muir and Over 1970).

The TAE is found at all orientations, although the variance for oblique test angles is higher than for horizontal and vertical orientations (Mitchell and Muir 1976). The

Fig. 7.2. Tilt aftereffect in human subjects. These plots show one TAE curve for each of the four subjects in Mitchell and Muir (1976): (a) DEM, (b) DWM, (c) JH, and (d) AC. The data were computed by averaging 10 trials before and 10 trials after adaptation. Error bars represent ±1 standard error of the mean (SEM); none were published for subject AC. Each trial consisted of a 3-minute adaptation to a sinusoidal grating, followed by a brief exposure to a test grating. The perceived orientation of the test grating was measured by having the subject adjust the orientation of a test line (presented in an unadapted portion of the visual field) until it appeared parallel to the test grating. For a given orientation difference counterclockwise between test and adaptation gratings, the TAE magnitude was then computed as the difference between the perceived orientations of the test grating before and after adaptation. In each case, a 0◦ orientation difference represents the orientation of the adaptation grating. For DEM the adaptation grating was horizontal, for AC it was vertical, and for DWM and JH it was oblique (135◦). Similar direct effects were observed in all subjects, i.e. they perceived small orientation differences larger than they actually were; indirect effects varied more, but all subjects reported some contraction of large orientation differences.

magnitude of the effect increases logarithmically with increasing adaptation time and decreases logarithmically with time elapsed since the adaptation period (Gibson and Radner 1937). The direct effect saturates at approximately 4◦ (Campbell and Maffei 1971; Greenlee and Magnussen 1987; Magnussen and Johnsen 1986; Mitchell and Muir 1976). The largest documented indirect effect in central (foveal) vision is ap-

158 7 Understanding Visual Performance: The Tilt Aftereffect

proximately 2.5◦, and reaches up to about 60% of the magnitude of the direct effect for a given subject and paradigm (Campbell and Maffei 1971; Mitchell and Muir 1976; Muir and Over 1970). The TAE is spatially localized: Adaptation for an input in one retinal location has no measurable effect on test inputs in other locations sufficiently distant (Gibson and Radner 1937).

The TAE has been studied extensively because it is reliable and easy to measure. It is also important in that it may provide a window into how V1 adapts to visual input, as will be reviewed next.

7.1.2 Theoretical Explanations

Over the years, a wide variety of explanations for the TAE have been put forth (Barlow 1990; Coltheart 1971; Dong 1995; Dragoi, Sharma, Miller, and Sur 2002; Dragoi, Sharma, and Sur 2000; Field 1994; Foldi¨ak´ 1990; Gibson and Radner 1937; Kohler¨ and Wallach 1944; Levine and Grossberg 1976; Tolhurst and Thompson 1975; Vaitkevicius, Karalius, Meskauskas, Sinius, and Sokolov 1983). Modern TAE theories are based on changes in how orientation-selective cells respond in V1 and other visual cortex areas. Hubel and Wiesel (1959, 1962, 1965, 1968) first observed that if the same oriented visual pattern is presented repeatedly, the neurons that at first responded strongly become more difficult to excite. This desensitization, called pattern adaptation, persists even after the pattern is removed. Conversely, neurons with orientation preferences orthogonal to the adaptation input actually respond below their resting level during this process, and may therefore become facilitated instead of desensitized (Hubel and Wiesel 1967). Based on these results, TAE theories have been developed from two perspectives: (1) neural fatigue, i.e. individual neurons become less responsive, and (2) lateral inhibition, i.e. interactions between multiple neurons change.

Neural Fatigue

The neural fatigue theory of the TAE is based directly on the results of Hubel and Wiesel (1959, 1962, 1965, 1968): If neurons with orientation preferences close to the adaptation input become fatigued, neurons that prefer more distant orientations will respond most strongly for similar input during testing. Assuming that the perceived orientation depends on the orientation preferences of the most activate neurons, a direct TAE results (Sutherland 1961). Similarly, the indirect TAE could be due to the (weaker) facilitation of neurons orthogonal to the adaptation input (Muir and Over 1970), or to fatigue of neurons with cross-shaped receptive fields (Coltheart 1971).

Although the fatigue theory was plausible originally, cortical neurons do not actually fatigue in this manner. They can be activated repeatedly in vitro with direct electrical simulation, without decreasing their output or reducing their sensitivity to input (Thomson and Deuchars 1994). Moreover, firing is not crucial for pattern adaptation: Cells continue to adapt even when their firing is prevented pharmacologically, and forcing the cells to fire does not cause them to adapt (Vidyasagar 1990). Thus, pattern adaptation, and therefore the TAE, is unlikely to be caused by fatigue from repeated firing.