Ординатура / Офтальмология / Английские материалы / Computational Maps in the Visual Cortex_Miikkulainen_2005

128 5 Development of Maps and Connections

Fig. 5.31. Combined OR/OD/DR map trained with natural images. The combined LISSOM map from natural images is similar to the one in Figure 5.29. More of the units are selective for orientations near horizontal and vertical, so the map has more red and cyan than with artificial training inputs. Not as many neurons are highly selective for direction, but many of the selective patches are located next to other patches selective for the same orientation but opposite direction. Overall, these results are similar to those from artificial stimuli, with greater variability reflecting the more complicated feature correlations in natural images.

Natural image input affects the RFs and lateral connections more strongly than the maps (Figure 5.32). As in the single-feature simulations, the afferent RFs develop a variety of shapes, including both two-lobe and three-lobe RFs, in contrast to the uniformly three-lobed RFs of the Gaussian-trained map. Because natural images have correlations with longer range, the lateral connections are also wider and patchier, as they are in the single-feature natural image maps.

Nature Noisy disks Gaussians

Fig. 5.32. Effect of training patterns on OR/OD/DR maps. From left to right, each row shows a sample retinal activation, final spatiotemporal receptive fields of sample neurons with lags 3, 2, 1, and 0 from left to right, their inhibitory lateral connections, the orientation preference and selectivity map, the ocular dominance map, and the direction preference and selectivity map. The top row shows a network trained with oriented Gaussians, the middle with noisy disks, and the bottom with natural images. All networks develop realistic orientation maps. Maps with Gaussian inputs and noisy disks develop only smooth, twoor three-lobed RFs, whereas networks with natural images develop a wide variety of RF types, corresponding to the wide range of patterns seen in natural images. In each case, the lateral connection patterns follow the features in the map. These results show that joint OR/OD/DR maps can develop from a variety of abstract and realistic input stimuli.

The simulations with noisy disk patterns show that joint maps of ocular dominance, orientation, and direction can also develop based on spontaneous activity patterns (Figure 5.32). Again, the RFs and lateral connections differed significantly from the Gaussians case, with primarily two-lobed RFs and more long-range lateral connections. The map was less selective for direction than with the other input types, primarily because the neurons became selective for moving curved edges whereas selectivity was measured using straight sine gratings. The ocular dominance map is more sharply delineated (as it is in strabismus) than with other inputs, primarily because the background noise was often stronger than the dim input in the other eye, reducing the overall correlation. Overall, these results suggest that if spontaneous activity patterns have enough motion and differ enough between the eyes, maps of all three preference types can develop.

Together, the results presented in this section show computationally for the first time how the OR, OD, and DR input features interact during development, and sug-

130 5 Development of Maps and Connections

gest a simple explanation for how the combined structures observed in animal maps can emerge. When multiple maps are overlaid, the global structure becomes more difficult to interpret, which is why the single-feature maps constitute a useful abstraction. The combined results are realistic and allow matching of anatomical differences with map organization in different species (as will be discussed in more detail in Section 17.2.2). The model also suggests which complexities are due to mapping multiple features and which are a result of complexities in natural inputs.

5.7 Discussion

The LISSOM model shows how a single local and unsupervised self-organizing process can be responsible for the development of topographic maps and the lateral connection structures in the primary visual cortex. The model is the first to account for the self-organization of lateral connections together with orientation maps, ocular dominance maps, and direction selectivity maps. Starting from random-strength connections, neurons develop oriented spatiotemporal receptive fields and lateral interaction profiles cooperatively and simultaneously. When the input varies in several dimensions at once, the cells and lateral connections become selective for multiple features, using the same architecture and learning rules that apply to individual feature dimensions. Such self-organization stores long-range activity correlations between feature-selective cells in the lateral connections. As will be demonstrated in more detail in Section 14.2, these correlations can be used to eliminate redundant information during visual processing, and to make cortical cells more selective.

LISSOM makes several testable predictions about specific lateral connection patterns in the cortex. The LISSOM orientation map predicts that the long-range connections at pinwheel centers, saddle points, and fractures have unselective, broad unimodal, and biaxial distributions, respectively. The direction map predicts that long-range lateral connections primarily link neurons with similar orientation and direction preference, extend along the orientation preference, and avoid orthogonal orientations and opposite directions. At DR fractures they connect with both directions and extend along their common orientation, in DR saddles they connect with the directions of the saddle, and in DR pinwheels they connect with all directions. The OD preferences are overlaid with the OR and DR preferences: The neurons most selective for one eye prefer connections from that eye, but such a preference is absolute only in strabismic animals. These predictions can be tested experimentally by combining optical imaging and injected tracers (like Bosking et al. 1997; Lowel¨ 1994; Lowel¨ and Singer 1992; Malach et al. 1993; Sincich and Blasdel 2001), as will be discussed in Section 16.4.2.

For clarity, the simulations shown in this chapter were each based on a single type of input pattern, such as Gaussians or natural images. The biological visual cortex, however, may be exposed to multiple sources of activity during development, including spontaneous, internally generated patterns and visual, externally evoked inputs. Biological development is thus likely to depend on a complex combination of such patterns. As will be shown in detail in Chapter 9, modeling both prenatal

internal activity and postnatal visual images allows the model to account for both the primitive orientation maps seen at birth in animals and the more refined adult maps. Such more complex simulations are a natural extension of the single-input experiments presented in this chapter, providing a solid basis for understanding how V1 develops in animals.

LISSOM simulations serve to identify the types of input patterns that lead to normal and abnormal development of maps and connections, which can in turn help focus future biological experiments. For instance, realistic LISSOM ocular dominance maps develop from strength or brightness differences between patterns in the two eyes, but not from large position differences. This result leads to interesting predictions about the roles of internal and external input in shaping the OD maps. In newborn mammals, a rudimentary OD map exists at birth, presumably organized based on prenatal internally generated input such as retinal waves (Crair, Horton, Antonini, and Stryker 2001; Horton and Hocking 1996). If retinal waves are the only source, the LISSOM model predicts that the map would be strabismic, because the retinal waves are uncorrelated between the two eyes. This prediction is difficult to verify directly, but because normal adult maps are not strabismic, two interesting possibilities follow: (1) If the newborn maps are not strabismic, the LISSOM results suggest that the internally generated input that constructs the newborn map must be correlated between the two eyes. Since the retinal waves are not, there must be some additional source of internally generated patterns, like the PGO waves, that plays this role. (2) If newborn maps are indeed strabismic, visual experience must play a crucial role in shaping a normal adult map. In effect, the OD map is then constructed in two phases, prenatal and postnatal, like the orientation map. Possibilities for identifying such patterns and processes experimentally, and for modeling them computationally, are described in more detail in Section 17.2.3.

The LISSOM direction selectivity model is based on a simple but effective model of moving inputs, i.e. translation in a random direction within the image plane at each presentation. In the future it can be made more realistic by interleaving input frames with propagation and settling. Such a process is more complicated but should lead to similar results. More importantly, it is not yet clear what types of moving input drive the development of direction-selective neurons and direction maps in animals. Possible candidates include drifting retinal waves before birth, moving objects in the environment (relative to a stationary background), and optic flow due to eye or head movements. Each of these types of natural motion have different statistical properties, which is important to take into account in a more detailed model. As suggested above for OD and OR, multiple types of internal and external time-varying input are likely to contribute to the developmental process. Future models will depend on measuring and characterizing sources of these inputs in detail, as will be discussed in Section 17.2.3.

In addition to the lagged LGN cells modeled in LISSOM, different species may utilize other mechanisms for establishing direction selectivity and motion preferences (see Clifford and Ibbotson 2002 for a review). For instance, some connection pathways within V1 (e.g. between cortical layers in a column) may have delays long enough to serve as a memory of previous activity levels, much like the lagged af-

132 5 Development of Maps and Connections

ferent inputs do. Once they are characterized in sufficient detail, such mechanisms can be implemented in LISSOM, and they can potentially help account for a wider variety of motion preference results within the same basic framework (as will be discussed in Section 17.1.5).

In a similar manner, the LISSOM model can be instrumental in understanding differences between species more generally. The simulations in this chapter were necessarily based on pooled data from several different species because comparable experimental data do not yet exist across species. As such data are obtained, the model can be parameterized to account for a specific species in detail. As will be discussed in Section 17.2.2, such computational experiments should allow us to understand which differences are significant and what their origins are.

The LISSOM results in this chapter focused on orientation, ocular dominance, and direction maps, which are experimentally the best understood feature dimensions. Future models may also be trained with input that vary in spatial frequency, color, and disparity. Such simulations should result in preferences and maps for these additional features without requiring significant changes to the LISSOM model itself. Because little is known about these dimensions in biology, the simulations could be use as a guideline for further experimental studies. Such simulations will be discussed in more detail in Section 17.2.1.

5.8 Conclusion

The results in this chapter demonstrate that a single local and unsupervised selforganizing process can develop both the afferent and lateral connection structures in the primary visual cortex. This model is the most complete computational simulation of V1 to date. The model suggests that the afferent connections represent visual features with the highest variance, such as topography, orientation, ocularity, and direction. The lateral connections represent correlations between such features, and implement an efficient coding of visual information.

The same self-organizing process may continue to operate in the adult, serving a different role: It may be responsible for plasticity and adaptation after damage, as will be described in the next chapter.

6

Understanding Plasticity

So far we have seen how LISSOM organizes visual information in the afferent receptive fields and lateral connections and develops a representation of visual features and their correlations. The resulting structures are similar to those seen in the visual cortex, and the model can be used to investigate phenomena in the adult cortex that are difficult to understand experimentally. One of the most intriguing such phenomena is that the maps in the adult sensory cortex are highly plastic, i.e. adaptable. How can the cortex stay plastic in the adult, yet maintain the structures necessary for visual processing? This chapter will demonstrate how the same self-organizing process that builds the cortical organization during early development can also maintain it in a state of continuously adapting dynamic equilibrium in the adult.

Two types of cortical plasticity have been observed in biology: reorganization following lesions in the receptor surface, and reorganization after lesions in the cortex. These experiments will be modeled with LISSOM, showing how the network compensates for lesions in the retina, why the reorganization behavior after retinal lesions produces dynamic changes in receptive fields, how the network reorganizes after cortical lesions, and how the reorganization mechanisms suggest techniques to accelerate recovery following cortical surgery or stroke. The results suggest that the same self-organizing mechanism that allows the cortex to learn and represent input information also makes it robust against damage.

6.1 Biological and Computational Background

Reorganization after retinal and cortical lesions is well documented in biological experiments, as will be reviewed in this section. A number of modeling studies also suggest how these processes might take place computationally.

6.1.1 Reorganization After Retinal Lesions

Early studies of cortical development had suggested that most of the cortical structures develop only during a critical period just after birth and are hardwired afterward

134 6 Understanding Plasticity

Fig. 6.1. Reorganization of receptive fields after a retinal lesion. The receptive field distributions in the intact and the lesioned eye of a single adult cat are plotted in visual space 4–8 hours after a retinal lesion in one eye. (a) The electrode penetration site (vertical line) is shown along with the locations (open circles) of neurons whose receptive fields are plotted in (b) and (c). The arrows on the left mark the boundary of the cortical region that represents the retinal lesion. The neurons are labeled “e” to “r” from top to bottom and span a distance of about 5 mm. (b) The receptive fields in the visual space of the intact eye are shown as rectangles labeled “e” to “r” according to the neuron’s position along the penetration site. The grid lines are spaced 10◦ apart and the black diamond marks the area centralis. The dashed contour in the middle marks the area that corresponds to the scotoma in the lesioned eye; the receptive fields within this region are colored gray. The receptive fields show an orderly progression from left to right. In another penetration before the lesion (not shown), a distribution similar to that in (b) was observed in both eyes; therefore, the distribution in the intact eye suggests how the receptive fields in the lesioned eye were located before the lesion. (c) In the lesioned eye, the scotoma is shown as a dark gray area. The receptive fields that used to respond to the area inside the scotoma (the gray rectangles labeled “e” to “o”) have moved outward, and now represent the perilesion area. Several of them (e.g. “g”, “h”, and “m”) have also aligned with the scotoma boundary. As a result, the unresponsive area in the cortex (a perceptual blind spot) has disappeared, even though damage persists in the retina. Reprinted with permission from Calford et al. (1999), copyright 1999 by the Royal Society of London.

(Albus and Wolf 1984; Bonds 1979; Braastad and Heggelund 1985; Hubel et al. 1977; Wiesel 1982). The unspoken assumption was that all the information-carrying pathways are firmly and immutably formed during this early period and the plasticity necessary for learning and compensating for damage exists only in higher cortical areas. However, subsequent results showed that the primary cortical areas are capable of substantial change even in the adult brain (see Buonomano and Merzenich 1998; Chklovskii, Mel, and Svoboda 2004; Kaas 1991 for reviews). For example, after a small artificial blind spot or a physical lesion (i.e. a scotoma) is induced in the retina, an unresponsive area appears in the visual cortex. Over time, this area gradually recovers function and starts to respond to stimuli outside the scotoma (Figure 6.1; Calford, Wang, Taglianetti, Waleszczyk, Burke, and Dreher 2000; Chino, Kaas, Smith, Langston, and Cheng 1992; Chino, Smith, Kaas, and Cheng 1995; Darian-Smith and

Gilbert 1995; Waleszczyk, Wang, Young, Burke, Calford, and Dreher 2003). Consequently, the blind spot becomes invisible, even though the damage persists in the retina. The receptive fields on each eye organize largely independently, i.e. monocular and binocular lesions lead to similar results (Section 6.5).

The cortical changes following a retinal lesion can be quite rapid, as experiments by Pettet and Gilbert (1992) showed. They simulated a retinal scotoma by artificially masking a portion of the visual field and stimulating only the surround of the mask. The mask prevented a part of the cortex from receiving visual stimulation, producing a corresponding cortical scotoma. After the surround had been stimulated for a period of several minutes, the retinal scotoma was removed, and the receptive fields of the unstimulated neurons were found to have enlarged. Psychophysical experiments further showed that after removing the scotoma, a stimulus at the edge of the scotoma appeared to be shifted toward the center (Kapadia et al. 1994). Similar results were later observed with physical lesions to the retina, including detaching it partially (Schmid, Rosa, and Calford 1995) and removing photoreceptors with a focal laser (Calford et al. 1999). In both cases, the receptive fields were observed to expand rapidly.

Prima facie, such a dynamic expansion of receptive fields of the unstimulated neurons and the perceptual shift accompanying it appear incompatible with Hebbian self-organization, which suggests that only active neurons should change substantially. It has therefore been difficult to explain these observations. However, recent experimental results indicate that corticocortical lateral interactions may play a critical role in such postlesion reorganization. For example, if the activities of neurons outside the cortical scotoma are artificially suppressed, the reorganized neurons become silent as well (Calford, Wright, Metha, and Taglianetti 2003), suggesting that lateral activation may be necessary for reorganization. In this chapter, the LISSOM model will be used to provide a concrete computational explanation for this phenomenon, based on lateral interactions and the reorganization of afferent receptive fields.

6.1.2 Reorganization After Cortical Lesions

Lesions to the visual cortex may result, for example, from a stroke or head injury: Neurons in a small area of the cortex become unresponsive to input. Similar to retinal lesions, the cortex gradually reorganizes and recovers some of the function that was initially lost.

Recovery following cortical lesions involves three phases (Merzenich, Recanzone, Jenkins, and Grajski 1990):

1.Immediately after the lesion, there is an immediate partial compensation for the cortical damage, as the receptive fields of neurons surrounding the lesion become larger (Wurtz, Yamasaki, Duffy, and Roy 1990) and expand inward (Sober, Stark, Yamasaki, and Lytton 1997). As a consequence, less function is lost than expected based on the prelesion map (Figure 6.2).

2.Over the next several days, the reorganization includes mixed effects: Some receptive fields shift outward (Merzenich et al. 1990), in addition to those that

136 6 Understanding Plasticity

Fig. 6.2. Reorganization of receptive fields after a cortical lesion. The receptive fields measured through microelectrode recording before and after lesion in the middle temporal (MT) area of a macaque monkey are shown in visual space; the axes indicate angles in the visual field. The retinal area represented by the lesioned cortex is shown in gray (labeled “Motion scotoma”). Before the lesion, receptive fields of neurons immediately outside the cortical lesion boundary (dashed rounded rectangles) responded to the lower right corner of the gray area. However, 3 hours to 3 days after the lesion, the receptive fields (solid rounded rectangles) became larger and covered a larger portion of the scotoma, suggesting that neurons outside the lesion boundary were compensating for the loss of function. Reprinted with permission from Sober et al. (1997), copyright 1997 by the American Physiological Society.

expand inward. Also, in some cases the receptive fields became smaller (Eysel and Schweigart 1999; Sober et al. 1997) in addition to those that enlarged. Some of the immediate compensation is lost, because some perilesion neurons do not respond as strongly as before, and in some cases more function is lost than expected based on the prelesion map (Merzenich et al. 1990). A similar pattern of transient performance improvement, followed by a decline, is also seen in motor cortex (Nudo, Wise, Fuentes, and Milliken 1996), suggesting that it is a general trend in cortical recovery.

3.Over subsequent weeks, the receptive fields expand inward again, as the perilesion neurons begin to respond to part of the receptor surface that used to be represented within the lesion (Eysel and Schweigart 1999; Wurtz et al. 1990; Zepeda, Sengpiel, Guagnelli, Vaca, and Arias 2004). Gradually, the cortex compensates for the lesion and regains much of the lost function.

The mechanisms underlying such reorganization, especially during the regressive phase in the days following the lesion, are not well understood (Sober et al. 1997; Zepeda et al. 2004). Apparently, reorganization following cortical lesions is based on different mechanisms than that following retinal lesions. With retinal lesions, the distribution of input to the cortex changes, and the recovery consists of reorganizing the afferents. With cortical lesions, the input distribution remains unchanged but the local network structure is altered, and this process is likely to depend crucially on

lateral connections. Using the LISSOM model, this hypothesis will be tested computationally in Section 6.4.

6.1.3 Computational Models

During the last few years, there have been several efforts to develop computational models of cortical map reorganization (Goodall, Reggia, Chen, Ruppin, and Whitney 1997; Grajski and Merzenich 1990; Kohonen 1989; Obermayer, Ritter, and Schulten 1990a,c; Pearson, Finkel, and Edelman 1987; Sober et al. 1997; Sutton, Reggia, Armentrout, and D’Autrechy 1994). Most of these were designed either to model isolated plasticity phenomena or to demonstrate general principles of plasticity, and did not aim at a computational theory of how the observed cortical structures organize and reorganize. For example, Sober et al. (1997) showed how changing the parameters in an otherwise non-adaptive model could induce a variety of changes with retinal lesions, including both receptive field expansion and contraction.

A more general theory is provided by models based on self-organizing maps (Andrade, Muro, and Moran´ 2001; Goodall et al. 1997; Obermayer et al. 1990a,c; Sirosh and Miikkulainen 1994b, 1996a). For example, Obermayer et al. (1990a,c) used a SOM-based model to study how the somatosensory cortex reorganizes following the amputation of a finger. In animal experiments, such a lesion is analogous to reorganization following retinal lesions: Soon after the amputation, the unresponsive cortical region gradually regains function and responds to the skin of the remaining digits (Merzenich, Nelson, Stryker, Cynader, Schoppmann, and Zook 1984). The skin surface was modeled with a set of receptors scattered in the shape of a hand, and the somatosensory cortex with a SOM network. Circular Gaussian spots were presented on the receptor surface as input stimuli, and the SOM self-organized into a topographic map of the hand. Subsequently, the receptors of one digit were deactivated, and the self-organization continued. Because the input distribution had changed, the map reorganized based on the stimuli on the remaining fingers. The receptive fields of the neurons in the inactive region gradually moved outward from the lesioned finger, and began to respond best to other stimuli. In the end, all neurons that originally had responded to the lesioned finger represented the surviving fingers.

The SOM-based model illustrates how a topographic map can reorganize, but it does not account for phenomena such as the dynamic expansion of receptive fields seen after peripheral lesions. Moreover, if a cortical lesion were to be simulated in a SOM model, it would reorganize monotonically and recover function completely, unlike the cortex. In addition, Goodall et al. (1997) demonstrated that for proper recovery to take place, the area surrounding the cortical lesion needs to activate more easily than before the lesion. The reason for each of these observations is that the models are driven by afferent adaptation only; as shown by Andrade et al. (2001) and Sirosh and Miikkulainen (1994b, 1996a), specifically adapting lateral connections can overcome these problems in principle, although their models did not include fully detailed map structures.

Building on these foundations, the remaining sections will demonstrate how a large-scale orientation map model with explicit, adapting lateral interactions can pro-