Ординатура / Офтальмология / Английские материалы / Computational Maps in the Visual Cortex_Miikkulainen_2005

138 6 Understanding Plasticity

V1

Retina

ON

Fig. 6.3. Architecture of the reduced LISSOM model. As long as Gaussian patterns are used as input, it is not necessary to include separate ON and OFF channels in the model. Instead, the input can be directly presented as ON channel activations. For simplicity and compatibility with other similar models, this channel is called the Retina, and its neurons are referred to as photoreceptors. The afferent connections form a local anatomical receptive field directly on the redefined retina; the lateral connections are similar to those in other LISSOM models. The reduced model is more efficient to simulate computationally than the full LISSOM, with equivalent results (as shown in Figures 6.4 and 6.5).

vide an accurate and general account for the observed reorganizing behavior with both retinal and cortical lesions.

6.2 The Reduced LISSOM Model

Any of the trained LISSOM networks discussed in Chapters 4 and 5 can be used to study plasticity. However, the fine details of those maps are not necessary, and a simplified, computationally more efficient model can be used as well. Since the focus is on reorganization of the map only, the model can be trained with artificial inputs, and the ON/OFF channels can be bypassed (Figures 6.3 and 6.4). Such a reduced model will first be described in detail below, and demonstrated to develop an OR map equivalent to the LISSOM model of Section 5.3. The role of ON/OFF channels is then analyzed experimentally, and shown to be important for natural images, but unnecessary for the artificial inputs used in this chapter.

6.2.1 Method of Self-Organization

As was mentioned in Section 5.3.5, the ON and OFF channels in LISSOM allow forming similar responses despite differences in background illumination in the input. This property is important with natural images, but not necessary when artificial patterns such as Gaussians are used as input. In such cases, the model can be made computationally more efficient by including only the ON channel. Further, this channel can be combined with the retina into a single sheet of neurons, activated by the input image like the ON sheet in the LGN. For simplicity and compatibility with other similar models (Section 3.4.1), this sheet will be called the retina, and its

Reduced ON/OFF

Fig. 6.4. Effect of ON/OFF channels on orientation maps. The two rows show the results for two LISSOM networks trained with the same stream of Gaussian inputs. The top network is the LISSOM OR map from Section 5.3, and the bottom network is the reduced LISSOM model of Figure 6.3. As in Figure 5.13, each row includes a sample retinal activation, the LGN response (for the ON/OFF LISSOM network), the final receptive fields of sample neurons, their inhibitory lateral connections, the orientation preference and selectivity map, and the histogram and the Fourier transform of the orientation preferences. For the ON/OFF model, the inputs consisted of photograph-like images of Gaussians such as those used in Chapters 4 and 5, shown in gray scale from black to white (low to high), with medium gray representing background activation. In contrast, the reduced LISSOM inputs were similar to the activations in the ON channel, i.e. gray scale from white to black (low to high), with white background. The reduced LISSOM RFs are shown in gray scale like ON weights from white to black (low to high), whereas the ON/OFF LISSOM RFs are combined by subtracting the OFF weights from the ON, as e.g. in Figure 4.6. The RF orientations, lateral connections, and map organization are almost identical in the two models. The RFs on the ON/OFF channels have multiple ON and OFF lobes, and become slightly more oriented. As a result, the ON/OFF map is somewhat more selective. The histogram of each orientation map is nearly flat for both networks, because the inputs were uniformly distributed. These results show that as long as the maps are trained with the same stream of Gaussian inputs, functionally similar maps develop with or without the LGN. However, Figure 6.5 will show that the ON/OFF channels of the LGN are necessary for processing natural images.

neurons will be referred to as photoreceptors. The resulting reduced LISSOM architecture is otherwise similar to that of Section 5.3, except the cortical neurons receive input directly from such a redefined retina (Figure 6.3).

In the reduced LISSOM simulations, the cortical network consisted of an array of 142×142 neurons, and a retina of 36×36 receptors. The neurons in the cortical sheet received afferent connections from broad overlapping circular patches on the retina. The center of the anatomical receptive field of each cortical neuron was placed at the location in the central 24 × 24 portion of the retina corresponding to the location of the neuron in the cortex, so that every neuron had a complete set of afferent connections (Figure A.1). The connection strengths were initially random in a circular area

140 6 Understanding Plasticity

within six units from the RF center. The lateral weights were initially set to a smooth Gaussian profile. The rest of the parameters are described in Appendix B.

The network was organized in 10,000 input presentations of two randomly located and oriented Gaussians. The input sequence was exactly the same as for the LISSOM OR map of Section 5.3. As will be demonstrated in more detail in Section 8.4, such identical training makes it possible to compare the two architectures in detail.

6.2.2 Orientation Maps with and without the LGN

In the self-organizing process, a well-formed orientation map emerged, with the typical oriented receptive fields, topographic order, and patchy lateral connections (Figure 6.4). In fact, this map is almost identical to that in Section 5.3, with the same iso-orientation patches and other map features roughly in the same locations. The similarities extend to the individual neuron level as well: The RF orientations and lateral connection patterns are usually very similar between corresponding neurons in the two maps.

When the inputs consist of oriented Gaussians, the V1 activity patterns are the same with or without ON and OFF channels (Figure 6.5). Since the self-organizing process is driven by these activity patterns, the same map results in both networks. In other words, with such inputs, the reduced model is functionally equivalent to LISSOM with ON/OFF channels.

This equivalency explains why models with and without ON and OFF cells have both been able to develop realistic orientation maps. It also suggests that if the computational experiment focuses on the organization of the map and its lateral connections, and is based on artificial inputs, the simulations can be made more efficient by bypassing the LGN. This simplification will be utilized in the remainder of Part II, as well as in Parts IV and V of the book.

6.2.3 The Role of ON/OFF Channels

It is also important to point out how the LISSOM models with and without the ON/OFF channels differ. Most obviously, although they have the same orientation, the RF shapes are drastically different. These shapes are determined by both the V1 and LGN activities, and the LGN activities in the OFF channel differ greatly from those in the ON channel.

The difference is not important as long as the inputs consist of oriented Gaussians: The maps still respond to the same input with a similar activation pattern. However, the LGN plays a crucial role in suppressing spurious activation with other types of inputs, such as natural images and retinal wave patterns, which cover substantial parts of the retina. In such large patterns, there are often large active areas, large gradual changes in brightness, and nonzero mean levels of illumination. The ON and OFF cells suppress spurious activation in such cases, and allow the map to respond based on orientation (Figure 6.5).

Natural image No edge or line Oriented edge Oriented line

Fig. 6.5. Role of ON/OFF channels in processing various kinds of inputs. Each row shows a sample retinal activation, the LGN response, the V1 response and its histogram (OH) for the ON/OFF LISSOM network, and the V1 response and its histogram (RH) for the reduced LISSOM network. The sample inputs are plotted in gray scale from black to white (low to high) and the LGN activations by subtracting the OFF cell responses from the ON. In the V1 plots (c,e), orientation preferences of those neurons that respond are color coded according to the key on top, and color saturation represents the activation level (selectivity is not shown to match the perceived orientation measure; Section 7.2.1). The two networks respond similarly to an oriented Gaussian input on a blank background (top row), which is why very similar orientation maps developed in Figure 6.4. As seen in the histograms, only neurons with orientation preferences matching the input line respond. However, the networks behave very differently for other types of input. The ON and OFF channels filter out nonzero background levels and smooth, gradual changes in brightness, which ensures that V1 ON/OFF responds only to oriented patterns and sharp edges (second row; the response is strongest on the bright side of the edge because only bright Gaussians were used in training, as shown in the top row of Figure 5.13). In contrast, overall background illumination with no edges is ignored by the ON/OFF network (third row), whereas it activates nearly all of the V1 neurons in the reduced model. Without the LGN, the response to most patterns is determined by the total amount of brightness in the input, rather than by the orientation preference of the V1 neurons. Nonzero background levels, gradual changes in illumination, and large, bright objects are all common in natural images (bottom row), and thus the ON and OFF channels are crucial for preserving orientation selectivity when processing such images. On the other hand, the ON and OFF channels can be omitted for networks that process only schematic patterns on a blank background. The natural image is a retina-size detail (as shown in Figure 8.4(e)) from National Park Service (1995).

142 6 Understanding Plasticity

Natural scenes and retinal waves contain many such features, and thus including ON and OFF cells in LISSOM is crucial for experiments that utilize such inputs, like those in Chapter 5 and in Part III.

6.2.4 Methods for Modeling Plasticity

The plasticity simulations were performed using the reduced LISSOM orientation map network described above. After self-organization has reached the settled state shown in Figure 6.4, both the lateral and afferent connections are in a dynamic equilibrium with the input distribution. They adapt each time an input is presented, but the overall organization does not change significantly, as long as the architecture stays intact and the input distribution does not change. This model forms the foundation for studying plasticity.

To study the effect of retinal lesions, the dynamic equilibrium is disrupted by introducing an artificial scotoma: The activity in a square region of the photoreceptor array is set to zero for all subsequent inputs (Figure 6.6). A cortical lesion can be introduced in the same way: The outputs of a set of neurons in the middle of the cortical network are set permanently to zero (Figure 6.9). In both cases, the lesion disrupts the dynamic equilibrium and forces the network to adapt, as described in detail in the next two sections.

6.3 Retinal Lesions

As a result of the retinal scotoma, neurons in the center of the corresponding cortical area no longer receive input (Figure 6.6). Their response is lost, the dynamic equilibrium is disturbed, and the self-organizing process adapts the map and the receptive fields accordingly.

6.3.1 Reorganization of the Map

As shown in Figure 6.7, the reorganization in LISSOM proceeds in the same manner as observed in the biological cortex (Section 6.1.1; Chino et al. 1992). Initially, the afferent RFs are laid out across the retina relatively uniformly, with local distortions due to OR patches (as described in Section 5.3.3). After the lesion, the afferent RFs of the central, unstimulated neurons remain in the same location as before, but those of the surrounding neurons move outward. These neurons receive input from the receptors surrounding the scotoma, but no stimulation from inside it. Through Hebbian adaptation, the connections from the outside become stronger and those from the inside weaker, resulting in the observed shift outward.

Gradually, almost all neurons that receive afferent input shift their afferent weights outside the scotoma. Most of the initially unresponsive neurons of the network now respond to the periphery of the scotoma. How complete this process is depends on the size of the scotoma. If the lesion is large enough, a set of central

Intact network

Lesioned network

Fig. 6.6. Retinal activation and V1 response before and after a retinal scotoma. The initial and settled responses of the intact network (top row) and the lesioned network (bottom row) to input in (a) are shown in (b) and (c). The activations are displayed in gray scale from white to black (low to high; the orientation preferences of active V1 neurons are not shown). The retinal lesion is simulated by setting the activity of a set of receptors to zero, as shown in the bottom row of (a). The dotted line in (a) marks the lesioned area on the retina (the retinal scotoma), and in (b) and (c) marks the corresponding portion of V1 (the cortical scotoma). The cortical scotoma is approximately as wide as the lateral connections, matching artificial scotomas in biological experiments. Many of the neurons that responded to the intact input do not receive sufficient activation in the lesioned network and remain silent (because the topography of the retinal preferences is not uniform around the edges, some neurons inside the cortical scotoma still respond). Such changes in activity disrupt the dynamic equilibrium, forcing the network to reorganize.

neurons are never stimulated again; they retain their old receptive fields (as shown in Figure 6.7) and appear as a dark area in the visual field. With smaller lesions, the combined effect of the afferent input and lateral excitation is enough to cause them to reorganize as well, eventually making the blind spot in the retina invisible. The LISSOM model therefore suggests a mechanism for the reorganization in response to the retinal scotoma, and allows predicting its extent.

Corresponding changes can be seen in the orientation map (Figure 6.7). At the center of the scotoma the map remains unchanged, but near the edge, where the neurons’ receptive fields have shifted, significant reorganization can be observed. Many neurons near the boundary of the scotoma become selective for the orientation of the boundary. In response, the neurons farther away from the boundary adapt so

144 6 Understanding Plasticity

Retinotopy

OR preference

Fig. 6.7. Reorganization of the orientation map after a retinal scotoma. In the top row, the RF centers of every third neuron in the network are plotted as a grid in the retinal space; the bottom row displays the corresponding map of orientation preferences (selectivity is not shown). The RF centers in the grids are calculated from the settled response (instead of the afferent weights as e.g. in Figure 5.11; Appendices G.2 and G.3), because the lesioned map is not in equilibrium with the input. The dotted white line shows the cortical scotoma, i.e., the region of V1 corresponding to the lesioned area of the retina. (a) Before the scotoma, the RF centers are organized into a retinotopic map with orientation-based distortions, as in Figure 5.11. (b) Shortly after the scotoma, neurons whose RFs were entirely covered by the scotoma retain their old RFs, but the surrounding neurons start to reorganize their afferent weights into the periphery of the scotoma. (c) Five thousand iterations after the scotoma, most of the receptive fields have moved out into the periphery of the lesion (with corresponding inward changes in perception as demonstrated in Figure 6.8); how many remain in the center depends on how large the scotoma is compared with the RFs and the lateral connections. The orientation map is unchanged within the central region of the scotoma, but along the cortical scotoma boundary (in white) many neurons have become selective for the orientation of the boundary, and the rest of the map has adapted to these changes. The reorganization of the retinotopic map provides a detailed computational account for the outward shift in the RF center found by Chino et al. (1992; Section 6.1.1), while the changes in the orientation map constitute predictions for future experiments. An animated demo of the reorganization process can be seen at http://computationalmaps.org.

that the orientation map remains smooth. Such reorganization of the orientation map has not been studied in biology, although the receptive fields have been observed to align with the lesion boundary (Figure 6.1). The results from the LISSOM model therefore constitute predictions for future biological experiments.

6.3.2 Dynamic Receptive Fields

As observed in biology (Section 6.1.1; Pettet and Gilbert 1992), the reorganization in the LISSOM map also causes rapid changes in the receptive field size of the central, unstimulated neurons. As the neurons surrounding the cortical scotoma reorganize their RFs to the periphery, they become insensitive to the center of the retinal scotoma. If the scotoma is now removed, and an input is presented in the scotoma region, only the previously unstimulated neurons (which did not reorganize) respond vigorously to the new input; the surrounding ones do not (Figure 6.8c). Therefore, there is considerably less lateral inhibition from the surrounding neurons to the central neurons. Whereas the central neurons previously responded only weakly to stimuli at the periphery of the scotoma, such responses are now unmasked. Consequently, their RFs have become larger. The expansion is greatest along the preferred orientation because the strongest afferent weights lie in this direction (Figure 5.12a), and any decrease of inhibition unmasks responses mainly in that direction.

This explanation for dynamic receptive fields could be verified in a simple biological experiment. If inhibition to the unresponsive region of the cortex were to be suppressed (by selectively blocking inhibitory neurotransmission using a chemical such as bicuculline), the influence of the surround would be removed from its response. The receptive fields would then have the same size before and after the scotoma. On the contrary, if lateral inhibition is not responsible for the expansion, the dynamic changes of receptive field size would still occur.

The reorganization in LISSOM can account for the psychophysical result of inward shift (Section 6.1.1; Kapadia et al. 1994) as well. The neurons whose receptive fields have moved outward now respond to inputs farther from the center than before. Therefore, an input at the edge of the retinal scotoma stimulates many neurons inside the cortical scotoma that previously would not have responded, and the response pattern is shifted inward (Figure 6.8c). After the scotoma is removed and the normal stimulation reestablished, the reorganized RFs gradually return to the normal state, and the shift disappears.

The LISSOM model thus shows how the same self-organizing processes and lateral interactions that shape the receptive fields during early development can maintain them in a continuously adapting, dynamic equilibrium with the visual environment in the adult. Damage to such a system then results in map reorganization and dynamic receptive fields, giving a detailed computational account for these biological phenomena.

Intact network

Lesioned network

Fig. 6.9. Retinal activation and V1 response before and after a cortical lesion. The initial and settled responses of the intact network (top row) and the lesioned network (bottom row) to the retinal activation in (a) are shown in (b) and (c), as in Figure 6.6. The cortical lesion is simulated by keeping the input intact but setting the activity of cortical neurons to zero in a central region of the map (indicated by the dotted line in b and c). As with a retinal scotoma, the changes in activity disrupt the dynamic equilibrium and force the network to reorganize.

6.4.1 Reorganization of the Map

Three phases of reorganization were observed, as in animal experiments (Section 6.1.2; Merzenich et al. 1990). Immediately after the lesion, the receptive fields of neurons in the perilesion zone became larger. The lesioned neurons no longer inhibit the surrounding neurons, and activation that was previously suppressed is now unmasked. This result can be seen by comparing the response to a typical input before and after the lesion (Figure 6.10a,b). The postlesion activity extends farther outside the lesioned area than before. In effect, the perilesion neurons now respond to a new part of the input space: They took over part of the job of the lesioned area, and the apparent loss of representation is smaller than expected based on the prelesion map.

As soon as the equilibrium was disrupted, both lateral and afferent connections of the active neurons started to adapt. There is a lopsided distribution of activity close to the lesion boundary, with no activity inside and normal activity outside. Therefore, neurons close to the boundary encounter an imbalance of lateral interaction. By Hebbian adaptation, the lateral weights of these neurons strengthen to the active regions outside the lesion, and eventually become concentrated in the perilesion zone (Figure 6.11).