Ординатура / Офтальмология / Английские материалы / Computational Maps in the Visual Cortex_Miikkulainen_2005

with the noisy disks model of retinal waves match biological data better than maps trained with idealized versions of internal inputs and inputs consisting of only noise.

9.2.1 Method

The prenatal HLISSOM network consisted of a 96×96 cortex, 36×36 LGN, 54×54 PGO sheet, and 108 × 108 retina. In three separate experiments, this same network was trained with three different kinds of inputs to match the newborn orientation maps as well as possible. The goal was to understand whether the large, noisy spots seen in retinal waves are sufficient for forming newborn orientation maps, and what role spatial correlation and noise might each play in their self-organization.

In the main experiment, retinal waves were modeled with light and dark noisy disks (Figure 8.4a; Section 8.3). To generate such input, a spot-like structure was first rendered based on a circular disk with smooth Gaussian fall-off in brightness around the edges. Although the retinal wave patterns are often elongated, making them circular in this experiment shows that elongation is not necessary for orientation selectivity to develop. Uniformly distributed random noise was added to represent neural activities realistically. Although some of the noise in the observed retinal wave patterns is most likely due to measurement error, it is reasonable to assume that at least some of it is due to genuine neural activity, and should be included in the model.

More specifically, each input pattern contained one noisy disk, and was specified by the brightness of the disk related to the background (either light or dark), the location of the disk center (xc, yc), the radius rd of the full-intensity central portion of the disk, and the width σd for Gaussian smoothing of its edge. To calculate the activity for each retinal location (x, y), the Euclidean distance d of that location from the disk center is first measured as

The centers (xc, yc) were chosen randomly and the brightness of each pattern was either positive or negative relative to the mean brightness, chosen randomly. Noise was then included in this disk pattern by adding a uniformly distributed value in the range ±0.5 to each pixel.

To evaluate the contributions of spatial correlations and noise in the results, in two separate experiments a similar network was trained with the disk-like patterns without the added noise, and with patterns that consisted of noise only (with each input pixel a random number within [0..1]).

In each experiment, the network was trained for 1000 iterations, since this amount of training was found experimentally to represent prenatal self-organization well. If the prenatal phase was concluded earlier and training continued with natural images, the postnatal training would override the prenatal organization; if instead

192 9 Understanding Low-Level Development: Orientation Maps

the prenatal phase lasted longer than 1000 iterations, the postnatal phase would have little effect in refining the maps. The rest of the simulation parameters are detailed in Appendix C.1.

In the next two subsections, the map organization, receptive fields, and lateral connections resulting from noiseless and noisy disks and from noise alone will be compared.

9.2.2 Map Organization

The main result from the prenatal experiments is that the HLISSOM model trained with patterns modeling retinal waves develops an orientation map very similar to that found in newborn ferrets and binocularly deprived kittens (Figures 9.1 and 9.2; Chapman et al. 1996; Crair et al. 1998). This result shows that even simple internally generated inputs can be responsible for the observed prenatal self-organization. This result also explains how identical orientation maps can form for both eyes even without shared visual experience (using reverse lid suture; Godecke¨ and Bonhoeffer 1996). The development is driven by the orientations of small patches around the edge of the circular spots. These oriented edges are visible in the LGN response to the disk pattern in Figure 9.1.

The map develops lateral connection patterns that are oriented and patchy, although to a lesser extent than in the adult. They are a good match with animal data (such as those of Ruthazer and Stryker 1996). Oriented receptive fields with ON and OFF subregions also develop. Both two-lobed and three-lobed receptive fields are common for simple cells in adult V1 (Hubel and Wiesel 1968), but the RF types in newborns are not known. In the HLISSOM simulation with noisy disks, most neurons develop two-lobed receptive fields because the input patterns consisted of edges only (and no lines or bars). These results suggest that if orientation map development in animals is driven by large, spatially coherent spots of activity, newborns will primarily have two-lobed V1 receptive fields.

9.2.3 Effect of Training-Pattern Variations

Comparing the above results with noiseless and noise-only versions of the retinal wave patterns leads to several insights. The noiseless patterns result in a more regular map and smoother RFs, making the neurons highly selective (as seen in the middle row of Figure 9.1). They are actually more selective than newborn maps. Adding spatially uncorrelated noise, as was done in the “Noisy disks” simulation (top row), makes it harder for the neurons to become highly selective, resulting in maps that faithfully replicate newborn maps.

Interestingly, orientation maps develop even from uniformly random noise (bottom row; this result and the longer simulation in Section 5.3.5 replicate that of Linsker (1986a,b,c) in a biologically more detailed model). However, the resulting V1 map is significantly less organized than typical animal maps, even at birth. Most neurons are also only weakly selective for orientation, as can be seen in the sample RFs, most of which would be a good match to many different oriented lines. These

Disks Noisy disks

Noise

Fig. 9.1. Effect of internally generated prenatal training patterns on orientation maps.

Three different networks were trained for 1000 iterations to match newborn orientation maps as well as possible. The networks and training parameters were otherwise identical except different training inputs were used. As in Figure 5.13, the columns show a sample retinal activation, the LGN response to that activation, self-organized receptive fields for sample neurons, lateral inhibitory weights of these same neurons, the organization of the orientation map with selectivity superimposed in gray scale, and the histogram and the Fourier transform of the OR preferences. Overall, the features seen in the corresponding fully organized maps of Figure 5.13 have already started to emerge in each of these maps, although they are less distinct at this stage. They contain linear zones, pairs of pinwheels, saddle points, and fractures, and their retinotopic organization and gradient (not shown) are roughly similar to adult maps. The ring-like shape of the Fourier transform is also starting to emerge with disk and noisy disk inputs. The map obtained with noisy disks is the best match with animal maps (Figure 9.2). Note that nearly all of the resulting receptive fields have two lobes (i.e. they are edge-selective) rather than three (line-selective), predicting that a similar distribution would also be found in newborns. With noiseless patterns (middle row), the RFs are very smooth, and the neurons become highly selective for orientation, unlike neurons seen in newborn maps. On the other hand, with uncorrelated random noise (bottom row), the neurons become significantly less selective and the RFs do not have regular shapes like they do in animals. The “Noisy disks” map therefore constitutes the most realistic model of prenatal self-organization, and will be used as a starting point for postnatal training.

results suggest that the inputs need to be spatially coherent for realistic receptive fields and maps to develop; noise alone is not sufficient.

In summary, the noisy disks model of internal training patterns leads to orientation maps that are a good match with those seen in newborns. These patterns have enough oriented edges to drive self-organization, and enough noise to prevent the

194 9 Understanding Low-Level Development: Orientation Maps

Fig. 9.2. Prenatal orientation maps in animals and in HLISSOM. (a) A 1.9 mm ×1.9 mm section measured through optical imaging in a 2-week-old binocularly deprived kitten, i.e. a kitten without prior visual experience. The map is not as smooth as in the adult, and many of the neurons are not as selective (not shown), but the map already has iso-orientation patches, linear zones, pairs of pinwheels, saddle points, and fractures (detail of a figure by Crair et al. 1998, reprinted with permission, copyright 1998 by the American Association for the Advancement of Science). (b) The central 30 × 30 region of the “Noisy disks” orientation map from Figure 9.1. The overall organization is very similar in the two maps, suggesting that prenatal training with internally generated patterns may be responsible for the observed maps at birth.

map from becoming too selective. Such maps form a good starting point for further refinement with natural images, as will be demonstrated next.

9.3 Postnatal Development

This section shows how a prenatally trained HLISSOM can continue learning with natural images. Instead of overwriting the prenatal order, the map gradually gets more refined, and eventually represents the statistical distribution of features in the training images.

9.3.1 Method

The prenatal HLISSOM model trained for 1000 iterations with noisy disks was used as a starting point for the postnatal simulations. In 9000 further iterations, 108 ×108 segments of natural images were presented to the retina, and the network was allowed to self-organize with the learning parameters listed in Appendix C.1.

In the main experiment, the network was trained postnatally with a dataset most closely matched with natural input (dataset “Nature”). This set consists of 25 256 ×256-pixel images of naturally occurring objects taken by Shouval et al. (1996, 1997). Nearly all of these images are short-range closeups, although there are a few wide-angle landscapes showing the horizon. All orientations are represented, but overall this dataset includes slightly more horizontal and vertical contours than other orientations. The main research question is then answered by observing how the map becomes gradually more refined during self-organization with these inputs.

The second goal was to understand what role postnatal training might play in helping the animal cope with its environment. As was discussed in Section 8.1, when animals are raised in artificial environments with only vertical lines, the numbers of orientation-selective cells in V1 will reflect this bias (Blakemore and Cooper 1970). The orientation maps of such animals also have enlarged domains for the overrepresented orientations (Sengpiel et al. 1999). Even when raised in normal environments, the maps become smoother and more selective through postnatal experience (Crair et al. 1998). To understand these phenomena computationally, the HLISSOM model was trained on two other natural image datasets as well.

The second postnatal training set, “Landscapes”, consisted of 58 stock photographs from the National Park Service (1995). Nearly half of the images in this set are wide-angle photographs showing the horizon or other strong horizontal contours; a few also include man-made objects, such as fences. Therefore, this dataset has significantly more horizontally oriented contours than other contours. The third postnatal set, “Faces”, consists of 30 frontal photographs of upright human faces (Achermann 1995), which contain more vertical orientations than do the other two sets. Example images from these three postnatal datasets are shown in Figure 9.3. Each of these three sets of natural images has different distributions of oriented edges, and the resulting self-organized maps should differ accordingly.

9.3.2 Map Organization

Starting from the rough prenatal orientation map, postnatal training with natural images gradually refines the map (Figure 9.6, top row). The neurons become more selective and the organization of the map changes slightly. Note, however, that the overall shape of the postnatal map remains similar to the prenatal map, as has been found to be the case in animals, but not in previous models of prenatal and postnatal development of orientation maps (Burger and Lang 1999).

The final adult map matches animal data very well (Figure 9.4). The overall organization of features on this map is similar to measurements from e.g. monkeys, cats, and ferrets. Whereas the prenatal map has a roughly uniform distribution of orientation preferences, the final map is biased for horizontal and vertical orientations. This is important because a similar bias has been found in adult animals (Figure 9.5; Chapman and Bonhoeffer 1998; Coppola et al. 1998).

Most of the RFs in the final map are orientation selective (Figure 9.3, top row), as found in V1 of animals. However, they are still less selective than those in maps

196 9 Understanding Low-Level Development: Orientation Maps

ND+Faces ND+Landscapes ND+Nature

Fig. 9.3. Effect of environmental postnatal training patterns on orientation maps. Each simulation started with the same initial map, trained prenatally for 1000 iterations on noisy disks (ND) as shown in the top row of Figure 9.1. Postnatally, this map was trained for 9000 iterations under the same parameters but with retina-size segments of three different kinds of natural image inputs (the full images for these examples are shown in Figure 8.4d–f ). In each case, maps with realistic features, RFs, lateral connections, and Fourier transforms developed. The final maps are less selective than those trained with artificial stimuli (Section 5.3), matching biological maps well. They also differ significantly on how the preferences are distributed. The network in the top row was trained on images of natural objects and primarily close-range natural scenes from Shouval et al. (1996, 1997). Like biological maps, this map is slightly biased toward horizontal and vertical orientations (as seen in the histogram), reflecting the edge statistics of the natural environment. The network in the second row was trained with stock photographs from the National Park Service (1995), consisting primarily of landscapes with abundant horizontal contours. The resulting map is dominated by neurons with horizontal orientation preferences (red), with a lesser peak for vertical orientations (cyan), which is visible in both the map plot and the histogram. The network in the bottom row was trained with upright human faces, by Achermann (1995). It has an opposite pattern of preferences, with a strong peak at vertical and a lesser peak at horizontal (bottom row). Thus, postnatal self-organization in HLISSOM depends on the statistics of the input images used, explaining why horizontal and vertical orientations are more prominent in animal maps, and how this distribution can be disturbed in abnormal visual environments. It also suggests that postnatal learning plays an important role in how visual function develops: It allows the animal to discover what the most important visual features are and allocate more resources for representing them.

Fig. 9.4. Postnatal orientation maps in animals and in HLISSOM. (a) A 5 mm×5 mm area of the orientation preference map in adult macaque (detail of Figure 2.4a, reprinted with permission from Blasdel 1992b, copyright 1992 by the Society for Neuroscience). After postnatal training on natural images, the HLISSOM map (b) replicates its structure very well. Thus, the HLISSOM model shows how both the prenatal and adult orientation maps can develop based on internally generated and environmental stimuli.

trained with artificial stimuli (i.e. Section 5.3), which is realistic and expected because the natural images contain many patterns other than pure edges. The receptive fields have a realistic multi-lobe structure similar to those observed in simple cells of monkeys and cats (Hubel and Wiesel 1962, 1968). Lateral connection patterns are patchy and oriented, as they are in the adult animal (Bosking et al. 1997; Sincich and Blasdel 2001).

Thus, postnatal training with natural images can explain how orientation maps develop during early life. The model can also help us understand why such postnatal learning is useful, as will be discussed next.

9.3.3 Effect of Visual Environment

The HLISSOM model suggests a computational explanation for the horizontal and vertical biases in the orientation preferences. As was discussed in Section 9.3.1, the “Nature” image set has slightly more horizontal and vertical edges than edges in other orientations. Because self-organizing maps allocate resources according to the input distribution (as was discussed in Section 3.4.3), these orientations become more prominent in the map. Since vertical and horizontal contours are overrepresented in the natural environment as well (Switkes et al. 1978), HLISSOM suggests a possible mechanism for how the observed biases could result.

198 9 Understanding Low-Level Development: Orientation Maps

Fig. 9.5. Distribution of orientation preferences in animals and in HLISSOM. The thin line with circles delineates a histogram of orientation preferences for a typical adult ferret visual cortex (replotted from Coppola et al. 1998; measured through optical imaging in an oval 8.4 mm × 3.3 mm area). The thick line shows a similar histogram for the “ND+Nature” network from Figures 9.3, 9.4, and 9.6. Both adult ferrets and the HLISSOM model have more neurons representing horizontal or vertical than oblique contours, reflecting the statistics of the natural environment. HLISSOM maps trained on internally generated patterns alone instead have an approximately flat distribution, as seen in the histograms of Figure 9.1.

Moreover, if the statistical properties of the inputs are altered, the resulting maps should reflect this change. To demonstrate this idea computationally, the HLISSOM networks trained with the three different postnatal natural image datasets (as described in Section 9.3.1) can be compared.

The postnatal development of the HLISSOM map during the first few hundred iterations turned out very similar regardless of the training patterns used. This result is in line with Crair et al.’s (1998) finding (discussed in more detail in the next section) that visual experience typically has only a small effect on early map development in kittens.

Yet, in continued HLISSOM training the maps start to diverge (Figure 9.3). Compared with the slight horizontal and vertical biases of the “ND+Nature” map, the final “ND+Landscapes” map is strongly biased toward horizontal contours, with a much weaker bias for vertical. These biases are visible in the orientation plot, which is dominated by red (horizontal) and, to a lesser extent, cyan (vertical). The opposite pattern of biases is found for the “ND+Faces” map, which is dominated by cyan (vertical) and, to a lesser extent, red (horizontal). These results are analogous to those with animals raised in artificial environments over the long term (Section 8.1; Blakemore and Cooper 1970; Sengpiel et al. 1999). They suggest that the visual system learns to encode the edge statistics of the visual environment in the orientation map, a result that to our knowledge has not been demonstrated computationally before.

The HLISSOM model therefore shows how postnatal learning can contribute to building an effective visual system. The most common contours in the environment are the best represented in visual cortex, which will result in more effective processing of typical visual input.

9.4 Prenatal and Postnatal Contributions

Although prenatal and postnatal training together can account for the experimental data, are both phases necessary in order to construct a realistic orientation map? Somewhat surprisingly it turns out that training with internally generated patterns and training with natural images alone are both sufficient in principle. However, the animal would either not be able to perform visually at birth, or would not be able to adapt its performance to fit the environment better.

9.4.1 Method

To understand whether prenatal and postnatal phases are both necessary, three further experiments were run. First, the newborn network was trained for another 9000 iterations (with the same parameters as the postnatal network) with noisy disk patterns to determine whether training with internally generated patterns only could result in an organization similar to the adult map.

Second, another network with the same architecture and learning parameters, called the “Nature” network, was trained for 10,000 iterations with the “Nature” set of inputs, but starting from an unordered, random initial organization at iteration 0. That is, both the prenatal and postnatal phases used the same dataset of natural images. This network is used to test whether HLISSOM can self-organize from natural images alone, and whether its final organization will be different from a network trained with a prenatal map as a starting point.

Third, another randomly initialized network called “Blank+Nature” was trained with the same set of natural image inputs starting from iteration 1000, i.e. after some of the maturation processes had already taken place (Sections 4.4.3 and 16.1.6). That is, the simulation parameters had been changed according to the schedule for the first 1000 iterations, even though all inputs were blank and no training had actually occurred. The initial network therefore had a shorter excitation radius, steeper sigmoid, and slower learning rate than it would have had at iteration 0 (Appendix C.1). The purpose was to see whether there was a critical period after which training from natural images only would fail to generate a realistic map. Together with the main experiment combining prenatal and postnatal learning, these three experiments allow identifying distinct roles for the prenatal and postnatal phases of orientation map development.

9.4.2 Effect of Training-Regime Variations

When trained with only internally generated inputs, an orientation map develops that is qualitatively and quantitatively very similar to HLISSOM maps that were trained fully or partly with natural images (Figure 9.6). This result is important because it suggests that a continual generation of internal inputs during early life could be responsible for the development, instead of actual visual experience.

There is indeed evidence that internally driven self-organization continues after birth (or eye opening). Crair et al. (1998) found that similar maps develop in kittens

200 9 Understanding Low-Level Development: Orientation Maps

Blank+Nature Nature Noisy disks ND+Nature

Fig. 9.6. Effect of prenatal and postnatal training on orientation maps. The different rows illustrate how the prenatal training phase affects the final self-organized maps. The state of each network at iteration 1000 is shown on the left half, and the final state at iteration 10,000 on the right half. In the “ND+Nature” simulation (the same as in Figures 9.1 and 9.3), postnatal training makes more neurons sensitive to horizontal and vertical contours and more selective in general. However, the overall map shape remains similar, as found experimentally in animals (Chapman et al. 1996; compare individual orientation patches between pairs of maps on the top row). However, even without any prenatal training (bottom row), or when the network is trained with natural images also prenatally (third row), HLISSOM develops a qualitatively similar final map. In these cases, its organization depends only on the properties of the natural images, not on the internally generated patterns under genetic control. Conversely, even when natural images are replaced by internally generated ones in postnatal training (second row), orientation maps still develop. However, they are not a good match to the visual environment: For example, the orientation histogram is essentially flat. These results suggests that prenatal training is useful mostly because it allows animals to have a functional visual system already at birth, forming a robust starting point for further development. Postnatal training, on the other hand, allows the animal to adapt to the actual visual environment.