Ординатура / Офтальмология / Английские материалы / Computational Maps in the Visual Cortex_Miikkulainen_2005

Fig. 7.9. Cortical response during adaptation and during direct and indirect tilt aftereffect. Using the same plotting conventions as in Figure 7.4, each row shows (a) an example input, (b) the initial response of the central 64×64 region of V1 to that input (before the lateral connections), (c) the response of that region settled through the lateral connections but before adaptation to a vertical input line (SB), (d) the histogram of SB (SBH), (e) the corresponding settled response after a very long period of adaptation (SA), and (f ) the histogram of SA (SAH). The top row (labeled “Adaptation”) shows these responses to the same input as used for adaptation. After adaptation, the settled response is weaker, broader, and includes a wider range of orientations, but the perceived orientation stays approximately the same (compare the black lines in each histogram). For an input with a slightly different orientation (row “Direct effect”), more units encode orientations greater than 10◦ (green areas), and fewer encode those less than 10◦ (blue areas) in the settled response after adaptation than before. The net effect is a direct TAE, with the perceived orientation shifting away from the adaptation orientation, from 8.8◦ to 21.1◦ (compare the black lines in each histogram). For an input with an orientation very different from the adaptation pattern, the changes are more subtle (row “Indirect effect”). Only the neurons around 0◦ were activated during adaptation. Their inhibition from other vertical-preferring neurons increased, but decreased from those not active during adaptation. As a result, the green-colored neurons nearest 0◦ are now less inhibited by the rest of the neurons responding than before adaptation, and so they respond more strongly. The net effect is an indirect TAE, with the perceived orientation shifting toward the adaptation orientation, from 57.4◦ to 54.2◦ (compare the black lines in each histogram). Animated demos of these examples can be seen at http://computationalmaps.org.

170 7 Understanding Visual Performance: The Tilt Aftereffect

ceived angle appears contracted. This effect is smaller than the direct effect because only a small number of such neurons are activated.

Thus, the LISSOM model shows computationally how both the direct and indirect effects can be caused by the same activity-dependent adaptation process. Further, this process is the same one that drives the development of the map in the first place, and its plasticity in the adult. The explanation for the indirect effect is novel and emerges automatically from the model, which was not tuned in any way to model the TAE. In this manner, computational modeling in general, and LISSOM in particular, can be used to obtain insights into functional phenomena that are otherwise difficult to explain at the neural level.

7.5 Discussion

LISSOM is the first computational model to account for how the complete angular function of the TAE can arise from interactions between neurons in V1. The model is also the first to simulate the spatial patterns of activity and connection strengths that underlie the effect and make it spatially localized. It suggests that the same selforganizing principles that result in sparse coding and reduce redundant activation during development may also operate over short time intervals in the adult, affecting perceptual performance. This finding demonstrates a potentially crucial computational link between development, structure, and function.

The TAE experiments were based on the reduced version of the LISSOM model, which allowed demonstrating the main effects clearly and efficiently. Further properties of the TAE could be replicated using the more complex versions of the model. For instance, maps trained on natural images would exhibit TAEs with a higher variance at oblique orientations (Mitchell and Muir 1976), because such maps have fewer neurons selective for oblique orientations (Section 9.3.2). Maps trained with moving patterns would result in a TAE that is specific to the direction of motion, and a version with frequency-selective neurons (to be discussed in Section 17.2.1) should limit the TAE to the spatial frequency of the adaptation input, both well-known features of the TAE in humans (Carney 1982; Ware and Mitchell 1974). Orientation maps trained with multiple retinas would allow transferring the TAE from one eye to the other (Campbell and Maffei 1971; Gibson and Radner 1937): Because the neurons that are most selective for orientation are binocular (Section 5.6), adapting one eye causes equal effects upon test lines in either eye. In each case, the properties of the neurons in the map should directly result in the observed properties of the TAE, without additional modifications to the model.

The LISSOM explanation of the indirect effect as a result of weight normalization is novel. Although normalization was originally included for computational reasons, recent experimental work suggests that such processes actually occur in biological neurons as they adapt (Section 3.3). Such regulation changes the efficacy of each synapse while keeping their relative weights constant, as in the LISSOM model. The process depends on adapting connections like the direct effect, but may

172 7 Understanding Visual Performance: The Tilt Aftereffect

neurons, and provide a novel explanation of the indirect effect based on synaptic normalization. Importantly, these effects emerge from the same decorrelating process that determines how the cortex initially develops, and keeps it plastic in the adult. This process tends to deemphasize constant features of the input, resulting in shortterm perceptual anomalies such as aftereffects.

A crucial component of understanding the TAE in the LISSOM model consisted of direct visualizations of cortical activity and weight changes as they were occurring in the simulated cortex. This method made it clear exactly which processes contributed to the effect. In general, computational models can demonstrate many visual phenomena in detail that are difficult to measure experimentally, thus presenting a view of the cortex that is otherwise not available. Such an analysis can complement both high-level theories and experimental work with humans and animals, significantly contributing to our understanding of the cortex.

Part III

CONSTRUCTING VISUAL FUNCTION

176 8 HLISSOM: A Hierarchical Model

of the visual system is constructed by a general-purpose learning process, driven by inputs from the environment. On the other hand, there is also considerable evidence that many aspects of the visual system are hardwired, i.e. constructed from a specific blueprint encoded in the genome. The conflict between these two positions is generally known as the nature–nurture debate, which has been raging for centuries in various forms (Diamond 1974).

The idea of a specific blueprint does seem to apply to the largest scale organization of the visual system, at the level of areas and their interconnections. These patterns are largely similar across individuals of the same species, and their development does not generally depend on neural activity, visually evoked or otherwise (Miyashita-Lin, Hevner, Wassarman, Martinez, and Rubenstein 1999; Rakic 1988; Shatz 1996). But at smaller scales, such as maps and lateral connections within them, there is considerable evidence for both internally and environmentally controlled development. Thus, debates center on how this seemingly conflicting evidence can be reconciled. Orientation processing is the clearest example, and will be the topic of Chapter 9; in this section, evidence for genetic and environmental influences on OR is summarized, suggesting how computational modeling can be used to account for both. Similar evidence for other perceptual features will be discussed in Section 17.2.3 and for higher levels such as face detection in Section 10.1.

Experiments since the 1960s have shown that the environment can have a large effect on the structure and function of the early visual areas (see Movshon and van Sluyters 1981 for a review). For instance, Blakemore and Cooper (1970) found that if kittens are raised in environments consisting of only vertical contours during a critical period, most of their V1 neurons become responsive to vertical orientations. Similarly, orientation maps from kittens with such rearing devote a larger area to the orientation that was overrepresented during development (Sengpiel et al. 1999). Even in normal adult animals, the distribution of orientation preferences is slightly biased toward horizontal and vertical contours (Chapman and Bonhoeffer 1998; Coppola, White, Fitzpatrick, and Purves 1998). Such a bias would be expected if the neurons learned orientation selectivity from typical environments, which have a similar orientation bias (Switkes, Mayer, and Sloan 1978). Conversely, kittens who were raised without patterned visual experience at all, e.g. by suturing their eyelids shut, have few orientation-selective neurons in V1 as an adult (Blakemore and van Sluyters 1975; Crair et al. 1998). Thus, visual experience can clearly influence how orientation selectivity and orientation maps develop.

The lateral connectivity patterns within the map are also affected by visual experience. For instance, kittens raised without patterned visual experience in one eye (by monocular lid suture) develop nonspecific lateral interactions for that eye (Kasamatsu, Kitano, Sutter, and Norcia 1998). Conversely, lateral connections become patchier when inputs from each eye are decorrelated during development (by artificially inducing strabismus, i.e. squint; Gilbert et al. 1990; Lowel¨ and Singer 1992; Section 5.1.2).

In ferrets it is possible to reroute the connections from the eye that normally go to V1 via the LGN, so that instead they reach auditory cortex (see Sur et al. 1999; Sur and Leamey 2001 for reviews). As a result, the auditory cortex develops orientation-

selective neurons, orientation maps, and patchy lateral connections, although these structures are not as pronounced as in normal maps. Furthermore, the ferret can use the rewired neurons to make visual distinctions, such as discriminating between two grating stimuli (von Melchner, Pallas, and Sur 2000). Thus, afferent input to the cortex can profoundly affect its structure and function.

Therefore, the evidence from altered environments and rewiring experiments suggests that the structure and function of V1 could simply be learned from experience with the visual environment. This experience is presumably mediated by neural activity, and experiments have shown that blocking neural activity, or blocking activity-dependent plasticity, prevents the development of orientation selectivity (Chapman and Stryker 1993; Ramoa, Mower, Liao, and Jafri 2001). Together, these findings suggest that the visually driven neural activity patterns in the LGN might be sufficient to direct the development of V1 and other cortical areas.

However, there is also significant evidence on the contrary, suggesting that visual cortex structure is genetically determined. For example, it has been known for a long time that individual orientation-selective cells exist in newborn kittens and ferrets even before they open their eyes (Blakemore and van Sluyters 1975; Chapman and Stryker 1993). Psychological studies further suggest that human newborns can already discriminate between patterns based on orientation (Slater and Johnson 1998; Slater et al. 1988). Recent advances in experimental imaging technologies have even made it possible to measure the full map of orientation preferences in young animals. Such experiments show that large-scale orientation maps exist prior to visual experience, and that these maps have many of the same features found in adults (Figure 9.2a; Chapman, Stryker, and Bonhoeffer 1996; Crair et al. 1998; Godecke¨ et al. 1997). The lateral connections within the orientation map are also already patchy before eye opening (Godecke¨ et al. 1997; Luhmann et al. 1986; Ruthazer and Stryker 1996).

Furthermore, the global pattern of orientation preferences in the maps changes very little with normal visual experience, even as the individual neurons gradually become more selective for orientation, and lateral connections become more patchy (Chapman and Stryker 1993; Crair et al. 1998; Godecke¨ et al. 1997). Thus, despite the clear influence of environmental input on visual cortex structure, normal visual experience primarily preserves and fine-tunes the existing structures, rather than drives their development.

How can the same circuitry be both genetically hardwired, yet also capable of significant learning and adaptation based on the environment? As was discussed in Section 2.3, new experiments are finally starting to shed light on this question: Many of the structures present at birth could result from learning of spontaneous, internally generated neural activity, such as retinal waves and PGO waves. The same activitydependent learning mechanisms that can explain postnatal learning may simply be functioning before birth, driven by activity from internal instead of external sources. In this way, “hardwiring” may actually be learned. This explanation provides a possible way to reconcile the evidence in the nature vs. nurture debate in orientation processing. The answer is not simply that both components have an effect; there is only one developmental process, and it consists of a synergy of nature and nurture.

178 8 HLISSOM: A Hierarchical Model

The goal of Part III is to test this hypothesis in detailed computational experiments. Two developmental phenomena are studied: (1) How orientation maps develop, as described above, and (2) how human infants prefer facelike visual input already at birth and how these preferences change in early life. Although the experimental data about both of these phenomena are rather confusing and even contradictory, they make perfect sense under the synergy hypothesis. The computational experiments are based on the LISSOM model, expanded outward to include lower and higher level areas of visual processing.

8.2 The Hierarchical Architecture

HLISSOM consists of the key areas of the visual system that are necessary to model how orientation and face processing develops based on internal and external inputs. The architecture is illustrated in Figure 8.1. HLISSOM extends LISSOM in three ways: (1) It includes input patterns arising from the brainstem (the PGO generator) in addition to retinal input; (2) it includes a higher level cortical face-selective area (FSA) in addition to V1; and (3) it includes divisive normalization on the afferent input. As described in Section 6.2.3, the ON and OFF channels of the LGN are necessary to process natural images.

8.2.1 Brainstem Input Area

As was reviewed in Section 2.3, the photoreceptors are not the only source of neural activity in the visual system. Spontaneous waves of activity in the retina and the PGO waves generated in the brainstem have both been shown to affect the development of the LGN, and may also influence how V1 develops. To investigate the role of such patterns, HLISSOM includes the PGO generator as an additional input area representing spontaneous activity from the brainstem.

The PGO pathway has not yet been mapped in detail in animals, but the activity that results from the PGO waves appears to be similar to that from visual input (Marks et al. 1995). Thus, for simplicity, the PGO pathway is modeled with an area like the retina, connecting to the LGN in the same way (Figure 8.1).

8.2.2 Face-Selective Area

It is easy to see how internally generated patterns could influence how orientation maps develop. However, behavioral tests with human infants (discussed in detail in Section 10.1) also suggest that internal activity may be important for the development of high-level processing, such as face perception. As was reviewed in Section 2.1.3, in adults there are face-selective cortical regions that receive input from V1. HLISSOM includes such a face-selective region called the FSA so that the development of face perception can be studied.

The FSA represents the first region in the ventral processing pathway above V1 that has receptive fields spanning approximately 45◦ of visual arc, i.e. large enough

Face−selective

area (FSA)

V1

LGN

Fig. 8.1. Architecture of the HLISSOM model. Each sheet of units in the model visual pathway is shown with a sample activation pattern and the connections to one example unit. The activities are shown in gray scale as in Figure 4.1. Visual inputs are presented on the retina, and the resulting activity propagates through afferent connections to each of the higher levels. Internally generated PGO input propagates similarly to visual input. Activity in the model at any one time originates either in the PGO sheet or the retina, but not both at once. As in the LISSOM model, the activity in the cortical levels (V1 and FSA) is focused by lateral connections, which are initially excitatory between nearby neurons (dotted circles) and inhibitory between more distant neurons (dashed circles). The final patterns of lateral and afferent connections in the cortical areas develop through an unsupervised self-organizing process, as in LISSOM. After self-organization is complete, each stage in the hierarchy represents a different level of abstraction. The LGN responds best to edges and lines, suppressing areas with no information. The V1 response is further selective for the orientation of each contour; the response is patchy because neurons preferring other orientations do not respond. The FSA represents the highest level of abstraction — a neuron in the FSA responds when there appears to be a face in its receptive field on the retina.