Ординатура / Офтальмология / Английские материалы / Computational Maps in the Visual Cortex_Miikkulainen_2005

Fig. 2.7. Lateral connections in the tree shrew orientation map. (a) The vertical and horizontal orientation preferences in a 8 mm × 5 mm section of V1 in the adult tree shrew, measured using optical imaging. The areas responding to vertical stimuli are plotted in black and those responding to horizontal stimuli in white. Vertical in the visual field (90◦) corresponds to a diagonal line at 135◦ in this plot. The small green dot in the middle marks the site where a patch of vertical-selective neurons were injected with a tracer chemical. The neurons to which that chemical propagated through lateral connections are colored red. Short-range lateral connections target all orientations equally, but long-range connections go to neurons that have similar orientation preferences and are extended along the orientation preference of the source neurons. (b) The same information plotted on a 2.5 mm × 2 mm section of the full orientation map to the right and below the injection site. The injected neurons are colored greenish cyan (80◦), and connect to other neurons with similar preferences. Measurements in monkeys show similar patchiness, but in monkey the connections do not usually extend as far along the orientation axis of the neuron (Sincich and Blasdel 2001). These results, theoretical analysis, and computational models suggest that the lateral connections play a significant role in orientation processing (Bednar and Miikkulainen 2000b; Gilbert 1998; Sirosh 1995). Reprinted with permission from Bosking et al. (1997), copyright 1997 by the Society for Neuroscience.

2.2.2 Development

Lateral connectivity patterns have been found to form gradually during early development. Before eye opening, lateral connections grow exuberantly and to long distances in a short period (Callaway and Katz 1990). The connections are then pruned into well-defined clusters (Callaway and Katz 1990, 1991; Dalva and Katz 1994; Gilbert 1992; Katz and Callaway 1992; Lowel¨ and Singer 1992; Luhmann et al. 1986). What process drives such pruning? Enormous amounts of genetic information would be required to specify each connection and each synaptic weight of the neurons in a cortical map. Instead, lateral connections develop in an activity-dependent manner. Several observations support this view:

1.When activity in ferret V1 is silenced using tetrodotoxin during early development, lateral connections remain broad and unspecific, and do not become patchy (Ruthazer and Stryker 1996).

262 Biological Background

2.If kittens are deprived of visual input during early development, the connections are much less patchy than normal (Callaway and Katz 1991; Ruthazer and Stryker 1996).

3.The patchy patterns can be altered by changing the input to the developing cortex. The resulting patterns reflect correlations in the input. For example, when a kitten is made strabismic, thereby removing correlations between the visual inputs in the two eyes, the lateral connections in the primary visual cortex organize differently, linking only the regions responding to the same eye (Lowel¨ and Singer 1992).

4.In the mouse somatosensory barrel cortex, sensory deprivation (by sectioning the input nerve) results in shorter and sparser lateral connections compared with a normally reared animal (McCasland, Bernardo, Probst, and Woolsey 1992).

These observations suggest that lateral connections, like afferent connections, develop based on correlations in the input. The development of these different types of connections may actually be strongly related. Lateral connection patterns form approximately at the same time as the afferent connections organize into topographic maps (Burkhalter et al. 1993; Dalva and Katz 1994; Katz and Callaway 1992). Although each individual lateral connection is weak, their total effect on neural activity can be substantial (Gilbert et al. 1990), and they can thereby affect how the afferent connections develop. Changes in afferent connections then change the activity patterns in the cortex, which in turn influences the organization of lateral connections. In this manner, the two sets of connections develop synergetically, eventually evolving to a state of equilibrium in the adult animal. This principle is formalized and tested in detail in the LISSOM model.

2.2.3 Computational and Functional Hypotheses

Given the above observations, several possible functions have been proposed for the long-range lateral connections in the cortex. The list below is by no means complete, but it represents several of the views currently debated, including those put forward in later chapters of this book.

Modulating and Controlling Cortical Responses

1.Lateral connections may amplify weak stimuli and suppress strong stimuli, thus normalizing cortical activity (Somers, Toth, Todorov, Rao, Kim, Nelson, Siapas, and Sur 1996; Stemmler, Usher, and Niebur 1995).

2.They may modulate responses to achieve sharp orientation tuning and hyperacuity (Edelman 1996; Sabatini 1996; Somers et al. 1996).

3.They may combine responses to establish rotational and scaling invariance (Edelman 1996; Wiskott and von der Malsburg 1996).

4.They may mediate competition and synchronization over large distances of cortex (Taylor and Alavi 1996; Usher, Stemmler, and Niebur 1996; Wang 1996).

5.They may selectively enhance and suppress responses to implement attention and control (Taylor and Alavi 1996).

Representing and Associating Information

1.Lateral connections may store information that allows decorrelating visual input and filtering out known statistical redundancies in the cortical representations (Barlow and Foldi¨ak´ 1989; Dong 1996; Ghahramani and Hinton 1998; Sirosh, Miikkulainen, and Bednar 1996a).

2.They may help establish direction selectivity and motion sensitivity (Ernst, Pawelzik, Sahar-Pikielny, and Tsodyks 2001; Marshall 1990).

3.They alone may be responsible for orientation selectivity in the cortex (Adorjan,´ Levitt, Lund, and Obermayer 1999; Ernst et al. 2001).

4.They may store information for feature binding and grouping, such as Gestalt rules (Choe and Miikkulainen 1997; Edelman 1996; Polat, Norcia, and Sagi 1996; Prodohl,¨ Wurtz,¨ and von der Malsburg 2003; Singer et al. 1990; von der Malsburg and Singer 1988; Wang 1996).

5.They may associate representations at different sensory cortices, serving as a foundation for multi-modal integration (Choe 2002; Lewis and Van Essen 2000; Shipp, Blanton, and Zeki 1998).

Mediating Development, Plasticity, and Learning

1.Lateral interactions may play a crucial role in the development of cortical columns, such as those representing orientation, ocular dominance, spatial frequency, and direction selectivity (Bednar and Miikkulainen 2003b; Dong 1996; Edelman 1996; Sirosh et al. 1996a).

2.They may mediate reorganization of the cortex in response to drastic changes in the input environment (such as retinal lesions and input deprivation; Gilbert and Wiesel 1992; Kapadia et al. 1994; Pettet and Gilbert 1992; Sirosh et al. 1996a).

3.They may mediate the perceptual learning processes observed as early as the primary visual cortex by encoding local associations (Dong 1996; Edelman 1996; Usher et al. 1996).

4.They may form the substrate for encoding memories as attractors in the cortical network (Miikkulainen 1992; Taylor and Alavi 1996).

5.Shared lateral connections may explain how similar orientation maps can develop for both eyes, even if the eyes are alternately sutured shut so that they never experience similar input (Kim and Bonhoeffer 1994; Shouval, Goldberg, Jones, Beckerman, and Cooper 2000).

Mediating Visual Phenomena

1.Lateral connections may mediate visual comparisons, such as those necessary for object recognition, figure-ground discrimination, and segmentation (Edelman 1996; Marshall and Alley 1996; Somers et al. 1996; Sporns, Tononi, and Edelman 1991; Wang 1996; Wiskott and von der Malsburg 1996).

282 Biological Background

2.They may mediate perceptual filling in, such as compensating for blind spots, perceptual completion and illusory contours (Choe 2001; Finkel and Edelman 1989; Grossberg and Mingolla 1985; Li 1998, 1999; Somers et al. 1996; Usher et al. 1996).

3.They may be responsible for visual illusions, such as the tilt illusion, brightnesscontrast illusion, and Poggendorf illusion, which involve interactions between neighboring feature detectors (Bednar and Miikkulainen 2000b; Usher et al. 1996; Yu and Choe 2004; Yu, Yamauchi, and Choe 2004).

4.Adaptation of lateral connections may be responsible for temporary, patternspecific visual aftereffects, due to increasing lateral inhibition between activated neurons (Barlow 1990; Bednar 1997; Bednar and Miikkulainen 2000b).

5.Lateral connections between different ocular dominance areas and disparityselective neurons may contribute to binocular fusion, depth perception and stereo vision (Cormack and Riddle 1996; Lowel¨ 1994; Lowel¨ and Singer 1992; Petrov 2002).

The LISSOM model is based on the idea that lateral connections are crucial for the computations that take place in the visual cortex. In Part II of the book, inhibitory long-range lateral connections are shown to play a central role in self-organization. The LISSOM visual cortex is in a dynamic equilibrium, constantly adapting to both external and internal input. As a result, the observed structures of feature preferences develop, as well as patchy lateral connectivity between them (Chapter 5). The mechanisms are also active in the adult, implementing repair after retinal or cortical damage (Chapter 6), and resulting in psychophysical phenomena such as visual illusions and aftereffects (Chapter 7). The experimental data specific to these phenomena will be reviewed in the beginning of those chapters. Part IV will focus on excitatory lateral connections, showing how they can mediate binding and segmentation in a spiking-neuron model of the visual cortex. Part III, however, will focus on how environmentally and internally directed self-organization can implement a synergy of nature and nurture in development. The theoretical and biological foundations for this idea are reviewed next.

2.3 Genetic Versus Environmental Factors in Development

The LISSOM model will demonstrate how input-driven self-organization can account for the afferent and lateral connection structures in the visual cortex. As was discussed in Chapter 1, the most obvious source for such inputs is the visual environment during early life. However, the visual cortex already has a significant amount of structure before the visual experience begins, i.e. at birth or at eye opening. Such structure must be at least partially determined genetically. Why is it useful to include both genetic and environmental influences in constructing the visual cortex, and how can a developmental process combine them? These issues will be discussed in this section, providing the motivation for the computational studies of prenatal and postnatal development in Part III.

2.3.1 Bias/Variance Tradeoff

Why did evolution result in a developmental process that utilizes both genetic and environmental information, as opposed to a pure hardwiring or a pure tabula rasa learning process? This issue can be understood in terms of the well-known bias/variance tradeoff in machine learning (Geman, Bienenstock, and Doursat 1992; Utgoff and Mitchell 1982). Given a set of example inputs and outputs (the training set), a learning system needs to construct a mapping that produces correct outputs for new examples (the test set). There is often a very large number of possible mappings consistent with the training set, and they result in different outputs for the same test inputs. Which mapping will be selected is determined by the bias of the learner. The best results are obtained if the bias matches the problem and is strong (Haussler 1988). That way, the outputs for new examples are likely to be correct. Also, the same mapping is selected with different training sets and even with noisy training examples, i.e. the learner will have a low variance.

Unfortunately, it is not usually clear what the right bias is, and it is necessary to make the bias weaker. Which mapping will be selected will then depend more on the training data. As a result, the variance is increased: The selection of the mapping becomes unpredictable, determined based on which examples were included in the training set and the noise in those examples. Choosing an appropriate point in the bias/variance tradeoff therefore depends on how much is known about the problem in advance.

Biological systems face the same tradeoff: Neural structures can be determined genetically or learned from environmental inputs. A strong genetic bias is appropriate for organisms whose environment is predictable over many individual lifetimes, such as most invertebrates. For instance, the nematode worm Caenorhabditis elegans develops a nervous system with exactly 302 neurons in the same configuration in every individual (Sulston and Horvitz 1977). Such a strong bias allows the worm to function in its environment reliably and immediately.

However, the environment faced by mammals is much more complex and variable, and only the large-scale structures can be specified with a strong bias. The same sensory and motor areas appear in the same cortical locations in all individuals of the same species (Rakic 1988; Shatz 1996). These structures can still vary, but only in extreme cases such as prenatal injury (Goldman-Rakic 1980). The smallscale structures, on the other hand, are constructed primarily through interaction with the environment, and have a high variance. The number of neurons, their specific arrangements, and the patterns of connections differ between individuals of the same species (Shatz 1996).

The reason for the lower bias and higher variance in higher animals is that their environment is less predictable. If the individual were to be constructed with a strong bias, it would not be able to adapt to the different environments during its lifetime, and would perform poorly. On the other hand, learning is unreliable; if the right kind of input is not received at the right time, the individual may not develop a crucial skill (Blakemore and Cooper 1970; Hirsch and Spinelli 1970; Hubel and Wiesel 1974; Issa et al. 1999). Evolution has therefore determined a point in the bias/variance

30 2 Biological Background

tradeoff that allows constructing a reliable but flexible system by combining genetic and environmental information.

How this idea can be utilized in constructing complex natural or artificial systems in general will be discussed in Sections 16.2.3 and 17.3.5. How it can be implemented specifically to construct the visual system of higher animals will be analyzed next.

2.3.2 Combining Genetic and Environmental Information

The large-scale structures of the brain, such as the pattern of the different brain areas, are constructed primarily through chemical gradients (Molnar,´ Higashi, and Lopez´- Bendito 2003; Rakic 1988; von der Malsburg and Willshaw 1977; Willshaw and von der Malsburg 1979). These gradients direct the growing connections to a general location on the cortical sheet. The gradients are largely unaffected by environmental stimuli, making the bias very strong. Incorporating environmental information into this process would be difficult, requiring a transduction mechanism between an environmental stimulus and the developmental hardware.

On the other hand, at the level of individual neurons and connections between small groups, sensory systems act as just such a transduction mechanism. In a sensory system, patterns in the environment are represented as patterns in neural activity, and these patterns in turn change how the orientation, ocular dominance, and similar map-level organizations in the cortex develop (as discussed above). At this level, the question becomes how genetic cues could be expressed. First, the system is structured to utilize information in input activity; second, the amount of information necessary to specify individual connections may be too large to store genetically.

The recent discovery of spontaneous activation provides an important clue: Much of the neural activity in developing sensory systems is not caused by the external environment, but generated internally in many cortical and subcortical sensory areas, such as the visual cortex, the retina, the auditory system, and the spinal cord (Feller et al. 1996; Lippe 1994; Meister, Wong, Baylor, and Shatz 1991; Peinado, Yuste, and Katz 1993; Wong, Meister, and Shatz 1993; Yuste, Nelson, Rubin, and Katz 1995; see O’Donovan 1999; Sengpiel and Kind 2002; Wong 1999 for reviews). This activity may express a genetic bias within a system that is designed to learn from the environment (Constantine-Paton et al. 1990; Maffei and Galli-Resta 1990; Marks et al. 1995; Roffwarg et al. 1966; Shatz 1990, 1996). The genetic information is represented in the same way at the neural level: as patterns of activity in the input seen by a brain area. The genome thus needs to specify only a pattern generator, a mechanism capable of producing visual-like patterns, rather than specifying individual connections.

The result is a genetic specification of potentially complex neural hardware. Such a specification is desirable in an evolutionary sense, because different functional architectures can be obtained by changing only a small part of the genome (Jouvet 1980). Random mutations in that portion of the genetic code would cause different patterns to be generated, which might lead to different cortical structures. Such a

mechanism would facilitate evolutionary search, because it increases the chance that a chance mutation leads to a meaningful change.

The pattern generation hypothesis can potentially explain much of the experimental data on innate visual capabilities. The following two subsections present evidence that two specific types of internally generated activity patterns, retinal waves and ponto-geniculo-occipital waves, implement a genetic bias on visual cortex structures. These patterns will be crucial for the LISSOM model of how V1 and faceselective cortical areas are constructed, as will be discussed in detail in Part III.

2.3.3 Retinal Waves

In the developing retina of e.g. cats and ferrets, internally generated activity occurs as intermittent, local waves across groups of ganglion cells (Figure 1.2; Meister et al. 1991; Sirosh 1995; Wong et al. 1993). The waves begin before photoreceptors have developed (Maffei and Galli-Resta 1990), so they cannot result from visual input. Instead, they arise from spontaneous recurrent activity in networks of developing amacrine cells that provide input to the ganglion cells (Catsicas and Mobbs 1995; Feller et al. 1996; Shatz 1996). Like visual images, these waves are locally coherent in space and time (i.e. nearby ganglion cells are likely to be active continuously), and thus they could act as training input for the developing LGN and visual cortex (Shatz 1990).

Several experimenters have shown that interfering with the spontaneous activity can change how the visual system develops (Grubb, Rossi, Changeux, and Thompson 2003; McLaughlin, Torborg, Feller, and O’Leary 2003; Shatz 1990; Stellwagen and Shatz 2002). For instance, when the retinal waves are abolished, the inputs from the two eyes are no longer segregated in the LGN (Chapman 2000; Shatz 1996)). Similarly, when activity is silenced at the V1 level during early development, V1 neurons in mature animals are much less selective for orientation (Chapman and Stryker 1993). These results suggest that spontaneous activity is crucial for normal development of low-level vision.

Recent experiments have focused on whether spontaneous activity is merely permissive for development, perhaps by keeping newly formed connections alive until visual input occurs, or whether it is truly instructive, determining how the structures develop (Chapman, Godecke,¨ and Bonhoeffer 1999; Crair 1999; Katz and Shatz 1996; Miller, Erwin, and Kayser 1999; Penn and Shatz 1999; Sur, Angelucci, and Sharma 1999; Sur and Leamey 2001; Thompson 1997). For instance, Weliky and Katz (1997) artificially activated a large number of axons in the optic nerve of ferrets, thereby disrupting the pattern of spontaneous retinal activity. Even though this manipulation increased the total amount of activity, thereby making sure it was as permissive as before, V1 actually became less selective for orientation. Thus, spontaneous activity cannot only be permissive; it has at least some specific instructional role.

Similarly, pharmacologically increasing the number of retinal waves in one eye has been shown to prevent the LGN from developing normally (Stellwagen and Shatz 2002; cf. Crowley and Katz 2000). Yet, when waves are increased in both eyes, the

32 2 Biological Background

V1

LGN

Fig. 2.8. Spontaneous activity in the cat PGO pathway. Each line shows a 65-second electrode recording from a cell in the indicated area during REM sleep in the cat. Spontaneous REM sleep activation in the pons of the brainstem is relayed to the LGN of the thalamus (bottom), to the primary visual cortex (top), and to many other regions in the cortex. It is not yet known what spatial patterns of visual cortex activation are associated with this temporal activity, or with other types of internally generated activity during sleep. However, such activity is largely genetically determined and could affect how the visual system develops. Reprinted with permission from Marks et al. (1995), copyright 1995 by Elsevier.

LGN develops normally, which again shows that the type of activity is important, not simply whether there is activity or not. However, what features of the activity are important are not known, because it has not yet been possible to manipulate the activity precisely. The LISSOM model will be used in Chapter 9 to study this issue computationally.

2.3.4 Ponto-Geniculo-Occipital Waves

Retinal waves are the best-studied source of spontaneous activity, because they are easily accessible to experimenters. However, other internally generated patterns may also be important for the development of the visual cortex. One example is the ponto- geniculo-occipital (PGO) waves that are the hallmark of rapid-eye-movement (REM) sleep in at least cats, ferrets, monkeys, and humans (see Steriade, Pare,´ Bouhassira, Deschenes,ˆ and Oakson 1989 for a review; Figure 2.8).

During and just before REM sleep, PGO waves originate in the brainstem and travel to the LGN, many areas of the visual cortex, and a variety of subcortical areas (see Callaway, Lydic, Baghdoyan, and Hobson 1987 for a review). In adults, PGO waves are strongly correlated with eye movements and with vivid visual imagery in dreams, suggesting that they activate the visual system as if they were visual inputs (Marks et al. 1995). Experimental studies also suggest that PGO waves are under genetic control: They elicit different activity patterns in different species (Datta 1997), and the eye movement patterns that are associated with PGO waves are more similar in identical twins than in unrelated age-matched subjects (Chouvet, Blois, Debilly, and Jouvet 1983). Thus, PGO waves are a possible source for genetically controlled training patterns for the visual system. But do they actually serve this role?

REM sleep has long been believed to be important for development, for two reasons (Roffwarg et al. 1966): Developing mammalian embryos spend a large percentage of their time in states that look much like adult REM sleep, and the duration of REM sleep is strongly correlated with how plastic the neural system is, both over development and across different species (also see the more recent review by Siegel 1999, as well as Jouvet 1980). Also consistent with Roffwarg et al.’s hypothesis,

blocking REM sleep or the PGO waves alone has been found to increase the effect of visual experience during development (Marks et al. 1995; Oksenberg, Shaffery, Marks, Speciale, Mihailoff, and Roffwarg 1996; Pompeiano, Pompeiano, and Corvaja 1995). When the visual input to one eye of a normal kitten is blocked for a short time during a critical period, the cortical and LGN area devoted to signals from the other eye increases (Blakemore and van Sluyters 1975). When REM sleep (or just the PGO waves) is interrupted as well, the effect of blocking one eye’s visual input is even stronger (Marks et al. 1995). This result suggests that REM sleep, and PGO waves in particular, limits or counteracts the effects of visual experience.

All of these characteristics suggest that PGO waves and other REM-sleep activity may be instructing development, like the retinal waves do (Jouvet 1980, 1998; Marks et al. 1995). However, due to limitations in experimental imaging equipment and techniques, it has not yet been possible to measure their two-dimensional spatial structure (Rector, Poe, Redgrave, and Harper 1997). Chapters 9 and 10 in Part III of this book will evaluate different candidates for internally generated activity, including retinal and PGO waves, and show what structure they would need to have to explain how maps and their connections develop in the visual cortex.

2.4 Temporal Coding

Part IV of the book will present a theory of perceptual grouping in the visual cortex, demonstrating that self-organized lateral excitatory connections play a crucial role in this process. The model assumes that binding and segmentation are based on temporal coding, i.e. timing of neuronal spiking events. In this section, experimental evidence for temporal coding will be reviewed. Computational models derived from these observations will be described and compared in the next chapter.

2.4.1 Binding Through Synchronization

Neurons are cells with the special property of being able to convey information in terms of electrical pulses, or spikes. Traditional neural network theories have hypothesized that the level of activation, or the spiking rate of neurons, forms the representation for perceptual events. However, as von der Malsburg (1981, 1986a,b) pointed out, such static representations suffer from the superposition catastrophe (Figure 2.9). This problem arises when distributed neural representations of two (or more) separate objects overlap: It is no longer clear which neuron represents which object (Figure 2.9a).

In contrast, if the representations for the individual objects alternate in time, binding and segmentation can occur naturally through temporal coding (Figure 2.9b). Von der Malsburg (1986b; 1987) hypothesized that perceptual grouping can be achieved in this way through synchronized and desynchronized firing of neurons. Temporal coding is therefore one way in which perceptual grouping can occur in the brain, but is there reason to believe that it does?

342 Biological Background

Cortex

. . .

Time 0 Time 1 Time 2 Time 3

Fig. 2.9. Solving the superposition catastrophe through temporal coding. If firing rates of neurons alone are used to represent objects, multiple objects in the scene can result in confusion. (a) A square and a triangle are presented in the retina. In the cortex plot, the neurons responding to the square are colored gray, and those responding to the triangle white. When both populations of neurons are active at once, it is impossible to know which neuron is representing which object. This problem is known as the superposition catastrophe (von der Malsburg 1981, 1986a,b). One solution is temporal coding, where temporal information is used to separate the two populations. Neurons representing one object activate at one time step, and neurons representing the other object activate at the next time step, as shown in (b).

2.4.2 Experimental Evidence

Experimental studies have shown that coherent oscillations do indeed arise within populations of neurons. Such oscillations are usually observed as synchronized highfrequency waves near the 40 Hz γ-band (see Buzsaki´ and Draguhn 2004; Jefferys, Traub, and Whittington 1996 for reviews). To test whether such temporal representations are used in the visual system to present grouping, two approaches can be taken. One way is to present inputs to the visual system and measure the oscillations that result. The other is to alter the temporal properties in the input, preventing or enhancing synchronization, and measure the effect on perceptual performance.

Using the first approach, it has been possible to determine that activities of two populations with similar properties, such as the same orientation preference, do indeed synchronize when stimulated with a common input (Eckhorn et al. 1988; Gray et al. 1989; Gray and Singer 1987; Singer 1993). In one such study, electrical recordings were made on two sites in the cat visual cortex with non-overlapping receptive fields, while moving light bar(s) were swept across these receptive fields (Figure 2.10). When a single long bar was used as the input, the two populations representing distant sections of the long bar fired synchronously. However, when two short bars were swept in the same location as before but in the opposite direction of each other, the firing of the two populations was no longer synchronized. Interestingly, when two separate short bars were swept in the same direction, the two populations showed a weak but synchronized activity (Engel, Konig,¨ Kreiter, and Singer 1991a; Engel, Kreiter, Konig,¨ and Singer 1991b; Gray et al. 1989; Singer