Ординатура / Офтальмология / Английские материалы / Computational Maps in the Visual Cortex_Miikkulainen_2005

4

LISSOM: A Computational Map Model of V1

The neurobiological observations and computational principles from Part I will be brought together in this chapter to construct a laterally connected map model of the primary visual cortex. The model is based on self-organizing maps, but its connectivity, activation, and learning mechanisms are designed to capture the essential biological processes in more detail. As will be seen in later chapters of Part II, these mechanisms lead to a detailed computational account for how the visual cortex develops, adapts, and functions. In Part III, this model is further extended to include subcortical areas and higher visual areas, and in Part IV, a temporal coding mechanism and connectivity necessary for perceptual grouping. In this chapter, the motivation for the model is first reviewed, followed by a detailed description of its organization, activation, and learning processes. A simple example of self-organization is given, forming a retinotopic map of visual input.

4.1 Motivation: Cortical Maps

As was discussed in Sections 3.4 and 3.5, self-organizing maps are a powerful abstraction of learning and knowledge organization in the cortex. However, much of the behavior of the visual cortex depends on structures and processes (reviewed in Sections 2.1 and 2.2) that have been abstracted away in many self-organizing map models. In contrast, the LISSOM model (laterally interconnected synergetically selforganizing map; Miikkulainen 1991; Miikkulainen, Bednar, Choe, and Sirosh 1997; Sirosh 1995) is designed specifically to capture those processes. More specifically, LISSOM is based on five principles:

1.The central layout of the LISSOM model is a two-dimensional array of computational units, corresponding to vertical columns in the cortex. Such columns act as functional units in the cortex, responding to similar inputs, and therefore form an appropriate level of granularity for a functional model.

2.Each unit receives input from a local anatomical receptive field in the retina, mediated by the ON-center and OFF-center channels of the LGN. Such connectivity corresponds to the neural anatomy; it also allows modeling a large area of

68 4 LISSOM: A Computational Map Model of V1

the visual cortex and processing large realistic visual inputs, which in turn allows studying higher visual function such as visual illusions, grouping, and face detection.

3.The cortical units are connected with excitatory and inhibitory lateral connections that adapt as an integral part of the self-organizing process. Before LISSOM, the function of these lateral interactions had not been analyzed in detailed computational models. They can play a central role in establishing an efficient visual representation, and in mediating functions such as illusions and perceptual grouping.

4.The units respond by computing a weighted sum of their input, limited by a logistic (sigmoid) nonlinearity. This is a standard model of computation in the neuronal units that matches their biological characteristics well.

5.The learning is based on Hebbian adaptation with divisive normalization. Hebbian learning is well supported by neurobiological data, and recent biological experiments have also suggested how normalization could occur in animals. These mechanisms are found to be computationally both necessary and powerful in explaining functional phenomena, such as the indirect tilt aftereffect.

In other words, LISSOM takes the central idea of self-organizing maps (1), and implements it at a level of known visual cortex structures (2 and 3) and processes (4 and 5). Although each of these principles has been tested in other models (Section 3.4.1), their combination is novel and allows LISSOM to account for a wide range of phenomena in the development, plasticity, and function of the primary visual cortex. The details of the model are described next, with examples to illustrate the processing that occurs.

4.2 The LISSOM Architecture

The LISSOM model of V1 is based on a simulated network of cortical neurons with afferent connections from the external world and recurrent lateral connections between neurons. These connections adapt based on correlated activity. The result is a self-organized structure where afferent connection weights form a map of the input space, and lateral connections store long-term correlations in neuronal activity. In this section, the layout and connectivity of the LISSOM model are described in detail.

4.2.1 Overview

LISSOM is intended to model accurately the biological structures and processes that are most important for the observed developmental and functional phenomena. Other biological features are abstracted in the model, in order to reduce confounding factors and to make an efficient, systematic analysis possible.

The V1 network in LISSOM is a sheet of N × N interconnected computational units, or “neurons” (Figure 4.1). Because the focus is on the two-dimensional organization of the cortex, each neuron in V1 corresponds to a vertical column of cells

V1

LGN

Retina

Fig. 4.1. Architecture of the basic LISSOM model. LISSOM consists of a hierarchy of two-dimensional sheets of neural units, including an array of retinal receptors, ON and OFF channels in the LGN, and a cortical network representing V1. The LGN and V1 activation is shown in gray-scale coding from white to black (low to high). The activity on the retina (a single oriented Gaussian) is presented like natural images: Light areas are strongly activated, dark areas are weakly activated, and medium gray represents background activation. This input gray scale will be used for all models that include the LGN and which can therefore process natural images. Sample connections are shown for one unit in each LGN sheet and one in V1. The LGN afferents form a local anatomical receptive field on the retina, and cause ON-center LGN units to respond to light areas surrounded by dark, and OFF-center units to dark areas surrounded by light. Neighboring LGN neurons have different but overlapping RFs. Similarly, V1 neurons have afferent receptive fields on the LGN sheets. V1 neurons also receive lateral excitatory and lateral inhibitory connections from nearby V1 neurons; these connections are shown as dotted and dashed circles around the V1 neuron, respectively. V1 activity is patchy because only those neurons respond whose feature preferences match the orientation, eye of origin, and direction of movement of the pattern currently in their receptive fields.

through the six layers of the biological cortex. This columnar organization helps make the problem of simulating such a large number of neurons tractable, and is viable because the cells in a column generally fire in response to the same inputs (Section 2.1.2). The activity of each neuron is represented by a continuous number within [0..1]; individual spiking is not modeled in basic LISSOM. Therefore, it is important to keep in mind that LISSOM neurons are not strictly identifiable with single cells in the biological cortex; instead, LISSOM models biological mechanisms at an aggregate level.

Each cortical neuron receives external input from two types of neurons in the LGN: ON-center and OFF-center. The LGN neurons in turn receive input from a small area of the retina, represented as an R × R array of photoreceptor cells. The afferent input connections from the retina to LGN and LGN to V1 are all excitatory. In addition to the afferent connections, each cortical neuron has reciprocal excitatory and inhibitory lateral connections with other neurons. Lateral excitatory connections have a short range, connecting only close neighbors in the map. Lateral inhibitory connections run for long distances, but may be patchy, connecting only selected neurons.

70 4 LISSOM: A Computational Map Model of V1

The ON and OFF neurons in the LGN represent the entire pathway from photoreceptor output to the V1 input, including the ON/OFF processing in the retinal ganglion cells and the LGN. Although the ON and OFF neurons are not always physically separated in the biological pathways, for conceptual clarity they are divided into separate channels in LISSOM. Each of these channels is further organized into an L × L array corresponding to the retinotopic organization of the LGN. For simplicity and computational efficiency, only single ON and OFF channels are used in LISSOM simulations in this book, but multiple channels could be included to represent different spatial frequencies. Also, the photoreceptors are uniformly distributed over the retina; since the inputs are relatively small in the LISSOM experiments, the fovea/periphery distinction is not crucial for the basic model.

Each neuron develops an initial response as a weighted sum (scalar product) of the activation in its afferent input connections. The lateral interactions between cortical neurons then focus the initial activation pattern into a localized response on the map. After the pattern has stabilized, the connection weights of cortical neurons are modified. As the self-organization progresses, these neurons grow more nonlinear and weak connections die off. The result is a self-organized structure in a dynamic equilibrium with the input.

The following subsections describe the specific components of the LISSOM model in more detail. They focus on the basic version of the model trained with unoriented Gaussian inputs, to highlight the basic principles as clearly as possible. In Chapter 5 this model is extended to two retinas, more complex Gaussian inputs, and natural images, in order to study how orientation preference, ocular dominance, and direction selectivity develops in the visual cortex. In Chapters 6 and 7, the model is temporarily abstracted further by bypassing the LGN, and this more efficient model is used to study plasticity and visual illusions. In Part III, the model is extended to include subcortical and higher level areas, in order to understand the synergy of genetically and environmentally driven development. Finally, Part IV will describe an extension that includes spiking neurons and long-range excitatory connections, to explain how perceptual grouping occurs.

4.2.2 Connections to the LGN

Previous models have explained how the connections from the retina to the LGN could develop from internally generated activity in the retina (Eglen 1997; Elliott and Shadbolt 1999; Haith 1998; Keesing, Stork, and Shatz 1992; Lee, Eglen, and Wong 2002a). LISSOM instead focuses on learning at the cortical level, so all connections to neurons in the ON and OFF channels are set to fixed strengths.

The strengths were chosen to approximate the receptive fields that have been measured in adult LGN cells, using a standard difference-of-Gaussians model (DoG; Cai, DeAngelis, and Freeman 1997; Rodieck 1965; Tavazoie and Reid 2000). First, the center of each LGN receptive field is mapped to the location in the retina corresponding to the location of the LGN unit (appendix Figure A.1). This mapping ensures that the LGN will have the same two-dimensional topographic organization as the retina. Using that location as the center, the weights are then calculated from

Fig. 4.2. Afferent weights of ON and OFF neurons in the LGN. The receptive fields of a sample ON neuron and a sample OFF neuron, both located at the center of the corresponding 36 × 36 LGN sheet, are shown as gray-scale values on a 54 × 54 retina. The jagged black line traces the anatomical boundary of the RF, that is, connections exist only from retinal receptors located inside the boundary. Medium gray represents zero weight, dark gray stands for inhibitory connections, and light gray represents excitatory connections. Each RF shape is a difference of two Gaussians, the center and the surround. The Gaussians are normalized to have the same total strength, but the center Gaussian concentrates that strength in a much smaller region; in this example, σc = 0.5 and σs = 2. ON cells have an excitatory center and an inhibitory surround (a), and OFF cells have an inhibitory center and an excitatory surround (b), as in Figure 2.2a,b. These RFs perform edge detection at a spatial frequency determined by the width of the center; they highlight areas of the input image that have edges and lines, and do not respond to large areas of constant illumination.

the difference of two normalized Gaussians. More precisely, the weight Lxy,ab from receptor (x, y) in the receptive field of an ON-center cell (a, b) with center (xc, yc) is given by

where σc determines the width of the central Gaussian and σs the width of the surround Gaussian. The weights for an OFF-center cell are the negative of the ON-center weights, i.e. they are calculated as the surround minus the center. Figure 4.2 shows examples of such ON and OFF receptive fields.

Note that even though the OFF cells have the same weights as ON cells (differing only by the sign), their activities are not redundant. Since the firing rates in biological systems cannot be negative, each cell is thresholded to have only positive activations, as described in more detail in Section 4.3.2. As a result, the ON and OFF cells will never be active at the same cortical location. They therefore provide complementary information, both in the model and in the visual system. Separating the ON and OFF channels in this way makes it convenient to compare the model with experimental results.

72 4 LISSOM: A Computational Map Model of V1

Fig. 4.3. Initial V1 afferent and lateral weights. The initial incoming weights of a sample neuron at the center of V1 are plotted in gray-scale coding from white to black (low to high). (a) The afferent RF center for each neuron was determined by first finding the location on the LGN that corresponds to the location of the neuron in V1, then randomly scattering the center by ±1.2 retinal units (5%) from that location. The neuron was then connected to LGN units within a radius of 6.5 from the center with random normalized weights. These weights are shown on the 36 × 36 LGN sheet, with the jagged black line tracing the RF boundary. The weights from the ON and OFF LGN sheets were initially identical. Plots (b) and (c) similarly show the lateral weights of this neuron, plotted on the 142 × 142 V1 and outlined with a black line. The neuron itself is marked with a small white square in the middle. The excitatory connections were initially random within a radius of 14.2, and inhibitory within 34.5 units. Later figures will show how these connections become organized through input-driven selforganization.

4.2.3 Connections in the Cortex

In contrast to the fixed connection weights in the LGN, all connections in cortical regions in LISSOM are modifiable by neural activity. They are initialized according to the gross anatomy of the visual cortex, with weight values that provide a neutral starting point for self-organization.

Each neuron’s afferent receptive field center is located randomly within a small radius of its optimal position, i.e. the point corresponding to the neuron’s location in the cortical sheet. The neuron is connected to all ON and OFF neurons within radius rA from the center (Figures 4.3a and A.1). For proper self-organization to occur, the radius rA must be large compared with the scatter of the centers, and the RFs of neighboring neurons must overlap significantly, as they do in the cortex (Sirosh and Miikkulainen 1997).

Lateral excitatory connections are short range, connecting each neuron to itself and to its neighbors within a close radius. The extent of lateral excitation should be comparable to the activity correlations in the input. Lateral inhibitory connections extend in a larger radius, and also include connections from the neuron itself and from its neighbors (Figure 4.3b,c). The range of lateral inhibition may vary as long as it is greater than the excitatory radius. This overall center–surround pattern is crucial for self-organization, and approximates the lateral interactions that take place at high contrasts in the cortex (Section 2.2.1). Long-range excitatory connections can

74 4 LISSOM: A Computational Map Model of V1

Fig. 4.4. Example input and response. At each self-organization iteration in LISSOM, the photoreceptors in the retina are activated according to the gray-scale values of the input image (as described in Figure 4.1). In this example, two unoriented Gaussians with width σu = 3.0 were drawn on random spatially separated locations on a 54 × 54 array of receptors (a). The 36 × 36 ON and OFF cell responses are plotted in the central square of (b) by subtracting the OFF cell responses from the ON: Dark areas represent higher activity in the OFF channel, light areas higher activity in the ON channel, and medium gray represents equal activation in both channels. The LGN responds to edges and lines in the input, with high ON cell activations where the input is brighter than the surround, and high OFF cell activations where the input is darker than the surround. Before self-organization (i.e. iteration 0), the 142 × 142 V1 map initially responds broadly and unspecifically to the input patterns (c); V1 activations are represented in the central square in gray scale from white to black (low to high). The lateral connections focus the response into discrete activity “bubbles” in (d), and connections are then modified. After 10,000 input presentations and learning steps, the initial and settled V1 responses are more focused, forming a sparse representation of the input (e and f ). This figure also illustrates how the retina is mapped to the LGN and the LGN mapped to V1 in LISSOM. The LGN and V1 networks are drawn to the same scale as the retina (as indicated by the outside squares), so that activity at a given location in the LGN and V1 corresponds to a stimulus at the corresponding location in the retina. The retina is larger than the LGN and the LGN larger than V1 so that all LGN and V1 neurons have complete receptive fields (i.e. they are not cut off by the network boundary; Figure A.1). In subsequent activity figures, such padding is omitted and only the retinal and LGN area that matches the V1 network is shown. An animated demo of the map response can be seen at http://computationalmaps.org.

where χxy is the activation of cell (x, y) in the receptive field of (a, b), Lxy,ab is the afferent weight from (x, y) to (a, b), and γL is a constant scaling factor. The squashing function σ(·) (Figure 4.5) is a piecewise linear approximation of the sigmoid activation function mentioned in Section 3.1.4:

As in other models, this approximation is used because it implements the essential thresholding and saturation behavior, and can be computed more quickly than a smooth logistic function.

Changing γL in Equation 4.3 by a factor m is equivalent to dividing θl and θu by m. Even so, γL is treated as a separate parameter to make it simpler to use the same values of θl and θu for different networks. The specific value of γL is set man-

Fig. 4.5. Neuron activation function σ(s). The neuron requires an input as large as the threshold θl before responding, and saturates at the ceiling θu. The output activation values are limited to [0..1]. This activation function is an efficient approximation of the logistic (sigmoid) function.

ually so that the LGN outputs approach 1.0 in the highest-contrast regions of typical input patterns. This allows each subsequent level to use similar parameter values in general, other than γL.

Because of its DoG-shaped receptive field, an LGN neuron will respond whenever the input pattern is a better match to the central portion of the RF than to the surrounding portion. The positive and negative portions of the RF thus have a push– pull effect (Hirsch, Alonso, Reid, and Martinez 1998a; Hubel and Wiesel 1962). That is, even if an input pattern activates the ON portion of the LGN RF, the neuron will not fire unless the OFF portion is not activated. This balance ensures that the neurons will remain selective for edges over a wide range of brightness levels. Section 6.2.3 will show that this push–pull effect is crucial when natural images are used as input to the model. Overall, the LGN neurons respond to image contrast, subject to the minimum and maximum activity values enforced by the activation function.

4.3.3 Cortical Activation

The cortical activation mechanism is similar to that of the LGN, but extended to support self-organization and to include lateral interactions. The total activation is computed by combining the afferent and lateral contributions. First, the afferent stimulation sij of V1 neuron (i, j) is calculated as a weighted sum of activations in its receptive fields on the LGN:

where ξab is the activation of neuron (a, b) in the receptive field of neuron (i, j) in the ON or OFF channels, Aab,ij is the corresponding afferent weight, and γA is a constant scaling factor. The afferent stimulation is squashed using the sigmoid activation function, forming the neuron’s initial response as

76 4 LISSOM: A Computational Map Model of V1

where σ(·) is a piecewise linear sigmoid as in Equation 4.4.

After the initial response, lateral interaction sharpens and strengthens the cortical activity over a very short time scale. At each of these subsequent discrete time steps, the neuron combines the afferent stimulation s with lateral excitation and inhibition:

where ηkl (t − 1) is the activity of another cortical neuron (k, l) during the previous time step, Ekl,ij is the excitatory lateral connection weight on the connection from that neuron to neuron (i, j), and Ikl,ij is the inhibitory connection weight. All connection weights have positive values. The scaling factors γE and γI represent the relative strengths of excitatory and inhibitory lateral interactions, which determine how easily the neuron reaches full activation.

The cortical activity pattern starts out diffuse and spread over a substantial part of the map (Figure 4.4c,e). Within a few iterations of Equation 4.7, it converges into a small number of stable focused patches of activity, or activity bubbles (Figure 4.4d,f ). Such settling results in a sparse final activation, which allows representing visual information efficiently (Section 14.2; Barlow 1972; Field 1994). It also ensures that nearby neurons have similar patterns of activity and therefore encode similar information, as seen in the cortex.

While the cortical response is settling, the afferent input remains constant. However, the lateral interaction is not strong enough to maintain the activity bubble when the input changes. A change in the input will cause the net input of the neurons (i.e. the sums of the afferent and lateral activations) to fall below the threshold θl. A new response will then form with little interference from the previous response. The LISSOM network could therefore be trained even with continuously changing inputs, without explicitly resetting the network to zero activity in between.

4.4 Learning

Self-organization of the connection weights takes place in successive input iterations, usually 5000–20,000 in total. Each iteration consists of presenting an input image, computing the corresponding settled activation patterns in each neural sheet, and modifying the weights. Weak lateral connections are periodically removed, modeling connection death in biological systems. In order to achieve smooth maps, the lateral excitation radius, sigmoid, and learning rate parameters can be gradually adjusted over the course of learning.

4.4.1 Weight Adaptation

After the activity has settled, the connection weights of each cortical neuron are modified. Both the afferent and lateral weights adapt according to the same biologically motivated mechanism: the Hebb rule (Hebb 1949) with divisive postsynaptic normalization (Section 3.3):