Ординатура / Офтальмология / Английские материалы / Computational Maps in the Visual Cortex_Miikkulainen_2005

C

HLISSOM Simulation Specifications

The HLISSOM model extends basic LISSOM by including afferent normalization (Equation 8.1) that allows processing natural images, as well as a PGO sheet and a face-selective area. The PGO sheet is connected to the LGN and V1 is connected to the FSA so that each unit has a full receptive field (Figure A.1). The simulation parameters are based on the default LISSOM parameters of Appendix A, except as specified in the subsections that follow.

C.1 V1 Only

The V1-only simulations showing the effect of γn (Figures 8.2 and 8.3) were based on disk-shaped input patterns. The parameters were the same as in the “Disks” simulation in Section A.5.1, except the area was very large (sg = 8) to allow large, detailed retinal stimuli to be tested, the cortical density was very low (Nd = 24) to reduce memory and computational requirements, and the LGN radius was slightly smaller (σc = 0.4) for historical reasons. During self-organization, γn was zero; after self-organization, that parameter was adjusted as shown in the figure.

The small-scale V1-only HLISSOM simulations in Figure 8.5 and in Chapter 9 were based on the default LISSOM simulation parameters of Appendix A; in particular, afferent normalization was not used (γn = 0). In the input and weight stream simulations (Figure 8.5) the cortical density was lower (Nd = 64) so that the random initial weights would be visible.

The V1 simulation parameters were always evolved on a schedule for a 10,000iteration run; in the prenatal simulations of Figure 9.1, the training was simply interrupted at 1000. The prenatal “Disks” and “Noisy Disks” simulations were based on the same parameters as the full-length simulations described in Section A.5.1, up to iteration 1000. The prenatal “Noise” simulation was identical to “Noisy disks”, except no disks were drawn.

The “Nature” simulation in Figure 9.6 was based on the “Images” simulation described in Section A.5.2, except the inhibitory and excitatory strengths were fixed (γI = 2.0 and γE = 1.2), and the input density scale was higher (sd = 8) without

Table C.4. Parameter change schedule for combined V1 and FSA simulations. During iterations 20,000–28,000st, the network with both V1 and FSA (Section 10.2) was trained using the schedule for iterations 0–8000st in Table A.3. Beyond 28000st, it was trained as shown in this table. Because each cortical area has an independent set of parameters, the values for V1 and the FSA are listed separately.

In the statistical significance tests discussed in Figures 10.16 and 10.17, each input image was presented at 16 different locations chosen from the nodes of a regular 4 × 4 grid centered on the retina, with each grid step four retinal units wide. The example images shown in these figures are located at the center of the retina.

C.3 Combined V1 and Face-Selective Area

In the simulations of Section 10.2, V1 and FSA were combined into a single model. The retina, LGN, and V1 parameters were identical to the “Disks” simulation in Section A.5.1, except V1 density was very low (Nd = 24) to reduce memory and computational requirements, the V1 area was very large (sg = 8) to allow large retinal stimuli to be tested, the LGN radius was slightly smaller (σc = 0.4) to match earlier simulations, and the simulation continued past 20,000st. The FSA parameters were identical to the default FSA-only parameters (Section C.2), except the FSA was slightly more dense (Nd = 36/0.94), the input region was significantly more dense (Ld = 170) because the FSA connects to V1 and not to LGN, the area scale corresponded to the full area of V1 (sg = 0.94), the afferent radius was smaller (rA = 0.375Ld + 0.5) and the weights initially random because only small training patterns were used, the afferent scale was larger (γA = 3) to compensate for the patchy activity in the input region (V1), and the parameters were changed on the schedule listed in Table C.4. The FSA also had only one input (V1) instead of two (the ON and OFF LGN channels), i.e. nA = 1 for the FSA.

Because FSA training followed that of V1, V1 training iteration 20,000st was treated as if it was iteration zero in FSA training, and thus the parameters e.g. for iteration 28000st were determined like parameters for iteration 8000st of the default FSA simulation. After iteration 28000st, the V1 and FSA parameters followed the schedule shown in Table C.4. The V1 parameters were chosen to ensure that the network responded well to large natural images. First, the value of γn was gradually

increased from zero to make the responses less dependent on image contrast. As a result, similar shapes that have different contrasts in different areas of an image would lead to comparable V1 responses. The other parameters were adjusted to compensate for the effect of γn, ensuring that V1 responses to the highest-contrast patterns were near 1.0, and lower contrasts resulted in lower V1 responses. The parameters of the FSA were chosen similarly, except the FSA sigmoid threshold θl was also gradually increased so that the response would be nearly binary. In this way, the FSA response was used to decide whether a facelike pattern had occurred in the input, as described in Section 10.2.

D

PGLISSOM Simulation Specifications

The PGLISSOM simulations focused on demonstrating valid self-organization of the orientation map, and on grouping visual features. These simulations were based on the reduced LISSOM specifications of Appendix B, with slightly different parameter values to reduce the required computational resources.

D.1 Self-Organization

The self-organization simulations in Section 11.5 form the baseline for all PGLISSOM simulations. The parameter values for this simulation are specified in Table D.1 and the schedule for parameter adaptation over the course of self-organization in Table D.2. The parameter values were found to be effective by running several experiments, and small changes to them did not affect the global behavior of the model.

All weights were initialized with uniformly random values distributed within [0..1]; the afferent connections were mapped to the retina so that each neuron had a complete receptive field (Figure A.1). GMAP was smaller than SMAP to make simulations faster and more compact. The γC was lower in SMAP than in GMAP so that activity caused by high excitation in GMAP would not interfere with selforganization in SMAP, and to allow the self-organized global order of SMAP to be transferred to GMAP. The value γE was lower and γI higher in GMAP than in SMAP to prevent the map from becoming too active.

The inhibitory connections in GMAP did not adapt (αIG = 0); the initial broad connectivity remains to provide background inhibition, as explained in Section 11.2. The afferent and intracolumnar learning rates αA and αC in both maps were decreased over time, so that the order in the map could gradually start stabilizing. The long-range lateral inhibitory connections in SMAP and long-range excitatory lateral connections in GMAP were pruned on the same schedule, specified by wd and td.

The base thresholds θb in both maps were set to θb0 when the input was first presented, the activation-based values in Table D.1 were then computed, and the base thresholds were fixed to those values for the remaining settling iterations. As a result, the network would not become too active or completely silent for any of the training

438 D PGLISSOM Simulation Specifications

in such higher resolution; it also makes synchronization more robust, as discussed in Section 12.4.3.

Each contour integration test pattern was generated to approximate the patterns used with human subjects (such as that in Figure 13.1) as well as possible within the small model retina and cortex. The input patterns consisted of oriented Gaussians with σa = 1.87 and σb = 1.22. The elements of each contour were placed on the retina at approximately collinear locations (cocircular in the curvature experiment of Section 13.4) with a minimum separation of 0.5σa2 and an orientation that corresponded to the desired degree of jitter. In addition, the neurons in SMAP and GMAP had to be highly selective for that input in that location. The background elements were then placed on the retina at random locations and orientations with the same distance and selectivity constraints until no more possible such locations existed.

In the input distribution experiments in Section 13.4, slightly broader Gaussians were used to train the networks, and the Gaussians became elongated slightly slower. Initially, σa = 3.9 and σb = 1.1, and they were increased to σa = 7.1 and σb = 0.9 at iteration 10,000. As a result, self-organization was slower; on the other hand, the connectivity patterns remained broader, allowing the networks to perform robustly on the wider variety of inputs.

D.3 Synchronization

The synchronization and desynchronization simulations in Chapter 12 were all based on a simplified PGLISSOM network where synchronization behavior could be clearly demonstrated. The network consisted of a one-dimensional array of neurons, connected one-to-one to input and output. The input neurons spiked at every time step, and the membrane potential of each neuron was initialized to uniformly random distribution within [0..1]. The afferent weights were fixed at 1.0, the lateral excitatory weights at 1.0/nE (where nE is the number of incoming excitatory lateral connections of the neuron), and the lateral inhibitory weights at 1.0/nI (where nI is the number of incoming inhibitory lateral connections). The afferent contribution was set to γA = 0.63 and decay rate to λA = 1.0, spike generator decay rate was λθ = 0.05, sigmoid threshold and ceiling θl = 0.0, θu = 3.0, relative refractory threshold proportion γθ = 0.65, and base threshold θb = 0.1.

The remaining parameters were systematically varied in the experiments: (1) lateral excitatory and inhibitory connection patterns and radii rE and rI, (2) their contributions to the neuron activation γE and γI, (3) their synaptic decay rates λE and λI,

(4) the size and pattern of the afferent input, (5) the degree of noise in the membrane potential, and (6) the duration tr of the absolute refractory period.

E

SOM Simulation Specifications

In order to establish a baseline for comparison with LISSOM, the simulations in Chapter 3 were based on the SOM version of self-organizing maps, where the response is based on Euclidean distance similarity measure, and the weights are changed to minimize Euclidean difference between the input and the weight vector (Equation 3.15).

The default parameters, used in the basic self-organization simulation of Figure 3.6, are listed in Table E.1. The receptor surface was fully connected to each map unit. The connections were initialized to uniformly random values within [0..1]. The input consisted of single unoriented Gaussians whose centers were chosen from a uniformly random distribution so that they were evenly scattered over the receptor surface.

The magnification simulation (Figure 3.7) was run with the same network and learning parameters; the only difference was the distribution of the input Gaussians. The network was trained up to 10,000 iterations as before, at which point the input distribution was changed. Instead of a uniformly random distribution, in Figure 3.7a a Gaussian high-density area with center (0.5, 0.5) and radius 0.01 was added to a uniformly random distribution of range [0..0.2]. In Figure 3.7b, the distribution

consisted of a uniform distribution and two Gaussian high-density areas, one with center (0.5, 0.25) and the other (0.5, 0.75), and the major and minor axes of 0.05 and 0.0125 for both areas.

The three-dimensional model of ocular dominance (Figure 3.10) was abstracted further by representing the input location as a two-dimensional variable (x, y), instead of an activation pattern on a two-dimensional surface as in the two previous simulations. These values were drawn from a uniformly random distribution within [0..1]. The third variable, representing ocular dominance, was also uniformly randomly distributed, but within [−0.15..0.15]. The inputs were thus three-dimensional vectors of (x, y, z) values, fully connected to the map units. The network was

self-organized in tf = 120, 000 input presentations, with the adaptation rate = 0.1 exp(−4.0 t/tf ) and neighborhood width = max[13.3 exp(−5.0 t/tf ), 0.8].

The specifications for the SOM in handwritten digit recognition simulations will be described in Appendix F.2.