Ординатура / Офтальмология / Английские материалы / Computational Maps in the Visual Cortex_Miikkulainen_2005

294 13 Understanding Perceptual Grouping: Contour Integration

close by on the map. As shown in Figure 11.5, the excitatory connections not only connect to neurons with similar orientation preferences, but at a close range also to those with somewhat different orientation preferences. Thus, proximity of the inputs, as well as the good continuation of contours, determines the degree of synchronization. At the vertices, the two abutting inputs cause the corresponding cortical areas to synchronize, which in turn causes the three sides of the triangle to synchronize. As a result, the network represents the whole triangle as a coherent object.

These results show that PGLISSOM indeed performs contour completion, and also forms representations for whole objects. Inputs can be grouped through proximity as well as through good continuation. Such mechanisms arise automatically from the properties of afferent and lateral connections in the model, and may form a general principle for grouping (ordinary and illusory) in the visual system.

13.3.5 Salience of Closed Versus Open Contours

With a thorough understanding of how contour completion occurs in the model, let us now return to the psychophysical observation that closed contours are easier to detect than open contours (Kovacs and Julesz 1993; Pettet et al. 1998; Tversky et al. 2004). As discussed in Section 13.1.1, the most recent evidence suggests that there is no special reverberatory mechanism around the closed contour: The advantage is due to proximity and good continuation between elements (Tversky et al. 2004).

While it is difficult to replicate the control conditions in a small retina of PGLISSOM, the illusory triangle experiment in Section 13.3.2 can be used to test the fundamental principle of this theory computationally. The complete and incomplete triangles in Figure 13.16 form closed and open contours. Indeed, the perception of an illusory triangle breaks when one component is removed from the Kanizsa triangle (Figure 13.17).

To measure how salient the two objects are, the average correlation coefficients between the nine elements of the complete illusory triangle (elements 1 through to 9) and seven elements of the incomplete triangle (1–5 and 8–9) were calculated. The results are shown in Figure 13.19. The activities in the network for the closed contour are significantly more synchronized than those of the open contour, indicating that the closed contour is more salient.

The explanation for this effect in PGLISSOM is straightforward. Every part of the closed contour receives excitatory lateral contribution from both neighboring areas, and strong synchronization results along the contour. In contrast, at the two ends of an open contour the neurons only receive lateral excitation from one neighboring area, and the synchrony does not reach the same level of salience.

In this simple experiment, PGLISSOM has provided an independent computational confirmation of the current psychophysical theory on closed vs. open contours. The difference arises from local interactions based on proximity and good continuation, without a separate reverberatory mechanism. In the future, larger PGLISSOM models can be used to replicate the actual conditions in human experiments, leading to more detailed predictions and insights into this phenomenon.

Fig. 13.19. Contour completion performance in closed vs. open contours. The average correlation coefficients between the MUA sequences in the complete and incomplete triangles (labeled “closed” and “open”) of Figure 13.16 are shown, calculated over two trials and different groups of contour elements. Although the differences in correlation within each side are not significant (p > 0.2), the elements in the closed contour are significantly more correlated across the sides and within the whole object than in the open contour (p < 0.004), consistent with psychophysical results (Section 13.1.1). The correlations between contour and background elements and within the background were significantly weaker in both cases (p < 0.03), indicating that both contours were perceived as single objects. The PGLISSOM model therefore provides independent computational support to the theory that closed contours are salient because of proximity and good continuation rather than a special reverberatory mechanism.

13.4 Influence of Input Distribution on Anatomy and Performance

In Section 13.2 we saw how lateral connectivity plays a central role in contour integration in PGLISSOM. Because these connections are learned through input-driven self-organization, different input distributions during development result in different anatomy and performance. The self-organizing process can therefore potentially account for the observed differences in human visual performance across different parts of the visual field, such as upper vs. lower hemifield and fovea vs. periphery (Section 13.1.1). This hypothesis can be tested computationally by training PGLISSOM with inputs of varying frequency and complexity.

13.4.1 Method

The visual inputs that the cortex receives during training may vary in several ways. For example, inputs in the fovea and lower hemifield may be more frequent, shorter, sharper, curved, textured, or have higher contrast. As was discussed in Section 13.1.1, distributions of these features across the visual field have not yet been fully characterized, and it would be somewhat premature to test the model with a selection of such variations. However, at a more abstract level, two distinctly different dimensions of variation can be identified: (1) the amount of training each area receives,

296 13 Understanding Perceptual Grouping: Contour Integration

and (2) the complexity of the training inputs it sees. In this section, these dimensions, represented by input frequency and curvature, are varied systematically, leading to verifiable predictions about the resulting anatomy and performance.

In the frequency experiment, two PGLISSOM networks trained with different input presentation frequencies were compared. The first one was similar to the contour integration network described in Section 13.2, i.e. trained with single randomly located and oriented elongated Gaussians. The second one was otherwise the same, except every other input presentation was skipped during training. The simulation parameters (excitatory radius, learning rate, thresholds, and connection death) were adapted according to the same schedule as before, modeling maturation based on time and trophic factors (Sections 4.4.3 and 16.1.6). In other words, the second network received input half as frequently as the first network during its maturation.

In the curvature experiment, each training input consisted of three short elongated Gaussian bars that together formed a smooth contour, located and oriented randomly on the retina. The angles between the bars were changed to adjust the curvature of the input. For the first network, the training inputs had uniformly randomly distributed curvature in the range [0o..25o], and for the second network in the range [0o..10o].

Over the course of self-organization, the input Gaussians were slightly broader and became elongated slightly slower than those used in Section 11.5. As a result, the learning was slower but the resulting network performed more robustly on the wider variety of inputs (Appendix D.2). Other than the difference in input distributions, all PGLISSOM networks were trained as in Section 11.5. After training, the afferent and lateral connectivity patterns and contour integration performance of each network were measured and the differences analyzed as in Sections 11.5 and 13.2, as will be described next.

13.4.2 Differences in Connection Patterns

All four simulations resulted in a similar map of orientation preferences, matching the results of previous self-organization experiments (Section 11.5). Two interesting observations can be made based on orientation selectivity, as shown in the distributions in Figure 13.20. First, neurons in the more frequently stimulated network were more selective than those in the less frequently stimulated one (Figure 13.20a), suggesting that its initial responses are sparser but stronger for specific inputs. Second, the networks trained with different curvature are equally selective (Figure 13.20b), suggesting that any performance differences are likely to be due to lateral connections.

To uncover any differences between the resulting lateral connection patterns, the (φ, θ, δ) statistics were calculated on the four networks as in Section 13.2.3. In the frequency experiment, two major differences emerged: (1) The high probability areas extend out longer in the high-frequency network (Figure 13.21a) than in the lowfrequency network (Figure 13.21b), i.e. the network with more frequent exposure to oriented edges can group together more distant inputs. (2) The most probable θ for a given (φ, δ) location tends to be cocircular in the high-frequency network, whereas

13.4.3 Differences in Contour Integration

For each of the four networks trained in Section 13.4.1, two contour integration experiments were performed, with orientation jitters 0◦ and 40◦. The 40◦ test case was chosen because contour integration performance in both humans and the model degrades most rapidly at around 40◦ (Figure 13.7), making the differences due the input distributions most clearly visible. The method described in Section 13.2.1 was used for all experiments. Figures 13.23 and 13.24 display the MUA sequences in each case. The MUAs are significantly more synchronized for the high-frequency network than for the low-frequency one when the orientation jitter is the same (compare Figure 13.23a vs. b and c vs. d). For the networks trained with different curvature range, the degree of synchrony was similar for 0◦ orientation jitter (Figure 13.24a vs. b), but the network trained with a broader range was significantly more synchronized in the 40◦ case (Figure 13.24c vs. d). The correlation coefficients between the MUA sequences confirm these observations (Figure 13.25). Frequency makes a difference both in perceiving collinear and cocircular contours, whereas curvature matters only with the cocircular ones.

Such performance differences are predicted by the afferent and lateral connection patterns described in the previous section. Each of the four networks has lateral connections that can group collinear contours, so any difference in performance with 0◦ orientation jitter must be due to the afferent connections. The neurons in the highfrequency network have more selective afferent connections, and therefore activate and synchronize more strongly for inputs that match their preferences. On the other hand, the afferent weights do not differ significantly in the curvature experiment, and neither does the performance of the two networks in the 0◦ case. In contrast, with 40◦ of jitter, the shape of the lateral connections makes a big difference. Each neighboring pair of contour elements is aligned on a cocircular path, and integration requires cocircular connections. Because the lateral connections in the high-frequency network and the high-curvature network are more cocircular, they can detect such contours with high orientation jitter much better than the low-frequency and lowcurvature networks.

In summary, differences in the input distribution, even as simple as presentation frequency or curvature, can change how the maps are organized, which in turn can affect performance in contour integration. Such differences in structure and function are due to the input-driven nature of self-organization. This principle provides a possible developmental explanation for the differences in contour integration performance across different areas of the visual field found in psychophysical experiments.

13.5 Discussion

The results in this chapter suggest that contour integration, segmentation, and completion can be due to synchronization mediated by self-organized afferent and lateral connections, and may form a general principle for grouping (ordinary and illusory) in the visual system.

those generated internally before birth (Section 2.3) could already prepare the animal for essential tasks in the actual visual environment. As demonstrated in Part III, training after birth with natural images will then further refine the circuitry for more accurate performance.

The PGLISSOM model can be extended in several ways to model a wider range of phenomena and to make it biologically more accurate. For example, the process of forming edge-induced illusory contours demonstrated in Section 13.3 could be extended to line-end-induced contours as well. As will be described in more detail in Section 17.2.12, the model could be extended with a V2 network, containing neurons with end-stopped receptive fields, and connected in a manner similar to GMAP in the current model. Synchronized activation in a group of such neurons in V2 would then be interpreted as a line-end-induced contour. In this manner, the same binding mechanism based on lateral interactions would account for both types of contours, providing an alternative to the bipole model discussed in Section 13.1.2.

The range of illusory contours could be expanded further by including feedback from higher levels of visual processing. For example, contours that establish an illusory object (such as the Kanizsa triangle of Figure 13.3a) cannot be explained entirely by low-level mechanisms (Hoffman 1998); they appear to be partly driven by object representations and cognitive factors as well. In fact, connections between lower and higher visual areas are reciprocal (Felleman and Van Essen 1991; Nelson 1995) and well suited for carrying out such computations. Extending PGLISSOM to include such high-level feedback is an interesting future research direction, as will be discussed in Sections 17.2.13 and 17.2.14.

The PGLISSOM model can also be extended in size. The retina and cortical maps are currently limited by the available computational resources, which makes it difficult to replicate the exact contour integration experiments done with humans, and especially those involving illusory contours and the perception of closed vs. open contours. The retina and the cortex would need to be an order of magnitude larger to approximate typical inputs consisting of about 200 line segments. Such simulations are currently not feasible, but the scaling techniques described in Chapter 15 and Section 17.2.9, coupled with the expected growth of computing power, should make them possible in a few years. While the current PGLISSOM model with smallscale inputs is a valid demonstration of the underlying processing principles, such larger-scale models would allow making detailed predictions that match actual psychophysical measurements. Such a large-scale model could also be trained with natural images, or with a combination of prenatal and postnatal inputs, further enhancing the realism of the model. It would then be possible to study new phenomena, such as interaction of multiple stimulus dimensions, as described in Section 17.2.8.

The model can also be extended with a more accurate representation of the input to the different visual areas. As was discussed in Section 13.1.1, contour integration is stronger in the fovea than in the periphery, and in the lower vs. upper hemifield. Section 13.4 demonstrated how such functional differences can result from different distributions of training inputs in these areas. To verify that such differences indeed exist, a method similar to that of Reinagel and Zador (1999) could be used: Input statistics from different parts of the visual field could be collected using eye-tracking