3.3 Speed |
23 |
spatial frequency of the visual Image and thereby perform some sort of Fourier-like analysis. This view was propelled after it was discovered that the response characteristics of V1 cells are more subtle, when they are stimulated with sinusoidal gratings - instead of a single line or bar only (De Valois and De Valois, 1988). For example, the orientation tuning curve looked as shown in figure 10d. The horizontal dashed line crossing the orientation tuning curve represents the ‘spontaneous’ firing rate, that is the mean firing rate when a neuron is not stimulated. This spontaneous firing rate can be in the range of less than one spike per second (0.5 Hertz) to up to several spikes a second. For the preferred grating orientation the cell fires with a high frequency - as recorded in experiments using single line orientations. But for grating angles that deviate a few degrees off the preferred orientation, the response of the neuron is actually below the spontaneous firing rate. A second ‘refined’ finding was, that the receptive field profile looked more like a Gabor1 function than a Gaussian (not shown). A third intricate discovery was that different cells showed a preference for gratings of specific frequency. Taken together, it looks like there are cells that code for different spatial (or resolutional) scales and for different orientations, which suggests that some sort of wavelet encoding may occur in the primary visual cortex. If one translated that into the picture of the orientation columns, then one would add the dimension of spatial scale to each orientation (figure 10e). Many vision models perform image-filtering operations inspired by the above framework. They use filters whose exact form is motivated by those recordings. For example, an elegant model by Perona and Malik performs texture segregation and produces a contour image as output (Malik and Perona, 1990). A number of neurophysiology studies have tried to find filter detectors in higher areas (V2 and upwards in the cortical hierarchy). They used polar and hyperbolic patterns that could possibly serve as complex filter detectors (e.g. (Hegde and Essen, 2000; Gallant et al., 1996)). Such patterns are shown in the bottom row of figure 11.
3.3 Speed
Latency Code Many of the neurophysiological experiments are carried out using (visual) presentation times of several hundreds of milliseconds, which is a long time span compared to the fast-paced dynamics of visual selection: saccades are launched every 200-300ms, attentional shifts are carried out several times between saccades (Parasuraman, 1998). One may therefore question the interpretation of experiments with long stimulation durations. Indeed, it has long been neglected that visual processing occurs blazingly fast. Although Potter had already shown this with behavioral experiments (see section
1A Gaussian multiplied by a sinusoid (Gabor, 1946)
24 |
Neuroscientific Inspiration |
1.2), it was Thorpe who sharply pointed out this characteristic using event-related potential (ERP) studies (Thorpe et al., 1996) (see also (Schendan et al., 1998)). In his experiments, subjects had to decide whether a picture contained an animal or not. The picture was presented for only 20ms. The analysis of the ERP patterns showed, that after 150ms only, a subject has already made a reliable decision and that this decision was made in the frontal cortex, indicating that visual information has made the loop from V1 over IT to frontal cortex somehow. It should be mentioned however, that Thorpe’s and Potter’s experiments work with expectation: the subject knows in about what to look for - in other terms certain frames may have been activated already and this preactivation could possibly reduce the reaction time by some amount. Still, 150ms is blazingly fast and without expectation it would probably take only a few tens of milliseconds more.
Figure 13: A latency code. The amount of input determines the onset of firing: a large input triggers early onset, a small input triggers late onset (0: onset of stimulus, t: time). This type of timing code would swiftly convert a complex pattern into a subtly timed spike pattern. Proposed in variants by Thorpe and Hopfield. Adapted from Thorpe, 1990.
Because visual processing happens so rapidly - and possibly any neural processing -, one may doubt whether there is a frequency code at work, because such a code may be simply too slow. Thorpe himself therefore proposed a timing code in which computation is performed by precisely timed spikes (Thorpe, 1990). Figure 13 expresses the idea. The amount of input determines when the cell fires its first spike after stimulus onset: a large input triggers early firing, a small input triggers late firing. VanRullen et al. developed a network that is made of a large stack of neuronal layers using this type of encoding. The network is designed to mimic Thorpe’s visual experiment (Rullen et al., 1998). The idea of such a latency code has also been proposed by Hopfield (Hopfield, 1995).
3.4 Alternative ‘Codes’ |
25 |
Global-to-local Some psychologists were aware of the the blazing processing speed long before the above debates and were wondering how the visual system may analyze visual information so rapidly. They argue, that because we can recognize the gist of a scene so rapidly, that the visual system processes a scene by decomposing it in a global- to-local manner, basically the reverse to the neuroscientific paradigm. Neisser has formulated this idea in the 60’s already (Neisser, 1967), Navon was the first to seriously investigate this concept (Navon, 1977). Navon described this view concisely with ‘forest before trees’: In experiments with large letters made of small letters (figure 14), Navon tried to prove that first a global perception of the large letter takes place, followed by a local perception of the small letters. If such a ‘global-first’
Figure 14: Navon’s task to test for a global-to-local recognition evolvement: Is the global ‘H’ or the local ‘S’ perceived first?
evolvement would take place in the visual system, one may wonder how. Because it had to happen fast, global processing may have to take place in low cortical areas already. This stands in apparent contrast to the long-assumed picture that only a spatially local analysis occurs in V1. But this picture has already been criticized from two sides. One side are the neurophysiological recordings on the characteristics of the receptive field (previous section). Another side are the psychophysical measurements on contour integration, which evidence that perceptual grouping operations may take place in V1 (Kovacs, 1996; Hess and Field, 1999). It may therefore be worth considering whether a possible global-to-local analysis starts in V1 already. On a complementary note, there is an attempt to model a global-to-local analysis (see (GERRISSEN, 1984; GERRISSEN, 1982)).
3.4 Alternative ‘Codes’
The dominant view of how neurons communicate with each other is guided by regarding it as some sort of Morse code or also called a ‘neural code’ (Koch, 1999). The sequence of spikes, that a neuron generates, is interpreted as being part of a rate or timing code. Both,
26 |
Neuroscientific Inspiration |
rate and timing codes, come in many different variants (deCharms and Zador, 2000). We have mentioned a few examples previously. An alternative to this Morse code thinking is to regard spikes merely as connectors between far distant sites: the sequence of spikes would have no meaning, but each single spike may be part of a computation connecting distal sites.
Cortical Potential Distributions One such alternative is Tuckwell’s theory, in which the critical measure is a potential distribution across cortex (Tuckwell, 2000). One component of the cortical potential distribution can be the field potential, which is a voltage that reflects primarily post-synaptic potentials and only to a minor extent the spikes themselves. Another component can be the magnetic fields. Both potentials can be regarded as a general electromagnetic field description for cortical states. According to Tuckwell, such global cortical potential distributions could define cognitive states. A change between these states could occur very rapidly simply by the simultaneous change of the entire distribution at each locus.
Waves Another alternative would be to regard an observed (or measured) spike as being part of a traveling wave. Traveling waves exist in the nervous system of many animals (e.g. (Hughes, 1995; Prechtl et al., 1997; Wilson et al., 2001; Shevelev and Tsicalov, 1997). Generally, they are considered as non-functional, accidentally emerging from networks of neurons. Recently, there has been some effort to attribute computational functions to traveling waves. Jacobs and Werblin measured traveling waves in the salamander retina in response to visual stimulation, and speculated that it may have edge enhancing effects (Jacobs and Werblin, 1998). They envision that such waves are possibly involved in different neural computations. Barch and Glaser use excitable membranes to detect motion: different motion signals leave traveling waves of characteristic shape on an excitable membrane (Barch and Glaser, 2002).
Yet, long before the discovery and reinterpretation of traveling waves, there existed some computational reflections on coding and visual representations, which have never really been pursued in neuroscience. One is the broadcast receiver principle and the other is the idea of self-interacting shape. Both use the idea of wave propagation, which can be readily interpreted as traveling waves.
Broadcast Receiver Principle This idea has been expressed by several people (e.g. (Blum, 1967; Deutsch, 1962), see (Blum, 1967) for references). Blum has given the most illustrative example, see figure 15a. The schematic depicts an excitable 3D volume that propagates waves. He imagined that a feature extraction process places filtered properties (of a visual field for example) onto this propagation medium.