Temporal and Spatial Sensations in Vision

Henning et al., 1975; Nachmias and Rogowitz, 1983). Henning et al. (1975) reported that in their simultaneous visual masking experiment that a masker consisting of only upper harmonics of a “missing” F0 could nevertheless affect detection of a sinusoidal test stimulus with energy only at F0, despite the lack of any spectral overlap between the two signals. They used 1.9 cycles per degree (c/deg) sinusoidal patterns as the test stimulus and an amplitude modulation pattern whose components are 7.6, 9.5, and 11.4 c/deg (i.e., the fourth, fifth, and sixth harmonics of the 1.9 c/deg) as the masking stimulus. That is, the missing fundamental component in the masking stimulus (1.9 c/deg) was perceived and it then disturbed the detection of the test stimulus. Nachmias and Rogowitz (1983) found similar results.

In the following experiment, we measured the subjective flicker rates for compound waveforms consisting of harmonic components without fundamental frequency F0 (Fujii et al., 2000). Because the complex components were combined linearly, there was no Fourier energy at F0. The experiment was conducted under the condition that the complex components were an in-phase and a random-phase waveform. If the perceived rates are based on the actual waveform itself, observers could not detect the rates in the random-phase condition with any clear periodicity. On the other hand, if the F0 component is perceived for both in-phase and random-phase stimuli, there may well exist a correlation mechanism to detect it. Such a correlation mechanism would be similar to the neural mechanism thought to be responsible for periodicity pitch in the auditory system.

Four subjects, ages 23–26 years old, participated in the experiment. All had normal or corrected-to-normal vision. They were well trained before starting the experiment, because they had never participated in such an experiment before. They dark-adapted for about 1 min before all sessions. The light source was a 7-mm- diameter green light-emitting diode (LED), set at a distance of 80 cm from the observer in dark surroundings. The LED stimulus field was spatially uniform, and the size of it corresponded with 0.5 deg. The stimulus waveform was produced with a 16-bit digital-to-analog converter. The mean luminance was set to 20 candela per square meter (cd/m2) and kept constant during the sessions. To prove the linearity of the apparatus, the luminance waveforms of the stimuli with a luminance meter (TOPCON BM-8, Tokyo, Japan) with a response time of 1 ms were measured. A nonlinear LED output was observed; however, such nonlinear components were sufficiently smaller than signal components (–30 dB) in this experiment.

Stimuli in the current study were compound waveforms consisting of five complex components. The frequency of each component corresponded with the n-th harmonic of the fundamental frequency F0. In series A, we selected four stimuli in terms of the complex frequency range with F0 = 1 Hz. Stimulus 1 consisted of 3, 4, 5, 6, and 7 Hz, and for stimuli 2, 3, and 4, the frequency ranges (11, 12. . .15), (21, 22. . .25), and (31, 32. . .35) Hz, respectively, were selected. In series B, for stimuli 5, 6, 7, and 8, complex components were selected in the frequency range for 30–40 Hz, in which we cannot detect any flickering rate if only a single component is presented. Stimulus 5 with F0 = 0.75 Hz consisted of 30, 30.75, 31.5, 32.25, and 33 Hz. For the stimuli 6, 7, and 8 (with F0 = 2, 2.5, and 3 Hz, respectively), the components were (30, 32. . .38), (30, 32.5. . .40), and (27, 30. . .39) Hz, respectively.

Fig. 13.1 An example of the spectrum of the complex flicker signal used in the experiment. Left: Complex components are 30, 32, 34, 36, and 38 Hz, where the energy of the fundamental frequency (F0 = 2 Hz) is absent. Right: Real waveforms for (above) the in-phase condition exhibiting prominent peaks corresponding to the F0 and (below) the random phase condition in which F0 peaks are not at all obvious

The waveforms of the complex signals used in the experiment are illustrated in Fig. 13.1. The real waveform of the stimuli was affected by the phase of components, so that the in-phase and random-phase stimuli had different waveforms. The in-phase waveform had remarkable peaks corresponding to the F0. For the randomphase condition, each component was compounded with different phases so that the waveforms had no significant peaks.

The subjective flicker rate of the stimulus was obtained by means of the method of limits, with a reference stimulus of sinusoidal flicker. These two stimuli were presented in pairs with a blank interval. The task of observers was to judge which of these two stimuli seemed to flicker at the faster rate. As the reference stimulus, we used ascending and descending series. That was, the comparison stimulus was varied in steps, from a low frequency to a high frequency (or vice versa) to measure the value at which the observers’ responses reversed. The mean of the two values before and after reversal of the observers’ responses was determined as the matched frequency of the test stimulus. When the observers perceived two or more rates for one test stimulus, they were asked to judge with the rate perceived most strongly. This means that the observers matched the sinusoid to the most prominent component of the compound waveforms, and thus, one matched frequency was obtained through one trial. Intervals of the comparison stimulus were 0.1 Hz step for frequencies below 1 Hz, 0.2 Hz step for 1 to 3 Hz, and a 1 Hz step for frequencies above 3 Hz. In the descending series, trials started from a value of a few hertz above the highest frequency of the components in the test stimulus. There were two series of the comparison stimulus (ascending and descending) and two orders of presentation (test-comparison and comparison-test), giving a total of four conditions. For each

Fig. 13.2 Subjective flicker rate of missing fundamental visual stimuli with different harmonics (a, above) and different F0 values (b, next page) for in-phase and random-phase conditions. Response probability distributions for 4 subjects. (a) F0 = 1 Hz (b) F0 = 0.75, 2, 2.5, and 3 Hz. Response probability distributions for flicker rate judgments of flicker stimuli with different missing F0s and phase spectra. From top to bottom, F0 = 0.75, 2, 2.5, and 3 Hz

condition, four trials were repeated. Thus, 16 matched frequencies were obtained for each test stimulus.

Results of the probability of responses to each matched frequency are shown in Fig. 13.2 as a histogram. For the in-phase stimuli, observers perceived the rates at F0. This frequency is easily detected, because it is consistent with the time inter-

Fig. 13.2 (continued)

val between the periodic peaks appearing in the temporal waveforms as shown in Fig. 13.1. For the random-phase stimuli, matched frequencies were comparable with the several aperiodic peaks, which correspond with the component frequencies. We could detect the flicker rates from local peaks in the waveforms in this low-frequency range only (3–7 Hz). In the high-frequency range, however, the fundamental frequency F0 was perceived most frequently for both in-phase and random-phase stimuli, which is called the missing fundamental phenomenon, even allowing some exception such as certain multiples of F0.

Fig. 13.3 The probability of matching the flicker rate to a value within 10% of the fundamental frequency F0 as a function of the fundamental frequency. Filled circles and open circles represent in-phase and random-phase conditions, respectively

Figure 13.3 shows the observers’ responses within (1 ± 0.1) F0 as a function of the fundamental frequency F0. Both curves have a similar profile, except that the probability was about 10% higher for the in-phase condition. Although probability was affected by the phase, the most frequently perceived rates were about F0 in all cases. The highest probability is seen at F0 = 2 and 2.5 Hz for the random-phase and in-phase conditions, respectively. These values correspond with the periods of 500 ms and 400 ms, which are similar to the “sensitive range” reported by Fraisse (1984). He reported that in the sensitive range (500 ms to 700 ms), the sensitivity increased to the periodicity of successive presentation of the stimuli. Our observers might also have responded sensitively to the periodicity of the flickering stimuli in this range. Thus, observers may detect the rates at fundamental frequency, which are not included in the power spectrum of the stimuli. One promising operation that gives the phase-independent prediction for our empirical evidence is the ACF (Fig. 13.4). Actually, the ACF of the real stimulus waveforms had identical profiles for both phase conditions used in the experiment. This result is consistent with the fact that the ACF has particular peaks corresponding to the F0. One can postu-

Fig. 13.4 An example of the temporal ACF of the flicker stimuli for both conditions, in-phase and random phase. The value of τ1 corresponds to the “missing” fundamental frequency, which has no energy in the power spectrum representation