Chapter 1

Introduction

1.1 Auditory Temporal and Spatial Factors

For the past four decades, we have pursued a theory of architectural acoustics based on acoustics and auditory theory. In the summer of 1971, I visited the III Physics Institute at the University of Goettingen and its director, Professor Manfred R. Schroeder, with the intent of investigating aspects of spatial hearing most relevant to design of concert halls. Peter Damaske and I were interested in explaining the subjective diffuseness of sound fields using the interaural crosscorrelation function (IACF). The magnitude (maximum value) of the IACF defined by IACC is an indication of how subjectively diffuse a given sound will be perceived. We played sounds into a room using a multiple loudspeaker-reproduction system and recorded them at the two ears of a dummy head (Damaske and Ando, 1972). Because the IACC was known to be an important determinant in the horizontal localization of sounds, we thought it likely then that it would also play an important role for subjective diffuseness. Two years later, in 1974, a comparative study of European concert halls performed by Schroeder, Gottlob, and Siebrasse showed that the IACC was an important factor for subjective preferences and established a consensus across individuals.

In early 1975 at Kobe University, we had a special aural experience. By optimizing the horizontal direction and the delay time of a single reflection of a speech signal through successive adjustment, we were able to achieve a superior sound field. A loudspeaker in front of a single listener reproduced the direct sound. In the optimal setup, we found that the angle of the single reflection measured from the front was 30 degrees in the horizontal plane, the single reflection delay time was about 20 ms, and the amplitude of the reflection was equal to that of the direct sound. For our continuous speech signal, the effective duration was also close to this same value of 20 ms. The effective duration (τe) of a signal is a measure of its temporal coherence, the duration over which the signal is self-similar and less degraded by room reverberations. Subjective preference therefore seemed to us to be characterized in terms of both temporal and spatial factors. These working hypotheses were reconfirmed in the fall of 1975 while the author was an Alexander-von-Humboldt Fellow in Goettingen (Ando, 1977; Ando and Gottlob, 1979). During that period (1975–1977), we

also were able to explain sound colorations produced by single reflections in terms of the envelope of the autocorrelation function (ACF) of the source signal (Ando and Alrutz, 1982).

In 1983, a method of calculating subjective preference at each seat in a concert hall was published (Ando, 1983). Soon after, a theory of designing concert halls was formulated based on a model of the auditory system. We assumed that some aspects depended on the auditory periphery (the ear), whereas others depended on processing in the auditory central nervous system (Ando, 2003). The model takes into account both temporal factors and spatial factors that determine the subjective preference of sound fields (Ando, 1985). The model consists of a monaural autocorrelation (ACF) mechanism and an interaural crosscorrelation (IACF) mechanism for binaural processing. These two representations are used to describe spatial hearing operations that we presume to be taking place in several stations in the auditory pathway, from the auditory brainstem to the hemispheres of the cerebral cortex.

Once one has an explicit model of subjective preference alongside a model of auditory perception, then rational design of spaces for listening becomes possible. Special attention is paid in this volume to computing optimal individual preferences by adjusting the weighting coefficient of each temporal and spatial factor. “Subjective preference” is important to us for philosophical and aesthetic as well as practical, architectural acoustics reasons. We consider preference as the primitive response of a living creature that influences its direction and judgment in the pursuit of maintaining life: of body, of mind, and of personality (Ando, 2004).

1.2Auditory System Model for Temporal and Spatial Information Processing

We have attempted to learn how sounds are represented and processed in the cochlea, auditory nerve, and in the two cerebral hemispheres in order to develop a theory of spatial hearing for architectural acoustics that is grounded in the human auditory system. Once effective models of auditory processing are developed, designs for concert halls can proceed in a straightforward fashion, according to the guidelines derived from the model. In addition, understanding the basic operations of the auditory system may lead to development of automatic systems for recognizing speech and analyzing music, as well as identifying environmental noise and its subjective effects.

In our attempts to understand how the auditory system works, we have sought the neural response correlates of auditory percepts that are most relevant to architectural acoustics. Table 1.1 summarizes the different perceptual attributes associated with sound sources and sound fields as well as several dimensions of listening preferences. The table lists the perceptual attribute, its acoustic correlate, the nature of the acoustic and neuronal representations thought to be involved, the features in these representations that subserve the attribute, as well as related signs of neuronal response and the presumed locations of their generation.

Temporal sensations are those auditory qualities that are most directly related to temporal patterns and time delays, while spatial sensations are auditory qualities most directly related to spatial aspects of sound and sound fields. Temporal sensations naturally lend themselves to autocorrelation-like acoustical descriptions and putative neuronal representations, while spatial sensations lend themselves to interaural crosscorrelation descriptions of sound fields and internal representations. It is striking how much of what we hear can be accounted for in terms of features of these two kinds of representations, the monaural autocorrelation function (ACF) and the binaural interaural crosscorrelation function (IACF). It is perhaps equally surprising that observable neuronal correlates for all these perceptual attributes can be found in the summed activity of large numbers of auditory neurons.

It is remarkable that the temporal discharge patterns of neurons at the level of the auditory nerve and brainstem include sufficient information to effectively represent the ACF of an acoustic stimulus. Mechanisms for the neural analysis of interaural time differences through neural temporal crosscorrelation operations and for analysis of stimulus periodicities through neural temporal autocorrelations were proposed more than 50 years ago (Jeffress, 1948; Licklider, 1951; Cherry and Sayers, 1956). Since then, a host of electrophysiological studies recording from single neurons and neural populations has more clearly elucidated the neuronal basis for these operations. Binaural crosscorrelations are computed by axonal tapped delay transmission lines that feed into neurons in the medial superior nucleus of the auditory brainstem that act as coincidence detectors (Colburn, 1996). If one examines the temporal patterning of discharges in the auditory nerve (Secker-Walker and Searle, 1990), one immediately sees the basis for a robust time-domain representation of the acoustic stimulus. Here the stimulus autocorrelation is represented directly in the interspike interval distribution of the entire population of auditory nerve fibers (Cariani and Delgutte, 1996a,b). This autocorrelation-like neural representation subserves the perception of pitch and tonal quality (aspects of timbre based on spectral contour) (Meddis and O’Mard, 1997; Cariani, 1999).

Higher in the auditory pathway, at brainstem, midbrain, and cortical levels, neural correlates of both temporal and spatial sensations can also be found. In our own laboratory in Japan we found these correlates at several levels of the auditory pathway – in auditory brainstem responses (ABRs) and in several measures of cortical response (SVRs, EEG, MEG).

We have observed neural response correlates of temporal aspects of sensation, such as pitch or the missing fundamental (Inoue et al., 2001), loudness (Sato et al., 2001), and in addition duration sensation (Saifuddin et al., 2002). These are well described by the temporal factors extracted from the ACF (Ando et al., 2000; Ando, 2002). We found that the temporal factors of sound fields such as t1 and Tsub are mainly associated with left hemisphere responses (Ando et al, 1987; Ando, 1992; Ando and Chen, 1996; Chen and Ando, 1996; Soeta et al., 2002). In contrast, aspects of sound fields involving spatial sensations such as subjective diffuseness (Ando and Kurihara, 1986) and apparent source width (ASW) as well as subjective preferences (Sato and Ando, 1996; Ando et al., 2000) are mainly associated with right hemisphere responses (Ando and Hosaka, 1983; Ando, 1985, 1992, 1998; Ando et al.,

1987; Sato et al., 2003). Some qualities involve both spatial and temporal sensations. For example, the timbre of the sound field in an existing hall was investigated by a dissimilarity judgment in relation to both temporal and spatial factors (Hotehama et al., 2002).

Observable correlates of spatial sensations can also be found at many levels of auditory processing, in evoked auditory brainstem responses (ABRs) and slow vertex responses (SVRs). ABRs are short latency (0–10 ms) averaged evoked electrical potentials that reflect the successive excitation of neuronal populations in the ascending auditory pathway (Section 4.1). Waves I–V in ABRs are produced by near-synchronous firing of large numbers of neurons in the cochlea, brainstem, and midbrain. The later waves in this response reflect the contributions of binaural neural processing stations in the auditory brainstem. We found that the magnitudes of Wave IV on either side (IVl,r) varied with the sound energies present at the entrances of the two ears. We also observed that the magnitude of wave V in the ABR, which reflects the amount of synchronized neural activity at the level of the midbrain, covaries with the magnitude (IACC) of the interaural crosscorrelation function (IACF) (Ando et al., 1991). Slow vertex responses (SVRs) are longer latency (10–500 ms) averaged auditory-evoked responses that are computed from scalp electroencephalogram (EEG) signals. These potentials reflect the electrical contributions of neuronal populations mainly in auditory cortical regions. We found SVR correlates of subjective diffuseness and apparent source width that are related to the interaural correlation magnitude (IACC) in the latencies of the negative right hemisphere cortical response peak N2 (260–300 ms latency, Fig. 4.9). Systematic shifts in the latencies and/or amplitudes of SVR peaks were also observed for first reflection time ( t1), sensation level (SL), and interaural correlation magnitude (IACC) (Section 4.2).

We also carried out a long series of experiments directed at finding observable neural correlates of subjective preferences of sound fields. Overall subjective preference has been well described by four orthogonal acoustic factors, two of them temporal and two spatial. The two temporal factors are (1) the initial time delay gap between the direct sound and the first reflection ( t1) and (2) the subsequent reverberation time (Tsub). The two spatial factors are (1) listening level (LL) and (2) the maximum magnitude of IACF (IACC), the latter being an index of the sound’s apparent spatial compactness in azimuth. The SVRand EEG-based neural correlates of the two temporal factors were found to be associated with the left hemisphere, while the two spatial factors were associated with the right hemisphere. For example, we reconfirmed by magnetoencephalography (MEG) that the left cerebral hemisphere is associated with first reflection time t1. The right cerebral hemisphere was activated by the typical spatial factors, i.e., the magnitude of interaural crosscorrelation (IACC) (Sato et al., 2003). Not surprisingly, the neural correlates of listener preferences for these temporal and spatial factors were also found in the corresponding hemispheres. Neural correlates for preferences typically involved prolongation of alpha wave activity under preferred listening conditions as seen in both EEG and MEG. More specifically, we found that the duration of coherent, repetitive alpha wave activity (effective duration of the ACF of the response signal)

directly corresponds to how much a given stimulus is preferred, i.e., the scale value of individual subjective preference (Soeta et al., 2002; 2003). This evidence ensures that the basic theory of subjective preference can be applied to each individual preference as well (Ando, 1998).

Because individual differences of subjective preference for spatial factors are small (nearly everyone has the same basic preferences), given the type of program material involved, we can a priori determine the optimal architectural space form of the room by taking common preferences into account. The temporal factors that involve successive delays produced by sets of reflective surfaces are closely related to the dimensions of the specific concert hall under design. These dimensions can potentially be altered to optimize the space for specific types of music, such as organ music, chamber music or choral works (Ando, 1998).