Hearing

 

hearing: 1: to perceive or apprehend by the ear; [...] 1: to have the capacity of apprehending sound; [...] 1: the process, function, or power of perceiving sound; specif: the special sense by which noises and tones are received as stimuli; [...] [217]

The main attributes used for describing a hearing event are:

Pitch

is the auditory attribute on the basis of which tones may be ordered on a musical scale. Two aspects of the notion pitch can be distinguished in music: one related to the frequency (or fundamental frequency) of a sound which is called pitch height, and the other related to its place in a musical scale which is called pitch chroma. Pitch heights vary directly with frequency over the range of audible frequencies. This 'dimension' of pitch corresponds to the sensation of 'high' and 'low'. Pitch chroma, on the other hand, embodies the perceptual phenomenon of octave equivalence, by which two sounds separated by an octave (and thus relatively distant in terms of pitch height) are nonetheless perceived as being somehow equivalent. This equivalence is demonstrated by the fact that almost all scale systems in the world in which the notes are named give the same names to notes that are roughly separated by an octave. Thus pitch chroma is organized in a circular fashion, with an octave-equivalent pitches considered to have the same chroma. Chroma perception is limited to the frequency range of musical pitch (50-4000 Hz) [218].

Loudness

is the subjective intensity of a sound. Loudness depends mainly on five stimulus variables: intensity, spectral content, time, background, and spatial distribution of sound sources (binaural loudness) [296].

Timbre,

also referred to as sound quality or sound color. The classic negative definition of timbre is: the perceptual attribute of sound that allows a listener to distinguish among sounds that are otherwise equivalent to pitch, loudness, and subjective duration. Contemporary research has begun to decompose this attribute into several perceptual dimensions of a temporal, spectral and spectro-temporal nature [218].

Spatial attributes

of a hearing event may be divided into distance and direction:

The perception of a direction of a sound source depends on the differences in the signals between the two ears ( interaural cues: interaural level difference (ILD) and interaural time difference (ITD)) and the spectral shape of the signal at each ear ( monaural cues). Interaural and monaural cues are produced by reflections, diffractionsa and damping caused by the body, head, and pinna. The transfer function from a position in space to a position in the ear canal is called head-related transfer function (HRTF). The perception of distance is influenced by changes in the timbre and distance dependencies in the HRTF. In echoic environments the time delay and directions of direct sound and reflections affect the perceived distance.

Somatic senses

 

somatic: 1: of, relating to, or affecting the body [...]; 2. of or relating to the wall of the body [217]

The main keywords related to somatic senses are tactile and haptic which are both related to the sense of touch (see 1.1.3 ). Concerning this sense, there is a lot more than only touch itself. For example, [120] distinguishes five ``senses of skin'': the sense of pressure, of touch, of vibration, of cold, and of warmth.

[120] indicates two more senses, called ``sense of position'' and ``sense of force'', related to the proprioceptors. The proprioceptors are receptors (special nerve-cells receiving stimuli) within the human body. They are attached to muscles, tendons, and joints. They measure for example the activity of muscles, the stressing of tendons, and the angle position of joints. This sense of proprioception is called kinesthesis and [120] calls the accompanying modality kinesthetical:

Kinesthesis (perception of body movements): (physiology, psychology) Kinesthesis is the perception that enables one person to perceive movements of the own body. It is based on the fact that movements are reported to the brain (feedback), as there are:

• angle of joints

• activities of muscles

• head movements (reported by the vestibular organ within the inner ear)

• position of the skin, relative to the touched surface

• movements of the person within the environment (visual kinesthesis)

Kinesthesis supports the perception of the sense organs. If some informations delivered by a sense organ and by kinesthesis are contradictory, the brain will prefer the information coming up from the sense organ.

Human computer interaction makes use of kinesthesis, e.g. if a key has a perceptible point of pressure or if the hand performs movements with the mouse to position the mouse-pointer on the screen, and so on [67].

For a multimodal system, kinesthesis is not as relevant as it is for a VR system, because aspects like experiencing an outer influence to the sense of balance, e.g. when wearing a head-mounted display (HMD), will not be a major topic here. Therefore, the most relevant somatic sense is the sense of touch, which can be addressed by special output devices either with tactile or with force feedback (see 2.2.4 ).

According to [309], four different types of touch receptors (or mechanoreceptors) are present at the human hand . They have different characteristics which will not be addressed within the scope of this report. As tests have shown, the output response of each receptor decreases over time (called stimulation adaptation) for a given input stimulus. The 2-point discrimination ability is also very important. The index finger pulp is able to sense all points with a distance of more than 2 mm whereas in the center of the palm two points that are less than 11 mm apart feel like only one. Other relevant factors are the amplitude and the vibration frequency of the contactor.