3.8 Discussion

as well (see also Section 3.5.2). Note that we deliberately did not correct these values for their baseline, because - although useful in academic research - the baseline-correction procedure is not easily applicable in future emotionally aware systems.

For each parameter of each physiological measure, a repeated measures ANOVA was conducted, with the four emotions, each measured with two film scenes, as within-subject factors. So, a total of 24 (i.e., 4 × 6) repeated measures ANOVAs were conducted. As measure of effect size partial eta squared (η2) is reported, which indicates the proportion of variance accounted for (i.e., a generalization of r/r2 and R/R2 in correlation/regression analysis) [211, 737]. The classification of the film scenes into the four emotion classes was based on the participants’ ratings as provided in Table 3.1, which were perfectly in line with the findings reported by Gross and Levenson [235, 237].

The EMG of the frontalis did not provide a significant discrimination between the 4 emotion classes on any of the statistical parameters. Of all physiological measures, the zygomaticus major signal is the most discriminative biosignal (see Table 3.2). The mean, absolute deviation, standard deviation and variance calculated over the zygomaticus major EMG signal showed strong significant effects of emotions. Significant effects did also show in the skewness and kurtosis of the EDA signal and the skewness of the corrugator supercilii EMG signal (Table 3.2). For the skewness of the EMG zygomaticus signal a trend was present over the four emotions (F (3, 18) = 3.013, p = .057), which explained rather a lot of the variance present between the 4 emotion classes (η2 = .334).

3.8 Discussion

3.8.1 Comparison with the literature

Most 120 seconds averaged values of the biosignals yielded no significant effects of emotion class, in contrast to what is generally reported in the literature. One of the reasons might be that we chose not to correct our data for baseline values, as is common in the psychophysiological literature. Another factor is that the present analysis was chosen to extend over a relatively long period of time including the beginning of the video fragment in which the targeted emotions were still in the process of being elicited, which might have diminished the differences between categories of emotions.

For the zygomaticus major, we did find an effect for the average value, even when not corrected for baseline and averaged over 120 seconds. This is in line with results of previous research by Larsen, Norris, and Cacioppo [380], who concluded that valence influences both the corrugator supercilii and the zygomaticus major. They found that valence had a stronger effect on the corrugator supercilii than on the zygomaticus major in experiencing

51

3 Statistical moments as signal features

Table 3.2: The discriminating statistical parameters for the EDA, EMG corrugator supercilii, and EMG zygomaticus signals. For each parameter, the average value for all four emotion classes (i.e., neutral: 0; positive: +; mixed: +/-; negative: -.) is provided as well as the strength and significance of its discriminating ability. Additionally, as measure of effect size partial eta squared (η2) is reported, which indicates the proportion of variance accounted for [211, 737].

standardized affective pictures, sounds, and words, while our research shows a stronger effect of the four emotion classes on the mean zygomaticus major signal, than on the corrugator supercilii. In addition, the effect is present with four statistical parameters of the zygomaticus major, where it is only present in one statistical parameter (skewness) of the corrugator supercilii.

The difference in strength of the effects found between the current research and that of Larsen, Norris, and Cacioppo [380] can possibly be explained by the absence of a baseline correction in our procedure. Another difference between the two researches is the type of stimuli (cf. [8]). Film scenes are dynamic and multi-modal, they induce emotions by both auditory, and dynamic visual stimuli, as well as affective words, in some fragments. The dynamic and multi-modal characteristics of the film scenes also provide good means to build up emotions, or to create a shock effect [570, 700, 701]. This is almost not possible with affective words, sounds or pictures of a static character, as their use lacks the opportunity to built up emotions. On the one hand, all these factors give film scenes a relatively high degree of ecological validity [235, 237, 700, 701]. On the other hand, it is not possible to determine which modality influences the emotional state of the subjects to the highest extent.

For three of the 4 biosignals the parameter skewness turned out to be important as a significant effect or as a trend. To the authors best knowledge, the skewness (and kurtosis) of EMG signals as discriminating descriptor have been discusses in only three studies. In 1983, Cacioppo, Marshall-Goodell and Dorfman [82] analyzed among a number of parameters, the skewness and kurtosis of skeletal muscle patterns, recorded through EMGs. Four years later, an article by Cacioppo and Dorfman [81] that discussed “waveform moment

52

3.8 Discussion

analysis in psychophysiological research” in general. In 1989, Hess et al. [278] conducted research toward experiencing and showing happy feelings, also using video segments. Hess et al. [278] recorded four facial EMG signals and extracted the mean, variance, skewness and kurtosis of these signals. The current research is distinct from that of Hess et al. [278] since it distinguishes four emotion classes instead of the presence or absence of only one. Each of these three studies identified skewness and kurtosis of EMG signals as potentially interesting for the discrimination between emotions. However, surprisingly little attention has been given to moments of order 3 and higher in ASP.

3.8.2 Use in products

Not all investigated parameters of all measures proved to be equally suited for sensing human’s emotional states. This is no doubt due to the demanding analysis conditions we imposed: no baseline correction and averages over relatively long time intervals. Nevertheless, even under these demanding analysis conditions, some of the measures still succeeded in distinguishing between the respective emotion classes.

For three of the four biosignals used, the parameter skewness proved to be an interesting source of information. The skewness of the distributions of the data of two of the biosignals differs significantly over the four emotions, where a trend is present for a third signal. To inspect more distribution details of the signals, additional analyses could be conducted. Measures such as the slope of the signal and the peak density could be taken into account for further analysis. Such analysis could help understanding to what extent emotions were indeed built up during the movie scenes.

In addition to adding more descriptors of the biosignals, the time windows of measurement can be changed. In the current setup, the time window enclosed the complete length of the film scene. However, smaller time windows (e.g., 10 or 30 seconds) can be applied to conduct more detailed analysis of biosignals’ behavior in relation to the movie content. Moreover, dynamic time windows can be applied that enclose the time directly after a critical event (if any) appeared in the film scene. The drawback of the latter approach is that it cannot be applied in practice, while it may be expected to provide good results for data gathered through experimentation, as in the current research.

A more general notion that can have a significant impact on measurement of emotions is that the emotional state of people changes over time, due to various circumstances. Moreover, different persons have different emotional experiences over the same events, objects, or actions. This variance in experienced emotions is determined by a person’s personality. Personality traits correlate with affective states, especially with the personality traits extraversion and neuroticism, which have been linked both theoretically and empirically to the fundamental affective states of positive and negative affect, respectively [442]. Hence,

53

3 Statistical moments as signal features

to enable tailored communication strategies in HCI, not only the emotional state of a person should be determined but also his personality. When the system possesses a personality profile of its user, it will be able to react appropriately to its user’s emotions by selecting a suitable communication strategy. We will explore this issue in Chapters 5 and 6.

The next chapter will continue the analyses presented in this chapter. Analysis will be conducted on the same data set using other time windows. Events in the movie fragments will be traced and their effects on the EMG and EDA signals will be unveiled. Moreover, the possible influence of scene changes will be addressed.

54