Summary

B Summary

276

B Summary

Slowly computers are becoming dressed, huggable and tangible operating interfaces. They are being personalized and are expected to understand more of their users’ feelings, emotions, and moods. Consequently, concepts, such as emotions, which were originally the playing field of philosophers, sociologists, and psychologists, have also become entangled with computer science. In 1997, Picard baptized the topic affective computing. Moreover, she identified biosignals as an important covert channel to capture emotions, in addition to channels such as speech and computer vision. This monograph explores several factors that have been posed to be of key importance to affective computing, in particular to affective signal processing (ASP) : the use of biosignals for emotion-aware systems. It is divided into five parts: i) a prologue, ii) basic research on baseline-free ASP, iii) basic research on bimodal ASP, iv) three studies towards affective computing, and v) an epilogue. Additionally, an appendix provides a description of all statistical and pattern recognition techniques used. Here is a concise summary of each of the five parts.

In the introduction, Chapter 1 of the prologue (Part I ), a brief introduction of emotion theory, the field of affective computing, A SP, and their relevance for computer science is provided. Human-Computer Interaction, Artificial Intelligence, and health informatics are described. The monograph’s working model, a closed loop model (i.e., a control system with an active feedback loop), is introduced and its signal processing and classification components are described. A concise overview of the biosignals investigated is given. In Chapter 2 a review of affective computing is presented, with an emphasis on ASP using biosignals. The conclusion of this chapter is that ASP lacks the progress it needs. Possible angles of view that can aid ASP’s progress are explored in the next three parts.

In Part II two basic studies on baseline-free ASP using statistical moments are presented. These two studies address a number of key issues for ASP. Chapter 3 covers research for which dynamic real world stimuli (i.e., movie scenes) were used to induce emotions. The ElectroMyoGraphy (EMG) of three facial muscles was recorded, which is often done to establish a ground truth measurement. In addition, the participants’ ElectroDermal Activity (EDA) was recorded. EDA is a robust well documented biosignal that reveals the level of experienced arousal. In Chapter 4 analyses on the same data set as in Chapter 3 are reported. The studies differ in the choice of time windows, which enabled research towards the impact and usage of this parameter for ASP. Moreover, events in the movie scenes were directly linked to affective responses.

Part III Two studies are presented that employed bi-modal ASP by the rare combination of ElectroCardioGram (ECG) and speech. These studies only differ from each other with respect to the stimuli that were used for emotion elicitation, which has recently been shown to be a factor of importance [8]. The research presented in these two chapters also assessed the influence of emotion representations by analyzing the obtained data using both the dimensional valence-arousal model and the six basic emotions. Moreover, the impact

277

B Summary

of the environment (or context), the personality traits neuroticism and extroversion, and demographics on ASP was explored. In Chapter 5 research is reported that employed a (or perhaps even the) reference set for affective computing: Lang, Bradley, and Cuthbert’s (1994) International Affective Picture System (IAPS). In Chapter 6 a study is presented that used the same set of stimuli (i.e., movie fragments) as was used in the research described in Chapters 3 and 4. This enabled a comparison of static versus dynamic stimuli and, as such, assessed their validity.

Part IV consists of three chapters that present studies that work towards affective computing. First, in the research in Chapter 7, a complete signal processing + classification processing pipeline for ASP is executed on the data already presented in Chapters 3 and 4. Several preprocessing strategies and automatic classifiers are explored. Second, in Chapter 8, two clinical case studies on ASP are presented that aim to explore the feasibility of Computer Aided Diagnosis (CAD) for patients suffering from a post-traumatic stress disorder (PTSD). Third, in Chapter 9, the data of the studies presented in Chapter 8 are used to develop a complete signal processing + pattern recognition processing pipeline, similar to the one presented in Chapter 7. As such, this chapter explores the feasibility of the envisioned emotion-aware systems, in this case: A SP-based Computer-Aided Diagnosis (CAD) for mental health care.

This monograph’s epilogue, Part V, consists of two chapters. In the first one, Chapter 10, the lessons learned from the research presented in the previous chapters is described. It formulates a set of prerequisites and guidelines of which the author hopes that it can serve as a user manual for other researchers who are interested in research on ASP. In this manual, the following issues are discussed: physical sensing characteristics, temporal construction, normalization, context, validation, triangulation, and user identification. In the second and last chapter of this part, Chapter 11, the monograph ends with a wrap-up of the work, which is followed by a historical reflection. Next, a triplet of applications is presented that is (almost) ready to be brought to the market here and now, which are followed by two possible future applications. This monograph closes with a brief conclusion: The work presented in this monograph revealed several factors of importance for ASP, which helps the scientific community to understand ASP better. Moreover, I expect that the manual that resulted from the work presented in this monograph will guide future research on ASP to higher levels.

278