- •Stellingen
- •Propositions
- •List of Figures
- •List of Tables
- •1 Introduction
- •Introduction
- •Affect, emotion, and related constructs
- •Affective Computing: A concise overview
- •The closed loop model
- •Three disciplines
- •Human-Computer Interaction (HCI)
- •Health Informatics
- •Three disciplines, one family
- •Outline
- •2 A review of Affective Computing
- •Introduction
- •Vision
- •Speech
- •Biosignals
- •A review
- •Time for a change
- •3 Statistical moments as signal features
- •Introduction
- •Emotion
- •Measures of affect
- •Affective wearables
- •Experiment
- •Participants
- •Equipment and materials
- •Procedure
- •Data reduction
- •Results
- •Discussion
- •Comparison with the literature
- •Use in products
- •4 Time windows and event-related responses
- •Introduction
- •Data reduction
- •Results
- •Mapping events on signals
- •Discussion and conclusion
- •Interpreting the signals measured
- •Looking back and forth
- •5 Emotion models, environment, personality, and demographics
- •Introduction
- •Emotions
- •Modeling emotion
- •Ubiquitous signals of emotion
- •Method
- •Participants
- •International Affective Picture System (IAPS)
- •Digital Rating System (DRS)
- •Signal processing
- •Signal selection
- •Speech signal
- •Heart rate variability (HRV) extraction
- •Normalization
- •Results
- •Considerations with the analysis
- •The (dimensional) valence-arousal (VA) model
- •The six basic emotions
- •The valence-arousal (VA) model versus basic emotions
- •Discussion
- •Conclusion
- •6 Static versus dynamic stimuli
- •Introduction
- •Emotion
- •Method
- •Preparation for analysis
- •Results
- •Considerations with the analysis
- •The (dimensional) valence-arousal (VA) model
- •The six basic emotions
- •The valence-arousal (VA) model versus basic emotions
- •Static versus dynamic stimuli
- •Conclusion
- •IV. Towards affective computing
- •Introduction
- •Data set
- •Procedure
- •Preprocessing
- •Normalization
- •Baseline matrix
- •Feature selection
- •k-Nearest Neighbors (k-NN)
- •Support vector machines (SVM)
- •Multi-Layer Perceptron (MLP) neural network
- •Discussion
- •Conclusions
- •8 Two clinical case studies on bimodal health-related stress assessment
- •Introduction
- •Post-Traumatic Stress Disorder (PTSD)
- •Storytelling and reliving the past
- •Emotion detection by means of speech signal analysis
- •The Subjective Unit of Distress (SUD)
- •Design and procedure
- •Features extracted from the speech signal
- •Results
- •Results of the Stress-Provoking Story (SPS) sessions
- •Results of the Re-Living (RL) sessions
- •Overview of the features
- •Discussion
- •Stress-Provoking Stories (SPS) study
- •Re-Living (RL) study
- •Stress-Provoking Stories (SPS) versus Re-Living (RL)
- •Conclusions
- •9 Cross-validation of bimodal health-related stress assessment
- •Introduction
- •Speech signal processing
- •Outlier removal
- •Parameter selection
- •Dimensionality Reduction
- •k-Nearest Neighbors (k-NN)
- •Support vector machines (SVM)
- •Multi-Layer Perceptron (MLP) neural network
- •Results
- •Cross-validation
- •Assessment of the experimental design
- •Discussion
- •Conclusion
- •10 Guidelines for ASP
- •Introduction
- •Signal processing guidelines
- •Physical sensing characteristics
- •Temporal construction
- •Normalization
- •Context
- •Pattern recognition guidelines
- •Validation
- •Triangulation
- •Conclusion
- •11 Discussion
- •Introduction
- •Hot topics: On the value of this monograph
- •Applications: Here and now!
- •TV experience
- •Knowledge representations
- •Computer-Aided Diagnosis (CAD)
- •Visions of the future
- •Robot nannies
- •Digital Human Model
- •Conclusion
- •Bibliography
- •Summary
- •Samenvatting
- •Dankwoord
- •Curriculum Vitae
- •Publications and Patents: A selection
- •Publications
- •Patents
- •SIKS Dissertation Series
Stellingen
behorende bij het proefschrift
A ective Signal Processing (ASP):
Unraveling the mystery of emotions
1.ASP zal op termijn middelen verscha en om mensen te manipuleren (maar misschien vinden ze dat niet eens erg).
2.ASP lijdt onder het uitblijven van standaarden.
3.Een voldoende voorwaarde om de kansverdeling van een continue stochast X op een oneindig interval te karakteriseren door haar centrale momenten (bekend als
het Hamburger momentenprobleem) is:
∞ |
|
−1 |
|
1 |
|||
X inf (E[X 2I]) |
|
= ∞. |
|
2I |
|||
N=1 I≤N |
|
|
|
Daaraan is voldaan door biosignalen die een Laplace of normale verdeling hebben, hetgeen meestal het geval is. Hierbij zijn X ’s centrale momenten gedefinieerd als:
Z +∞
E[(X − X¯)N] = (X − X)NFX (X)DX,
−∞
waarbij X¯ de gemiddelde waarde van X is en FX de dichtheidsfunctie van X . Het eindige rijtje van de eerste N centrale momenten (bijvoorbeeld N = 4) is een compacte representatie van X en geeft, als zodanig, een alternatief voor andere signaal decompositie technieken (bijvoorbeeld Fourier en wavelets), wat ook interessant is voor ASP, vanuit zowel a ectief en computationeel oogpunt.
4.“Als je kunt meten waarover je spreekt en je het uit kunt drukken in getallen dan weet je er iets over.” (William Thomson, beter bekend als Lord Kelvin, 1824– 1907, 1883). Toch is het, om cognitieve engineering (zoals ASP) van theorie naar de praktijk te brengen, nodig om ook van onduidelijke modellen gebruik te maken.
5.Nu de samenleving ICT omarmt, worden ethische kwesties in verband met ICT belangrijker. Helaas worstelt de ethiek nog met het veroveren van een plaats binnen de techniek.
6.Multidisciplinair onderzoek is nog geen interdisciplinair onderzoek. In het eerste geval is vaak nog sprake van onbegrip voor elkaars methoden, theorie¨en en cultuur; in het tweede geval zijn deze problemen grotendeels opgelost.
7.Onderwijs is nog steeds het ondergeschoven kindje van de Nederlandse universiteiten.
Egon L. van den Broek Wenen, Oostenrijk, 1 augustus 2011
Propositions
belonging to the Ph.D.-thesis
A ective Signal Processing (ASP):
Unraveling the mystery of emotions
1.ASP will eventually provide the means to manipulate people (but, perhaps they won’t even mind).
2.ASP su ers from a lack of standardization.
3.A su cient condition for the probability distribution of a continuous random variable X to be characterized by its central moments (i.e., the Hamburger moment problem) for an infinite interval is given by:
∞ |
1 |
−1 |
|
|
|||
N=1 I≤N |
|
||
X |
inf (E[X 2I]) |
|
= ∞, |
|
2I |
||
which holds for biosignals that have a Laplace or normal distribution, as is usually the case. With X ’s central moments being defined as:
Z +∞
E[(X − X¯)N] = (X − X)NFX (X)DX,
−∞
where X¯ is the average value of X and FX is the density function of X . The finite series of the first N central moments (e.g., N = 4) is a compact representation of X and provides, as such, an alternative to other signal decomposition techniques (e.g., Fourier and wavelets), which is also interesting for ASP, from both an a ective and a computational point of view.
4.“. . . when you can measure what you are speaking about, and express it in numbers, you know something about it . . . ” (William Thomson; a.k.a. Lord Kelvin, 1824–1907, 1883). Although true, to bring cognitive engineering (e.g., ASP) from theory to practice, ill defined models must also be embraced.
5.With society embracing ICT, ethical issues in relation to ICT are increasing in importance. Regrettably, they are still struggling to find their way into engineering.
6.Multidisciplinary research is not the same as interdisciplinary research. With the first, incomprehension for each other’s methods, theories, and culture is often still present; with the latter, these problems have largely been resolved.
7.Education is still the red headed stepchild of the Dutch universities.
Egon L. van den Broek Vienna, Austria, August 1, 2011
AFFECTIVE SIGNAL PROCESSING (ASP)
UNRAVELING THE MYSTERY OF EMOTIONS
Egon L. van den Broek
Ph.D. dissertation committee:
Chairman and Secretary
prof. dr. M. C. Elwenspoek, University of Twente, The Netherlands Promotores:
prof. dr. ir. A. Nijholt, University of Twente, The Netherlands
prof. dr. T. Dijkstra, Radboud University Nijmegen, The Netherlands Assistent-promotor:
dr. J. H. D. M. Westerink, Philips Research, The Netherlands Members:
prof. dr. P. M. G. Apers, University of Twente, The Netherlands prof. dr. A. Esposito, Second University of Naples, Italy
International Institute for Advanced Scientific Studies, Italy prof. dr. ir. H. J. Hermens, University of Twente, The Netherlands /
Roessingh Research and Development, The Netherlands prof. dr. ir. E. Hoenkamp, Queensland University of Technology, Australia
prof. dr. L. R. B. Schomaker, University of Groningen, The Netherlands
Paranimfen:
Joris H. Janssen, M.Sc., Eindhoven University of Technology, The Netherlands / Philips Research, The Netherlands
Frans van der Sluis, M.Sc., University of Twente, The Netherlands /
Radboud University Medical Center Nijmegen, The Netherlands
CTIT Ph.D.-thesis series No. 11-204 (ISSN: 1381-3617)
Centre for Telematics and Information Technology (CTIT)
P.O. Box 217, 7500 AE Enschede, The Netherlands
SIKS Dissertation series No. 2011-30
The research reported in this thesis has been carried out under the auspices of SIKS, the Dutch Research School for Information and Knowledge Systems.
This book was typeset by the author using LATEX 2ε.
Cover: Design and graphics by Wilson Design, Uden, The Netherlands. Printing: Ipskamp Drukkers, Enschede, The Netherlands.
AFFECTIVE SIGNAL PROCESSING (ASP)
UNRAVELING THE MYSTERY OF EMOTIONS
PROEFSCHRIFT
ter verkrijging van
de graad doctor aan de Universiteit Twente, op gezag van de rector magnificus,
prof. dr. H. Brinksma,
volgens besluit van het College voor Promoties in het openbaar te verdedigen
op vrijdag 16 september 2011 om 14.45 uur
door
Egidius Leon van den Broek
geboren op 22 augustus 1974 te Nijmegen
This dissertation is approved by:
Promotores: |
prof. dr. ir. A. Nijholt, University of Twente, The Netherlands |
|
prof. dr. T. Dijkstra, Radboud University Nijmegen, The Netherlands |
Assistent-promotor: |
dr. J. H. D. M. Westerink, Philips Research, The Netherlands |
Copyright c 2011 by Egon L. van den Broek.
All rights reserved. No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopy, recording, or any information storage or retrieval system, without written permission from the author.
ISSN: |
1381-3617; CTIT Ph.D.-thesis series No. 11-204 |
ISBN: |
978-90-365-3243-3 |
DOI: |
10.3990/1.9789036532433 |
Contents
|
List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
xi |
|
|
List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
xv |
|
I. |
PROLOGUE |
1 |
|
1 |
Introduction |
3 |
|
|
1.1 |
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
5 |
|
1.2 |
Affect, emotion, and related constructs . . . . . . . . . . . . . . . . . . . . . . . |
7 |
|
1.3 |
Affective Computing: A concise overview . . . . . . . . . . . . . . . . . . . . . |
8 |
|
1.4 |
Affective Signal Processing (ASP) : A research rationale . . . . . . . . . . . . . |
12 |
|
1.5 |
The closed loop model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
12 |
|
1.6 |
Three disciplines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
18 |
1.6.1Human-Computer Interaction (HCI) . . . . . . . . . . . . . . . . . . . . 18
|
1.6.2 |
Artificial Intelligence (AI) . . . . . . . . . . . . . . . . . . . . . . . . . . |
19 |
|
1.6.3 |
Health Informatics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
19 |
|
1.6.4 |
Three disciplines, one family . . . . . . . . . . . . . . . . . . . . . . . . |
20 |
1.7 |
Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
20 |
|
2 A review of Affective Computing |
23 |
||
2.1 |
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
25 |
|
2.2 |
Vision . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
25 |
|
2.3 |
Speech . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
27 |
|
2.4 |
Biosignals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
30 |
|
|
2.4.1 |
A review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
30 |
|
2.4.2 Time for a change . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
37 |
|
II. BASELINE-FREE ASP |
39 |
||
3 Statistical moments as signal features |
41 |
||
3.1 |
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
43 |
|
3.2 |
Emotion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
43 |
|
3.3 |
Measures of affect . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
44 |
|
v
Contents
3.4 |
Affective wearables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
45 |
|
3.5 |
Experiment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
46 |
|
|
3.5.1 |
Participants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
46 |
|
3.5.2 |
Equipment and materials . . . . . . . . . . . . . . . . . . . . . . . . . . |
46 |
|
3.5.3 |
Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
49 |
3.6 |
Data reduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
49 |
|
3.7 |
Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
50 |
|
3.8 |
Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
51 |
|
|
3.8.1 |
Comparison with the literature . . . . . . . . . . . . . . . . . . . . . . . |
51 |
|
3.8.2 |
Use in products . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
53 |
4 Time windows and event-related responses |
55 |
||
4.1 |
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
57 |
|
4.2 |
Data reduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
59 |
|
4.3 |
Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
59 |
|
|
4.3.1 |
The influence of scene changes . . . . . . . . . . . . . . . . . . . . . . . |
60 |
|
4.3.2 |
The film fragments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
60 |
|
4.3.3 Mapping events on signals . . . . . . . . . . . . . . . . . . . . . . . . . . |
62 |
|
4.4 |
Discussion and conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
67 |
|
4.4.1Interpreting the signals measured . . . . . . . . . . . . . . . . . . . . . . 67
|
4.4.2 Looking back and forth . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
69 |
|
III. BI-MODAL ASP |
71 |
||
5 Emotion models, environment, personality, and demographics |
73 |
||
5.1 |
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
75 |
|
5.2 |
Emotions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
76 |
|
|
5.2.1 |
On defining emotions . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
77 |
|
5.2.2 |
Modeling emotion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
77 |
5.3 |
Ubiquitous signals of emotion . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
78 |
|
5.4 |
Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
80 |
|
|
5.4.1 |
Participants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
80 |
5.4.2International Affective Picture System (IAPS) . . . . . . . . . . . . . . . 80
5.4.3 Digital Rating System (DRS) . . . . . . . . . . . . . . . . . . . . . . . . . |
81 |
|
5.5 Signal processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
83 |
|
5.5.1 |
Signal selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
83 |
5.5.2 |
Speech signal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
84 |
5.5.3 Heart rate variability (HRV) extraction . . . . . . . . . . . . . . . . . . . |
86 |
|
5.5.4Normalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
5.6 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
vi
Contents
5.6.1 Considerations with the analysis . . . . . . . . . . . . . . . . . . . . . . 88
5.6.2The (dimensional) valence-arousal (VA) model . . . . . . . . . . . . . . 89
5.6.3 |
The six basic emotions . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
91 |
5.6.4 |
The valence-arousal (VA) model versus basic emotions . . . . . . . . . |
93 |
5.7 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
94 |
|
5.7.1 |
The five issues under investigation . . . . . . . . . . . . . . . . . . . . . |
94 |
5.7.2Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
6 Static versus dynamic stimuli |
99 |
||
6.1 |
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
101 |
|
6.2 |
Emotion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
102 |
|
6.3 |
Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
103 |
|
6.4 |
Preparation for analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
103 |
|
6.5 |
Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
105 |
|
|
6.5.1 |
Considerations with the analysis . . . . . . . . . . . . . . . . . . . . . . |
105 |
|
6.5.2 |
The (dimensional) valence-arousal (VA) model . . . . . . . . . . . . . . |
106 |
|
6.5.3 |
The six basic emotions . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
107 |
|
6.5.4 |
The valence-arousal (VA) model versus basic emotions . . . . . . . . . |
108 |
6.6 |
Static versus dynamic stimuli . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
110 |
|
6.7 |
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
111 |
|
IV. TOWARDS AFFECTIVE COMPUTING |
113 |
||
7 Automatic classification of affective signals |
115 |
||
7.1 |
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
117 |
|
7.2 |
Data set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
118 |
|
|
7.2.1 |
Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
118 |
7.3 |
Preprocessing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
119 |
|
7.3.1Normalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
|
7.3.2 |
Baseline matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
121 |
|
7.3.3 |
Feature selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
121 |
7.4 |
Classification results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
122 |
|
|
7.4.1 |
k-Nearest Neighbors (k-NN) . . . . . . . . . . . . . . . . . . . . . . . . |
123 |
|
7.4.2 Support vector machines (SVM) . . . . . . . . . . . . . . . . . . . . . . . |
123 |
|
|
7.4.3 |
Multi-Layer Perceptron (MLP) neural network . . . . . . . . . . . . . . |
124 |
|
7.4.4 |
Reflection on the results . . . . . . . . . . . . . . . . . . . . . . . . . . . |
125 |
7.5 |
Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
126 |
|
7.6 |
Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
128 |
|
8 Two clinical case studies on bimodal health-related stress assessment |
129 |
||
8.1 |
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
131 |
|
vii
Contents
8.2 |
Post-Traumatic Stress Disorder (PTSD) . . . . . . . . . . . . . . . . . . . . . . . |
131 |
|
8.3 |
Storytelling and reliving the past . . . . . . . . . . . . . . . . . . . . . . . . . . |
134 |
|
8.4 |
Emotion detection by means of speech signal analysis . . . . . . . . . . . . . . |
134 |
|
8.5 |
The Subjective Unit of Distress (SUD) . . . . . . . . . . . . . . . . . . . . . . . |
135 |
|
8.6 |
Design and procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
136 |
|
8.7 |
Features extracted from the speech signal . . . . . . . . . . . . . . . . . . . . . |
137 |
|
8.8 |
Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
141 |
|
|
8.8.1 |
Results of the Stress-Provoking Story (SPS) sessions . . . . . . . . . . . |
142 |
|
8.8.2 |
Results of the Re-Living (RL) sessions . . . . . . . . . . . . . . . . . . . |
142 |
|
|
8.8.2.A Overview of the features . . . . . . . . . . . . . . . . . . . . . |
143 |
8.9 |
Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
144 |
|
|
8.9.1 |
Stress-Provoking Stories (SPS) study . . . . . . . . . . . . . . . . . . . . |
145 |
|
8.9.2 |
Re-Living (RL) study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
145 |
|
8.9.3 |
Stress-Provoking Stories (SPS) versus Re-Living (RL) . . . . . . . . . . |
146 |
8.10 |
Reflection: Methodological issues and suggestions . . . . . . . . . . . . . . . . |
148 |
|
8.11 |
Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
149 |
|
9 Cross-validation of bimodal health-related stress assessment |
151 |
||
9.1 |
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
153 |
|
9.2Speech signal processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
9.2.1Outlier removal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
9.2.2Parameter selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
9.2.3Dimensionality Reduction . . . . . . . . . . . . . . . . . . . . . . . . . . 155
9.3Classification techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156
9.3.1k-Nearest Neighbors (k-NN) . . . . . . . . . . . . . . . . . . . . . . . . 156
9.3.2Support vector machines (SVM) . . . . . . . . . . . . . . . . . . . . . . . 156
9.3.3Multi-Layer Perceptron (MLP) neural network . . . . . . . . . . . . . . 157
9.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
9.4.1Cross-validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
|
|
9.4.2 |
Assessment of the experimental design . . . . . . . . . . . . . . . . . . |
159 |
|
9.5 |
Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
161 |
|
|
9.6 |
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
162 |
|
V. |
EPILOGUE |
|
165 |
|
10 |
Guidelines for ASP |
167 |
||
|
10.1 |
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
169 |
|
|
10.2 |
Signal processing guidelines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
169 |
|
|
|
10.2.1 |
Physical sensing characteristics . . . . . . . . . . . . . . . . . . . . . . . |
169 |
|
|
10.2.2 |
Temporal construction . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
172 |
viii
Contents
10.2.3Normalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
10.2.4Context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
10.3Pattern recognition guidelines . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178
|
10.3.1 |
Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
179 |
|
10.3.2 |
Triangulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
180 |
|
10.3.3 |
User identification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
182 |
10.4 |
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
184 |
|
11 Discussion |
185 |
||
11.1 |
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
187 |
|
11.2Historical reflection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188
11.3Hot topics: On the value of this monograph . . . . . . . . . . . . . . . . . . . . 191
11.4 |
Impressions / expressions: Affective Computing’s I/O . . . . . . . . . . . . . |
193 |
|
11.5 |
Applications: Here and now! . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
194 |
|
|
11.5.1 |
TV experience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
195 |
|
11.5.2 |
Knowledge representations . . . . . . . . . . . . . . . . . . . . . . . . . |
196 |
11.5.3Computer-Aided Diagnosis (CAD) . . . . . . . . . . . . . . . . . . . . . 196
11.6Visions of the future . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
11.6.1Robot nannies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
|
11.6.2 Digital Human Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
198 |
11.7 |
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
200 |
BIBLIOGRAPHY |
201 |
|
A Statistical techniques |
261 |
|
A.1 |
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
263 |
A.2 |
Principal component analysis (PCA) . . . . . . . . . . . . . . . . . . . . . . . . |
264 |
A.3 |
Analysis of variance (ANOVA) . . . . . . . . . . . . . . . . . . . . . . . . . . . |
265 |
A.4 |
Linear regression models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
268 |
A.5 |
k-nearest neighbors (k-NN) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
269 |
A.6 |
Artificial neural networks (ANN) . . . . . . . . . . . . . . . . . . . . . . . . . . |
270 |
A.7 |
Support vector machine (SVM) . . . . . . . . . . . . . . . . . . . . . . . . . . . |
271 |
A.8 |
Leave-one-out cross validation (LOOCV) . . . . . . . . . . . . . . . . . . . . . |
272 |
Summary |
275 |
|
Samenvatting |
279 |
|
Dankwoord |
283 |
|
Curriculum Vitae |
287 |
|
|
ix |
|
Contents
Publications and Patents: A selection |
291 |
Publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
293 |
Patents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
298 |
SIKS Dissertation Series |
299 |
x
