- •Stellingen
- •Propositions
- •List of Figures
- •List of Tables
- •1 Introduction
- •Introduction
- •Affect, emotion, and related constructs
- •Affective Computing: A concise overview
- •The closed loop model
- •Three disciplines
- •Human-Computer Interaction (HCI)
- •Health Informatics
- •Three disciplines, one family
- •Outline
- •2 A review of Affective Computing
- •Introduction
- •Vision
- •Speech
- •Biosignals
- •A review
- •Time for a change
- •3 Statistical moments as signal features
- •Introduction
- •Emotion
- •Measures of affect
- •Affective wearables
- •Experiment
- •Participants
- •Equipment and materials
- •Procedure
- •Data reduction
- •Results
- •Discussion
- •Comparison with the literature
- •Use in products
- •4 Time windows and event-related responses
- •Introduction
- •Data reduction
- •Results
- •Mapping events on signals
- •Discussion and conclusion
- •Interpreting the signals measured
- •Looking back and forth
- •5 Emotion models, environment, personality, and demographics
- •Introduction
- •Emotions
- •Modeling emotion
- •Ubiquitous signals of emotion
- •Method
- •Participants
- •International Affective Picture System (IAPS)
- •Digital Rating System (DRS)
- •Signal processing
- •Signal selection
- •Speech signal
- •Heart rate variability (HRV) extraction
- •Normalization
- •Results
- •Considerations with the analysis
- •The (dimensional) valence-arousal (VA) model
- •The six basic emotions
- •The valence-arousal (VA) model versus basic emotions
- •Discussion
- •Conclusion
- •6 Static versus dynamic stimuli
- •Introduction
- •Emotion
- •Method
- •Preparation for analysis
- •Results
- •Considerations with the analysis
- •The (dimensional) valence-arousal (VA) model
- •The six basic emotions
- •The valence-arousal (VA) model versus basic emotions
- •Static versus dynamic stimuli
- •Conclusion
- •IV. Towards affective computing
- •Introduction
- •Data set
- •Procedure
- •Preprocessing
- •Normalization
- •Baseline matrix
- •Feature selection
- •k-Nearest Neighbors (k-NN)
- •Support vector machines (SVM)
- •Multi-Layer Perceptron (MLP) neural network
- •Discussion
- •Conclusions
- •8 Two clinical case studies on bimodal health-related stress assessment
- •Introduction
- •Post-Traumatic Stress Disorder (PTSD)
- •Storytelling and reliving the past
- •Emotion detection by means of speech signal analysis
- •The Subjective Unit of Distress (SUD)
- •Design and procedure
- •Features extracted from the speech signal
- •Results
- •Results of the Stress-Provoking Story (SPS) sessions
- •Results of the Re-Living (RL) sessions
- •Overview of the features
- •Discussion
- •Stress-Provoking Stories (SPS) study
- •Re-Living (RL) study
- •Stress-Provoking Stories (SPS) versus Re-Living (RL)
- •Conclusions
- •9 Cross-validation of bimodal health-related stress assessment
- •Introduction
- •Speech signal processing
- •Outlier removal
- •Parameter selection
- •Dimensionality Reduction
- •k-Nearest Neighbors (k-NN)
- •Support vector machines (SVM)
- •Multi-Layer Perceptron (MLP) neural network
- •Results
- •Cross-validation
- •Assessment of the experimental design
- •Discussion
- •Conclusion
- •10 Guidelines for ASP
- •Introduction
- •Signal processing guidelines
- •Physical sensing characteristics
- •Temporal construction
- •Normalization
- •Context
- •Pattern recognition guidelines
- •Validation
- •Triangulation
- •Conclusion
- •11 Discussion
- •Introduction
- •Hot topics: On the value of this monograph
- •Applications: Here and now!
- •TV experience
- •Knowledge representations
- •Computer-Aided Diagnosis (CAD)
- •Visions of the future
- •Robot nannies
- •Digital Human Model
- •Conclusion
- •Bibliography
- •Summary
- •Samenvatting
- •Dankwoord
- •Curriculum Vitae
- •Publications and Patents: A selection
- •Publications
- •Patents
- •SIKS Dissertation Series
8 Two clinical case studies on bimodal health-related stress assessment
|
5 |
Anxiety |
|
|
|
Happy |
|
|
4 |
|
|
SUD |
3 |
|
|
|
|
|
|
|
2 |
|
|
|
1 |
|
|
|
1 |
2 |
3 |
Time (minutes)
Figure 8.4: Reported stress over time per session (i.e., anxiety triggering and happy) for the Stress-Provoking Stories (SPS) study.
8.8.1 Results of the Stress-Provoking Story (SPS) sessions
Changes with respect to the SUD in the course of the sessions of the SPS study were analyzed first. No main effects of the SPS session (happy or anxious) or measurement moment (first, second, or third minute of storytelling) on the SUD scores were found in an ANOVA, nor did any significant interaction effect between these factors appear. A closer look at the SUD scores in the stress-provoking session showed that the experienced stress reported by the patients increased in the course of storytelling, as indicated by a trend in the ANOVA for the factor measurement moment, F (2, 67) = 2.59, p < .010. Figure 8.4 illustrates this trend. In addition, Figure 8.4 shows the confidence intervals, only without variability associated with between-subjects variance (cf. [128]).
Next, a robust acoustic profile was created of the speech characteristics sensitive to stress. This profile was generated after 20 iterations of the backward method, leaving 30 significant predictors explaining 81.00% of variance: R2 = .810, R2 = .757, F (30, 109) = 15.447, p < .001. Before applying the backward method (i.e., before any predictors were removed), 50 predictors explained 82.60% of variance: R2 = .826, R2 = .728, F (50, 89) = 8.445, p < .001. These results indicate that the amount of variance explained through the acoustic profile is high (i.e., R2 > .75), as was expected based on the literature [369].
8.8.2 Results of the Re-Living (RL) sessions
Similar to the analyses performed for the SPS sessions, the analyses for the RL sessions start with an ANOVA of the changes in SUD during the course of the sessions. The results
142
8.8 Results
|
5 |
Anxiety |
|
|
|
Happy |
|
|
4 |
|
|
SUD |
3 |
|
|
|
|
|
|
|
2 |
|
|
|
1 |
|
|
|
1 |
2 |
3 |
Time (minutes)
Figure 8.5: Reported stress over time per session (i.e., anxiety triggering and happy) for the Re-Living (RL) study.
were similar to the SPS analyses: no main effects of the RL session (happy or anxious) or time (first, second, or third minute of storytelling) on the SUD scores were found, nor did a significant interaction effect appear. Again, there was a trend in the anxiety triggering condition for patients to report more experienced stress later-on in the course of re-living, as indicated by a trend in the ANOVA for the factor time, F (2, 69) = 2.69, p < .010. This trend is also evident in Figure 8.5. Note that Figure 8.5 shows the confidence intervals without between-subjects variance (cf. [128]).
A strong acoustic profile for the RL session was created by means of the speech characteristics that are sensitive to stress. An LRM based upon all relevant features and their parameters (49 predictors) explained 69.10% of variance: R2 = .691, R2 = .530, F (49, 94) = 4.29, p < .001. A smaller LRM, based only on the significant features, used 23 predictors explaining 64.80% of variance: R2 = .648, R2 = .584, F (22, 121) = 10.12, p < .001. These results indicate that, for the RL sessions, the subjectively reported stress could be explained very well, as was expected based on the literature [369]. However, the explained variance was lower than for the SPS sessions.
8.8.2.A Overview of the features
A comparison of the LRM of the RL sessions and the SPS sessions shows that there are
13 shared predictors: pitch iqr25 and var; amplitude q75, var, and std ; power iqr25, q25, and std ; zero-crossings q25 and q10; high-frequency power var, std, and mean. However, this comparison is misleading due to the role of the interdependency of the predictors in specifying whether or not they have a significant contribution to the estimate. Hence, for
143
