- •Stellingen
- •Propositions
- •List of Figures
- •List of Tables
- •1 Introduction
- •Introduction
- •Affect, emotion, and related constructs
- •Affective Computing: A concise overview
- •The closed loop model
- •Three disciplines
- •Human-Computer Interaction (HCI)
- •Health Informatics
- •Three disciplines, one family
- •Outline
- •2 A review of Affective Computing
- •Introduction
- •Vision
- •Speech
- •Biosignals
- •A review
- •Time for a change
- •3 Statistical moments as signal features
- •Introduction
- •Emotion
- •Measures of affect
- •Affective wearables
- •Experiment
- •Participants
- •Equipment and materials
- •Procedure
- •Data reduction
- •Results
- •Discussion
- •Comparison with the literature
- •Use in products
- •4 Time windows and event-related responses
- •Introduction
- •Data reduction
- •Results
- •Mapping events on signals
- •Discussion and conclusion
- •Interpreting the signals measured
- •Looking back and forth
- •5 Emotion models, environment, personality, and demographics
- •Introduction
- •Emotions
- •Modeling emotion
- •Ubiquitous signals of emotion
- •Method
- •Participants
- •International Affective Picture System (IAPS)
- •Digital Rating System (DRS)
- •Signal processing
- •Signal selection
- •Speech signal
- •Heart rate variability (HRV) extraction
- •Normalization
- •Results
- •Considerations with the analysis
- •The (dimensional) valence-arousal (VA) model
- •The six basic emotions
- •The valence-arousal (VA) model versus basic emotions
- •Discussion
- •Conclusion
- •6 Static versus dynamic stimuli
- •Introduction
- •Emotion
- •Method
- •Preparation for analysis
- •Results
- •Considerations with the analysis
- •The (dimensional) valence-arousal (VA) model
- •The six basic emotions
- •The valence-arousal (VA) model versus basic emotions
- •Static versus dynamic stimuli
- •Conclusion
- •IV. Towards affective computing
- •Introduction
- •Data set
- •Procedure
- •Preprocessing
- •Normalization
- •Baseline matrix
- •Feature selection
- •k-Nearest Neighbors (k-NN)
- •Support vector machines (SVM)
- •Multi-Layer Perceptron (MLP) neural network
- •Discussion
- •Conclusions
- •8 Two clinical case studies on bimodal health-related stress assessment
- •Introduction
- •Post-Traumatic Stress Disorder (PTSD)
- •Storytelling and reliving the past
- •Emotion detection by means of speech signal analysis
- •The Subjective Unit of Distress (SUD)
- •Design and procedure
- •Features extracted from the speech signal
- •Results
- •Results of the Stress-Provoking Story (SPS) sessions
- •Results of the Re-Living (RL) sessions
- •Overview of the features
- •Discussion
- •Stress-Provoking Stories (SPS) study
- •Re-Living (RL) study
- •Stress-Provoking Stories (SPS) versus Re-Living (RL)
- •Conclusions
- •9 Cross-validation of bimodal health-related stress assessment
- •Introduction
- •Speech signal processing
- •Outlier removal
- •Parameter selection
- •Dimensionality Reduction
- •k-Nearest Neighbors (k-NN)
- •Support vector machines (SVM)
- •Multi-Layer Perceptron (MLP) neural network
- •Results
- •Cross-validation
- •Assessment of the experimental design
- •Discussion
- •Conclusion
- •10 Guidelines for ASP
- •Introduction
- •Signal processing guidelines
- •Physical sensing characteristics
- •Temporal construction
- •Normalization
- •Context
- •Pattern recognition guidelines
- •Validation
- •Triangulation
- •Conclusion
- •11 Discussion
- •Introduction
- •Hot topics: On the value of this monograph
- •Applications: Here and now!
- •TV experience
- •Knowledge representations
- •Computer-Aided Diagnosis (CAD)
- •Visions of the future
- •Robot nannies
- •Digital Human Model
- •Conclusion
- •Bibliography
- •Summary
- •Samenvatting
- •Dankwoord
- •Curriculum Vitae
- •Publications and Patents: A selection
- •Publications
- •Patents
- •SIKS Dissertation Series
8.11 Conclusions
other and against the SUD. Extrapolations were made using the data sets of both studies and a set of common discriminating speech features were identified. Moreover, the SUD was used as ground truth. However, this required introspection of the patients, which is generally not considered as the most reliable measure.
This research has used one signal; hence, no multi-model integration of signals has been applied. However, for both studies, the features and their parameters were all integrated in one LRM. Additional other signals were omitted on purpose since they could contaminate the ecological validity of the research, as they would interfere with the actual tasks the patients had to perform.
CAD should be able to function in a setting such as in which this research was conducted; hence, having the same characteristics. In general, these are average office settings. Within reason, the speech signal processing scheme 8.2 should be able to handle changing characteristics of an office, which could influence the room’s acoustics. However, there are no indications for any problems that could occur as a results of this.
8.11 Conclusions
This chapter has presented two studies in which the same PTSD patients participated. This provided us with two unique data sets. This has revealed interesting common denominators as well as differences between both studies, which are of concern for several theoretical frameworks, as was denoted in the previous section. Moreover, a thorough discussion has been presented, in two phases. First, the results of both studies were discussed and, subsequently, related to each other. Second, a range of aspects concerning the complete research were discussed. This emphasized the strength of the research presented and also provided interesting pointers for follow-up research.
A Linear Regression Model (LRM) was developed, derived from the data of each of the studies. These LRMs explained respectively 83% of the variance for the SPS study and 69% of the variance for the RL study, which are both high. Founded on the results of both studies, a set of generic features has been defined; see also Table 8.2. This set could serve as the foundation for the development of models that enable stress identification in a robust and generic manner.
It would also be of interest to apply such a model on patients suffering from other related psychiatric disorders, such as depression [9, 333] and insomnia [9, 267]. Probably, even for less related psychiatric disorders, the current approach would be a good starting point. In such a case, the general framework and speech signal processing scheme (see Figure 8.2), as presented in this chapter, could be employed. Most likely, only the set of parameters used for the LRM would have to be tailored to the specific disorders.
149
8 Two clinical case studies on bimodal health-related stress assessment
The speech signal processing approach used in this research could also be linked to approaches that measure physiological responsiveness of PTSD in other ways; for example, using biosignals or computer vision techniques (see Chapter 2). This would facilitate a triangulation of the construct under investigation, providing even more reliable results [680]. Furthermore, more specific analyses can be conducted; for example, in terms of either the valence and arousal model or discrete emotion categories [680] (cf. Chapters 6 and 5). However, it also has its disadvantages, as discussed in the previous section.
Taken together, an important and significant step has been made towards CAD for treatment of patients suffering from a PTSD in particular and stress-related psychiatric disorders in general. Through the design of the research, it was made sure that “real” emotions were measured. Subsequently, their objective measurement through speech signal processing was shown to be feasible. Models were constructed, founded on a selection from 65 parameters of five speech features. With up to 83% explained variance, the models were shown to provide reliable, robust classification of stress. As such, the foundation was developed for an objective, easily usable, unobtrusive, and powerful CAD.
With this chapter, theory has been brought to (clinical) practice. Through two studies, it is shown how rich speech is as an indirect biosignal. As such, it can be valuable even without other biosignals added to it. This provides us with an indirect completely unobtrusive biosignal on which models were build that can serve as an expert system in psychiatric practice. The next chapter has little in common with the current chapter except that it also explores the feasibility of building emotion-aware systems. In the current chapter, the research presented in this chapter will be fed to the signal processing + pattern recognition pipeline, aw was introduced in Section 1.5 (see also Figure 1.2) and already employed in Chapter 7. Again, a range of signal processing and machine learning techniques will be presented, which will bring us close to the envisioned emotion-aware systems: ASP-based Computer-Aided Diagnosis (CAD) for mental health care.
150
