- •Stellingen
- •Propositions
- •List of Figures
- •List of Tables
- •1 Introduction
- •Introduction
- •Affect, emotion, and related constructs
- •Affective Computing: A concise overview
- •The closed loop model
- •Three disciplines
- •Human-Computer Interaction (HCI)
- •Health Informatics
- •Three disciplines, one family
- •Outline
- •2 A review of Affective Computing
- •Introduction
- •Vision
- •Speech
- •Biosignals
- •A review
- •Time for a change
- •3 Statistical moments as signal features
- •Introduction
- •Emotion
- •Measures of affect
- •Affective wearables
- •Experiment
- •Participants
- •Equipment and materials
- •Procedure
- •Data reduction
- •Results
- •Discussion
- •Comparison with the literature
- •Use in products
- •4 Time windows and event-related responses
- •Introduction
- •Data reduction
- •Results
- •Mapping events on signals
- •Discussion and conclusion
- •Interpreting the signals measured
- •Looking back and forth
- •5 Emotion models, environment, personality, and demographics
- •Introduction
- •Emotions
- •Modeling emotion
- •Ubiquitous signals of emotion
- •Method
- •Participants
- •International Affective Picture System (IAPS)
- •Digital Rating System (DRS)
- •Signal processing
- •Signal selection
- •Speech signal
- •Heart rate variability (HRV) extraction
- •Normalization
- •Results
- •Considerations with the analysis
- •The (dimensional) valence-arousal (VA) model
- •The six basic emotions
- •The valence-arousal (VA) model versus basic emotions
- •Discussion
- •Conclusion
- •6 Static versus dynamic stimuli
- •Introduction
- •Emotion
- •Method
- •Preparation for analysis
- •Results
- •Considerations with the analysis
- •The (dimensional) valence-arousal (VA) model
- •The six basic emotions
- •The valence-arousal (VA) model versus basic emotions
- •Static versus dynamic stimuli
- •Conclusion
- •IV. Towards affective computing
- •Introduction
- •Data set
- •Procedure
- •Preprocessing
- •Normalization
- •Baseline matrix
- •Feature selection
- •k-Nearest Neighbors (k-NN)
- •Support vector machines (SVM)
- •Multi-Layer Perceptron (MLP) neural network
- •Discussion
- •Conclusions
- •8 Two clinical case studies on bimodal health-related stress assessment
- •Introduction
- •Post-Traumatic Stress Disorder (PTSD)
- •Storytelling and reliving the past
- •Emotion detection by means of speech signal analysis
- •The Subjective Unit of Distress (SUD)
- •Design and procedure
- •Features extracted from the speech signal
- •Results
- •Results of the Stress-Provoking Story (SPS) sessions
- •Results of the Re-Living (RL) sessions
- •Overview of the features
- •Discussion
- •Stress-Provoking Stories (SPS) study
- •Re-Living (RL) study
- •Stress-Provoking Stories (SPS) versus Re-Living (RL)
- •Conclusions
- •9 Cross-validation of bimodal health-related stress assessment
- •Introduction
- •Speech signal processing
- •Outlier removal
- •Parameter selection
- •Dimensionality Reduction
- •k-Nearest Neighbors (k-NN)
- •Support vector machines (SVM)
- •Multi-Layer Perceptron (MLP) neural network
- •Results
- •Cross-validation
- •Assessment of the experimental design
- •Discussion
- •Conclusion
- •10 Guidelines for ASP
- •Introduction
- •Signal processing guidelines
- •Physical sensing characteristics
- •Temporal construction
- •Normalization
- •Context
- •Pattern recognition guidelines
- •Validation
- •Triangulation
- •Conclusion
- •11 Discussion
- •Introduction
- •Hot topics: On the value of this monograph
- •Applications: Here and now!
- •TV experience
- •Knowledge representations
- •Computer-Aided Diagnosis (CAD)
- •Visions of the future
- •Robot nannies
- •Digital Human Model
- •Conclusion
- •Bibliography
- •Summary
- •Samenvatting
- •Dankwoord
- •Curriculum Vitae
- •Publications and Patents: A selection
- •Publications
- •Patents
- •SIKS Dissertation Series
8
Two clinical case studies on bimodal health-related stress assessment
Abstract
This chapter is the first of a set of two chapters that aim towards bringing affective computing to practice. As has been denoted in the Introduction, health informatics is on of ASP’s application domains. This chapter describes two studies that share the underlying idea that ASP can initialize Computer-Aided Diagnosis (CAD) for mental health care. To explore the feasibility of this idea, 25 patients suffering from a Post-Traumatic Stress Disorder (PTSD) participated in both studies. To this date, the treatment of PTSD is a great challenge for therapists. CAD is envisioned to enable objective and unobtrusive stress measurement, provide decision support on whether or not the level of stress is excessive, and, consequently, be able to aid in its treatment. Speech was chosen as an objective, unobtrusive stress indicator, considering that most therapy sessions are already recorded anyway. The two studies concerned a (controlled) stress-provoking storytelling (SPS) and a(n ecologically valid) re-living (RL) study, each consisting of a ‘happy’ and an ‘anxiety triggering’ session. The SUD was determined for subjective assessment, which enabled the validation of derived speech features. For both studies, a Linear Regression Model (LRM) was developed, founded on patients’ average acoustic profiles. It used five speech features: amplitude, zero crossings, power, high-frequency power, and pitch. From each feature, 13 parameters were derived; hence, in total 65 parameters were calculated. Using the LRM, respectively 83% and 69% of the variance was explained for the SPS and RL study. Moreover, a set of generic speech signal parameters was presented. Together, the models created and parameters identified can serve as the foundation for future CAD tools.
This chapter is an adapted version of:
Broek, E.L. van den, Sluis, F. van der, and Dijkstra, T. (2011). Telling the story and re-living the past: How speech analysis can reveal emotions in post-traumatic stress disorder (PTSD) patients. In J.H.D.M. Westerink, M. Krans, and M. Ouwerkerk (Eds.), Sensing Emotions: The impact of context on experience measurements (Chapter 10), p. 153–180. Series: Philips Research Book Series, Vol. 12. Dordrecht, The Netherlands: Springer Science+Business Media B.V.
[invited]
8.1 Introduction
No laga duele bieu: |
Let not woes of old |
Skavisábo di nobo. |
enslave you anew. |
– Nydia Ecury – |
|
8.1 Introduction
In our modern society, many people experience stress, sometimes for just a brief moment, at other times for prolonged periods of time. Stress can be defined as a feeling of pressure or tension, caused by influences from the outside world [140, Chapter 6]. It can be accompanied by positive and by negative feelings. It affects our physical state, for instance by increasing our heart rate and blood pressure, and freeing stress hormones like (nor)adrenaline and (nor)epinephrine [359], stimulating autonomic nerve action. Stress may become harmful if it occurs for too long or too frequently, or if it occurs during a traumatic experience. It may, for instance, result in depression, insomnia, or PTSD [178, 365, 547, 562]. To make things even worse, such stress related disorders stigmatize the people suffering from them, which in itself is an additional stressor [563, 564].
Depression cannot always be related to a specific cause, though several contributing factors have been identified: for example, genetic vulnerability and unavoidability of stress [232]. More specifically, certain stressful life events (e.g., job loss, widowhood) can lead to a state of depression. Furthermore, chronic role-related stress is significantly associated with chronically depressed mood [333]. Note that the experience of stress is associated with the onset of depression, and not with the symptoms of depression. Insomnia often has a fairly sudden onset caused by psychological, social, or medical stress [267]. Nevertheless, in some cases, it may develop gradually and without a clear stressor. Insomnia is characterized by sleep deprivation, and associated with increased physiological, cognitive, or emotional arousal in combination with negative conditioning for sleep [9]. Traumas can originate from a range of situations, such as warfare, natural disaster, and interpersonal violence such as sexual, physical, and emotional abuse, intimate partner violence, or collective violence (e.g., a bank robbery) [547]. In such cases, a PTSD may arise, which can be characterized by a series of symptoms and causes [178, 365, 547, 562], summarized in Table 8.1.
8.2 Post-Traumatic Stress Disorder (PTSD)
In our study, we studied the emotions in PTSD patients, who suffered from panic attacks, agoraphobia, and panic disorder with agoraphobia [365, 572].
A panic attack is a discrete period in which there is a sudden onset of intense apprehen-
131
8 Two clinical case studies on bimodal health-related stress assessment
Table 8.1: Introduction to (the DSM-IV TR [9] criteria for) Post-Traumatic Stress Disorder (PTSD).
Trauma can cause long-term physiological and psychological problems. This has been recognized for centuries. Such suffering (e.g., accompanying a Post-Traumatic Stress Disorder, PTSD), can be characterized in terms of a series of symptoms and causes. Traumas can originate from a range of situations, either short or long lasting; for example, warfare, natural disasters such as earthquakes, interpersonal violence such as sexual, physical, and emotional abuse, intimate partner violence, and collective violence.
Diagnostic criteria as defined by the DSM-IV TR [9] comprise six categories of symptoms, each denoting their various indicators:
1.Exposure of the person to a traumatic event.
2.Persistent reexperience of the traumatic event.
3.Persistent avoidance of stimuli, associated with the trauma, and numbing of general responsiveness (not present before the trauma).
4.Persistent symptoms of increased arousal, not present before the trauma.
5.Duration of the disturbance (symptoms in criteria 2, 3, and 4) is more than one month.
6.The disturbance causes clinically significant distress or impairment in social, occupa-
tional, or other important areas of functioning.
Many other symptoms have also been mentioned; for example, weakness, fatigue, loss of willpower, and psychophysiological reactions such as gastrointestinal disturbances. However, these are not included in the DSM-IV TR diagnostic criteria.
Additional diagnostic categories are also suggested for victims of prolonged interpersonal trauma, particularly early in life. These concern problems are related to: 1) regulation of affect and impulses, 2) memory and attention, 3) self-perception, 4) interpersonal relations, 5) somatization, and 6) systems of meaning. Taken together, PTSD includes a broad variety of symptoms and diagnostic criteria. Consequently, the diagnosis is hard to make, as is also the case for various other mental disorders.
sion, fearfulness or terror, often associated with feelings of impending doom. During these Panic Attacks, symptoms such as shortness of breath, palpitations, chest pain or discomfort, choking or smothering sensations, and fear of ‘going crazy’ or losing control are present. The panic attack has a sudden onset and builds rapidly to a peak (usually in 10 minutes or less). Panic attacks can be unexpected (uncued), situationally bound (cued), or situationally predisposed [572]. Agoraphobia is anxiety about, or avoidance of, places or situations from which escape might be difficult (or embarrassing), or in which help may not be available in the event of having a panic attack or panic-like symptoms [572]. Panic disorder with agoraphobia is characterized by both recurrent and unexpected panic attacks, followed by at least one month of persistent concern about having another panic attack, worries about the possible implications or consequences of such attacks, or a significant behavioral change related to these attacks. The frequency and severity of Panic attacks vary widely, but panic
132
8.2 Post-Traumatic Stress Disorder (PTSD)
disorder as described here has been found in epidemiological studies throughout the world. Panic disorders without and with agoraphobia are diagnosed two to three times as often in women as in men. The age of onset of panic disorders varies considerably, but most typically lies between late adolescence and the mid-thirties. Some individuals may have episodic outbreaks with years of remission in between, and others may have continuous severe symptomatology [572].
Due to its large inter-individual variability and its broad variety of symptoms, the diagnosis of PTSD is hard to make [178, 365, 547, 562]. At the same time, it is clear that an efficient treatment of PTSD requires an objective and early diagnosis of the patients’ problems and their therapeutic progress. Assessing the emotional distress of a patient is therefore of the utmost importance. Therapists have developed a range of questionnaires and diagnostic measurement tools for this purpose [353, 572]. Regrettably, these may be experienced as a burden by clients, because it takes the time and willingness of the clients to complete them.
In addition, several other problems arise when a clinician tries to assess the degree of stress in the patient. First, during the appraisal of a stress response, a stressor may not always be seen as stressful enough to be a cause for the mental illness. In other words, although the client may experience it as hugely stressful, the clinician might not always acknowledge it as such. Second, when measuring the response to a stressor, the clinician may rely on introspection and expertise, but these are always to some extent subjective and they also rely on the communicative abilities, truthfulness, and compliance of the client in question. Third, at times it may not be completely clear which (aspect of) the experienced stressor led to the excessive stress response. Finally, the evaluation of the progress in treatment is complicated by its gradualness and relativity.
Given these considerations, it is abundantly clear why researchers have searched for more objective, unobtrusive ways to measure emotions in patient populations. In other words, in addition to standardizing their professional approaches, therapists have sought for new sorts of Computer-Aided Diagnosis (CAD) that are applicable to real-life situations and measure real emotions.
In the following sections, we will first describe both the storytelling and trauma reliving techniques themselves. They provided us with stretches of speech, which we analyzed with respect to a series of signal characteristics to detect emotions. After discussing our speech analysis technique, we will explain how the Subjective Unit of Distress is standardly measured. This will then be followed by a more detailed report of our experimental study. We will end the chapter with an evaluation of our novel approach to stress and emotion measurement.
133
