To build a versatile machine learning algorithm, one needs to gather a large amount of data.The data has to describe as accurately as possible the wide spectrum of emotional responses to a stimulus. For this task we planned two different experiments, one focuses on stress specifically and the other focuses on affective responses.

Stress inducing protocols are already very popular in the psychology field. Probably the most common and validated one is the Trier Social Stress Test (TSST), in which the subjects have to deliver a free speech and perform mental arithmetic. For our experiment, though, we chose to use The Montreal Imaging Stress Task (MIST) as protocol reference. When conducting experiments at scale, the logistics are very much to think about and TSST is much more demanding than MIST. The main concept is still the same, both are very cognitively demanding and put on the subjects a lot of social pressure.
The experiment is mostly based on a proprietary software which displays a mental arithmetic task (in various difficulties), 10 buttons for each digit to submit the answer, a feedback on the submitted response and a comparison of the subject’s score vs. the average subject’s score. Each task is limited by time and it is continuously shown to the subject. 

Stress Experiment

Affective Experiment

Important aspect of affect recognition studies is the selection of suitable stimuli that will be used for eliciting emotions in human subjects. Stimuli may include audio-only stimuli, video-only, or a combination of audio and video in the form of film clips or music video clips. We chose to follow the DREAMER protocol as a reference for our affective experiment.

Audio and visual stimuli in the form of film clips is employed in order to elicit emotional reactions to the subjects. A dataset consisting of 18 film clips selected and evaluated by Gabert-Quillen et al. was utilised for eliciting emotions. These film clips contain cut out scenes from different films that have been shown to evoke a wide range of emotions. From these 18 film clips, two of each targeted one of the following nine emotions: amusement, excitement, happiness, calmness, anger, disgust, fear, sadness and surprise. See table 1.

The dimension approach of emotions using the valence/arousal/dominance scale is not intuitive for humans. To ensure that the subjects provide correct ratings, the rating scale is thoroughly explained both verbally and through examples. Moreover, they are asked to provide an emotional assessment derived by what emotions they actually felt and not what they thought that the video clip was intended to elicit.


The experiment started by showing a neutral film clip (a video clip considered to have no valence in order to establish the baseline signals). The use of neutral signals as baseline signals have been proven to allow the removal of daily dependencies when recording physiological data. As a result, the neutral clip was shown before each film clip in order to help the subject return to a neutral emotional state, in addition to establishing the baseline signal. After viewing each film clip, a graphical user interface is presented in order to evaluate the emotion by reporting the felt arousal and valence on five point scales. Self-assessment manikins (for more information click here) are used in order to facilitate the subject’s assessments of arousal and valence.

The arithmetic task algorithm creates a task using up to 4 numbers ranging from 0 to 99 and up to 4 operands (+, -, *, /), depends on the difficulty. The task solution is always a single digit number such that one of the 10 buttons mentioned above is the correct solution. Upon the response submission an appropriate feedback is presented (“correct”, “wrong”), if no response was submitted within the time limit, the response “wrong” is displayed. When the subject is wrong a loud buzzer sound is played. The time limit is first calibrated in the training phase (explained later on) and adjusted through the trial according to the subject’s performance.


A single trial consists of several phases, instructions, training, cognitive and 4 cycles of stress and relax.

The instructions phase is simply a briefing of the subject and an introduction to the system and how to use it. Very simple just to make sure the subject understands everything.

Training is used for calibrating the time limit of arithmetic tasks according to the subject’s abilities and performance so they will not be too hard to too easy to solve. In addition, the subject can get used to answering the tasks before presenting the time limit. In this phase tasks are presented in various difficulties but without a time limit or any other stressor.

The cognitive phase looks the same as the training but the subject cannot tell that the phase has changed. The only difference is that the cognitive tasks are all the same difficulty but still with no time limit or other stressors. The concept here is to compare the performance of the subject in a “neutral” situation to a “stressful” situation. The same tasks will be presented later on the same trial but this time with all the stressors and the time limit.

The stress phase already described above, the subject has to solve the arithmetic tasks of the same level and has to face a lot of stressors, the feedback to its answer, time limit, social pressure (personal vs average score) and others.

  Last is the relaxation phase where a soothing music is played and subject is asked to lay back and clear the mind.

Figure 1. The stress experiment user interface

Figure 1. The stress experiment user interface

Figure 2. The stress experiment protocol

Figure 2. The stress experiment protocol

Table 1. Average intensity ratings of target emotions by movies

Table 1. Average intensity ratings of target emotions by movies



Science   >

Measuring Emotions