How to extract clues from emotions for scientific research?

Emotion is an important indicator in human activity feedback and one of the inevitable important topics in many research fields, such as emotional verification, psychology, criminal investigation, research on lies, objective feedback on food and beverage in the commercial field, market research. There are extensive researches on usability evaluation, education, games, human-computer interaction, etc. What is emotion and how do scholars now perform facial expression analysis?

What is emotion?

Emotion is a mixed psychological phenomenon. It consists of a unique subjective experience, external representation and physiological arousal (Izard, 1977).

I. Subjective experience is the individual's self-feeling of different emotions based on different objective stimuli, and there are individual differences in the same objective stimuli.

II. The external manifestation of emotions, often referred to as emotional expressions. It is a quantitative form of motion of various parts of the body when the emotional state occurs, including facial expressions, gesture expressions, and voice expressions.

Facial expression is a quantitative form of motion for all facial muscle changes. It can express emotions of different natures in detail, so it is the main sign of identifying emotions. When it comes to expressions, it is the American psychologist Paul Ekman who mainly studies the face. Expression recognition, emotion and interpersonal deception, and the common facial expressions of different cultures have commonality. According to facial anatomy, he divides it into several action units that are independent and related to each other and analyzes them. The motion characteristics of these motion units and the main areas they control and the expressions associated with them, and give a lot of photo explanations. Many face animations are based on the Facial Action Coding System, which is our Said micro-expression.

During his more than forty years of research, he studied the faces of New Guinean tribes, elite patients, spies, serial killers, and professional killers. FBI, CIA, etc. often ask him to be an expression consultant. A gesture expression refers to an expression movement of other parts of the body other than the face, including gestures, body postures, and the like. Tone is also an important form of expressing emotions, and the tone of tone is also closely related to emotions. In Ekman's "Telling Lies" book, the relationship between expression, words, voice, physical behavior, etc., when people's emotions change, is introduced in detail. If some scholars have deeper research on emotions, they may wish to refer to them.

III. Physiological arousal refers to the physiological response of emotions. It involves a wide range of neural structures, such as the brainstem of the central nervous system, central gray matter, thalamus, amygdala, blue spot, pineal gland, prefrontal cortex, and peripheral nerves and endocrine and exocrine glands. Physiological arousal is a physiological activation level. The physiological response patterns of different emotions are different. If you are satisfied and happy, your heartbeat rhythm is normal. When you are furious, your heart beats faster, your blood pressure rises, your breathing rate increases, and even intermittent or paused. In modern judicial psychology, the polygraphs that are often used are designed according to the principle that individuals cannot control physiological changes under emotion.

How do academics study emotions today?

According to the concept of emotion, we can evaluate it based on three aspects:

1. Emotional Subjective Experience: At present, questionnaires and interviews are used to understand the inner experience of the participants. Validity and reliability are subjective self-reported scales, rating scales, etc., such as PANAS (Positive and Negative Affect Schedule).

2. Emotional external performance

Facial expression : The current measurement methods include manual coding and computer automatic measurement. All are based on FACS theory to analyze the relationship between micro-expressions, basic expressions, emotional attitudes and other external stimuli. Manual coding mainly uses Ekman's FACS system to manually encode facial expressions of photos and videos. The computer's fully automated analysis system, the first and most accurate, is the Noldus Facial Expression Analysis System (FaceReader).

Attitude expression : It mainly measures the change of limb behavior when emotion changes. At present, it can only be measured by manual marker coding. Solution – Noldus' Behavioral Observation and Analysis System (The Observer XT). In addition, it is possible to investigate the use of a motion capture system to measure gesture expressions.

Voice expression : At present, the computer automatic measurement method is adopted. The best solution is Israeli technology, and two companies are dedicated to speech sentiment analysis. Noldus' Sound Behavior Analysis System (UltraVox) is also an option.

3. Arousal of emotional physiology: At present, EEG, heart rate, respiration, and skin electricity are used to measure emotional physiology awakening. Noldus recommends EEG to use the NeuroScan system, which is commonly used in applications. Of course, dry electrode technology can also be used. Other physiological signals are multi-channel physiological recorder (BIOPAC). Of course, we have a more comprehensive solution - data synchronization and integration, if you want to know more applications and other information, please feel free to leave a message or call ~

Applications

Emotional research is very broad, and I will introduce a study on emotions and eating disorders:

Research background :

Facial expressions are part of our body language and play an important role in interpersonal communication. The processing and expression of social emotions is considered to be an important factor in the development and maintenance of dietary disorders. Patients with Anorexia Nervosa have less facial expression and often report when communicating with the outside world, and about bulimia nervosa ( We don't know much about the facial expressions of Bulimia Nervosa and rehabilitated patients. Researchers such as Jenni Leppanen of the University of London's Neuroscience Institute have done relevant validation experiments with FaceReader.

Research methods :

The researchers used the FaceReader and Diet Disorders Questionnaire (EDEQ) to study a total of 297 subjects in AN, BN, RecAN (Recovered from AN) and HC (Healthy control). The subjects watched the film fragments that caused sadness or happiness, and then passed FaceReader analyzes and draws conclusions.

in conclusion:

The positive emotion of HC in the positive segment of Movie 1 (humorous scene) is significantly higher than that of the RecAN group, while there is no significant difference in the sad mood in the sad segment of Movie 2 (tragic scene).

References :

[1] Peng Yuling. General Psychology [M]. 4th Edition. Beijing: Beijing Normal University Press, 2012 (5) 406-408

[2] (US) Paul Ekman, Deng Bojun translation. Lying [M]. Beijing: Xinzhi Sanlian Bookstore, 2008 (12)

[3] Wang Yan. Mind Reading FBI and CIA teach you to crack the super psychological code [M]. Beijing: Tsinghua University Press, 2013 (8) 107-205.

[4] Jenni Leppanen1, Marcela Marin Dapelo et al. Computerised analysis of facial emotion expression in eating disorders [J]. Public Library of Science, 2017 (6) 1-10

Pay attention to the QR code below to get more product information and news!