Analyzing How Blended Emotions are Expressed using Machine Learning Methods

Abstract: Blended emotion is a classification of emotional experiences that involve the combination of multiple emotions. Research on the expression of blended emotions allows researchers to understand how different emotions interact and coexist in an individual’s emotional experience. Using machine learning to analyze mixed emotions may indeed bring new insights to the study of blended emotions. This thesis aims to explore blended emotion expression by testing machine learning models (SVM, Decision Tree, and Naive Bayes) trained on the single motion dataset on the blended emotion datasets and vice versa, to analyze the relationship between blended emotions and their constituent emotions. Furthermore, this thesis explores whether there is a dominant emotion in blended emotions and conducts an ablation study to investigate the importance of various facial features within each emotion. The results of testing models’ generalization capabilities propose that blended emotion expressions are highly likely to result from the overlapping combinations of features from their constituent emotions or the combination of some features from one constituent emotion with some from another. Furthermore, based on the dataset used, this thesis also finds that happiness predominated in the blended emotion ’disgust & happiness’. Additionally, an ablation study is conducted to identify the features that have the most significant impact on the accuracy and F1 score of single/pure emotion and blended emotion recognition across various recognition models.