3D Facial Modelling for Valence Estimation

Abstract: We, as humans, purposely alter our facial expression to convey information during our daily interactions. However, our facial expressions can also unconsciously change based on external stimuli. In the current thesis, we focus on visual stimuli and hypothesize that our facial expression is indicative of the perceived valence -namely the pleasantness- of the former. To evaluate our hypothesis, we experiment with different neural network architectures on a 3D facial mesh-valence dataset. At first, various VAE-based architectures, adapted accordingly to operate on 3D meshes, were employed to extract representation embeddings of facial meshes. Thereafter an LSTM head was utilized to address the different sequential downstream tasks, including the valence estimation. In addition, representation disentanglement approaches were considered, aiming at representing the facial shape and expression independently. Although our experiments suggest that the facial expression is not a reliable estimator of the perceived valence, we demonstrate that mesh VAE-based architectures can be employed to extract competent mesh representations and address less ambiguous downstream tasks such as expression classification. Additionally, we observed that representation disentanglement boosts the performance in both terms of facial expression classification and valence estimation. Finally, we highlight the mesh VAE capabilities in morphing between existing meshes as well as generating novel samples.