Visualizing the Body Language of a Musical Conductor using Gaussian Process Latent Variable Models

Abstract: In this bachelors’ thesis we investigate and visualize a Gaussian process latent variable model (GP-LVM), used to model high dimensional motion capture data of a musical conductor in a lower dimensional space. This work expands upon the degree project of K. Karipidou, ”Modelling the body language of a musical conductor using Gaussian Process Latent Variable Models”, in which GP-LVMs are used to perform dimensionality reduction of motion capture data of a conductor conducting a string quartet, expressing four different underlying emotional interpretations (tender, angry, passionate and neutral). In Karipidou’s work, a GP-LVM coupled with K-means and an HMM are used for classification of unseen conduction motions into the aforementioned emotional interpretations. We develop a graphical user interface (GUI) for visualizing the resulting lower dimensional mapping performed by a GP-LVM side by side with the motion capture data. The GUI and the GP-LVM mapping is done within Matlab, while the open source 3D creation suite Blender is used to visualize the motion capture data in greater detail, which is then imported into the GUI. Furthermore, we develop a new GP-LVM in the same manner as Karipidou, but based on the angles between the motion capture nodes, and compare its accuracy in classifying emotion to that of Karipidou’s location based model. The evaluation of the GUI concludes that it is a very useful tool when a GP-LVM is to be examined and evaluated. However, our angle-based model does not improve the classification result compared to Karipidou’s position-based. Thus, using Euler angles are deemed inappropriate for this application. Keywords: Gaussian process latent variable model, motion capture, visualization, body language, musical conductor, euler angles.