Local collaboration in a Mixed Reality environment : Adding virtual heads to improve social presence

Abstract: This thesis investigates the use of virtual avatar heads to enhance local video see-through collaboration in a mixed reality environment. When users engage with each other using a head-mounted display, the device sits atop their head and obstructs their view, concealing their gaze and facial expressions while collaborating. This makes communication harder by removing non-verbal hints. The proposed solution aims to restore them by rendering virtual avatar heads on top of users heads. The study examines the impact of different avatar styles, including the potential effects of the uncanny valley, as well as the use of lip syncing versus facial tracking to animate avatar mouths. An application was developed using Unity3D to implement this solution, allowing two users to collaborate in a mixed reality environment with avatars on their heads. An experiment was conducted with 56 participants, where users collaborated in two tasks: the twenty question game and a collaborative object placement task. A between-subject design was used to compare with and without avatars, avatar rendering type and avatar lip animation. During the experiment, social presence, user experience and performance were measured through questionnaires (Networked Minds of social presence, NASA TLX and User Experience Questionnaire) and eye gaze data. The study found that although there was not always a strong difference between no avatar and avatars, the use of cartoon avatars with lip syncing was the most favorable option, enhancing the user’s comfort and facilitating interpretation of their partner’s emotions and feelings, as well as receiving more attention from them, which could be due to the Uncanny Valley. However, no evidence of performance improvement was found. The findings of this study have important implications for the design of collaborative mixed reality environments, highlighting the potential benefits of using virtual avatars to enhance communication and social presence. The study also underscores the importance of avatar style and facial animation as well as the potential impact of the uncanny valley on user experience.