An Explorative Study of Interaction with Tracked Objects in a Virtual Reality Game

Abstract: In recent years, Internet enabled objects (Internet of Things) and their augmentation through virtual reality has become both technically possible and increasingly advanced. This paper explores how interaction between physical IoTs and their virtual and digital twins can be advanced. Three virtual reality games focusing on three different interactions were created. One game focused on rotating, another on pushing and pulling, and a third on lifting and dropping. All games revolved around the use of real tracked cardboard boxes which were represented in the games by a virtual box in the same relative position and rotation to the player, allowing participants to manipulate the virtual boxes by manipulating the physical boxes. 14 participants were asked to play the games and were afterwards interviewed in regards to the games, their interactions with them, their enjoyability and their thoughts regarding them and the concept in general. The results were acquired from game performance, the reflections of the participants, and the subsequent analysis of recorded audio and video. The study presents the difficulties, challenges and opportunities of such a system, while also providing insight into lessons learned from the creation of the system and the games. The main contributions of the paper are the lessons learned in creating the games and experiences in addition to a few specific areas of interest for future research on the area, namely the importance of ergonomic consideration and affordance evaluation.