Design and Development of a Virtual Reality Application to Introduce Gesture-Based Interaction

Abstract: This thesis provides insight into the development of Virtual Reality(VR) applications with the purpose of introducing users to a new gesture interaction platform. Gesturebased interaction and VR are two rapidly evolving technologies with a great potential to complement each other. The gestures used are based on a cognitive model of holding a sphere in your hands and tapping on its surface. By using the hand tracker Leap Motion and the Head Mounted Display Oculus Rift the sphere shaped controller is placed in the user's virtual hands. The thesis is a collaboration with Erghis Technologies AB who developed the concept of the sphere and software to track the gestures. Gesture based interfaces are called natural user interfaces. But a natural experience should be easy to understand and meet the users' expectation to interactions. To meet this design challenge the interface was broken down into the actions the user can perform and making sure that they were conveyed to the user and reinforced with feedback. The ow between user action and system feedback was ne tuned for every gesture to improve the feeling of the interaction. When using the system the users are rst shown the gestures on a video played in the virtual environment. After the short introduction they can form an understanding about the gestures by interacting with the system. For the interaction loop to ow the gestures needs to be reliably tracked. The gestures are easier to track when the hands are easy for the camera to see. This means that gesture interfaces designed for the users to position their hands in a way that can easily bee seen by the tracking camera will improve the user's experience. By placing the tracking camera of the Leap Motion on the Oculus Rift the volumes of their separate tracking systems can be calibrated to match in the real and virtual world. This can be achieved by using a mount and software released during the development of this thesis. The move of the Leap Motion from the table to the HMD was an unforeseen requirement found at the end of development. The change in orientation had a negative impact on tracking reliability since the gestures of the sphere were poorly adapted to this new position of the camera. The developed application "Control Sphere" got a very positive response from user evaluation and has seen more than 200 downloads. The gestures combined with the visual feedback was seen as an engaging method of interaction. A version of the application was created for a regular monitor and user tests conrmed that performing the gestures in VR was a more natural experience.