SOUND HUNTER : Developing a Navigational HRTF-Based Audio Game for People with Visual Impairments

Abstract: In this thesis, I propose a framework for designing 3D-based audio-only games in which all navigation is based on perceiving the 3D-audio, as opposed to relying on other navigational aids or imagining the audio as being spatial, where additional sounds may be added later on in the development process. To test the framework, a game named Sound Hunter was developed in an iterative process together with both sighted and visually impaired participants in three focus groups, 8 usability tests, and a final evaluation. Sound Hunter currently features 20 levels progressively becoming more difficult, and relies on no stationary equipment. Instead, all navigation is based on the smartphone’s accelerometer data, where the only requirement is headphones to properly perceive the HRTF filtering, being delivered through the Pure Data external [earplug~], using a generalized HRTF and linear interpolation. The results indicate that the suggested framework is a successful guidance tool when wanting to develop faster perception-based and action-filled 3D-audio games, and the learning curve for the navigation was approximately 15 minutes, after which the participants navigated with very high precision. Furthermore, the results showed that there is a high need for audio-only games intended for visually impaired smartphone users, and that with only minor alterations to game menus and adjustments to the iPhone’s accelerometer function, both older and younger visually impaired smartphone users can navigate through 3D-audio environments using simple hand movements. The results also strongly support that Sound Hunter may be used to train people’s spatial hearing in an entertaining way with full experimental control, as the participants felt that they focused more on their hearing and even trained their hearing while playing the game, and sounds were perceived as more externalized than lateralized. Also, the results strongly suggest that there are two main factors affecting the learning curve for adapting to a foreign HRTF during virtual interactive gaming experiences; the adaptation to the navigational controls, and the experience of front/back confusion, where control adaptation is promoted by having a strong default setting with customizable sensitivity, and the experience of front/back confusion can be greatly reduced by introducing complex distance-dependent meta-level communication in synthesized sounds. Using distance-dependent meta-level communication in the wrong way, however, can lead to illusions of false distance, making navigation impossible. All of the participants would have recommended Sound Hunter to others, and they were very impressed by both the quality of the 3D rendering, and the way in which it could be used to navigate, where one of the participants, a blind expert audio-only game developer, also being highly experienced with audio-only games, claimed that Sound Hunter offered the best representation of 3D audio he had ever experienced in an audio-only game