Using Leap Motion for the Interactive Analysis of Multivariate Networks

Abstract: This work is an interdisciplinary study involving mainly the fields of information visualisation and human-computer interaction. The advancement of technology has expanded the ways in which humans interact with machines, which has benefited both the industry as well as several fields within science. However, scientists and practitioners in the information visualisation domain remain working, mostly, with classical setups constituted of keyboard and standard computer mouse devices. This project investigates how a shift in the human-computer interaction aspect of visualisation software systems can affect the accomplishment of tasks and the overall user experience when analysing two-dimensionally displayed multivariate networks. Such investigation is relevant as complex network structures have seen an increase in use as essential tools to solve challenges that directly affect individuals and societies, such as in medicine or social sciences. The improvement of visualisation software’s usability can result in more of such challenges answered in a shorter time or with more precision. To answer this question, a web application that enables users to analyse multivariate networks through interfaces based both on hand gesture recognition and mouse device was developed. Also, a number of gesture designs were developed for several tasks to be performed when visually analysing networks. Then, an expert in the field of human-computer interaction was invited to review the proposed hand gestures and report his overall user experience of using them. The results show that the expert had, overall, similar user experience for both hand gestures and mouse device. Moreover, the interpretation of the results indicates that the accuracy offered by gestures has to be carefully taken into account when designing gestures for selection tasks, particularly when the selection targets are small objects. Finally, our analysis points out that the manner in which the software’s graphical user interface is presented also affects the usability of gestures, and that both factors have to be designed accordingly.