Investigating Hand Gestures as Additional Input in a Multimodal Input Interface

Abstract: At the time of writing, touch is revolutionizing the market with devices that have touch as the primary input modality. In this thesis we have been able to explore how another input modality, touchless, can complement touch in everyday applications. Touchless refers to an input method that is able to recognize hand gestures and act upon them. To investigate how these two input modalities can be combined, an Android prototype application featuring both touch and touchless was created. We chose to create a recipe reader because we found that a lot of people can relate to having soggy hands in the kitchen. The outcome of this thesis is assembled to form design principles that we find worthy to consider when designing multimodal input interfaces that includes both touch and touchless.