Combining touch & mid-air gestures on public displays

Abstract: This thesis investigates the impact of different factors on public displays that use touch and mid-air gestures. We present a novel application for public displays: MirrorTouch — a game combining touch interaction with mid-air gesturing through depth sense. We studied the impact of these factors through a series field studies. We show that clear affordances are important to communicate both modalities clearly and also how to improve the conversion rate of passers-by. We present our findings of how social situations around such a display varied with the location where the display was placed. We also present quantitative data on group behavior around our public display, such as a measuring of the honeypot effect and how people made transitions between the different modalities. Our results can be of great value to designers and researchers of public displays that want to deploy displays with similar techniques.