Touch Gesture Visualizations in Screencasts – Designing a Pictorial Language Using an Autobiographical Design Method

Abstract: A screencast is a digital recording of computer screen output. The format is frequently used to demonstrate and teach the usage of software functions, and is then often accompanied by narrative audio. In order for a screencast to be considered effective in these cases, it is required that it clearly conveys the interaction between user and computer program. This is problematic in the case of gesture based user interfaces since input by touch gestures lack visual representation other than feedback from the user interface. Thus, from visual information only, it is hard for the audience to interpret which specific gesture, such as touch, long touch and double touch, is used for a specific program function. Instead they have to rely on the producer communicating the information in some other way. In this master’s thesis, a pictorial language with the purpose of visualizing touch gestures in screencasts is designed, since it could be a potential solution to the above dilemma. This is done by combining an autobiographical design method with external user evaluations. The research suggest that users interpret this type of visualizations as interface functions rather than physical touch gestures, and that some of the language’s design attributes are highly dependent on external factors. Thus, for the pictorial language to function as an effective tool in screencasts, it is needed for the producers using the tool to understand both its limitations as well as possibilities.