Jamming with the Plot Twister : Designing virtual worlds for improvised performances

Abstract: Using a research through design approach, this thesis presents a first prototype of Plot Twister, a novel interactive, audiovisual interface specifically designed for solo musicians improvising alongside computers. This highly simplified and intuitive interface allows the performer to simultaneously manipulate visual and musical content using one single controller, thus enabling the other hand to play an instrument. The design is based on the concept of adaptive music used in video games, where progression of musical form and expression over time is produced through movement in virtual 3D-space. This thesis describes the design process of two interface iterations and audiovisual compositions which are first evaluated using interviews, and then through auto-ethnographic multimodal interaction analysis by stimulated recall to identify its functions and potentials in the context of improvised live performances. The results implicate different ways how the physical interface can both influence as well as enable the user to effortlessly adapt musical form and content to fit her improvised musical exploration. Both audience and performer share access to the visual interface that provides real-time visual feedback of the musical content in a “gamified” style. The technical framework strongly favours looped, continuous, gradual and non-linear progressions of computer-based media where the concept of Plot Twister potentially can be implemented in other interactive contexts in the future.