The Player as a Conductor : Utilizing an Expressive Performance System to Create an Interactive Video Game Soundtrack

Abstract: Music is commonly applied in art and entertainment to enhance the emotional experience. In video-games and other non-linear mediums this task must be achieved dynamically at run-time, as the timeline of events is unknown in advance. Different techniques have been developed to solve this issue, but most commercial applications still rely on pre-rendered audio. In this study, I investigate using a Computer System for Expressive Music Performance (CSEMP) to dynamically shape a computer performance of a pre-composed track to a small platforming game. A prototype environment utilising the KTH Rule System was built and evaluated through semistructured interviews and observations with 7 participants. The results suggest that changes in the musical performance can successfully reflect smaller changes in the experience such as character movement, and are less effective for sound effects or more dramatic changes, such as when the player is engaging in combat or when the player loses. All participants preferred the interactive soundtrack over a non-interactive version of the same soundtrack.