Expressive Automatic Music Transcription : Using hard onset detection to transcribe legato slurs for violin

Abstract: Automatic Music Transcriptions systems such as ScoreCloud aims to convert audio signals to sheet music. The information contained in sheet music can be divided into increasingly descriptive layers, where most research on Automatic Music Transcription is restricted on note-level transcription and disregard expressive markings such as legato slurs. In case of violin playing, legato can be determined from the articulated, "hard" onsets that occur on the first note of a legato slur. We detect hard onsets in violin recordings by three different methods — two based on signal processing and one on Convolutional Neural Networks. ScoreCloud notes are then labeled as articulated or slurred, depending on the distance to the closest hard onset. Finally, we construct legato slurs between articulated notes, and count the number of notes where the detected slur label matches ground-truth. Our best-performing method correctly labels notes in 82.9% of the cases, when averaging on the test set recordings. The designed system serves as a proof-of-concept for including expressive notation within Automatic Music Transcription. Vibrato was seen to have a major negative impact on the performance, while the method is less affected by varying sound quality and polyphony. Our system could be further improved by using phase input, data augmentation, or high-dimensional articulation representations.