Transfer learning techniques in time series analysis

Abstract: Deep learning works best with vast andd well-distributed data collections. However, collecting and annotating large data sets can be very time-consuming and expensive. Moreover, deep learning is specific to domain knowledge, even with data and computation. E.g., models trained to classify animals would probably underperform when they classify vehicles. Although techniques such as domain adaptation and transfer learning have been popularised recently, tasks in cross-domain knowledge transfer have also taken off. However, most of these works are limited to computer vision. In the domain of time series, this is relatively underexplored. This thesis explores methods to use time series data from one domain to classify data generated from another domain via transfer learning. It focuses on using accelerometer data from running recordings to improve the classification performance on jumping data based on the apparent similarity of individual recordings. Thus, transfer learning and domain adaptation techniques were used to use the learning acquired through deep model training on running sequences. This thesis has performed four experiments to test this domain similarity. The first one consists of transforming time series with the continuous wavelet transform to get both time and frequency information. The model is then pre-trained within a contrastive learning framework. However, the continuous wavelet transformation (CWT) did not improve the classification results. The following two experiments consisted of pre-training the models with self-supervised learning. The first one with a contrastive pretext-task improved the classification results, and the resilience to data decrease. The second one with a forward forecasting pretext-task improved the results when all the data was available but was very sensitive to data decrease. Finally, the domain adaptation was tested and showed interesting performances on the classification task. Although some of the employed techniques did not show improvement, pre-training using contrastive learning on the running dataset has shown great improvement to classify the jumping dataset.