Assessing the Viability of Random Indexing in Song Recommender Systems

Abstract: This thesis assesses how Random Indexing performs as a recommender system for music recommendations. Recommender systems have gotten more and more important as the amount of content provided gets larger and larger. They are usually focused on either product traits, and how they relate, or users and their past consumption. Random indexing is a method that originates in natural language processing, and it works by defining some context where the Distributional Hypothesis is used. The Distributional hypothesis says that words that occur in the similar contexts, have similar meanings. The thought is that this should also be the case for songs in playlists. A model is trained and used for participation in the Spotify Million Playlist Dataset Challenge. The recommendations made are evaluated and compared to the state-of-the-art, and an analysis of the e-CSI model is done to see how this may impact customer satisfaction and customer loyalty. The results were unexpected, as training on a smaller subset of the whole dataset performed best. Compared to the state-of-the-art, it is shown that Random Indexing on its own is not comparable performance wise. However, there might be privacy benefits, but this is speculative. This lesser performance would according to the e-CSI model affect customer satisfaction and customer loyalty negatively. Finally, there is a discussion about possible reasons to why the smaller dataset performed better, as well as limitations and possible improvements.