Learning Style Compatibility on Fashion Data

University essay from KTH/Skolan för elektroteknik och datavetenskap (EECS)

Author: Lukas Frösslund; [2021]

Keywords: ;

Abstract: Fashion Recommendation can be defined as a set of systems that tries to predict and retrieve a curated and often ranked selection of fashion items based on the preference of one or more target consumers. Traditional systems relied on providing substitute recommendations, meaning that they were centered around finding similarities between fashion items. However, recent approaches have aimed to provide complementary recommendations that are instead built on item compatibility. Outfit Matching has recently emerged as a popular task when modelling compatibility between fashion items. The objective of the task is to retrieve a set of fashion items, each of a different item category, such that the items collectively can be considered visually compatible. In this thesis, two state-of-the-art deep neural network models, earlier used for the task of matching outfits, were implemented to investigate their performance on a novel task of matching fashion styles. This more unconstrained task accepted duplicate item categories as well as mixed demographics, enabling the retrieval of a larger and more diverse selection of fashion items. A fashion dataset was constructed for the thesis, where the two models were evaluated on the data using the Fill-in-the-blank (FITB) experiment commonly used in fashion compatibility modelling. Additionally, an item retrieval test was conducted, evaluated using recall @ top k to determine the ability of the models to learn style compatibility in a retrieval setting. Results showed that both models struggled when introducing fewer constraints, with an FITB accuracy of 48.97% when matching fashion styles, compared to 63.73% on the outfit matching task. However, an increase in the embedding dimension of the data yielded a significant increase in accuracy. When performing experiments using previously unseen classes of data, no significant decrease in performance was noted, suggesting an ability in both models to generalize well to new fashion styles. Retrieval tests could show a clear preference in both models to retrieve relevant items, with recall values reaching 54.30% for a k-value of 50. Suggestions for future work include efforts to be put on improving shortcomings in the data by ensuring all samples to be distinct in style, and as well to move beyond solely visual data and include semantic textual data in the embedding representation. Finally, the construction of a benchmark dataset for style compatibility modelling would be beneficial in drawing attention to the task. 

  AT THIS PAGE YOU CAN DOWNLOAD THE WHOLE ESSAY. (follow the link to the next page)