Complementary Labels and Their Impact on Deep Learning of a Target Class : Evaluated on Object Detection in the Low Data Regime

Abstract: In specialized object detection tasks and domains, it is sometimes only possible to collect and annotate a small amount of data for training and evaluation, which constrains training to a low data regime that can lead to poor generalization. In this thesis, the impact of annotations from additional classes, referred to as complementary labels, when learning a target class is studied as a potential approach to improve performance in the low data regime, for object detection. In particular, the thesis aims to investigate in which data regimes complementary labels seem beneficial, whether labels from different complementary classes contribute equally to the performance on the target class, and how varying the number of complementary classes can affect the performance on the target class. Two datasets were studied; CSAW-S, a medical dataset, and MSCOCO, a natural dataset. For each of these datasets, three experiments were conducted to examine various aspects of complementary labels. First, an experiment that compares the use of all available complementary labels and no complementary labels is conducted for various data regimes. Second, an experiment that leaves out individual complementary classes during training is performed. Third, an experiment that varies the number of complementary classes used during training is performed. The results suggest that complementary labels are helpful in the low data regime, provided the complementary classes have sufficient representation in the dataset. Furthermore, complementary classes that have clear context and interaction with the target class seem to be beneficial, and the impact of individual complementary classes does not seem to be cumulative. Lastly, increasing the number of complementary classes used seems to have a stabilizing effect on the target class performance, provided enough classes are used. Due to limitations in the methodology and choice of experiments, these findings are not conclusive. Nevertheless, various improvements to the methodology of studying complementary labels have been identified, which can help future studies present stronger conclusions.