Relevance in the eye of the beholder

Abstract: This thesis examines a method for how humans can assess quality of classifications by image based neural network. Through examples visualisations of layerwise relevance propagation (Bach et al. 2015) test participants are tasked with learning to differentiate correctly and incorrectly classified images. To test their learning, they are tasked with evaluating visualisations based on whether they believe the underlying classification to be correct or not. The test was conducted for 20 test participants who after 10 learning examples each were tasked with evaluating the same 72 visualisations. Three different types of visualisations are developed according to different mathematical principles. The results show that the test was often confusing for the test participants indicating both that better preparation and training of test participants might be appropriate but also that the subject is inaccessible to many. There are also small beneficial effects which are expected to increase with better training and preparation.