Transferability of features from multiple depths of a deep convolutional neural network

Abstract: Deep convolutional neural networks are great at learning structures in signals and sequential data. Their performance have surpassed most non-convolutional algorithms on classical problems within the field of image analysis. A reason behind their success is that even though these networks generally need a great amount of examples to learn from, they can be used to learn smaller tasks through different types of transfer learning techniques. When having small amounts of data, a common approach is to remove the output layer and use the remaining network as a feature extractor. In this work we attempt to quantify how network layers go from general to specific through extracting features from multiple depths. The transition was measured on some different object classification problems by training classifiers both directly on the feature vectors and on combinations of the vectors. The reached conclusion was that the feature from the very last layer of a deep convolutional network are very specific to the source task and using it to learn other classification problems is often sub-optimal. The best depth to extract features from depends on how similar the problem you want to learn is to the source task.