Categorization of Historical Photographs using Convolutional Neural Networks

Abstract: The goal of this project is to explore the possibilities of using Convolutional Neural Networks (CNN) on historical photographs taken in Sweden to determine a feasible way of automatically detecting the studio that the photograph was taken in. Photographs supplied by The City Faces Project were used for this purpose. The CNN algorithm that was chosen for this project is version 8 of the "You Only Look Once" (YOLO) algorithm. The primary method that was explored was to analyse the objects located in historical photographs to answer the question of whether it is possible to detect the studio that a photograph was taken in based on the objects located within it. The primary results from this are two models: the General Model, that can find a number of different objects in the photographs with a mean average precision (mAP) score of 0.7523, and the Specific Model, which can find specific types of objects with a mAP score of 0.6086. The other methods that were explored revolved around training a classification model on the photographs to determine where they came from. The classification model that showed the most success was trained on detecting what region of Sweden the photos came from. It had a mAP score of 0.26 while the other classification models proved unsuccessful. In addition some experiments in explainable AI were conducted with the aim of finding the best methods to determine what was the primary part of an object that a model thought distinguished it from other objects. Those experiments indicated that explainable AI is a growing field with potential to become a key component in object detection but at this time the reliability of those methods leaves one with more questions than answer.