GreenML : A methodology for fair evaluation of machine learning algorithms with respect to resource consumption

Abstract: Impressive results can be achieved when stacking deep neural networks hierarchies together. Several machine learning papers claim state-of-the-art results when evaluating their models with different accuracy metrics. However, these models come at a cost, which is rarely taken into consideration. This thesis aims to shed light on the resource consumption of machine learning algorithms, and therefore, five efficiency metrics are proposed. These should be used for evaluating machine learning models, taking accuracy, model size, and time and energy consumption for both training and inference into account. These metrics are intended to allow for a fairer evaluation of machine learning models, not only looking at accuracy. This thesis presents an example of how these metrics can be used by applying them to both text and image classification tasks using the algorithms SVM, MLP, and CNN.