A neural network performance analysis with three different model structures

Abstract: This report analyzes three neural network structures: dense, convolutional and recurrent. One data set example and problem has been chosen for each type of structure: a multi-class classification problem, an image classifier and a time sequence prediction, respectively. This report also aims at understanding which structure performs better for different problem statements, and how the different parameters they depend on affect their performance. The most common parameters that have been analyzed are the following: the number of intermediate layers, the number of neurons, the number of epochs, the batch size, the activation function, the loss function and the optimizer. The results showed that a dense structure has high dependencies between the values of its internal operations. Hence, the average execution time for CPU and GPU are similar. However, accelerated algorithms for GPUs made a substantial difference for convolutional and recurrent structures in comparison to CPU launches. Furthermore, the results of each model showed that most of their attributes vary the performance of the model during training, obtaining a combination of values that are suitable for each structure.