Building a Deep Neural Network From Scratch

Abstract: Machine learning is becoming increasingly common in our society and is predictedto have a major impact in the future. Therefore, it would be both interesting and valuable tohave a deep understanding of one of the most used algorithms in machine learning, deepneural network. In this project, the goal is to implement a neural network from scratch anduse it to solve an image recognition problem. Specifically, instead of using common machinelearning libraries, such as Pytorch or TensorFlow, we only use a linear algebra library(numpy) and implement the deep learning models and algorithms ourselves. In addition to afundamental implementation of a neural network we implement performance enhancingalgorithms and study the effects these have on our neural network. The finished program is a neural network that can be customized to any size and canimplement different activation functions. It also has several options for different trainingfeatures, called hyperparameters, such as batch-size, learning rate and momentum whichcan be tweaked to optimize the training process. Furthermore, we train neural networks tosolve two different image classification problems each with a different complexity. On theeasier problem MNIST, our neural network correctly classifies handwritten digits with a testaccuracy of 95.75% . On the more complex dataset CIFAR10, the neural network achieved antest accuracy of 41.94%.