Comparing energy efficiency of Leaky integrate-and-fire and Spike response neuron models in Spiking Neural Networks

Abstract: Spiking Neural Networks (SNNs) are a type of neural network that is designed to mimic the way neurons function in our brains. While there have been notable advancements in developing SNNs, energy consumption hasn't been studied to the same extent. This gets especially relevant with steadily increasing network sizes. Finding the ideal hyperparameters for the SNN is a way to minimize the energy usage without significantly compromising its performance. In this study, we investigate how two different spiking neuron models compare in terms of accuracy and energy consumption. Two different SNNs are used, the Spike Response model (SRM) against the Leaky Integrate-and-fire model (LIF). The models are trained on two different datasets, N-MNIST and DVS128 Gesture, for better generalization. The results show that the LIF model was more energy efficient on the M-NNIST dataset that had a smaller number of pixels and lacked any temporal dynamics. However, SRM’s consideration of a larger set of previous time instances proved more energy efficient on datasets with a temporal dimension, similar to complex data that resemble real-world scenarios. In order to reach a comprehensive verdict of the ideal neuron model configuration, the models need to be compared to more types of data, as well as further tweaking more hyperparameters used for the training of the SNN.