Implementation and Robustness of Hopfield Networks with Spiking Neurons

Abstract: Computational models of neural activity and neural networks have been an active area of research as long as there have been computers, and have led several important discoveries in the eld of machine learning. One kind of articial network proposed by John J. Hopeld in 1982 has been among the more successful ones, and is still in active use today. It has been suggested that in addition to its merits in machine learning, it could also serve as a foundation of the explanation of human ability of recollection and association. However, Hopeld's original design used a very simplied model of neurons. By using so called integrate-and-remodels, higher realism can be achieved. This report begins with a discussion of mechanistic and quantitative description of neurons, in particular the induction of action potentials, and furthermore why an integrate-and-re model is a reasonable choice for a model of intermediate complexity. By explicitly describing individual spikes, a fundamental but often neglected characteristic of communication between neurons is captured. Integrate-and-re models are included in the Neural Simulation Tool (NEST), and in this report such a neural model is applied to Hopeld networks. Both spike-rate coding and temporal coding are studied, as well as a simple model of synaptic Spike-Timing DependentPlasticity (STDP) for online learning. The networks' robustness is evaluated with respect to changes in (a) global scaling of the synaptic weights, (b) delays in the synaptic connections, (c) level of noise and (d) strength of input stimuli. They are found to be somewhat sensitive, with (a) giving the most denite results, suggesting that the used description of Hopfield networks might not be an immediately plausible biological model. In particular, networks using temporal coding are found to be especially difficult to calibrate. This could reveal a potential weakness in relatively recent and apparently successful models.