A Bayesian Bee Colony Algorithm for Hyperparameter Tuning of Stochastic SNNs : A design, development, and proposal of a stochastic spiking neural network and associated tuner

Abstract: With the world experiencing a rapid increase in the number of cloud devices, continuing to ensure high-quality connections requires a reimagining of cloud. One proponent, edge computing, consists of many distributed and close-to-consumer edge servers that are hired by the service providers. This thesis considers one problem in edge computing, edge user allocation, which involves assigning as many users to as few edge servers as possible. This problem can be formulated as a constraint satisfaction problem (CSP). With many users and edge servers, the complexity of the problem is high, resulting in computationally expensive or possibly infeasible calculations using conventional solvers. We approach this problem using spiking neural networks. For a spiking neural network supplied with sufficient stochastic noise, the distribution of network states converges to a stationary distribution expressed in terms of an energy function. By appropriately designing the network, it is possible to encode the CSP in a stochastic spiking neural network such that the low energy, high probability, states are solutions to the CSP. To maximize the performance of the stochastic spiking neural network, the synaptic weights and neuron parameters require adjusting or tuning. However, the spiking dynamics of the network preclude computation of traditional derivatives, as neurons are governed by discrete and event-based dynamics rather than continuous activation functions. The performance of the spiking neural network is also stochastic. This means that even a poorly tuned network can return good solutions, and vice versa. In this thesis, a stochastic neural network of spiking neurons is designed to solve the edge user allocation problem. For this network a new hyperparameter tuner is proposed, combining aspects from the explorational artificial bee colony algorithm (ABC) with the exploitation of the tree-structured Parzen estimator algorithm (TPE). This new algorithm, ABC-BA, is designed with the aim of both exploring the solution space and exploiting the promising regions. It is also designed to be less sensitive to the inherent stochasticity of the stochastic spiking neural network. The network is tuned and evaluated on four problem sizes: 6, 100, 1000, and 10000 users. Results show that the network finds the optimal solution for the smaller problems while finding solutions slightly under optimum for the larger ones. While not guaranteeing optimal solutions, the stochastic network is, compared with the conventional solver, able to find good solutions for the largest problem. The networks with tuned parameters are also tested on unseen problem instances, results suggesting that the tuned parameters function well on similarly sized problems as the one there are tuned to. To evaluate ABC-BA, the developed algorithm is compared against its two parts. The experiments suggest that ABC-BA outperforms its building blocks in terms of desirable search patterns and parameter performance. An important future research direction is to evaluate whether this conclusion holds for other CSP-solving stochastic spiking neural networks.