Towards Embedded Implementations of Multi-Armed Bandit Algorithms for Optimised Channel Selection

Abstract: The proliferation of Internet of Things increases the demand for achieving a high performance for embedded devices on the Internet. The IEEE 802.15.4 wireless network standard provides ultra low complexity, cost, power consumption and data rate for inexpensive devises, although it has problems such as external interference and multi-path fading. The Time Slotted Channel Hopping (TSCH) and MiCMAC are examples of medium access control (MAC) protocols who mitigate these problems by employing channel hopping. However, their limitations are that they predefine channel hopping sequences at compile time and do not perform any optimisations. A  suggested solution to these issues are the use of Multi-Armed Bandit (MAB) algorithms, which have shown to give significantly better performances compared to TSCH. However, these algorithms have only been simulated and not implemented for embedded devices. In this thesis, we present embedded implementations of the MAB algorithms Sliding-Window Upper Confidence Bound (SW-UCB) and Discounted-UCB (D-UCB) in the C programming language with fixed-point arithmetic as an alternative for embedded microprocessors that lack a floating-point unit. We measure the performances of the implementations in terms of packet delivery ratio (PDR) by running them with different parameter settings in simulated stationary and non-stationary environments, where they are compared with corresponding floating-point implementations of these algorithms. The results of the experiments are very promising, no significant differences in PDR between the results of the floating-point and fixed-point implementations are shown, which further suggests that MAB algorithms are viable options for optimised channel selection. The results instead depended more on the proper parameter settings for each algorithm, where the optimal settings differed depending on the environment.